Home

execute_scripts.sh

execute_scripts.sh
- Purpose
- License
- History
- Operating system
- Language / type
- Prerequisites
- Usage
- Example
  - Other Examples
- Internas
- Notes
- Download

Purpose

The script execute_scripts.sh can be used to execute a number of scripts or executables on the localhost either sequentiel or parallel.
In the later case you can restrict the maximum number of parallel running processes and change the timeout values via parameter -w and -W:

-W timeout[/intervall] "timeout" is the timeout for background processes and "intervall" is the wait intervall for background processes. Use "-1" or "none" for the timeout value to disable the timeout. Current values: Timeout: 1h (= 3600 second(s) ) Intervall: 2s (= 2 second(s) ) Use a trailing "m" for times in minutes or a trailing "h" for times in hours; "s" is for seconds which is the default if neither "m" nor "h" is used. Use "default" for any value to use the default value. Use " -W /intervall" to only change the intervall. You can also use "," to separate the values. This parameter is only used if the commands are executed in parallel (parameter "-d") Long format: --timeout -w noOfBackgroundprocesses[/startIntervall[/maxStartTime]] "noOfBackgroundprocesses" is the max. number of background processes running at the same time, "startIntervall" is the wait intervall and "maxStartTime" is the timeout for starting the background processes. Use "-1" or "none" for an unlimited number of parallel background processes. Use "-1" or "none" for the timeout value to disable the timeout. Current values: Max. number of parallel running background processes: -1 Wait intervall: 5s (= 5 second(s) ) Timeout value: 5m (= 300 second(s) ) Use a trailing "m" for times in minutes or a trailing "h" for times in hours; "s" is for seconds which is the default if neither "m" nor "h" is used. Use "default" for any value to use the default value. Use " -w /startIntervall" to only change the start intervall. Use " -w //maxStartTime" to only change the timeout You can also use "," to separate the values. This parameter is only used if the commands are executed in parallel (parameter "-d") Long format: --noOfbackgroundProcesses

An example usage of the script is in the package JWM for Linux version 2.0.0 and newer (08.09.2014: not yet released but coming soon ..)

License

# CDDL HEADER START
#
# The contents of this file and the script are subject to the terms of the
# Common Development and Distribution License, Version 1.0 only
# (the "License"). You may not use this file except in compliance
# with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END

History

Version	Releasedate	Description
1.0.0	08.06.2014	initial release
1.0.1	09.01.2016	#### die will now end the script even if the parameter -f is used
1.0.2	01.01.2017	#### replaced "set -o noglob" with "set -f" and "set -o glob" with "set +f" to be compatible with shells #### like mirksh that to not support the glob/noglob syntax

Operating system

Tested with Linux; should also work on other Unix Operating systems

Language / type

Kornshell Script

Prerequisites

ksh must be installed

Usage

[xtrnaw7@t540p scripts]$ ./execute_scripts.sh -h -v [08.06.2014 08:29:56] execute_scripts.sh v1.0.0 started at So 8. Jun 08:29:56 CEST 2014. [08.06.2014 08:29:56] No config file ("execute_scripts.conf") found (use -C to create a default config file) execute_scripts.sh v1.0.0 - execute multiple excecutables either parallel or sequentiell Usage: execute_scripts.sh [-v|+v] [-q|+q] [-h] [-l logfile|+l] [-y|+y] [-n|+n] [-D debugswitch] [-a|+a] [-O|+O] [-f|+f] [-C] [-H] [-X] [-S n] [-V] [-T] [-d|+d] [-W [timeout[/intervall]] [-w NoOfBackgroundProcs[/intervall[/timeout]]]] [-c shell] [-k|+k] [-r|+r] [-B|+B] -I [listfile|directory|regex] [-i executable] [-x executable] [-o workdir] [-s startscript] [-z stopscript] The parameters -I or -i are mandatory. Use the parameter "-v -h [-v]" to view the detailed online help; use the parameter "-X" to view some usage examples. see also http://bnsmb.de/solaris/execute_scripts.html Note: Use -{switch} or --{longswitch} to turn an option on; use +{switch} or ++{longswitch} to turn an option off The long format of the parameter (--parameter/++parameter) is not supported by all ksh implementations Parameter: -v|+v - turn verbose mode on/off; current value: y Long format: --verbose / ++verbose -q|+q - turn quiet mode on/off; current value: n Long format: --quiet / ++quiet -h - show usage Long format: --help -l - set the logfile current value: /var/tmp/execute_scripts.LOG Long format: --logfile +l - do not write a logfile Long format: ++logfile -y|+y - assume yes to all questions or not Long format: --yes / ++yes -n|+n - assume no to all questions or not Long format: --no /++no -D|+D - debug switch current value: use "-D help" to list the known debug switches Long format: --debug / ++debug -a|+a - turn colors on/off; current value: n Long format: --color / ++color -O|+O - overwrite existing files or not; current value: n Long format: --overwrite / ++overwrite -f|+f - force; do it anyway; current value: n Long format: --force / ++force -C - write a default config file in the current directory and exit Long format: --writeconfigfile -H - write extended usage to STDERR and exit Long format: --doc -X - write usage examples to STDERR and exit Long format: --view_examples -S n - print error/warning summaries: n = 0 no summariess, 1 = print error msgs, 2 = print warning msgs, 3 = print error and warning mgs Current value: 0 Long format: --summaries -V - write version number to STDOUT and exit Long format: --version -T - append STDOUT and STDERR to the file "/var/tmp/execute_scripts.sh.15413.tee.log" Long format: --tee -I [listfile|directory|regex] The value for the parameter -I is either a file with a list of executables to execute, a directory, or a regular expression for files. If the value is a directory all executable files in in the directory will be executed. If the value is a regular expression all executables that match this regular expression will be executed. In both cases the script sorts the list of executables before adding them to the list of executables to execute A listfile is a file with a list of executables to execute The format of the file is: one executable per line, empty lines and lines beginning with a hash "#" are ignored. Parameter for the executables are NOT allowed. Use a comma "," to separate multiple listfiles or directories. or use the parameter "-I" more than one time. Missing listfiles or directories or regex that do not expand to a file are not treated as error. Current list of executables is: "" Long format: --list -i executable "executable" is an executable to execute. Use a comma "," to separate multiple executables or use the parameter "-i" more than one time. Current list of executables is: "" Long format: --exec -x executable exclude the executable "executable" from the execution Use a comma "," to separate multiple executables or use the parameter "-i" more than one time. Regular expressions for "executable" are allowed. Use "-x none" to clear the list of executables to exclude. Current value: "" Long format: --exclude -s startscript The executable "startscript" is executed before all other executables. And it is always executed in the foreground Current value: "" Long format: --startscript -z stopscript The executable "stopscript" is executed after all other executables finished. And it is always executed in the foreground Current value: "" Long format: --stopscript -o working directory This is the directory used for temporary files and the log files of the executables. Current value: /var/tmp/execute_scripts.sh.15413.work Long format: --workdir -c shell_to_use "shell_to_use" is the shell to execute scripts use "default" to use the shell from this script and use "none" to use no explicit shell Current value "/usr/bin/ksh" Long format: --shell -r|+r remove duplicates from the list of executables Current value "1" Long format: --remove_duplicates -d|+d execute the commands in parallel Current value: n Long format: --parallel -k|+k add comments to the output file Current value: y Long format: --nocomments -B|+B stop after an executable returns a non-zero return code (only if the executables are executed in sequentiell order) Current value: n Long format: --stop_after_error -W timeout[/intervall] "timeout" is the timeout for background processes and "intervall" is the wait intervall for background processes. Use "-1" or "none" for the timeout value to disable the timeout. Current values: Timeout: 1h (= 3600 second(s) ) Intervall: 2s (= 2 second(s) ) Use a trailing "m" for times in minutes or a trailing "h" for times in hours; "s" is for seconds which is the default if neither "m" nor "h" is used. Use "default" for any value to use the default value. Use " -W /intervall" to only change the intervall. You can also use "," to separate the values. This parameter is only used if the commands are executed in parallel (parameter "-d") Long format: --timeout -w noOfBackgroundprocesses[/startIntervall[/maxStartTime]] "noOfBackgroundprocesses" is the max. number of background processes running at the same time, "startIntervall" is the wait intervall and "maxStartTime" is the timeout for starting the background processes. Use "-1" or "none" for an unlimited number of parallel background processes. Use "-1" or "none" for the timeout value to disable the timeout. Current values: Max. number of parallel running background processes: -1 Wait intervall: 5s (= 5 second(s) ) Timeout value: 5m (= 300 second(s) ) Use a trailing "m" for times in minutes or a trailing "h" for times in hours; "s" is for seconds which is the default if neither "m" nor "h" is used. Use "default" for any value to use the default value. Use " -w /startIntervall" to only change the start intervall. Use " -w //maxStartTime" to only change the timeout You can also use "," to separate the values. This parameter is only used if the commands are executed in parallel (parameter "-d") Long format: --noOfbackgroundProcesses [08.06.2014 08:29:56] The log file used was "/tmp/execute_scripts.sh.15413.TEMP" [08.06.2014 08:29:56] execute_scripts.sh v1.0.0 started at So 8. Jun 08:29:56 CEST 2014 and ended at So 8. Jun 08:29:56 CEST 2014. [08.06.2014 08:29:56] The RC is 1. [xtrnaw7@t540p scripts]$

Example

Run the executables in sequential order:

Run the executables in parallel:

[xtrnaw7@t540p scripts]$ ./execute_scripts.sh -I "test" -x "*/start.sh" -x "*/stop.sh" -s test/start.sh -z test/stop.sh -I test/s00* -I listfile_not_there -I logs -I "test/00*no_files_for_this_mask" -D no_of_error_loglines=100 -D no_of_ok_loglines=50 -D no_of_logfile_lines=20 -i /usr/bin/uname -I neither_dir_nor_file -B -d -W ,10
[08.06.2014 08:36:34] execute_scripts.sh v1.0.0 started at So 8. Jun 08:36:34 CEST 2014.
[08.06.2014 08:36:34] No config file ("execute_scripts.conf") found (use -C to create a default config file)
[08.06.2014 08:36:34] WARNING: No executables found for the regular expression "test/s00*"
[08.06.2014 08:36:34] WARNING: File/Directory "listfile_not_there" NOT found.
[08.06.2014 08:36:34] WARNING: No executables found in the directory "logs"
[08.06.2014 08:36:34] WARNING: No executables found for the regular expression "test/00*no_files_for_this_mask"
[08.06.2014 08:36:34] WARNING: File/Directory "neither_dir_nor_file" NOT found.
[08.06.2014 08:36:34] Using the log file "/var/tmp/execute_scripts.LOG"
[08.06.2014 08:36:34]
[08.06.2014 08:36:34] Will process 12 executable(s). (Parameter -I and/or -i)
[08.06.2014 08:36:34]   The regular expression(s) for executables that should not be executed are (Parameter -x):
[08.06.2014 08:36:34]      */start.sh */stop.sh
[08.06.2014 08:36:34]   Duplicates are NOT removed from the list of executables (Parameter -r to change)
[08.06.2014 08:36:34] Using the start script "/data/develop/scripts/test/start.sh" (Parameter -s)
[08.06.2014 08:36:34] Using the stop script "/data/develop/scripts/test/stop.sh" (Parameter -z)
[08.06.2014 08:36:34] The working directory is "/var/tmp/execute_scripts.sh.19041.work". (Parameter -o)
[08.06.2014 08:36:34] Start and stop comment lines will be added to the logfiles (Parameter -k to change)
[08.06.2014 08:36:34]
[08.06.2014 08:36:34] The executables will run parallel in the background (Parameter -d)
[08.06.2014 08:36:34]
[08.06.2014 08:36:34] The maximum number of parallel background processes is -1 (Parameter -w -1,x,x -1 = not limited).
[08.06.2014 08:36:34]   The wait intervall for starting the background processes is 5 second(s) (Parameter -w x,5s,x);
[08.06.2014 08:36:34]   the timeout for starting the background processes is 300 second(s) (Parameter -w x,x,5m -1 = not limited).
[08.06.2014 08:36:34]
[08.06.2014 08:36:34] Waiting up to 3600 second(s) for the background processes to finish (Parameter -W 1h,x, -1 = not limited).
[08.06.2014 08:36:34]   The wait intervall for waiting for the background processes to finish is 10 second(s) (Parameter -W x,10 )
[08.06.2014 08:36:34]
[08.06.2014 08:36:34] The last 20 lines of the logfiles of the executables will be printed after the executable finished (Parameter -D no_of_logfile_lines=20)

*** Enter <l><return> to list the executables to execute, <x> to list the regex for executables not to execute, <p> to view the script parameter
*** <y><return> to start or <n><return> to abort. Default is <n>: y
[08.06.2014 08:36:36] Starting processing ...
[08.06.2014 08:36:36] Executing the start script "/data/develop/scripts/test/start.sh" ...
This is /data/develop/scripts/test/start.sh ...
[08.06.2014 08:36:36]   Number of running processes are: 0
[08.06.2014 08:36:36]   ---- Processing the executable "/data/develop/scripts/test/001test.sh" ... ( 1 from 12; 0 already started)
[08.06.2014 08:36:36]     The log file is "/var/tmp/execute_scripts.sh.19041.work/001test.sh.19041.log"
[08.06.2014 08:36:36]   Number of running processes are: 1
[08.06.2014 08:36:36]   ---- Processing the executable "/data/develop/scripts/test/002test.sh" ... ( 2 from 12; 1 already started)
[08.06.2014 08:36:36]     The log file is "/var/tmp/execute_scripts.sh.19041.work/002test.sh.19041.log"
[08.06.2014 08:36:37]   Number of running processes are: 2
[08.06.2014 08:36:37]   ---- Processing the executable "/data/develop/scripts/test/003test.sh" ... ( 3 from 12; 2 already started)
[08.06.2014 08:36:37]     The log file is "/var/tmp/execute_scripts.sh.19041.work/003test.sh.19041.log"
[08.06.2014 08:36:37]   Number of running processes are: 3
[08.06.2014 08:36:37]   ---- Processing the executable "/data/develop/scripts/test/004test.sh" ... ( 4 from 12; 3 already started)
[08.06.2014 08:36:37]     The log file is "/var/tmp/execute_scripts.sh.19041.work/004test.sh.19041.log"
[08.06.2014 08:36:37]   Number of running processes are: 4
[08.06.2014 08:36:37]   ---- Processing the executable "/data/develop/scripts/test/005test.sh" ... ( 5 from 12; 4 already started)
[08.06.2014 08:36:37]     The log file is "/var/tmp/execute_scripts.sh.19041.work/005test.sh.19041.log"
[08.06.2014 08:36:37]   Number of running processes are: 5
[08.06.2014 08:36:37]   ---- Processing the executable "/data/develop/scripts/test/006test.sh" ... ( 6 from 12; 5 already started)
[08.06.2014 08:36:37]     The log file is "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log"
[08.06.2014 08:36:37]   Number of running processes are: 6
[08.06.2014 08:36:37]   ---- Processing the executable "/data/develop/scripts/test/007test.sh" ... ( 7 from 12; 6 already started)
[08.06.2014 08:36:37]     The log file is "/var/tmp/execute_scripts.sh.19041.work/007test.sh.19041.log"
[08.06.2014 08:36:37]   Number of running processes are: 7
[08.06.2014 08:36:37]   ---- Processing the executable "/data/develop/scripts/test/008test.sh" ... ( 8 from 12; 7 already started)
[08.06.2014 08:36:37]     The log file is "/var/tmp/execute_scripts.sh.19041.work/008test.sh.19041.log"
[08.06.2014 08:36:37]   Number of running processes are: 8
[08.06.2014 08:36:37]   ---- Processing the executable "/data/develop/scripts/test/009test.sh" ... ( 9 from 12; 8 already started)
[08.06.2014 08:36:37]     The log file is "/var/tmp/execute_scripts.sh.19041.work/009test.sh.19041.log"
[08.06.2014 08:36:37] ++++ The executable "/data/develop/scripts/test/start.sh" is on the exclude list. Ignoring this executable.
[08.06.2014 08:36:37] ++++ The executable "/data/develop/scripts/test/stop.sh" is on the exclude list. Ignoring this executable.
[08.06.2014 08:36:37]   Number of running processes are: 9
[08.06.2014 08:36:37]   ---- Processing the executable "/usr/bin/uname" ... ( 10 from 12; 9 already started)
[08.06.2014 08:36:37]     The log file is "/var/tmp/execute_scripts.sh.19041.work/uname.19041.log"
[08.06.2014 08:36:37] The loop to start the 10 background processes ended at So 8. Jun 08:36:37 CEST 2014, the runtime is 0:00:00.
[08.06.2014 08:36:37]
[08.06.2014 08:36:37] Starting the loop to wait for the background processes at So 8. Jun 08:36:37 CEST 2014 ...
[08.06.2014 08:36:37]   Waiting up to 3600 seconds for the background processes to finish ...
[08.06.2014 08:36:37]     Process "19351" for /usr/bin/uname finished
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/uname.19041.log"
[08.06.2014 08:36:37] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/uname.19041.log" are:

[08.06.2014 08:36:37] -------------------------------------

# ### ---- Log of the executable "/usr/bin/uname" executed at So 8. Jun 08:36:37 CEST 2014--- start ---
/usr/bin/uname: extra operand ‘[’
Try '/usr/bin/uname --help' for more information.

[08.06.2014 08:36:37] -------------------------------------

[08.06.2014 08:36:37]     Process "19333" for /data/develop/scripts/test/009test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/009test.sh.19041.log"
[08.06.2014 08:36:37]     Process "19319" for /data/develop/scripts/test/008test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/008test.sh.19041.log"
[08.06.2014 08:36:37]     Process "19305" for /data/develop/scripts/test/007test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/007test.sh.19041.log"
[08.06.2014 08:36:37]     Process "19291" for /data/develop/scripts/test/006test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log"
[08.06.2014 08:36:37]     Process "19277" for /data/develop/scripts/test/005test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/005test.sh.19041.log"
[08.06.2014 08:36:37]     Process "19263" for /data/develop/scripts/test/004test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/004test.sh.19041.log"
[08.06.2014 08:36:37]     Process "19249" for /data/develop/scripts/test/003test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/003test.sh.19041.log"
[08.06.2014 08:36:37]     Process "19235" for /data/develop/scripts/test/002test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/002test.sh.19041.log"
[08.06.2014 08:36:37]     Process "19220" for /data/develop/scripts/test/001test.sh is still running
[08.06.2014 08:36:37]       The log file is "/var/tmp/execute_scripts.sh.19041.work/001test.sh.19041.log"
[08.06.2014 08:36:37]   9 background processes (from 10) are still running
[08.06.2014 08:36:37]   Waiting for 10 second(s) now (total wait time until now: 0 seconds; max. timeout is 3600 seconds ; remaining wait time is 3600 seconds) ...
[08.06.2014 08:36:47]     Process "19333" for /data/develop/scripts/test/009test.sh is still running
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/009test.sh.19041.log"
[08.06.2014 08:36:47]     Process "19319" for /data/develop/scripts/test/008test.sh finished
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/008test.sh.19041.log"
[08.06.2014 08:36:47] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/008test.sh.19041.log" are:

[08.06.2014 08:36:47] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/008test.sh" executed at So 8. Jun 08:36:37 CEST 2014--- start ---
This is /data/develop/scripts/test/008test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/008test.sh" executed at So 8. Jun 08:36:42 CEST 2014 --- end ---

[08.06.2014 08:36:47] -------------------------------------

[08.06.2014 08:36:47]     Process "19305" for /data/develop/scripts/test/007test.sh is still running
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/007test.sh.19041.log"
[08.06.2014 08:36:47]     Process "19291" for /data/develop/scripts/test/006test.sh is still running
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log"
[08.06.2014 08:36:47]     Process "19277" for /data/develop/scripts/test/005test.sh finished
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/005test.sh.19041.log"
[08.06.2014 08:36:47] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/005test.sh.19041.log" are:

[08.06.2014 08:36:47] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/005test.sh" executed at So 8. Jun 08:36:37 CEST 2014--- start ---
This is /data/develop/scripts/test/005test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/005test.sh" executed at So 8. Jun 08:36:42 CEST 2014 --- end ---

[08.06.2014 08:36:47] -------------------------------------

[08.06.2014 08:36:47]     Process "19263" for /data/develop/scripts/test/004test.sh is still running
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/004test.sh.19041.log"
[08.06.2014 08:36:47]     Process "19249" for /data/develop/scripts/test/003test.sh is still running
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/003test.sh.19041.log"
[08.06.2014 08:36:47]     Process "19235" for /data/develop/scripts/test/002test.sh is still running
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/002test.sh.19041.log"
[08.06.2014 08:36:47]     Process "19220" for /data/develop/scripts/test/001test.sh is still running
[08.06.2014 08:36:47]       The log file is "/var/tmp/execute_scripts.sh.19041.work/001test.sh.19041.log"
[08.06.2014 08:36:47]   7 background processes (from 10) are still running
[08.06.2014 08:36:47]   Waiting for 10 second(s) now (total wait time until now: 10 seconds; max. timeout is 3600 seconds ; remaining wait time is 3590 seconds) ...
[08.06.2014 08:36:57]     Process "19333" for /data/develop/scripts/test/009test.sh is still running
[08.06.2014 08:36:57]       The log file is "/var/tmp/execute_scripts.sh.19041.work/009test.sh.19041.log"
[08.06.2014 08:36:57]     Process "19305" for /data/develop/scripts/test/007test.sh finished
[08.06.2014 08:36:57]       The log file is "/var/tmp/execute_scripts.sh.19041.work/007test.sh.19041.log"
[08.06.2014 08:36:57] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/007test.sh.19041.log" are:

[08.06.2014 08:36:57] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/007test.sh" executed at So 8. Jun 08:36:37 CEST 2014--- start ---
This is /data/develop/scripts/test/007test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/007test.sh" executed at So 8. Jun 08:36:57 CEST 2014 --- end ---

[08.06.2014 08:36:57] -------------------------------------

[08.06.2014 08:36:57]     Process "19291" for /data/develop/scripts/test/006test.sh is still running
[08.06.2014 08:36:57]       The log file is "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log"
[08.06.2014 08:36:57]     Process "19263" for /data/develop/scripts/test/004test.sh is still running
[08.06.2014 08:36:57]       The log file is "/var/tmp/execute_scripts.sh.19041.work/004test.sh.19041.log"
[08.06.2014 08:36:57]     Process "19249" for /data/develop/scripts/test/003test.sh is still running
[08.06.2014 08:36:57]       The log file is "/var/tmp/execute_scripts.sh.19041.work/003test.sh.19041.log"
[08.06.2014 08:36:57]     Process "19235" for /data/develop/scripts/test/002test.sh finished
[08.06.2014 08:36:57]       The log file is "/var/tmp/execute_scripts.sh.19041.work/002test.sh.19041.log"
[08.06.2014 08:36:57] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/002test.sh.19041.log" are:

[08.06.2014 08:36:57] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/002test.sh" executed at So 8. Jun 08:36:36 CEST 2014--- start ---
This is /data/develop/scripts/test/002test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/002test.sh" executed at So 8. Jun 08:36:56 CEST 2014 --- end ---

[08.06.2014 08:36:57] -------------------------------------

[08.06.2014 08:36:57]     Process "19220" for /data/develop/scripts/test/001test.sh is still running
[08.06.2014 08:36:57]       The log file is "/var/tmp/execute_scripts.sh.19041.work/001test.sh.19041.log"
[08.06.2014 08:36:57]   5 background processes (from 10) are still running
[08.06.2014 08:36:57]   Waiting for 10 second(s) now (total wait time until now: 20 seconds; max. timeout is 3600 seconds ; remaining wait time is 3580 seconds) ...
[08.06.2014 08:37:07]     Process "19333" for /data/develop/scripts/test/009test.sh is still running
[08.06.2014 08:37:07]       The log file is "/var/tmp/execute_scripts.sh.19041.work/009test.sh.19041.log"
[08.06.2014 08:37:08]     Process "19291" for /data/develop/scripts/test/006test.sh is still running
[08.06.2014 08:37:08]       The log file is "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log"
[08.06.2014 08:37:08]     Process "19263" for /data/develop/scripts/test/004test.sh is still running
[08.06.2014 08:37:08]       The log file is "/var/tmp/execute_scripts.sh.19041.work/004test.sh.19041.log"
[08.06.2014 08:37:08]     Process "19249" for /data/develop/scripts/test/003test.sh is still running
[08.06.2014 08:37:08]       The log file is "/var/tmp/execute_scripts.sh.19041.work/003test.sh.19041.log"
[08.06.2014 08:37:08]     Process "19220" for /data/develop/scripts/test/001test.sh finished
[08.06.2014 08:37:08]       The log file is "/var/tmp/execute_scripts.sh.19041.work/001test.sh.19041.log"
[08.06.2014 08:37:08] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/001test.sh.19041.log" are:

[08.06.2014 08:37:08] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/001test.sh" executed at So 8. Jun 08:36:36 CEST 2014--- start ---
This is /data/develop/scripts/test/001test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/001test.sh" executed at So 8. Jun 08:37:06 CEST 2014 --- end ---

[08.06.2014 08:37:08] -------------------------------------

[08.06.2014 08:37:08]   4 background processes (from 10) are still running
[08.06.2014 08:37:08]   Waiting for 10 second(s) now (total wait time until now: 30 seconds; max. timeout is 3600 seconds ; remaining wait time is 3570 seconds) ...
[08.06.2014 08:37:18]     Process "19333" for /data/develop/scripts/test/009test.sh finished
[08.06.2014 08:37:18]       The log file is "/var/tmp/execute_scripts.sh.19041.work/009test.sh.19041.log"
[08.06.2014 08:37:18] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/009test.sh.19041.log" are:

[08.06.2014 08:37:18] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/009test.sh" executed at So 8. Jun 08:36:37 CEST 2014--- start ---
This is /data/develop/scripts/test/009test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/009test.sh" executed at So 8. Jun 08:37:17 CEST 2014 --- end ---

[08.06.2014 08:37:18] -------------------------------------

[08.06.2014 08:37:18]     Process "19291" for /data/develop/scripts/test/006test.sh is still running
[08.06.2014 08:37:18]       The log file is "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log"
[08.06.2014 08:37:18]     Process "19263" for /data/develop/scripts/test/004test.sh is still running
[08.06.2014 08:37:18]       The log file is "/var/tmp/execute_scripts.sh.19041.work/004test.sh.19041.log"
[08.06.2014 08:37:18]     Process "19249" for /data/develop/scripts/test/003test.sh is still running
[08.06.2014 08:37:18]       The log file is "/var/tmp/execute_scripts.sh.19041.work/003test.sh.19041.log"
[08.06.2014 08:37:18]   3 background processes (from 10) are still running
[08.06.2014 08:37:18]   Waiting for 10 second(s) now (total wait time until now: 40 seconds; max. timeout is 3600 seconds ; remaining wait time is 3560 seconds) ...
[08.06.2014 08:37:28]     Process "19291" for /data/develop/scripts/test/006test.sh is still running
[08.06.2014 08:37:28]       The log file is "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log"
[08.06.2014 08:37:28]     Process "19263" for /data/develop/scripts/test/004test.sh finished
[08.06.2014 08:37:28]       The log file is "/var/tmp/execute_scripts.sh.19041.work/004test.sh.19041.log"
[08.06.2014 08:37:28] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/004test.sh.19041.log" are:

[08.06.2014 08:37:28] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/004test.sh" executed at So 8. Jun 08:36:37 CEST 2014--- start ---
This is /data/develop/scripts/test/004test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/004test.sh" executed at So 8. Jun 08:37:22 CEST 2014 --- end ---

[08.06.2014 08:37:28] -------------------------------------

[08.06.2014 08:37:28]     Process "19249" for /data/develop/scripts/test/003test.sh finished
[08.06.2014 08:37:28]       The log file is "/var/tmp/execute_scripts.sh.19041.work/003test.sh.19041.log"
[08.06.2014 08:37:28] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/003test.sh.19041.log" are:

[08.06.2014 08:37:28] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/003test.sh" executed at So 8. Jun 08:36:37 CEST 2014--- start ---
This is /data/develop/scripts/test/003test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/003test.sh" executed at So 8. Jun 08:37:27 CEST 2014 --- end ---

[08.06.2014 08:37:28] -------------------------------------

[08.06.2014 08:37:28]   1 background process (from 10) is still running
[08.06.2014 08:37:28]   Waiting for 10 second(s) now (total wait time until now: 50 seconds; max. timeout is 3600 seconds ; remaining wait time is 3550 seconds) ...
[08.06.2014 08:37:38]     Process "19291" for /data/develop/scripts/test/006test.sh finished
[08.06.2014 08:37:38]       The log file is "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log"
[08.06.2014 08:37:38] The last 20 line(s) of the logfile "/var/tmp/execute_scripts.sh.19041.work/006test.sh.19041.log" are:

[08.06.2014 08:37:38] -------------------------------------

# ### ---- Log of the executable "/data/develop/scripts/test/006test.sh" executed at So 8. Jun 08:36:37 CEST 2014--- start ---
This is /data/develop/scripts/test/006test.sh ...
# ### ---- Log of the executable "/data/develop/scripts/test/006test.sh" executed at So 8. Jun 08:37:32 CEST 2014 --- end ---

[08.06.2014 08:37:38] -------------------------------------

[08.06.2014 08:37:38] The loop to wait for the background processes ended at So 8. Jun 08:37:38 CEST 2014, the runtime is 0:01:00.
[08.06.2014 08:37:38]
[08.06.2014 08:37:38] Executing the stop script "/data/develop/scripts/test/stop.sh" ...
This is /data/develop/scripts/test/stop.sh ...
[08.06.2014 08:37:38]
[08.06.2014 08:37:38] All done (10 executable(s) ).
[08.06.2014 08:37:38]
[08.06.2014 08:37:38]

[08.06.2014 08:37:38] 10 executable(s) executed:
/data/develop/scripts/test/001test.sh /data/develop/scripts/test/002test.sh /data/develop/scripts/test/003test.sh /data/develop/scripts/test/004test.sh /data/develop/scripts/test/005test.sh /data/develop/scripts/test/006test.sh /data/develop/scripts/test/007test.sh /data/develop/scripts/test/008test.sh /data/develop/scripts/test/009test.sh /usr/bin/uname

[08.06.2014 08:37:38] 2 executable(s) not executed because they are on the exclude list:
/data/develop/scripts/test/start.sh /data/develop/scripts/test/stop.sh

[08.06.2014 08:37:38] The working directory was "/var/tmp/execute_scripts.sh.19041.work".

[08.06.2014 08:37:38] The log file used was "/var/tmp/execute_scripts.LOG"
[08.06.2014 08:37:38] execute_scripts.sh v1.0.0 started at So 8. Jun 08:36:34 CEST 2014 and ended at So 8. Jun 08:37:38 CEST 2014.
[08.06.2014 08:37:38] The RC is 0.
[xtrnaw7@t540p scripts]$

Other Examples

Execute all scripts matching the regular expression "${SCRIPTDIR}/compile*sh"; use the start script "${SCRIPTDIR}/start_compiling.sh", the stop script "${SCRIPTDIR}/check_compile_status.sh". The working directory is "${WORKDIR}/", the timeout for the background processes is 2 hours and the intervall between the checks of the status of the running background processes is 30 seconds.

execute_script.sh \ -I "${SCRIPTDIR}/compile*sh" \ -s "${SCRIPTDIR}/start_compiling.sh" \ -z "${SCRIPTDIR}/check_compile_status.sh" \ -o "${WORKDIR}/" \ -d -W 2h,30 \ -D no_of_logfile_lines=0 \

Internas

Note: See also the documentation for the script template used for this script: scriptt.sh

execute_scripts.sh reads a config file if it exists.

To create an initial config file use:

[xtrnaw7@t540p scripts]$ ./execute_scripts.sh -C [08.06.2014 08:38:17] execute_scripts.sh v1.0.0 started at So 8. Jun 08:38:17 CEST 2014. [08.06.2014 08:38:17] Using the log file "/var/tmp/execute_scripts.LOG" [08.06.2014 08:38:17] Creating the config file "execute_scripts.conf" ... [08.06.2014 08:38:17] Writing the config file "execute_scripts.conf" ... [08.06.2014 08:38:17] Configfile "execute_scripts.conf" successfully written. [08.06.2014 08:38:17] The log file used was "/var/tmp/execute_scripts.LOG" [08.06.2014 08:38:17] execute_scripts.sh v1.0.0 started at So 8. Jun 08:38:17 CEST 2014 and ended at So 8. Jun 08:38:17 CEST 2014. [08.06.2014 08:38:17] The RC is 0.

The config file looks like:

# config file for execute_scripts.sh v1.0.0, created So 8. Jun 08:38:17 CEST 2014

__CONFIG_FILE_VERSION="v1.0.0"

# extension for backup files
#
DEFAULT_BACKUP_EXTENSION=".$$.backup"

# allow the debug shell in AskUser
#
__DEBUG_SHELL_IN_ASKUSER=${__TRUE}

#### __DUMP_ALREADY_CREATED - do not automatically create another dump if
####   this variable is ${__TRUE}
####
#    __DUMP_ALREADY_CREATED=${__TRUE}

#### __CREATE_DUMP - create an environment dump if the scripts exits with
####   error
####   (replace <dumpdir> with either 0 or the directory for
####   the dumps) to always create a dump at script end
####
#    __CREATE_DUMP=<dumpdir>

#### DEFAULT_DUMPDIR - default directory for environment dumps
####
DEFAULT_DUMP_DIR="${TMPDIR:-${TMP:-${TEMP:-/tmp}}}"

# variables that can be changed via parameter
#

# default for the parameter -o
DEFAULT_WORKDIR="/var/tmp/${__SCRIPTNAME##*/}.$$.work"

# execute the command in parallel? -- default for the parameter -d
DEFAULT_EXECUTE_PARALLEL=${__FALSE}

# wait time in seconds for the background processes
#   -- 1st default for the parameter -W
DEFAULT_MAX_RUN_WAIT_TIME=1h

# wait intervall in seconds for the background processes
#   -- 2nd default for the parameter -W
DEFAULT_RUN_WAIT_INTERVALL=2s

# maximum number of background processes
#   -- 1st default for the parameter -w
DEFAULT_MAX_NO_OF_BACKGROUND_PROCESSES=-1

# wait intervall for starting the background processes
#   -- 2nd default for the parameter -w
DEFAULT_START_PROC_WAIT_INTERVALL=5s

# timeout value for the start of the parallel background processes
# -- 3rd default for the parameter -w
DEFAULT_START_PROC_TIMEOUT=5m

# scripts to execute -- default for the parameter -i / -I
#
DEFAULT_SCRIPTLIST=""

# scripts NOT to execute -- default for the parameter -x
#
DEFAULT_SCRIPTEXCLUDE_LIST=""

# shell to use for script execution -- default for the parameter -c
#
DEFAULT_SHELL_TO_USE="${__SCRIPT_SHELL}"

# add comments to the logfiles? -- default for the parameter -k
#
DEFAULT_ADD_COMMENTS=${__TRUE}

# remove duplicate entries from the list of executables -- default for the parameter -r
#
DEFAULT_REMOVE_DUPLICATES=${__FALSE}

# start script -- default for the parameter -s
#
DEFAULT_STARTSCRIPT=""

# start script -- default for the parameter -z
#
DEFAULT_STOPSCRIPT=""

# stop after a script returns non-zero? -- default for the parameter -B
#
DEFAULT_STOP_AFTER_ERROR=${__FALSE}

# no of lines of the logfile to list in case of an error -- defuaulf for the parameter -D no_of_error_loglines=n
#
DEFAULT_NO_OF_ERROR_LOGLINES=10

# no of lines of the logfile to list in case of no error -- defuaulf for the parameter -D no_of_ok_loglines=n
#
DEFAULT_NO_OF_OK_LOGLINES=5

# no of lines of the logfile to list for parallel tasks -- defuaulf for the parameter -D no_of_logfile_lines=n
#
DEFAULT_NO_OF_LOGFILE_LINES=10

# variables that can NOT be changed with parameter

# only change the following variables if you know what you are doing #

# no further internal variables defined yet
#
# Note you can redefine any variable that is initialized before calling
# ReadConfigFile here!

Notes

Please see the source code of the script for additional information

Download

Download execute_scripts.sh