├── LICENSE
├── README.md
└── xargs-ssh.bash
/LICENSE:
--------------------------------------------------------------------------------
1 | Copyright (c) 2012, Kenny Rasschaert
2 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
3 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
4 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
5 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | xargs-ssh
2 | =============
3 |
4 | This simple Linux bash script runs multiple ssh commands in parallel by leveraging the parallel processing power of xargs. I first learned of this trick on [Jordan Sissel's blog](http://www.semicomplete.com/blog/articles/week-of-unix-tools/day-5-xargs.html).
5 |
6 | There are already a number of tools available to run ssh in parallel, such as
7 | * [pssh](http://www.theether.org/pssh/)
8 | * [sshpt](http://code.google.com/p/sshpt/)
9 | * [ClusterSSH](http://clusterssh.sourceforge.net/)
10 | * [Capistrano](http://capistranorb.com/)
11 | * [Fabric](http://fabfile.org/)
12 | * [Func](https://fedorahosted.org/func/)
13 | * [MCollective](https://puppetlabs.com/mcollective/)
14 |
15 | These tools all have numerous advantages and features that this simple script doesn't. The single advantage that xarg-ssh has over these alternatives is that it's just a bash script that you can run without having to install anything new on either server or client.
16 |
17 | At the heart of this script is this command
18 | ````bash
19 | xargs -a $serverlist -I"SERVER" -P${threads} -n1 sh -c "ssh SERVER $cmd"
20 | ````
21 | * $serverlist is a file containing a newline-separated list of hostnames.
22 | * ${threads} is the maximum number of threads you want it to run simultaneously at any given time.
23 | * $cmd is the command to be run on each server.
24 |
25 | If this script is a bit much for you, you could distill it down to just that one line. I recommend wrapping the ssh part in a command substitution in some sort of printf/echo statement if the output matters.
26 |
27 | Dependencies:
28 | -------------
29 | This script only works on Linux. The following tools are needed to run xargs-ssh.bash:
30 | * bash
31 | * openssh-client (also known as "ssh")
32 | * xargs
33 | * sed
34 | * awk
35 | * tput
36 | * find
37 | * POSIX shell (also known as "sh")
38 | * printf
39 |
40 | These are all tools that are commonly found on any GNU/Linux system.
41 |
42 | The versions of xargs and mktemp on OSX (the BSD versions) behave very differently to their Linux counterparts. I could adapt this script to work on OSX, but since I personally intend to use this only on Linux (I don't even own a Mac), I probably won't.
43 |
44 | In case you want to do it, here are some tips to help you on your way:
45 | Counting the cores on OSX is done like this:
46 | ````bash
47 | sysctl hw.ncpu | awk '{print $2}
48 | ````
49 | BSD mktemp requires a bit more effort to create a random file in /tmp/, but that probably doesn't even matter: BSD xargs doesn't have an option to read input from a file. The way to go here is to pipe your data straight to xargs.
50 |
51 | Usage:
52 | ------
53 | Either pipe a line-separated list of servers to xargs-ssh, or provide a file with the list with the -f option. You may optionally specifiy a command to run with the -c option and enable easily parsable output with the -s option.
54 | If server list is not provided, no -f option neither pipe, the script runs the command interactively as each server is typed (end with Ctrl-D).
55 |
56 |
57 |
58 | | Option | argument | Description |
59 |
60 |
61 | | -f | filename | specifies the input file. Use -f filename to read from a file. If no file is specified, stdin is used. |
62 |
63 |
64 | | -c | command | lets you specify a command to run on each server. By default this command is "uptime". |
65 |
66 |
67 | | -s | n/a | enables script mode, which keeps the output for each server on one line, new lines in the output are separated by "\n": |
68 |
69 |
70 | | -h | n/a | prints help message that explains how to use the script. |
71 |
72 |
73 |
74 | Examples:
75 | ---------
76 | Basic usage of the script using a list of servers in a file:
77 | ````bash
78 | ./xargs-ssh.bash -f /home/kenny/serverlist.txt
79 | ````
80 | An example of piping data to the script:
81 | ````bash
82 | for i in {1..4}; do echo server$i; done | ./xargs-ssh.bash
83 | ````
84 | These two commands would have the same output:
85 |
86 | ###### server3 ######
87 | 11:31:22 up 34 days, 17:09, 2 users, load average: 0.00, 0.00, 0.00
88 |
89 | ###### server4 ######
90 | 11:31:22 up 153 days, 22:44, 1 user, load average: 0.00, 0.00, 0.00
91 |
92 | ###### server2 ######
93 | 11:31:22 up 34 days, 17:13, 8 users, load average: 0.12, 0.11, 0.05
94 |
95 | ###### server1 ######
96 | 05:28:35 up 76 days, 12:29, 0 users, load average: 0.00, 0.00, 0.00
97 |
98 |
99 | Specify a custom command and enable script mode:
100 | ````bash
101 | ./xargs-ssh.bash -f /home/kenny/serverlist.txt -c "hostname --fqdn && uptime" -s
102 | ````
103 | The output of each individual ssh reduced to one line of output, where new lines in the output are separated by "\n":
104 |
105 | server4: server4.customer.local\n 11:31:22 up 153 days, 22:44, 1 user, load average: 0.00, 0.00, 0.00
106 | server1: server1.customer.local\n 05:28:35 up 76 days, 12:29, 0 users, load average: 0.00, 0.00, 0.00
107 | server2: server2.customer.local\n 11:31:22 up 34 days, 17:13, 8 users, load average: 0.12, 0.11, 0.05
108 | server3: server3.customer.local\n 11:31:22 up 34 days, 17:09, 2 users, load average: 0.00, 0.00, 0.00
109 |
110 |
--------------------------------------------------------------------------------
/xargs-ssh.bash:
--------------------------------------------------------------------------------
1 | #!/bin/bash
2 |
3 | # Description: Parallel processing of SSH commands with xargs
4 | # Github URL: https://github.com/rasschaert/xargs-ssh
5 | # License: MIT
6 |
7 | scriptmode=0
8 | command="uptime"
9 | input=""
10 | serverlist=""
11 |
12 | # Parse command line options
13 | while getopts "hsf:c:" flag; do
14 | case "$flag" in
15 | h) echo "Usage: $0 [-f ] [-c ] [-h] [-s]"
16 | echo " -f specifies the input file. Use -f filename to read from a file. If no file is specified, stdin is used."
17 | echo " -c lets you specify a command to run on each server. By default this command is \"uptime\"."
18 | echo " -s enables script mode, which keeps the output for each server on one line."
19 | echo " -h prints this help message."
20 | exit 0
21 | ;;
22 | s) scriptmode=1
23 | ;;
24 | f) serverlist=$OPTARG
25 | ;;
26 | c) command="$OPTARG"
27 | ;;
28 | *) exit 1
29 | ;;
30 | esac
31 | done
32 |
33 | # If no input file is specified, use stdin
34 | if [[ -z "$serverlist" ]]; then
35 | tempfile=$(mktemp)
36 | while read data; do
37 | echo $data >> $tempfile
38 | done
39 | serverlist=$tempfile
40 | fi
41 |
42 | # Count the number of "CPU's" or hyperthreads on this machine
43 | threads="$(awk '/^processor/ {cpu++} END {print cpu}' /proc/cpuinfo)"
44 |
45 | # Fancy colors in terminal output for increased legibility
46 | bluetxt=$(tput setaf 4)
47 | normaltxt=$(tput sgr0)
48 | boldtxt=$(tput bold)
49 |
50 | # This is where the command is encapsulated in a printf statement.
51 | # Doing this makes sure that the output of each server will remain together.
52 |
53 | # If scriptmode is not specified, format the output in a pretty and legible way
54 | if [[ "$scriptmode" == 0 ]]; then
55 | cmd="{ printf \"###### ${bluetxt}${boldtxt}SERVER${normaltxt} ######\n\$(ssh -o ConnectTimeout=3 SERVER \"$command\" 2>&1)\n\n\"; }"
56 | # If scriptmode is specified, format the output so that the output of each server is on one easiliy parsed line
57 | else
58 | cmd="{ printf \"SERVER: \$(ssh -o ConnectTimeout=3 SERVER \"$command\" 2>&1)\" | sed 's/$/\\\n/' | tr -d '\n' | sed 's/\\\n$/\n/'; }"
59 | fi
60 |
61 | # Use xargs for parallelism, use as many threads as the CPU supports
62 | # Inside the xargs, start subshells to run ssh with the specified or default command
63 | xargs -a $serverlist -I"SERVER" -P${threads} -n1 sh -c "$cmd"
64 |
65 | # If a temporary file was created to put the serverlist in, remove it
66 | if [[ ! -z "$tempfile" ]]; then
67 | find $tempfile -type f | xargs rm
68 | fi
69 |
--------------------------------------------------------------------------------