├── LICENSE ├── Makefile ├── README.md ├── pmonitor.1 ├── pmonitor.html ├── pmonitor.sh └── test-pmonitor.sh /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2006-2015 Diomidis Spinellis 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. 14 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright 2012-2015 Diomidis Spinellis 3 | # 4 | # Licensed under the Apache License, Version 2.0 (the "License"); 5 | # you may not use this file except in compliance with the License. 6 | # You may obtain a copy of the License at 7 | # 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | # 10 | # Unless required by applicable law or agreed to in writing, software 11 | # distributed under the License is distributed on an "AS IS" BASIS, 12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | # See the License for the specific language governing permissions and 14 | # limitations under the License. 15 | # 16 | 17 | INSTPREFIX?=/usr/local 18 | 19 | EXECUTABLES=pmonitor 20 | 21 | # Manual pages 22 | MANSRC=$(wildcard *.1) 23 | MANPDF=$(patsubst %.1,%.pdf,$(MANSRC)) 24 | MANHTML=$(patsubst %.1,%.html,$(MANSRC)) 25 | 26 | %.pdf: %.1 27 | groff -man -Tps $< | ps2pdf - $@ 28 | 29 | %.html: %.1 30 | groff -man -Thtml $< >$@ 31 | 32 | all: $(EXECUTABLES) 33 | 34 | pmonitor: pmonitor.sh 35 | install pmonitor.sh pmonitor 36 | 37 | clean: 38 | rm -f *.o *.exe $(EXECUTABLES) $(MANPDF) $(MANHTML) 39 | 40 | install: $(EXECUTABLES) 41 | install $(EXECUTABLES) $(INSTPREFIX)/bin 42 | install -m 644 $(MANSRC) $(INSTPREFIX)/share/man/man1 43 | 44 | test: 45 | ./test-pmonitor.sh 46 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # pmonitor 2 | The _pmonitor_ command allows you to monitor a job's progress by specifying 3 | the name of the corresponding command, its process id, or the file being processed. 4 | 5 | See the examples below. 6 | 7 | ### Progress of uncompressing a file 8 | ``` 9 | $ pmonitor --command=gzip 10 | /home/dds/data/mysql-2015-04-01.sql.gz 58.06% 11 | ``` 12 | 13 | ### Progress of resolving IP addresses in a web server log file 14 | ``` 15 | $ pmonitor -c logresolve -i 20 16 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 4.02% 17 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 12.05% ETA 0:03:38 18 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 16.06% ETA 0:04:38 19 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 20.08% ETA 0:04:58 20 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 24.10% ETA 0:05:02 21 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 28.11% ETA 0:04:58 22 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 36.14% ETA 0:03:58 23 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 44.18% ETA 0:03:14 24 | /var/log/apache2/www.balab.aueb.gr_access_ssl.log.1 52.21% ETA 0:02:38 25 | ``` 26 | 27 | ### Progress of a MariaDB file import 28 | ``` 29 | $ pmonitor -u -c mysql 30 | /home/dds/data/mysql-2018-01-01/project_commits.csv 96.50% 31 | ``` 32 | 33 | ### Progress of a MariaDB primary key index generation 34 | ``` 35 | $ sudo pmonitor -u -c mysqld -i 10 -f /home/mysql/ghtorrent/project_commits.MYD 36 | /home/mysql/ghtorrent/project_commits.MYD 7.88% 37 | /home/mysql/ghtorrent/project_commits.MYD 7.92% ETA 6:32:27 38 | /home/mysql/ghtorrent/project_commits.MYD 7.96% ETA 6:26:32 39 | /home/mysql/ghtorrent/project_commits.MYD 8.00% ETA 6:29:27 40 | /home/mysql/ghtorrent/project_commits.MYD 8.04% ETA 6:24:26 41 | /home/mysql/ghtorrent/project_commits.MYD 8.08% ETA 6:23:24 42 | 43 | ``` 44 | ### Progress of a MariaDB index generation 45 | ``` 46 | $ sudo pmonitor -u -c mysqld -i 10 -f /home/mysql/ghtorrent/\#sql-62f0_14.MYD 47 | /home/mysql/ghtorrent/#sql-62f0_14.MYD 9.44% 48 | /home/mysql/ghtorrent/#sql-62f0_14.MYD 9.44% ETA 552:52:34 49 | ``` 50 | 51 | ## Installation 52 | Run `make install` or simply copy the file `pmonitor.sh` as `pmonitor` in 53 | your path. 54 | 55 | The _pmonitor_ program requires a working version of _lsof_. 56 | 57 | ## Manual page 58 | You can find the command's manual page [here](http://htmlpreview.github.io/?https://github.com/dspinellis/pmonitor/blob/master/pmonitor.html). 59 | 60 | ## See also 61 | * [pv](http://www.ivarch.com/programs/pv.shtml) 62 | * [progress](https://github.com/Xfennec/progress) 63 | -------------------------------------------------------------------------------- /pmonitor.1: -------------------------------------------------------------------------------- 1 | .TH PMONITOR 1 "13 February 2018" 2 | .\" 3 | .\" (C) Copyright 2006-2018 Diomidis Spinellis 4 | .\" 5 | .\" Licensed under the Apache License, Version 2.0 (the "License"); 6 | .\" you may not use this file except in compliance with the License. 7 | .\" You may obtain a copy of the License at 8 | .\" 9 | .\" http://www.apache.org/licenses/LICENSE-2.0 10 | .\" 11 | .\" Unless required by applicable law or agreed to in writing, software 12 | .\" distributed under the License is distributed on an "AS IS" BASIS, 13 | .\" WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | .\" See the License for the specific language governing permissions and 15 | .\" limitations under the License. 16 | .\" 17 | .SH NAME 18 | pmonitor \- monitor a job's progress 19 | .SH SYNOPSIS 20 | \fBpmonitor\fP 21 | \fB\-c\fP \fIcommand\fP | 22 | \fB\-f\fP \fIfile\fP | 23 | \fB\-p\fP \fIpid\fP 24 | [\fB-i\fP \fIinterval\fP] 25 | [\fB-d\fP] 26 | [\fB-u\fP] 27 | .SH DESCRIPTION 28 | The \fIpmonitor\fP command will display the progress of a process 29 | as a percentage. 30 | It does this by examining the process's open files, 31 | and calculating the ratio between 32 | the position of the file's seek pointer offset and the file length. 33 | For processes that process files in a sequential fashion, 34 | such as file compression and database import, this ratio 35 | can be translated to the percentage of the job that has been completed. 36 | .PP 37 | The command may produce no output, if a process does not hold any open 38 | files with a non-zero seek offset. 39 | .SH OPTIONS 40 | .TP 41 | \fB\-c\fP, \fB\-\-command\fP=\fICOMMAND\fP 42 | Monitor the progress through the files opened by the specified running command. 43 | The command can be specified through (part) of its name 44 | (e.g. \fIgzip\fP), or through a regular expression if the 45 | argument starts with a \fC/\fP. 46 | See the documentation of the \fIlsof\fP(1) \fB\-c\fP option regarding 47 | the rules and flags associated with the use of regular expressions. 48 | 49 | .TP 50 | \fB\-d\fP, \fB\-\-diff\fP 51 | In repeated output mode, 52 | output only records that differ from one iteration to the next. 53 | 54 | .TP 55 | \fB\-f\fP, \fB\-\-file\fP=\fIFILE\fP 56 | Monitor a process's progress through its processing of the specified file. 57 | 58 | .TP 59 | \fB\-p\fP, \fB\-\-pid\fP=\fIPID\fP 60 | Monitor the progress of the process with the specified process id \fIPID\fP. 61 | 62 | .TP 63 | \fB\-i\fP, \fB\-\-interval\fP=\fIINTERVAL\fP 64 | Repeat the output every \fIINTERVAL\fP seconds. 65 | 66 | .TP 67 | \fB\-h\fP, \fB\-\-help\fP 68 | Display the program's usage information and exit. 69 | 70 | .TP 71 | \fB\-u\fP, \fB\-\-update\fP 72 | Also monitor files that are opened in update (read and write) mode, 73 | rather than only those that are opened in read-only mode. 74 | This is useful for monitoring table scans of RDBMS engines. 75 | 76 | .SH AUTHOR 77 | Diomidis Spinellis \(em 78 | 79 | .SH "REPORTING BUGS" 80 | Visit the utility's \fIGitHub\fP page at 81 | . 82 | Fixes and improvements are accepted through pull requests. 83 | 84 | .SH "SEE ALSO" 85 | \fIlsof\fP(1), 86 | .PP 87 | The \fIpmonitor\fP command is modelled after a similar facility 88 | that was available on the Permin-Elmer/Concurrent OS32. 89 | -------------------------------------------------------------------------------- /pmonitor.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 5 | 6 | 7 | 8 | 9 | 10 | 16 | PMONITOR 17 | 18 | 19 | 20 | 21 |

PMONITOR

22 | 23 | NAME
24 | SYNOPSIS
25 | DESCRIPTION
26 | OPTIONS
27 | AUTHOR
28 | REPORTING BUGS
29 | SEE ALSO
30 | 31 |
32 | 33 | 34 |

NAME 35 | 36 |

37 | 38 | 39 |

pmonitor 40 | − monitor a job’s progress

41 | 42 |

SYNOPSIS 43 | 44 |

45 | 46 | 47 |

pmonitor 48 | −c command | −f file | 49 | −p pid [-i interval]

50 | 51 |

DESCRIPTION 52 | 53 |

54 | 55 | 56 |

The 57 | pmonitor command will display the progress of a 58 | process as a percentage. It does this by examining the 59 | process’s open files, and calculating the ratio 60 | between the position of the file’s seek pointer offset 61 | and the file length. For processes that process files in a 62 | sequential fashion, such as file compression and database 63 | import, this ratio can be translated to the percentage of 64 | the job that has been completed.

65 | 66 |

The command may 67 | produce no output, if a process does not hold any open files 68 | with a non-zero seek offset.

69 | 70 |

OPTIONS 71 | 72 |

73 | 74 | 75 | 76 |

−c, 77 | −−command=COMMAND

78 | 79 |

Monitor the progress through 80 | the files opened by the specified running command. The 81 | command can be specified through (part) of its name (e.g. 82 | gzip), or through a regular expression if the 83 | argument starts with a /. See the documentation of 84 | the lsof(1) −c option regarding the 85 | rules and flags associated with the use of regular 86 | expressions.

87 | 88 |

−f, 89 | −−file=FILE

90 | 91 |

Monitor a process’s 92 | progress through its processing of the specified file.

93 | 94 |

−p, 95 | −−pid=PID

96 | 97 |

Monitor the progress of the 98 | process with the specified process id PID.

99 | 100 |

−i, 101 | −−interval=INTERVAL

102 | 103 |

Repeat the output every 104 | INTERVAL seconds.

105 | 106 |

−h, 107 | −−help

108 | 109 |

Display the program’s 110 | usage information and exit.

111 | 112 |

AUTHOR 113 | 114 |

115 | 116 | 117 |

Diomidis 118 | Spinellis — <http://www.spinellis.gr>

119 | 120 |

REPORTING BUGS 121 | 122 |

123 | 124 | 125 |

Visit the 126 | utility’s GitHub page at 127 | <https://github.com/dspinellis/pmonitor>. Fixes and 128 | improvements are accepted through pull requests.

129 | 130 |

SEE ALSO 131 | 132 |

133 | 134 | 135 | 136 |

lsof(1),

137 | 138 |

The 139 | pmonitor command is modelled after a similar facility 140 | that was available on the Permin-Elmer/Concurrent OS32.

141 |
142 | 143 | 144 | -------------------------------------------------------------------------------- /pmonitor.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Monitor the progress of a specified job 4 | # 5 | # Copyright 2006-2018 Diomidis Spinellis 6 | # 7 | # Licensed under the Apache License, Version 2.0 (the "License"); 8 | # you may not use this file except in compliance with the License. 9 | # You may obtain a copy of the License at 10 | # 11 | # http://www.apache.org/licenses/LICENSE-2.0 12 | # 13 | # Unless required by applicable law or agreed to in writing, software 14 | # distributed under the License is distributed on an "AS IS" BASIS, 15 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 | # See the License for the specific language governing permissions and 17 | # limitations under the License. 18 | # 19 | # For each file or file associated with the specified process is reading, 20 | # display the percentage associated with its seek pointer offset. For 21 | # files that are processed in a sequential fashion this can be translated 22 | # to the percentage of the job that has been completed. 23 | # 24 | # This command is modelled after a similar facility 25 | # available on Permin-Elmer/Concurrent OS32 26 | # 27 | # Requires: 28 | # - lsof(8) with offset (-o) printing functionality 29 | # 30 | # Submit issues and pull requests as https://github.com/dspinellis/pmonitor 31 | # 32 | 33 | # Run lsof with the specified options 34 | # The OPT1 and OPT2 variables are passed to lsof as arguments. 35 | opt_lsof() 36 | { 37 | lsof -w -o0 -o "$OPT1" "$OPT2" 38 | } 39 | 40 | # Display the scanned percentage of lsof files. 41 | display() 42 | { 43 | # Obtain the offset and print it as a percentage 44 | awk ' 45 | # Return current time 46 | function time() { 47 | "date +%s" | getline t 48 | close("date +%s") 49 | return t 50 | } 51 | 52 | # Return length of specified file 53 | function file_length(fname) { 54 | if (!cached_length[fname]) { 55 | if (fname ~ /^\/dev\/[^/]*$/ && system("test -b " fname) == 0) { 56 | getline < ("/sys/block/" substr(fname, 6) "/size") 57 | cached_length[fname] = $1 * 512 # sector size is always 512 bytes 58 | } else { 59 | "ls -l '\''" fname "'\'' 2>/dev/null" | getline 60 | cached_length[fname] = $5 + 0 61 | } 62 | } 63 | return cached_length[fname] 64 | } 65 | 66 | BEGIN { 67 | CONVFMT = "%.2f" 68 | start = time() 69 | } 70 | 71 | $4 ~ /^[0-9]+[r'$UPDATE']$/ && $7 ~ /^0t/ { 72 | now = time() 73 | offset = substr($7, 3) 74 | fname = $9 75 | len = file_length(fname) 76 | if (len > 0) { 77 | if (!start_offset[fname]) 78 | start_offset[fname] = offset 79 | delta_t = now - start 80 | delta_o = offset - start_offset[fname] 81 | if (delta_t > 2 && delta_o > 0) { 82 | bps = delta_o / delta_t 83 | t = (len - offset) / bps 84 | eta = "" 85 | if (t > 0) { 86 | eta_s = t % 60 87 | t = int(t / 60) 88 | eta_m = t % 60 89 | t = int(t / 60) 90 | eta_h = t 91 | eta = sprintf(" ETA %d:%02d:%02d", eta_h, eta_m, eta_s) 92 | } 93 | } 94 | out = fname "\t" offset / len * 100 "%" 95 | if (!'$ONLYDIFF' || !seen[out]) 96 | print out eta 97 | seen[out] = 1 98 | } 99 | } 100 | ' 101 | } 102 | 103 | # Report program usage information 104 | usage() 105 | { 106 | cat <<\EOF 1>&2 107 | Usage: 108 | 109 | pmonitor [-c command] [-d] [-f file] [-i interval] [-p pid] 110 | -c, --command=COMMAND Monitor the progress of the specified running command 111 | -d, --diff During continuous display show only records that differ 112 | -f, --file=FILE Monitor the progress of commands processing the 113 | specified file 114 | -h, --help Display this message and exit 115 | -i, --interval=INTERVAL Continuously display the progress every INTERVAL seconds 116 | -p, --pid=PID Monitor the progress of the process with the specified 117 | process id 118 | -u, --update Also monitor files opened in update (rather than read 119 | mode) 120 | 121 | Exactly one of the c, f, p options must be specified. 122 | 123 | Terminating... 124 | EOF 125 | } 126 | 127 | # Option processing; see /usr/share/doc/util-linux-ng-2.17.2/getopt-parse.bash 128 | # Note that we use `"$@"' to let each command-line parameter expand to a 129 | # separate word. The quotes around `$@' are essential! 130 | # We need TEMP as the `eval set --' would nuke the return value of getopt. 131 | 132 | # Allowed short options 133 | SHORTOPT=c:,d,f:,h,i:,p:,u 134 | 135 | if getopt -l >/dev/null 2>&1 ; then 136 | # Simple (e.g. FreeBSD) getopt 137 | TEMP=$(getopt $SHORTOPT "$@") 138 | else 139 | # Long options supported 140 | TEMP=$(getopt -o $SHORTOPT --long command:,diff,file:,help,interval:,pid:,update -n 'pmonitor' -- "$@") 141 | fi 142 | 143 | if [ $? != 0 ] ; then 144 | usage 145 | exit 2 146 | fi 147 | 148 | # Note the quotes around `$TEMP': they are essential! 149 | eval set -- "$TEMP" 150 | 151 | ONLYDIFF=0 152 | 153 | while : ; do 154 | case "$1" in 155 | -c|--command) 156 | test -z "$OPT1" || usage 157 | OPT1=-c 158 | OPT2="$2" 159 | shift 2 160 | ;; 161 | -d|--diff) 162 | ONLYDIFF=1 163 | shift 164 | ;; 165 | -f|--file) 166 | test -z "$OPT1" || usage 167 | if ! [ -r "$2" ] ; then 168 | echo "Unable to read $2" 1>&2 169 | exit 1 170 | fi 171 | OPT1=-- 172 | OPT2="$2" 173 | shift 2 174 | UPDATE=u 175 | ;; 176 | -i|--interval) 177 | INTERVAL="$2" 178 | shift 2 179 | ;; 180 | -h|--help) 181 | usage 182 | exit 0 183 | ;; 184 | -p|--pid) 185 | test -z "$OPT1" || usage 186 | OPT1=-p 187 | OPT2="$2" 188 | shift 2 189 | ;; 190 | -u|--update) 191 | UPDATE=u 192 | shift 193 | ;; 194 | --) 195 | shift 196 | break 197 | ;; 198 | *) 199 | echo "Internal error!" 200 | exit 3 201 | ;; 202 | esac 203 | done 204 | 205 | # No more arguments allowed and one option must be specified 206 | if [ "$1" != '' -o ! -n "$OPT1" -o ! -n "$OPT2" ] 207 | then 208 | usage 209 | exit 2 210 | fi 211 | 212 | if [ "$INTERVAL" ] ; then 213 | while : ; do 214 | opt_lsof 215 | sleep $INTERVAL 216 | done 217 | else 218 | ONLYDIFF=0 219 | opt_lsof 220 | fi | 221 | display 222 | -------------------------------------------------------------------------------- /test-pmonitor.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | DICT=/usr/share/dict/words 4 | SIZE=$(wc -c <$DICT) 5 | HALF=$(expr $SIZE / 2 / 1024 + 1) 6 | 7 | ( 8 | dd of=/dev/null count=$HALF bs=1024 >/dev/null 2>&1 9 | sleep 2 10 | ) <$DICT & 11 | 12 | PID=$! 13 | 14 | sleep 1 15 | 16 | RESULT=$(./pmonitor.sh -p $PID | sed -n '/dict/s/.* \(.*\)\.[0-9][0-9]%/\1/p') 17 | 18 | wait 19 | 20 | if [ "$RESULT" = 50 ] ; then 21 | echo "OK pmonitor" 22 | exit 0 23 | else 24 | echo "FAIL pmonitor: expected 50, got '$RESULT'" 25 | exit 1 26 | fi 27 | --------------------------------------------------------------------------------