├── .gitattributes ├── COPYING ├── Makefile ├── README ├── check-isutf8 ├── chronic ├── combine ├── debian ├── changelog ├── compat ├── control ├── copyright ├── docs └── rules ├── ifdata.c ├── ifdata.docbook ├── ifne.c ├── ifne.docbook ├── isutf8.c ├── isutf8.docbook ├── lckdo.c ├── lckdo.docbook ├── mispipe.c ├── mispipe.docbook ├── parallel.c ├── parallel.docbook ├── pee.c ├── pee.docbook ├── physmem.c ├── sponge.c ├── sponge.docbook ├── ts ├── vidir ├── vipe └── zrun /.gitattributes: -------------------------------------------------------------------------------- 1 | debian/changelog merge=dpkg-mergechangelogs 2 | -------------------------------------------------------------------------------- /COPYING: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc. 5 | 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Library General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License 307 | along with this program; if not, write to the Free Software 308 | Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 309 | 310 | 311 | Also add information on how to contact you by electronic and paper mail. 312 | 313 | If the program is interactive, make it output a short notice like this 314 | when it starts in an interactive mode: 315 | 316 | Gnomovision version 69, Copyright (C) year name of author 317 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 318 | This is free software, and you are welcome to redistribute it 319 | under certain conditions; type `show c' for details. 320 | 321 | The hypothetical commands `show w' and `show c' should show the appropriate 322 | parts of the General Public License. Of course, the commands you use may 323 | be called something other than `show w' and `show c'; they could even be 324 | mouse-clicks or menu items--whatever suits your program. 325 | 326 | You should also get your employer (if you work as a programmer) or your 327 | school, if any, to sign a "copyright disclaimer" for the program, if 328 | necessary. Here is a sample; alter the names: 329 | 330 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 331 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 332 | 333 | , 1 April 1989 334 | Ty Coon, President of Vice 335 | 336 | This General Public License does not permit incorporating your program into 337 | proprietary programs. If your program is a subroutine library, you may 338 | consider it more useful to permit linking proprietary applications with the 339 | library. If this is what you want to do, use the GNU Library General 340 | Public License instead of this License. 341 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | BINS=isutf8 ifdata ifne pee sponge mispipe lckdo parallel 2 | PERLSCRIPTS=vidir vipe ts combine zrun chronic 3 | MANS=sponge.1 vidir.1 vipe.1 isutf8.1 ts.1 combine.1 ifdata.1 ifne.1 pee.1 zrun.1 chronic.1 mispipe.1 lckdo.1 parallel.1 4 | CFLAGS=-O2 -g -Wall 5 | INSTALL_BIN?=install -s 6 | PREFIX=/usr 7 | 8 | DOCBOOK2XMAN=docbook2x-man 9 | 10 | all: $(BINS) $(MANS) 11 | 12 | clean: 13 | rm -f $(BINS) $(MANS) 14 | 15 | install: 16 | mkdir -p $(DESTDIR)$(PREFIX)/bin 17 | $(INSTALL_BIN) $(BINS) $(DESTDIR)$(PREFIX)/bin 18 | install $(PERLSCRIPTS) $(DESTDIR)$(PREFIX)/bin 19 | 20 | mkdir -p $(DESTDIR)$(PREFIX)/share/man/man1 21 | install $(MANS) $(DESTDIR)$(PREFIX)/share/man/man1 22 | 23 | check: isutf8 24 | ./check-isutf8 25 | 26 | isutf8.1: isutf8.docbook 27 | $(DOCBOOK2XMAN) $< 28 | 29 | ifdata.1: ifdata.docbook 30 | $(DOCBOOK2XMAN) $< 31 | 32 | ifne.1: ifne.docbook 33 | $(DOCBOOK2XMAN) $< 34 | 35 | pee.1: pee.docbook 36 | $(DOCBOOK2XMAN) $< 37 | 38 | sponge.1: sponge.docbook 39 | $(DOCBOOK2XMAN) $< 40 | 41 | mispipe.1: mispipe.docbook 42 | $(DOCBOOK2XMAN) $< 43 | 44 | lckdo.1: lckdo.docbook 45 | $(DOCBOOK2XMAN) $< 46 | 47 | parallel.1: parallel.docbook 48 | $(DOCBOOK2XMAN) $< 49 | 50 | %.1: % 51 | pod2man --center=" " --release="moreutils" $< > $@; 52 | -------------------------------------------------------------------------------- /README: -------------------------------------------------------------------------------- 1 | This is a collection of the unix tools that nobody thought to write 2 | long ago, when unix was young. Currently it consists of these tools: 3 | 4 | chronic: runs a command quietly unless it fails 5 | combine: combine the lines in two files using boolean operations 6 | ifdata: get network interface info without parsing ifconfig output 7 | isutf8: check if a file or standard input is utf-8 8 | ifne: run a command if the standard input is not empty 9 | lckdo: execute a program with a lock held (deprecated) 10 | mispipe: pipe two commands, returning the exit status of the first 11 | parallel: run multiple jobs at once 12 | pee: tee standard input to pipes 13 | sponge: soak up standard input and write to a file 14 | ts: timestamp standard input 15 | vidir: edit a directory in your text editor 16 | vipe: insert a text editor into a pipe 17 | zrun: automatically uncompress arguments to command 18 | 19 | Its web page is here: http://kitenet.net/~joey/code/moreutils/ 20 | 21 | Your suggestions of additional tools to add to this collection are 22 | appreciated. The web page lists some that are under consideration but 23 | have not yet been included, I also welcome feedback on which of these to 24 | include. 25 | 26 | -- Joey Hess 27 | -------------------------------------------------------------------------------- /check-isutf8: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Bash needed for the character encodings below. 3 | # 4 | # Run checks for ./isutf8. 5 | # 6 | # Lars Wirzenius 7 | 8 | failed=0 9 | 10 | check() { 11 | printf "$2" | ./isutf8 -q 12 | ret=$? 13 | if [ $ret != $1 ]; then 14 | echo "Failure:" 15 | echo " input: $2" 16 | echo " expected: $1" 17 | echo " got: $ret" 18 | failed=1 19 | fi 20 | printf "$2" > check-isutf8.tmp.$$ 21 | ./isutf8 -q check-isutf8.tmp.$$ 22 | ret=$? 23 | rm -f check-isutf8.tmp.$$ 24 | if [ $ret != $1 ]; then 25 | echo "Failure (from file):" 26 | echo " input: $2" 27 | echo " expected: $1" 28 | echo " got: $ret" 29 | failed=1 30 | fi 31 | } 32 | 33 | check 0 '' 34 | check 0 'a' 35 | check 0 'ab' 36 | check 0 '\xc2\xa9' 37 | 38 | check 1 '\xc2' 39 | check 1 '\xc2\x20' 40 | check 1 '\x20\xc2' 41 | check 1 '\300\200' 42 | check 1 '\xed\xa0\x88\xed\xbd\x85' # UTF-16 surrogates 43 | check 1 '\xef\xbf\xbe' # 0xFFFE 44 | check 1 '\xef\xbf\xbf' # 0xFFFF 45 | 46 | exit $failed 47 | -------------------------------------------------------------------------------- /chronic: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | =head1 NAME 4 | 5 | chronic - runs a command quietly unless it fails 6 | 7 | =head1 SYNOPSIS 8 | 9 | chronic COMMAND... 10 | 11 | =head1 DESCRIPTION 12 | 13 | chronic runs a command, and arranges for its standard out and standard 14 | error to only be displayed if the command fails (exits nonzero or crashes). 15 | If the command succeeds, any extraneous output will be hidden. 16 | 17 | A common use for chronic is for running a cron job. Rather than 18 | trying to keep the command quiet, and having to deal with mails containing 19 | accidental output when it succeeds, and not verbose enough output when it 20 | fails, you can just run it verbosely always, and use chronic to hide 21 | the successful output. 22 | 23 | 0 1 * * * chronic backup # instead of backup >/dev/null 2>&1 24 | 25 | =head1 AUTHOR 26 | 27 | Copyright 2010 by Joey Hess 28 | 29 | Original concept and "chronic" name by Chuck Houpt. 30 | 31 | Licensed under the GNU GPL version 2 or higher. 32 | 33 | =cut 34 | 35 | use warnings; 36 | use strict; 37 | use IPC::Run qw( start pump finish timeout ); 38 | 39 | if (! @ARGV) { 40 | die "usage: chronic COMMAND...\n"; 41 | } 42 | 43 | my ($out, $err); 44 | my $h = IPC::Run::start \@ARGV, \*STDIN, \$out, \$err; 45 | $h->finish; 46 | my $ret=$h->full_result; 47 | 48 | if ($ret >> 8) { # child failed 49 | showout(); 50 | exit ($ret >> 8); 51 | } 52 | elsif ($ret != 0) { # child killed by signal 53 | showout(); 54 | exit 1; 55 | } 56 | else { 57 | exit 0; 58 | } 59 | 60 | sub showout { 61 | print STDOUT $out; 62 | print STDERR $err; 63 | } 64 | -------------------------------------------------------------------------------- /combine: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | =head1 NAME 4 | 5 | combine - combine sets of lines from two files using boolean operations 6 | 7 | =head1 SYNOPSIS 8 | 9 | combine file1 and file2 10 | 11 | combine file1 not file2 12 | 13 | combine file1 or file2 14 | 15 | combine file1 xor file2 16 | 17 | _ file1 and file2 _ 18 | 19 | _ file1 not file2 _ 20 | 21 | _ file1 or file2 _ 22 | 23 | _ file1 xor file2 _ 24 | 25 | =head1 DESCRIPTION 26 | 27 | B combines the lines in two files. Depending on the boolean 28 | operation specified, the contents will be combined in different ways: 29 | 30 | =over 4 31 | 32 | =item and 33 | 34 | Outputs lines that are in file1 if they are also present in file2. 35 | 36 | =item not 37 | 38 | Outputs lines that are in file1 but not in file2. 39 | 40 | =item or 41 | 42 | Outputs lines that are in file1 or file2. 43 | 44 | =item xor 45 | 46 | Outputs lines that are in either file1 or file2, but not in both files. 47 | 48 | =back 49 | 50 | "-" can be specified for either file to read stdin for that file. 51 | 52 | The input files need not be sorted, and the lines are output in the order 53 | they occur in file1 (followed by the order they occur in file2 for the two 54 | "or" operations). Bear in mind that this means that the operations are not 55 | commutative; "a and b" will not necessarily be the same as "b and a". To 56 | obtain commutative behavior sort and uniq the result. 57 | 58 | Note that this program can be installed as "_" to allow for the syntactic 59 | sugar shown in the latter half of the synopsis (similar to the test/[ 60 | command). It is not currently installed as "_" by default, but you can 61 | alias it to that if you like. 62 | 63 | =head1 SEE ALSO 64 | 65 | join(1) 66 | 67 | =head1 AUTHOR 68 | 69 | Copyright 2006 by Joey Hess 70 | 71 | Licensed under the GNU GPL. 72 | 73 | =cut 74 | 75 | use warnings; 76 | use strict; 77 | 78 | sub filemap { 79 | my $file=shift; 80 | my $sub=shift; 81 | 82 | open (IN, $file) || die "$file: $!\n"; 83 | while () { 84 | chomp; 85 | $sub->(); 86 | } 87 | close IN; 88 | } 89 | 90 | sub hashify { 91 | my $file=shift; 92 | 93 | my %seen; 94 | filemap $file, sub { $seen{$_}++ }; 95 | return \%seen; 96 | } 97 | 98 | sub compare_or { 99 | my ($file1, $file2) = @_; 100 | 101 | filemap $file1, sub { print "$_\n" }; 102 | filemap $file2, sub { print "$_\n" }; 103 | } 104 | 105 | sub compare_xor { 106 | my ($file1, $file2) = @_; 107 | 108 | compare_not($file1, $file2); 109 | compare_not($file2, $file1); 110 | } 111 | 112 | sub compare_not { 113 | my ($file1, $file2) = @_; 114 | 115 | my $seen=hashify($file2); 116 | filemap $file1, sub { print "$_\n" unless $seen->{$_} }; 117 | } 118 | 119 | sub compare_and { 120 | my ($file1, $file2) = @_; 121 | 122 | my $seen=hashify($file2); 123 | filemap $file1, sub { print "$_\n" if $seen->{$_} }; 124 | } 125 | 126 | if (@ARGV >= 4 && $ARGV[3] eq "_") { 127 | delete $ARGV[3]; 128 | } 129 | 130 | if (@ARGV != 3) { 131 | die "Usage: combine file1 OP file2\n"; 132 | } 133 | 134 | my $file1=shift; 135 | my $op=lc shift; 136 | my $file2=shift; 137 | 138 | if ($::{"compare_$op"}) { 139 | no strict 'refs'; 140 | "compare_$op"->($file1, $file2); 141 | } 142 | else { 143 | die "unknown operation, $op\n"; 144 | } 145 | -------------------------------------------------------------------------------- /debian/changelog: -------------------------------------------------------------------------------- 1 | moreutils (0.45) unstable; urgency=low 2 | 3 | * ts: Support %.s for seconds sinch epoch with subsecond resolution. 4 | Closes: #619764 5 | 6 | -- Joey Hess Sun, 19 Jun 2011 15:44:28 -0400 7 | 8 | moreutils (0.44) unstable; urgency=low 9 | 10 | * pee: Propigate exit status of commands run. 11 | 12 | -- Joey Hess Thu, 10 Mar 2011 17:38:14 -0400 13 | 14 | moreutils (0.43) unstable; urgency=low 15 | 16 | * chronic: New command, runs a command quietly, unless it fails. 17 | * Now depends on IPC::Run, used by chronic. 18 | 19 | -- Joey Hess Fri, 29 Oct 2010 15:54:37 -0400 20 | 21 | moreutils (0.42) unstable; urgency=low 22 | 23 | * sponge: Guarantee that output file is always updated atomically, 24 | by renaming a temp file into place. Closes: #592144 25 | * sponge: Ensure that output file permissions are always preserved 26 | if it already exists. 27 | * Typo. Closes: #596032 28 | 29 | -- Joey Hess Wed, 06 Oct 2010 20:03:25 -0400 30 | 31 | moreutils (0.41) unstable; urgency=low 32 | 33 | * ifdata.docbook: Mark interface as required in synopsis. Closes: #588397 34 | * Add missing AUTHOR section to docbook man pages. 35 | * sponge: Correct bad use of fread that caused a trailing quantity of 36 | soaked data to be silently discarded when a temp file was used 37 | and sponge output to stdout. Closes: #595220 38 | 39 | -- Joey Hess Thu, 02 Sep 2010 16:47:10 -0400 40 | 41 | moreutils (0.40) unstable; urgency=low 42 | 43 | * lckdo: Now deprecated, since util-linux's flock(1) can do the same 44 | thing. 45 | * parallel: -i will now replace {} inside parameters, before the {} had 46 | to be a separate parameter. 47 | 48 | -- Joey Hess Tue, 06 Jul 2010 19:38:14 -0400 49 | 50 | moreutils (0.39) unstable; urgency=low 51 | 52 | * parallel: Fix exit code handling when commands are specified after -- 53 | * parallel: Make -j 0 do something reasonable (start all jobs at once). 54 | * parallel: Fix to really avoid running new jobs when load is too high. 55 | * parallel: Fix logic error in code handling -l that could make parallel 56 | return a bogus 255 exit code when all jobs succeeded. Closes: #569617 57 | * parallel: Allow a decimal load value to be specified with -l 58 | * Caps sillyness. Closes: #570815 59 | * zrun: Add support for .xz files. 60 | 61 | -- Joey Hess Tue, 23 Feb 2010 15:51:26 -0500 62 | 63 | moreutils (0.38) unstable; urgency=low 64 | 65 | * Description improvements. Closes: #549450 66 | (Thanks, Justin B Rye) 67 | * parallel: Allow running independent commands, 68 | like `parallel -j3 -- ls df "echo hi"` 69 | * ifdata: Add FreeBSD kernel support, although some of the more esoteric 70 | interface options are not currently supported in FreeBSD. 71 | * parallel: Define WEXITED to allow building on FreeBSD kernel. 72 | * Thanks Enrico Tassi for the FreeBSD kernel support, which should 73 | be enough to get moreutils built on Debian kFreeBSD. Closes: #562609 74 | 75 | -- Joey Hess Tue, 09 Feb 2010 15:50:42 -0500 76 | 77 | moreutils (0.37) unstable; urgency=low 78 | 79 | * parallel: Clarify man page regarding CPUs. Closes: #536597 80 | * parallel: Add -n option. Thanks, Pierre Habouzit. Closes: #537992 81 | (As a side effect, fixes a segfault if -- was omitted.) 82 | * parallel.1: Typo fixes. Closes: #538147 83 | 84 | -- Joey Hess Sat, 25 Jul 2009 10:18:40 +0200 85 | 86 | moreutils (0.36) unstable; urgency=low 87 | 88 | * parallel: New program, contributed by Tollef Fog Heen, 89 | that can run multiple jobs in parallel, optionally checking 90 | load average. 91 | * mispipe: Fix closing of extra pipe FD before starting command 92 | so it is not inherited by daemons. Closes: #533448 93 | (Thanks, Jeremie Koenig) 94 | 95 | -- Joey Hess Fri, 10 Jul 2009 11:01:41 -0400 96 | 97 | moreutils (0.35) unstable; urgency=low 98 | 99 | * ifdata: Don't assume that all interface names are 6 characters or less, 100 | for instance "wmaster0" is longer. Increase the limit to 20 characters. 101 | Closes: #526654 (Thanks, Alan Pope) 102 | * isutf8: Reject UTF-8-encoded UTF-16 surrogates. Closes: #525301 103 | (Thanks, Jakub Wilk and liw) 104 | 105 | -- Joey Hess Tue, 05 May 2009 15:06:41 -0400 106 | 107 | moreutils (0.34) unstable; urgency=low 108 | 109 | * vipe: Avoid dying on empty input. Thanks, Anders Kaseorg 110 | Closes: #508491 111 | 112 | -- Joey Hess Thu, 11 Dec 2008 15:11:23 -0500 113 | 114 | moreutils (0.33) unstable; urgency=low 115 | 116 | * Support installing moreutils into prefixes other than /usr (Evan Broder) 117 | * Fix zrun breakage introduced last version. Closes: #504129 118 | 119 | -- Joey Hess Fri, 31 Oct 2008 17:01:03 -0400 120 | 121 | moreutils (0.32) unstable; urgency=low 122 | 123 | * zrun: Can be linked to zsomeprog to run the equivilant of zrun someprog. 124 | Closes: #411623 (Stefan Fritsch) 125 | * zrun: Add support for lzma and lzo. (Stefan Fritsch) 126 | * Fix pod error in vidir(1). 127 | 128 | -- Joey Hess Sun, 26 Oct 2008 23:43:30 -0400 129 | 130 | moreutils (0.31) unstable; urgency=low 131 | 132 | * pee.1: Document difference with tee in stdout. 133 | * ts: Support displaying fractional seconds via a "%.S" conversion 134 | specification. Closes: #482789 135 | 136 | -- Joey Hess Sat, 28 Jun 2008 23:19:31 -0400 137 | 138 | moreutils (0.30) unstable; urgency=low 139 | 140 | * debhelper v7; rules file minimisation 141 | * Use DESTDIR instead of PREFIX. 142 | * Add a DOCBOOK2XMAN setting. (Greg KH) 143 | * ifne: Add -n which makes it run the command if stdin is empty. 144 | * ifne: If no command is specified, print usage information. 145 | 146 | -- Joey Hess Wed, 14 May 2008 19:37:42 -0400 147 | 148 | moreutils (0.29) unstable; urgency=low 149 | 150 | * Add ifne, contributed by Javier Merino. 151 | * sponge, ifne: Ensure that suspending/resuming doesn't 152 | result in partial writes of the data, by using fwrite() 153 | rather than write(). 154 | * sponge: Handle large data sizes by using a temp file rather than by 155 | consuming arbitrary amounts of memory. Patch by Brock Noland. 156 | * ts: Allow both -r and a format to be specified, to parse dates and output 157 | in a specified format. 158 | * ts: Fix bug in timezone regexp. 159 | 160 | -- Joey Hess Tue, 15 Apr 2008 15:30:52 -0400 161 | 162 | moreutils (0.28) unstable; urgency=low 163 | 164 | * Moved to a git repository. 165 | * vidir: Applied patch from Stefan Fritsch (one part of #412176): 166 | - Check for control characters (especially newlines) in filenames 167 | and error out, since this can greatly confuse the editor or vidir. 168 | - If the source of a rename does not exist (and thus the rename will fail 169 | anyway), vidir should not move an existing target file to a tmpfile. 170 | - If a directory is renamed, vidir should take that into account when 171 | renaming files in this directory. 172 | - If a directory name is passed as name/ to vidir, vidir should not 173 | add second slash after the name. 174 | * vidir: Add support for unlinking directories. To recursivly delete 175 | a directory and its contents, pipe find to vidir, and delete the directory 176 | and its contents in the editor. Closes: #412176 177 | * Add example to man page about recursive modification of directories. 178 | Closes: #390099 179 | 180 | -- Joey Hess Sat, 02 Feb 2008 17:26:23 -0500 181 | 182 | moreutils (0.27) unstable; urgency=low 183 | 184 | * vidir: Check exit codes of close. Closes: #463739 185 | 186 | -- Joey Hess Sat, 02 Feb 2008 16:42:18 -0500 187 | 188 | moreutils (0.26) unstable; urgency=low 189 | 190 | * isutf8: Correct inverted exit code when passed a file to check. 191 | Closes: #453306 192 | 193 | -- Joey Hess Wed, 28 Nov 2007 14:19:32 -0500 194 | 195 | moreutils (0.25) unstable; urgency=low 196 | 197 | * isutf8: Detect and reject overlong UTF-8 sequences. Closes: #440951 198 | Many thanks to liw for the patch. 199 | 200 | -- Joey Hess Mon, 12 Nov 2007 11:58:07 -0500 201 | 202 | moreutils (0.24) unstable; urgency=low 203 | 204 | * vidir: Force numbers to normalised integers. 205 | * vidir: Abort on unknown item numbers rather than deleting them. 206 | Closes: #442440 207 | 208 | -- Joey Hess Sun, 16 Sep 2007 13:05:54 -0400 209 | 210 | moreutils (0.23) unstable; urgency=low 211 | 212 | * Add pointer to join from combine's man page. Closes: #435516 213 | * Don't strip binaries for debian package if built with 214 | DEB_BUILD_OPTIONS=nostrip. Closes: #437577 215 | * Include Michael Tokarev's lckdo program and replace / conflict with the 216 | current lckdo package. (Robert Edmonds) Closes: #432906 217 | * lckdo: Don't clear other flags when setting close on exec. 218 | 219 | -- Joey Hess Wed, 05 Sep 2007 21:45:25 -0400 220 | 221 | moreutils (0.22) unstable; urgency=low 222 | 223 | * vidir, zrun: Don't put temp files in the current directory. Closes: #429367 224 | 225 | -- Joey Hess Mon, 25 Jun 2007 16:11:27 -0400 226 | 227 | moreutils (0.21) unstable; urgency=low 228 | 229 | * Patch from Sergej Pupykin fixing ifdata -pN. 230 | 231 | -- Joey Hess Mon, 25 Jun 2007 13:07:21 -0400 232 | 233 | moreutils (0.20) unstable; urgency=low 234 | 235 | * Typo fixes from Ralf Wildenhues. 236 | * ifdata: Add -bips and -bops options contributed by André Appel, 237 | to print the number of bytes of incoming/outgoing traffic per second. 238 | 239 | -- Joey Hess Sat, 23 Dec 2006 15:55:45 -0500 240 | 241 | moreutils (0.19) unstable; urgency=low 242 | 243 | * vidir: Don't ignore files whose names end in dots. Closes: #398141 244 | Thanks, Bram Sanders. 245 | 246 | -- Joey Hess Sat, 11 Nov 2006 22:02:17 -0500 247 | 248 | moreutils (0.18) unstable; urgency=low 249 | 250 | * mispipe.docbook: Typo. Closes: #386756 251 | * spongs: Output to stdout if no file is specified, useful in a pipeline 252 | such as: cvs diff | sponge | patch -R -p0 253 | Closes: #387501 254 | 255 | -- Joey Hess Thu, 14 Sep 2006 15:13:46 -0400 256 | 257 | moreutils (0.17) unstable; urgency=low 258 | 259 | * Add missing \n to sponge usage. Closes: #383944 260 | * Add mispipe, contributed by Nathanael Nerode. Pipes together two commands, 261 | returning the exit status of the first. 262 | 263 | -- Joey Hess Thu, 7 Sep 2006 20:49:16 -0400 264 | 265 | moreutils (0.16) unstable; urgency=low 266 | 267 | * Change the default ts format to include the month and day, for consistency 268 | with syslog format. 269 | * Add -r switch to ts, which makes it convert existing timestamps in 270 | the input into relative times, such as "15m2s ago". Many common date 271 | formats are supported. 272 | 273 | -- Joey Hess Sat, 19 Aug 2006 15:27:42 -0400 274 | 275 | moreutils (0.15) unstable; urgency=low 276 | 277 | * Remove notes about potential tools from README, moved to wiki. 278 | * vidir: Don't abort if it sees an empty or all-whitespace line. 279 | * vidir: If just a filename is removed and the number is left, 280 | treat this the same as removing the whole line, and delete the file, 281 | instead of trying to rename the file to "". 282 | * vidir: Remove the periods after the item numbers. 283 | * vidir: Man page improvements. Closes: #378122 284 | * combine: Man page improvements, to clarify even more that order does 285 | matter and that the operations are not commutative. Closes: #361123 286 | * combine: The behavior of "or" was fairly strange if lines were repeated in 287 | a file. Changed behavior to just print all lines from both files, even 288 | if this means printing dups. Not sure I like this behavior either, but 289 | it's consistent with the very useful behaviors of "and" and "not". 290 | * vidir, vipe: Use /usr/bin/editor if it's present, and EDITOR and VISUAL 291 | arn't set, to comply with Debian policy. For portability, fall back to vi 292 | if there's no /usr/bin/editor. Closes: #378623 293 | 294 | -- Joey Hess Tue, 25 Jul 2006 22:26:56 -0400 295 | 296 | moreutils (0.14) unstable; urgency=low 297 | 298 | * vidir: Also support EDITOR or VISUAL that contains spaces and parameters. 299 | 300 | -- Joey Hess Wed, 12 Jul 2006 13:35:49 -0400 301 | 302 | moreutils (0.13) unstable; urgency=low 303 | 304 | * ifdata: typo 305 | * vipe: Support EDITOR or VISUAL that contains spaces and parameters. 306 | 307 | -- Joey Hess Wed, 12 Jul 2006 12:53:44 -0400 308 | 309 | moreutils (0.12) unstable; urgency=low 310 | 311 | * Really fix typo. Closes: #369485 312 | * zrun: Add usage message. 313 | * ifdata: Fix bug in argument parsing that could make it segfault. 314 | * combine: Allow operators to be written in any case. 315 | 316 | -- Joey Hess Wed, 28 Jun 2006 14:04:49 -0400 317 | 318 | moreutils (0.11) unstable; urgency=low 319 | 320 | * combine: better synopsis. Closes: #360544 321 | * ifdata: basically rewritten by Adam Lackorzynski, code size is 40% 322 | smaller, macros removed, return checks added for all lib calls. 323 | Closes: #361728 324 | * vidir: behave properly if passed a directory name to edit. Closes: #369156 325 | * check-isutf8: needs to be a bash script, the way it encodes characters 326 | won't work with dash. 327 | * check-isutf8: exit nonzero if any tests fail 328 | 329 | -- Joey Hess Sat, 27 May 2006 19:12:50 -0400 330 | 331 | moreutils (0.10) unstable; urgency=low 332 | 333 | * ifdata: patch from Adam Lackorzynski to translate French error messages 334 | to English and to convert some #defines to enums. 335 | * ifdata: patch from Adam Lackorzynski to avoid printing info for 336 | bogus interface. Closes: #360433 337 | * Add zrun, contributed by Chung-chieh Shan (you might prefer to alias it to 338 | "z"). 339 | 340 | -- Joey Hess Sun, 2 Apr 2006 18:34:01 -0400 341 | 342 | moreutils (0.9) unstable; urgency=low 343 | 344 | * ifdata: robustness patch from Adam Lackorzynski, in particular deal with 345 | /proc not mounted. Closes: #358930 346 | * Typo fixes. Closes: #359039, #359040 347 | * pee: don't send a copy of every line sent to every command to stdout. 348 | Seems best for it not to output anything to stdout on its on at all. 349 | Closes: #359041 350 | 351 | -- Joey Hess Mon, 27 Mar 2006 13:45:45 -0500 352 | 353 | moreutils (0.8) unstable; urgency=low 354 | 355 | * Back to Mithandir's C sponge, now fixed. 356 | * tss.1, ifdata.1: typo fix. Closes: #358557, #358556 357 | * ifdata: patch from its author to make it behave properly on big endian 358 | systems. Closes: #358860 359 | 360 | -- Joey Hess Fri, 24 Mar 2006 20:52:21 -0500 361 | 362 | moreutils (0.7) unstable; urgency=low 363 | 364 | * Add pee (pipe tee) contributed by Miek Gieben. 365 | * ifdata: Patch from KELEMEN Peter to add support for printing hardware 366 | interface. Closes: #357646 367 | 368 | -- Joey Hess Sat, 18 Mar 2006 14:17:08 -0500 369 | 370 | moreutils (0.6) unstable; urgency=low 371 | 372 | * Revert to perl sponge since the C one corrupts larger files. 373 | Hopefully temporary.. 374 | 375 | -- Joey Hess Fri, 10 Mar 2006 15:33:43 -0500 376 | 377 | moreutils (0.5) unstable; urgency=low 378 | 379 | * Added ifdata, by Benjamin BAYART (originally called ifcfg). 380 | * Made ifdata -Wall clean. 381 | * Made ifdata support -h and print usage on unknown options. 382 | * Cleaned up ifdata's behavior when asked to print info for nonexistant 383 | devices. Still needs improvement. 384 | * Indentation improvements. 385 | * Replaced and(1) and not(1) by combine, based on an idea by Matt Taggart. 386 | 387 | -- Joey Hess Tue, 7 Mar 2006 23:02:14 -0500 388 | 389 | moreutils (0.4) unstable; urgency=low 390 | 391 | * Added versions of and(1) and not(1) that support arbitrary numbers of 392 | input files. 393 | 394 | -- Joey Hess Sun, 5 Mar 2006 22:28:13 -0500 395 | 396 | moreutils (0.3) unstable; urgency=low 397 | 398 | * Switch sponge to a C implementation by mithandir. 399 | * Added a list of prospective tools that I am considering adding to the 400 | README file. 401 | 402 | -- Joey Hess Fri, 3 Mar 2006 18:09:46 -0500 403 | 404 | moreutils (0.2) unstable; urgency=low 405 | 406 | * Build dep on docbook-xml. 407 | 408 | -- Joey Hess Sun, 19 Feb 2006 18:40:56 -0500 409 | 410 | moreutils (0.1) unstable; urgency=low 411 | 412 | * First release. 413 | 414 | -- Joey Hess Sun, 12 Feb 2006 17:11:21 -0500 415 | -------------------------------------------------------------------------------- /debian/compat: -------------------------------------------------------------------------------- 1 | 7 2 | -------------------------------------------------------------------------------- /debian/control: -------------------------------------------------------------------------------- 1 | Source: moreutils 2 | Section: utils 3 | Priority: optional 4 | Build-Depends: debhelper (>= 7), dpkg-dev (>= 1.9.0), docbook2x, docbook-xml 5 | Maintainer: Joey Hess 6 | Standards-Version: 3.9.2 7 | Vcs-Git: git://git.kitenet.net/moreutils 8 | Homepage: http://kitenet.net/~joey/code/moreutils/ 9 | 10 | Package: moreutils 11 | Architecture: any 12 | Depends: ${shlibs:Depends}, ${misc:Depends}, ${perl:Depends}, libipc-run-perl 13 | Suggests: libtime-duration-perl, libtimedate-perl 14 | Conflicts: lckdo 15 | Replaces: lckdo 16 | Description: additional Unix utilities 17 | This is a growing collection of the Unix tools that nobody thought 18 | to write long ago, when Unix was young. 19 | . 20 | So far, it includes the following utilities: 21 | - chronic: runs a command quietly unless it fails 22 | - combine: combine the lines in two files using boolean operations 23 | - ifdata: get network interface info without parsing ifconfig output 24 | - ifne: run a program if the standard input is not empty 25 | - isutf8: check if a file or standard input is utf-8 26 | - lckdo: execute a program with a lock held 27 | - mispipe: pipe two commands, returning the exit status of the first 28 | - parallel: run multiple jobs at once 29 | - pee: tee standard input to pipes 30 | - sponge: soak up standard input and write to a file 31 | - ts: timestamp standard input 32 | - vidir: edit a directory in your text editor 33 | - vipe: insert a text editor into a pipe 34 | - zrun: automatically uncompress arguments to command 35 | -------------------------------------------------------------------------------- /debian/copyright: -------------------------------------------------------------------------------- 1 | Format: http://dep.debian.net/deps/dep5/ 2 | Source: Contributed by many folks. 3 | 4 | Files: * 5 | Copyright: 2006-2011 Joey Hess 6 | License: GPL-2+ 7 | 8 | Files: isutf8.* 9 | Copyright: 2005 Lars Wirzenius 10 | License: GPL-2+ 11 | 12 | Files: sponge.* 13 | Copyright: 2006 Tollef Fog Heen 14 | License: GPL-2 15 | 16 | Files: ifdata.c 17 | Copyright: 2002 Benjamin BAYART 18 | License: GPL-2+ 19 | 20 | Files: pee.c 21 | Copyright: 2006 Miek Gieben 22 | License: GPL-2+ 23 | 24 | Files: zrun 25 | Copyright: 2006 Chung-chieh Shan 26 | License: GPL-2+ 27 | 28 | Files: mispipe.c 29 | Copyright: 2004 Nathanael Nerode 30 | License: GPL-2+ or Expat 31 | 32 | Files: lckdo.c lckdo.docbook 33 | Copyright: Michael Tokarev 34 | License: other 35 | Public domain 36 | 37 | Files: ifne.c ifne.docbook 38 | Copyright: 2008 Javier Merino 39 | License: GPL-2+ 40 | 41 | Files: parallel.c 42 | Copyright: 2008 Tollef Fog Heen 43 | License: GPL-2 44 | 45 | Files: physmem.c 46 | Copyright: 2000, 2001, 2003, 2005, 2006 Free Software Foundation, Inc. 47 | License: GPL-2+ 48 | 49 | License: GPL-2 50 | The full text of the GPL is distributed as COPYING in moreutils's source, 51 | and is distributed in /usr/share/common-licenses/GPL-2 on Debian systems. 52 | 53 | License: GPL-2+ 54 | The full text of the GPL is distributed as COPYING in moreutils's source, 55 | and is distributed in /usr/share/common-licenses/GPL-2 on Debian systems. 56 | 57 | License: Expat 58 | * Permission is hereby granted, free of charge, to any person obtaining a 59 | * copy of this software and associated documentation files (the "Software"), 60 | * to deal in the Software without restriction, including without limitation 61 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, 62 | * and/or sell copies of the Software, and to permit persons to whom the 63 | * Software is furnished to do so, subject to the following conditions: 64 | * 65 | * The above copyright notice and this permission notice shall be included in 66 | * all copies or substantial portions of the Software. 67 | * 68 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 69 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 70 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL 71 | * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 72 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 73 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 74 | * THE SOFTWARE. 75 | -------------------------------------------------------------------------------- /debian/docs: -------------------------------------------------------------------------------- 1 | README 2 | -------------------------------------------------------------------------------- /debian/rules: -------------------------------------------------------------------------------- 1 | #!/usr/bin/make -f 2 | 3 | # Prevent the makefile from stripping, in case it's being built in 4 | # unstripped mode. 5 | export INSTALL_BIN=install 6 | 7 | %: 8 | dh $@ 9 | 10 | # Not intended for use by anyone except the author. 11 | announcedir: 12 | @echo ${HOME}/src/joeywiki/code/moreutils/news 13 | -------------------------------------------------------------------------------- /ifdata.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | #if defined(__linux__) 9 | #include 10 | #include 11 | #endif 12 | 13 | #if defined(__FreeBSD_kernel__) 14 | #include 15 | #endif 16 | 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | 24 | enum { 25 | DO_EXISTS = 1, 26 | DO_PEXISTS, 27 | DO_PADDRESS, 28 | DO_PMASK, 29 | DO_PMTU, 30 | DO_PCAST, 31 | DO_PALL, 32 | DO_PFLAGS, 33 | DO_SINPACKETS, 34 | DO_SINBYTES, 35 | DO_SINERRORS, 36 | DO_SINDROPS, 37 | DO_SINALL, 38 | DO_SINFIFO, 39 | DO_SINFRAME, 40 | DO_SINCOMPRESSES, 41 | DO_SINMULTICAST, 42 | DO_SOUTALL, 43 | DO_SOUTBYTES, 44 | DO_SOUTPACKETS, 45 | DO_SOUTERRORS, 46 | DO_SOUTDROPS, 47 | DO_SOUTFIFO, 48 | DO_SOUTCOLLS, 49 | DO_SOUTCARRIER, 50 | DO_SOUTMULTICAST, 51 | DO_PNETWORK, 52 | DO_PHWADDRESS, 53 | DO_BIPS, 54 | DO_BOPS 55 | }; 56 | 57 | struct if_stat { 58 | unsigned long long in_packets, in_bytes, in_errors, in_drops; 59 | unsigned long long in_fifo, in_frame, in_compress, in_multicast; 60 | unsigned long long out_bytes, out_packets, out_errors, out_drops; 61 | unsigned long long out_fifo, out_colls, out_carrier, out_multicast; 62 | }; 63 | 64 | 65 | void print_quad_ipv4(in_addr_t i) { 66 | i = ntohl(i); 67 | printf("%d.%d.%d.%d", 68 | (i & 0xff000000) >> 24, 69 | (i & 0x00ff0000) >> 16, 70 | (i & 0x0000ff00) >> 8, 71 | (i & 0x000000ff)); 72 | } 73 | 74 | void print_quad_ipv6(uint16_t *a) { 75 | printf("%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x", 76 | a[0], a[1], a[2], a[3], a[4], a[5], a[6], a[7]); 77 | } 78 | 79 | void print_quad(struct sockaddr *adr) { 80 | switch (adr->sa_family) { 81 | case AF_INET: 82 | print_quad_ipv4(((struct sockaddr_in*)adr)->sin_addr.s_addr); 83 | break; 84 | case AF_INET6: 85 | print_quad_ipv6(((struct sockaddr_in6*)adr)->sin6_addr.s6_addr16); 86 | break; 87 | default: 88 | printf("NON-IP"); 89 | break; 90 | } 91 | } 92 | 93 | enum print_error_enum { 94 | PRINT_ERROR, 95 | PRINT_NO_ERROR, 96 | }; 97 | 98 | /** 99 | * return 0 success 100 | * 1 error 101 | */ 102 | static int do_socket_ioctl(const char *ifname, const unsigned long int request, 103 | struct ifreq *req, int *ioctl_errno, 104 | const enum print_error_enum print_error) { 105 | int sock, res; 106 | 107 | if ((sock = socket(PF_INET, SOCK_DGRAM, IPPROTO_IP)) == -1) 108 | return 1; 109 | strncpy(req->ifr_name, ifname, IFNAMSIZ); 110 | req->ifr_name[IFNAMSIZ - 1] = 0; 111 | 112 | if ((res = ioctl(sock, request, req)) == -1) { 113 | if (ioctl_errno) 114 | *ioctl_errno = errno; 115 | if (print_error == PRINT_ERROR) 116 | fprintf(stderr, "ioctl on %s: %s\n", ifname, strerror(errno)); 117 | close(sock); 118 | return 1; 119 | } 120 | 121 | close(sock); 122 | 123 | return 0; 124 | } 125 | 126 | int if_exists(const char *iface) { 127 | struct ifreq r; 128 | return !do_socket_ioctl(iface, SIOCGIFFLAGS, &r, NULL, PRINT_NO_ERROR); 129 | } 130 | 131 | #if defined(__linux__) 132 | 133 | void if_flags(const char *iface) { 134 | struct ifreq r; 135 | unsigned int i; 136 | const struct { 137 | unsigned int flag; 138 | char *name; 139 | } flags[] = { 140 | { IFF_UP, "Up" }, 141 | { IFF_BROADCAST, "Broadcast" }, 142 | { IFF_DEBUG, "Debugging" }, 143 | { IFF_LOOPBACK, "Loopback" }, 144 | { IFF_POINTOPOINT, "Ppp" }, 145 | { IFF_NOTRAILERS, "No-trailers" }, 146 | { IFF_RUNNING, "Running" }, 147 | { IFF_NOARP, "No-arp" }, 148 | { IFF_PROMISC, "Promiscuous" }, 149 | { IFF_ALLMULTI, "All-multicast" }, 150 | { IFF_MASTER, "Load-master" }, 151 | { IFF_SLAVE, "Load-slave" }, 152 | { IFF_MULTICAST, "Multicast" }, 153 | { IFF_PORTSEL, "Port-select" }, 154 | { IFF_AUTOMEDIA, "Auto-detect" }, 155 | { IFF_DYNAMIC, "Dynaddr" }, 156 | { 0xffff0000, "Unknown-flags" }, 157 | }; 158 | 159 | if (do_socket_ioctl(iface, SIOCGIFFLAGS, &r, NULL, PRINT_ERROR)) 160 | return; 161 | 162 | for (i = 0; i < sizeof(flags) / sizeof(flags[0]); i++) 163 | printf("%s%s%s", (r.ifr_flags & flags[i].flag) ? "On " : "Off ", 164 | flags[i].name, 165 | sizeof(flags) / sizeof(flags[0]) - 1 == i ? "" : "\n"); 166 | } 167 | 168 | void if_hwaddr(const char *iface) { 169 | struct ifreq r; 170 | unsigned char *hwaddr; 171 | 172 | if (do_socket_ioctl(iface, SIOCGIFHWADDR, &r, NULL, PRINT_ERROR)) 173 | return; 174 | 175 | hwaddr = (unsigned char *)r.ifr_hwaddr.sa_data; 176 | printf("%02X:%02X:%02X:%02X:%02X:%02X", 177 | hwaddr[0], hwaddr[1], hwaddr[2], hwaddr[3], hwaddr[4], hwaddr[5]); 178 | } 179 | 180 | #endif 181 | 182 | static struct sockaddr *if_addr_value(const char *iface, struct ifreq *r, 183 | unsigned long int request) { 184 | int e; 185 | 186 | if (do_socket_ioctl(iface, request, r, &e, PRINT_NO_ERROR)) { 187 | if (e == EADDRNOTAVAIL) 188 | return &r->ifr_addr; 189 | return NULL; 190 | } 191 | return &r->ifr_addr; 192 | } 193 | 194 | struct sockaddr *if_addr(const char *iface, struct ifreq *r) { 195 | return if_addr_value(iface, r, SIOCGIFADDR); 196 | } 197 | 198 | struct sockaddr *if_mask(const char *iface, struct ifreq *r) { 199 | return if_addr_value(iface, r, SIOCGIFNETMASK); 200 | } 201 | 202 | struct sockaddr *if_bcast(const char *iface, struct ifreq *r) { 203 | return if_addr_value(iface, r, SIOCGIFBRDADDR); 204 | } 205 | 206 | struct sockaddr *if_network(const char *iface) { 207 | struct sockaddr *saddr; 208 | static struct ifreq req; 209 | unsigned int mask; 210 | 211 | if (!(saddr = if_mask(iface, &req))) 212 | return NULL; 213 | 214 | mask = ((struct sockaddr_in*)saddr)->sin_addr.s_addr; 215 | 216 | if (!(saddr = if_addr(iface, &req))) 217 | return NULL; 218 | 219 | ((struct sockaddr_in*)saddr)->sin_addr.s_addr &= mask; 220 | return saddr; 221 | } 222 | 223 | int if_mtu(const char *iface) { 224 | static struct ifreq req; 225 | 226 | if (do_socket_ioctl(iface, SIOCGIFMTU, &req, NULL, PRINT_ERROR)) 227 | return 0; 228 | 229 | return req.ifr_mtu; 230 | } 231 | 232 | #if defined(__linux__) 233 | 234 | static void skipline(FILE *fd) { 235 | int ch; 236 | do { 237 | ch = getc(fd); 238 | } while (ch != '\n' && ch != EOF); 239 | } 240 | 241 | struct if_stat *get_stats(const char *iface) { 242 | FILE *fd; 243 | struct if_stat *ifstat; 244 | char name[10]; 245 | 246 | if (!(ifstat = malloc(sizeof(struct if_stat)))) { 247 | perror("malloc"); 248 | return NULL; 249 | } 250 | 251 | if ((fd = fopen("/proc/net/dev", "r")) == NULL) { 252 | perror("fopen(\"/proc/net/dev\")"); 253 | free(ifstat); 254 | return NULL; 255 | } 256 | 257 | /* Skip header */ 258 | skipline(fd); 259 | skipline(fd); 260 | 261 | do { 262 | int items = fscanf(fd, 263 | " %20[^:]:%llu %llu %llu %llu %llu %llu %llu %llu " 264 | "%llu %llu %llu %llu %llu %llu %llu %llu", 265 | name, 266 | &ifstat->in_bytes, &ifstat->in_packets, 267 | &ifstat->in_errors, &ifstat->in_drops, 268 | &ifstat->in_fifo, &ifstat->in_frame, 269 | &ifstat->in_compress, &ifstat->in_multicast, 270 | &ifstat->out_bytes, &ifstat->out_packets, 271 | &ifstat->out_errors, &ifstat->out_drops, 272 | &ifstat->out_fifo, &ifstat->out_colls, 273 | &ifstat->out_carrier, &ifstat->out_carrier 274 | ); 275 | 276 | if (items == -1) 277 | break; 278 | if (items != 17) { 279 | fprintf(stderr, "Invalid data read, check!\n"); 280 | break; 281 | } 282 | 283 | if (!strncmp(name, iface, sizeof(name))) { 284 | fclose(fd); 285 | return ifstat; 286 | } 287 | } while (!feof(fd)); 288 | 289 | fclose(fd); 290 | free(ifstat); 291 | return NULL; 292 | } 293 | 294 | #endif 295 | 296 | const struct { 297 | char *option; 298 | unsigned int flag; 299 | unsigned int is_stat; 300 | char *description; 301 | } options[] = { 302 | { "-e", DO_EXISTS, 0, "Reports interface existence via return code" }, 303 | { "-p", DO_PALL, 0, "Print out the whole config of iface" }, 304 | { "-pe", DO_PEXISTS, 0, "Print out yes or no according to existence" }, 305 | { "-pa", DO_PADDRESS, 0, "Print out the address" }, 306 | { "-pn", DO_PMASK, 0, "Print netmask" }, 307 | { "-pN", DO_PNETWORK, 0, "Print network address" }, 308 | { "-pb", DO_PCAST, 0, "Print broadcast" }, 309 | { "-pm", DO_PMTU, 0, "Print mtu" }, 310 | #if defined(__linux__) 311 | { "-ph", DO_PHWADDRESS, 0, "Print out the hardware address" }, 312 | { "-pf", DO_PFLAGS, 0, "Print flags" }, 313 | { "-si", DO_SINALL, 1, "Print all statistics on input" }, 314 | { "-sip", DO_SINPACKETS, 1, "Print # of in packets" }, 315 | { "-sib", DO_SINBYTES, 1, "Print # of in bytes" }, 316 | { "-sie", DO_SINERRORS, 1, "Print # of in errors" }, 317 | { "-sid", DO_SINDROPS, 1, "Print # of in drops" }, 318 | { "-sif", DO_SINFIFO, 1, "Print # of in fifo overruns" }, 319 | { "-sic", DO_SINCOMPRESSES, 1, "Print # of in compress" }, 320 | { "-sim", DO_SINMULTICAST, 1, "Print # of in multicast" }, 321 | { "-so", DO_SOUTALL, 1, "Print all statistics on output" }, 322 | { "-sop", DO_SOUTPACKETS, 1, "Print # of out packets" }, 323 | { "-sob", DO_SOUTBYTES, 1, "Print # of out bytes" }, 324 | { "-soe", DO_SOUTERRORS, 1, "Print # of out errors" }, 325 | { "-sod", DO_SOUTDROPS, 1, "Print # of out drops" }, 326 | { "-sof", DO_SOUTFIFO, 1, "Print # of out fifo overruns" }, 327 | { "-sox", DO_SOUTCOLLS, 1, "Print # of out collisions" }, 328 | { "-soc", DO_SOUTCARRIER, 1, "Print # of out carrier loss" }, 329 | { "-som", DO_SOUTMULTICAST, 1, "Print # of out multicast" }, 330 | { "-bips",DO_BIPS, 1, "Print # of incoming bytes per second" }, 331 | { "-bops",DO_BOPS, 1, "Print # of outgoing bytes per second" }, 332 | #endif 333 | }; 334 | 335 | void usage(const char *name) { 336 | unsigned int i; 337 | 338 | fprintf(stderr, "Usage: %s [options] iface\n", name); 339 | for (i = 0; i < sizeof(options) / sizeof(options[0]); i++) { 340 | fprintf(stderr, " %5s %s\n", 341 | options[i].option, options[i].description); 342 | } 343 | } 344 | 345 | void add_do(int *ndo, int **todo, int act) { 346 | *todo = realloc(*todo, (*ndo+1) * sizeof(int)); 347 | (*todo)[*ndo] = act; 348 | *ndo += 1; 349 | } 350 | 351 | static void print_addr(struct sockaddr *sadr) { 352 | if (!sadr) { 353 | fprintf(stderr, "Error\n"); 354 | exit(1); 355 | } 356 | print_quad(sadr); 357 | } 358 | 359 | struct if_stat *ifstats, *ifstats2 = NULL; 360 | 361 | void please_do(int ndo, int *todo, const char *ifname) { 362 | int i; 363 | static struct ifreq req; 364 | if (!ndo) return; 365 | // printf("I have %d items in my queue.\n",ndo); 366 | for (i=0; iin_packets); 408 | break; 409 | case DO_SINBYTES: 410 | printf("%llu",ifstats->in_bytes); 411 | break; 412 | case DO_SINERRORS: 413 | printf("%llu",ifstats->in_errors); 414 | break; 415 | case DO_SINDROPS: 416 | printf("%llu",ifstats->in_drops); 417 | break; 418 | case DO_SINFIFO: 419 | printf("%llu",ifstats->in_fifo); 420 | break; 421 | case DO_SINFRAME: 422 | printf("%llu",ifstats->in_frame); 423 | break; 424 | case DO_SINCOMPRESSES: 425 | printf("%llu",ifstats->in_compress); 426 | break; 427 | case DO_SINMULTICAST: 428 | printf("%llu",ifstats->in_multicast); 429 | break; 430 | case DO_SINALL: 431 | printf("%llu %llu %llu %llu %llu %llu %llu %llu", 432 | ifstats->in_bytes, ifstats->in_packets, 433 | ifstats->in_errors, ifstats->in_drops, 434 | ifstats->in_fifo, ifstats->in_frame, 435 | ifstats->in_compress, ifstats->in_multicast); 436 | break; 437 | case DO_SOUTBYTES: 438 | printf("%llu",ifstats->out_bytes); 439 | break; 440 | case DO_SOUTPACKETS: 441 | printf("%llu",ifstats->out_packets); 442 | break; 443 | case DO_SOUTERRORS: 444 | printf("%llu",ifstats->out_errors); 445 | break; 446 | case DO_SOUTDROPS: 447 | printf("%llu",ifstats->out_drops); 448 | break; 449 | case DO_SOUTFIFO: 450 | printf("%llu",ifstats->out_fifo); 451 | break; 452 | case DO_SOUTCOLLS: 453 | printf("%llu",ifstats->out_colls); 454 | break; 455 | case DO_SOUTCARRIER: 456 | printf("%llu",ifstats->out_carrier); 457 | break; 458 | case DO_SOUTMULTICAST: 459 | printf("%llu",ifstats->out_multicast); 460 | break; 461 | case DO_BIPS: 462 | if (ifstats2 == NULL) { 463 | sleep(1); 464 | ifstats2 = get_stats(ifname); 465 | } 466 | printf("%llu", ifstats2->in_bytes-ifstats->in_bytes); 467 | break; 468 | case DO_BOPS: 469 | if (ifstats2 == NULL) { 470 | sleep(1); 471 | ifstats2 = get_stats(ifname); 472 | } 473 | printf("%llu", ifstats2->out_bytes-ifstats->out_bytes); 474 | break; 475 | case DO_SOUTALL: 476 | printf("%llu %llu %llu %llu %llu %llu %llu %llu", 477 | ifstats->out_bytes, ifstats->out_packets, 478 | ifstats->out_errors, ifstats->out_drops, 479 | ifstats->out_fifo, ifstats->out_colls, 480 | ifstats->out_carrier, ifstats->out_multicast); 481 | break; 482 | #endif 483 | default: 484 | printf("Unknown command: %d", todo[i]); 485 | break; 486 | } 487 | printf("\n"); 488 | } 489 | } 490 | 491 | int main(int argc, char *argv[]) { 492 | int ndo=0; 493 | int *todo=NULL; 494 | char *ifname=NULL; 495 | int narg = 0; 496 | int do_stats = 0; 497 | unsigned int i, found; 498 | 499 | if (argc == 1) { 500 | usage(*argv); 501 | return 1; 502 | } 503 | 504 | while (narg < argc - 1) { 505 | narg++; 506 | 507 | found = 0; 508 | 509 | for (i = 0; i < sizeof(options) / sizeof(options[0]); i++) { 510 | if (!strcmp(argv[narg], options[i].option)) { 511 | add_do(&ndo, &todo, options[i].flag); 512 | do_stats |= options[i].is_stat; 513 | found = 1; 514 | break; 515 | } 516 | } 517 | 518 | if (found) 519 | continue; 520 | 521 | if (argv[narg][0] == '-') { 522 | usage(*argv); 523 | return 1; 524 | } 525 | else { 526 | ifname = argv[narg]; 527 | break; 528 | } 529 | } 530 | 531 | if (narg + 1 < argc || !ifname) { 532 | usage(*argv); 533 | return 1; 534 | } 535 | 536 | #if defined(__linux__) 537 | if (do_stats && (ifstats = get_stats(ifname)) == NULL) { 538 | fprintf(stderr, "Error getting statistics for %s\n", ifname); 539 | return 1; 540 | } 541 | #endif 542 | 543 | please_do(ndo, todo, ifname); 544 | 545 | return 0; 546 | } 547 | -------------------------------------------------------------------------------- /ifdata.docbook: -------------------------------------------------------------------------------- 1 | 2 | 3 | 22 | 23 | 26 | 27 | 28 | 29 | 30 | 31 | Joey 32 | Hess 33 | 34 | 2006-03-07 35 | 36 | 37 | 38 | ifdata 39 | 1 40 | 41 | 42 | 43 | ifdata 44 | get network interface info without 45 | parsing ifconfig output 46 | 47 | 48 | 49 | 50 | ifdata 51 | options 52 | iface 53 | 54 | 55 | 56 | 57 | DESCRIPTION 58 | 59 | 60 | ifdata can be used to check for 61 | the existence of a network interface, or to get 62 | information abut the interface, such as its IP 63 | address. Unlike ifconfig or 64 | ip, ifdata 65 | has simple to parse output that is designed to be 66 | easily used by a shell script. 67 | 68 | 69 | 70 | 71 | 72 | OPTIONS 73 | 74 | 75 | 76 | 77 | 78 | 79 | Print out a help summary. 80 | 81 | 82 | 83 | 84 | 85 | 86 | Test to see if the interface exists, 87 | exit nonzero if it does not. 88 | 89 | 90 | 91 | 92 | 93 | 94 | Prints out the whole configuration of 95 | the interface. 96 | 97 | 98 | 99 | 100 | 101 | 102 | Prints "yes" or "no" if the interface 103 | exists or not. 104 | 105 | 106 | 107 | 108 | 109 | 110 | Prints the IPv4 address of the 111 | interface. 112 | 113 | 114 | 115 | 116 | 117 | 118 | Prints the netmask of the 119 | interface. 120 | 121 | 122 | 123 | 124 | 125 | 126 | Prints the network address of the 127 | interface. 128 | 129 | 130 | 131 | 132 | 133 | 134 | Prints the broadcast address of the 135 | interface. 136 | 137 | 138 | 139 | 140 | 141 | 142 | Prints the MTU of the interface. 143 | 144 | 145 | 146 | 147 | 148 | Following options are Linux only. 149 | 150 | 151 | 152 | 153 | 154 | 155 | Prints the hardware address of the 156 | interface. 157 | 158 | 159 | 160 | 161 | 162 | 163 | Prints the flags of the 164 | interface. 165 | 166 | 167 | 168 | 169 | 170 | 171 | Prints out all the input statistics 172 | of the interface. 173 | 174 | 175 | 176 | 177 | 178 | 179 | Prints the number of input packets. 180 | 181 | 182 | 183 | 184 | 185 | 186 | Prints the number of input bytes. 187 | 188 | 189 | 190 | 191 | 192 | 193 | Prints the number of input errors. 194 | 195 | 196 | 197 | 198 | 199 | 200 | Prints the number of dropped input 201 | packets. 202 | 203 | 204 | 205 | 206 | 207 | 208 | Prints the number of input fifo overruns. 209 | 210 | 211 | 212 | 213 | 214 | 215 | Print the number of compressed input 216 | packets. 217 | 218 | 219 | 220 | 221 | 222 | 223 | Prints the number of input 224 | multicast packets. 225 | 226 | 227 | 228 | 229 | 230 | 231 | Prints out all the output statistics 232 | of the interface. 233 | 234 | 235 | 236 | 237 | 238 | 239 | Prints the number of output packets. 240 | 241 | 242 | 243 | 244 | 245 | 246 | Prints the number of output bytes. 247 | 248 | 249 | 250 | 251 | 252 | 253 | Prints the number of output errors. 254 | 255 | 256 | 257 | 258 | 259 | 260 | Prints the number of dropped 261 | output packets. 262 | 263 | 264 | 265 | 266 | 267 | 268 | Prints the number of output fifo overruns. 269 | 270 | 271 | 272 | 273 | 274 | 275 | Print the number of output collisions. 276 | 277 | 278 | 279 | 280 | 281 | 282 | Prints the number of output carrier 283 | losses. 284 | 285 | 286 | 287 | 288 | 289 | 290 | Prints the number of output multicast 291 | packets. 292 | 293 | 294 | 295 | 296 | 297 | 298 | Prints the number of bytes of 299 | incoming traffic measured in one second. 300 | 301 | 302 | 303 | 304 | 305 | 306 | Prints the number of bytes of 307 | outgoing traffic measured in one second. 308 | 309 | 310 | 311 | 312 | 313 | 314 | 315 | AUTHOR 316 | 317 | 318 | Benjamin BAYART 319 | 320 | 321 | 322 | -------------------------------------------------------------------------------- /ifne.c: -------------------------------------------------------------------------------- 1 | /* 2 | * 3 | * Copyright 2008 Javier Merino 4 | * This program is free software; you can redistribute it and/or modify it 5 | * under the terms of the GNU General Public License as published by the 6 | * Free Software Foundation; either version 2 of the License, or (at your 7 | * option) any later version. 8 | * 9 | * This program is distributed in the hope that it will be useful, but 10 | * WITHOUT ANY WARRANTY; without even the implied warranty of 11 | * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General 12 | * Public License for more details. 13 | * 14 | * You should have received a copy of the GNU General Public License along 15 | * with this program; if not, write to the Free Software Foundation, Inc., 16 | * 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 17 | * 18 | */ 19 | 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #define streq(a, b) (!strcmp((a), (b))) 27 | 28 | static void stdin_to_stream(char *buf, ssize_t r, FILE *outf) { 29 | while (r > 0) { 30 | if (fwrite(buf, r*sizeof(char), 1, outf) < 1) { 31 | fprintf(stderr, "Write error\n"); 32 | exit(EXIT_FAILURE); 33 | } 34 | r = read(0, buf, BUFSIZ*sizeof(char)); 35 | } 36 | if (r == -1) { 37 | perror("read"); 38 | exit(EXIT_FAILURE); 39 | } 40 | } 41 | 42 | int main(int argc, char **argv) { 43 | ssize_t r; 44 | int run_if_empty; 45 | char **argv_exec; 46 | int fds[2]; 47 | int child_status; 48 | pid_t child_pid; 49 | char buf[BUFSIZ]; 50 | FILE *outf; 51 | 52 | if ((argc < 2) || ((argc == 2) && streq(argv[1], "-n"))) { 53 | fprintf(stderr, "Usage: ifne [-n] command [args]\n"); 54 | return EXIT_FAILURE; 55 | } 56 | 57 | if (streq(argv[1], "-n")) { 58 | run_if_empty = 1; 59 | argv_exec = &argv[2]; 60 | } else { 61 | run_if_empty = 0; 62 | argv_exec = &argv[1]; 63 | } 64 | 65 | r = read(0, buf, BUFSIZ*sizeof(char)); 66 | 67 | if ((r == 0) && !run_if_empty) 68 | return EXIT_SUCCESS; 69 | else if (r == -1) { 70 | perror("read"); 71 | return EXIT_FAILURE; 72 | } 73 | 74 | if (pipe(fds)) { 75 | perror("pipe"); 76 | return EXIT_FAILURE; 77 | } 78 | 79 | if (r && run_if_empty) { 80 | /* don't run the subcommand if we read something from stdin and -n was set */ 81 | /* But write stdin to stdout so ifne -n can be piped without sucking the stream */ 82 | stdin_to_stream(buf, r, stdout); 83 | return EXIT_SUCCESS; 84 | } 85 | 86 | child_pid = fork(); 87 | if (!child_pid) { 88 | /* child process: rebind stdin and exec the subcommand */ 89 | close(fds[1]); 90 | if (dup2(fds[0], 0)) { 91 | perror("dup2"); 92 | return EXIT_FAILURE; 93 | } 94 | 95 | execvp(*argv_exec, argv_exec); 96 | perror(*argv_exec); 97 | close(fds[0]); 98 | return EXIT_FAILURE; 99 | } else if (child_pid == -1) { 100 | perror("fork"); 101 | return EXIT_FAILURE; 102 | } 103 | 104 | /* Parent: write stdin to fds[1] */ 105 | close(fds[0]); 106 | outf = fdopen(fds[1], "w"); 107 | if (! outf) { 108 | perror("fdopen"); 109 | exit(1); 110 | } 111 | 112 | stdin_to_stream(buf, r, outf); 113 | fclose(outf); 114 | 115 | if (waitpid(child_pid, &child_status, 0) != child_pid) { 116 | perror("waitpid"); 117 | return EXIT_FAILURE; 118 | } 119 | if (WIFEXITED(child_status)) { 120 | return (WEXITSTATUS(child_status)); 121 | } else if (WIFSIGNALED(child_status)) { 122 | raise(WTERMSIG(child_status)); 123 | return EXIT_FAILURE; 124 | } 125 | 126 | return EXIT_FAILURE; 127 | } 128 | -------------------------------------------------------------------------------- /ifne.docbook: -------------------------------------------------------------------------------- 1 | 2 | 3 | 22 | 23 | 26 | 27 | 28 | 29 |
30 | cibervicho@gmail.com 31 |
32 | 33 | Javier 34 | Merino 35 | 36 | 2008-05-01 37 | 38 | 39 | 40 | ifne 41 | 1 42 | 43 | 44 | 45 | ifne 46 | Run command if the standard input is not empty 47 | 48 | 49 | 50 | 51 | ifne [-n] command 52 | 53 | 54 | 55 | 56 | DESCRIPTION 57 | 58 | ifne runs the following command if and only if 59 | the standard input is not empty. 60 | 61 | 62 | 63 | OPTIONS 64 | 65 | 66 | 67 | 68 | 69 | Reverse operation. Run the command if the standard input is empty. 70 | Note that if the standard input is not empty, it is passed through ifne 71 | in this case. 72 | 73 | 74 | 75 | 76 | 77 | 78 | EXAMPLE 79 | 80 | find . -name core | ifne mail -s "Core files found" root 81 | 82 | 83 | 84 | 85 | AUTHOR 86 | 87 | Copyright 2008 by Javier Merino <cibervicho@gmail.com> 88 | Licensed under the GNU GPL 89 | 90 | 91 | 92 | -------------------------------------------------------------------------------- /isutf8.c: -------------------------------------------------------------------------------- 1 | /* 2 | * isutf8.c - do the input files look like valid utf-8 byte streams? 3 | * 4 | * Copyright (C) 2005 Lars Wirzenius 5 | * 6 | * This program is free software; you can redistribute it and/or modify 7 | * it under the terms of the GNU General Public License as published by 8 | * the Free Software Foundation; either version 2 of the License, or 9 | * (at your option) any later version. 10 | * 11 | * This program is distributed in the hope that it will be useful, 12 | * but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | * GNU General Public License for more details. 15 | * 16 | * You should have received a copy of the GNU General Public License 17 | * along with this program; if not, write to the Free Software 18 | * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 19 | */ 20 | 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | 28 | 29 | #define VERSION "1.1" 30 | 31 | 32 | /* 33 | * Code to indicate an invalid UTF8 character. 34 | */ 35 | enum { INVALID_CHAR = 0xffffffff }; 36 | 37 | 38 | /* 39 | * Produce shortest UTF8 encoding of a 31-bit value in 'u', returning it 40 | * in the array 'buf'. Return the number of bytes in the encoded value. 41 | * If the value is too large (more than 32 bits or would take more than 42 | * 'maxbytes' bytes), return -1. 43 | */ 44 | static int encodeutf8(unsigned long u, unsigned char *buf, size_t maxbytes) 45 | { 46 | static const struct { 47 | int nbytes; 48 | unsigned long max; 49 | } tab[] = { 50 | { 1, 0x0000007F }, 51 | { 2, 0x000007FF }, 52 | { 3, 0x0000FFFF }, 53 | { 4, 0x001FFFFF }, 54 | { 5, 0x03FFFFFF }, 55 | { 6, 0x7FFFFFFF }, 56 | }; 57 | static const int ntab = sizeof(tab) / sizeof(tab[0]); 58 | int i, j; 59 | 60 | if (u > tab[ntab-1].max) 61 | return -1; 62 | 63 | for (i = 0; i < ntab; ++i) { 64 | if (u <= tab[i].max) 65 | break; 66 | } 67 | assert(i < ntab); 68 | 69 | if (tab[i].nbytes > maxbytes) 70 | return -1; 71 | 72 | if (tab[i].nbytes == 1) { /* Special case */ 73 | buf[0] = u; 74 | } else { 75 | for (j = tab[i].nbytes-1; j > 0; --j) { 76 | buf[j] = 0x80 | (u & 0x3f); 77 | u >>= 6; 78 | } 79 | 80 | unsigned char mask = ~(0xFF >> tab[i].nbytes); 81 | buf[0] = mask | u; 82 | } 83 | 84 | return tab[i].nbytes; 85 | } 86 | 87 | 88 | /* 89 | * Return number of ones at the top of a byte. 90 | * 91 | * I'm pretty sure there is a fancy trick to do this without a loop, 92 | * but I'm too tired to figure it out now. --liw 93 | */ 94 | static int high_ones(int c) { 95 | int n; 96 | 97 | for (n = 0; (c & 0x80) == 0x80; c <<= 1) 98 | ++n; 99 | return n; 100 | } 101 | 102 | 103 | /* 104 | * Decode a UTF8 character from an array of bytes. Return character code. 105 | * Upon error, return INVALID_CHAR. 106 | */ 107 | static unsigned long decodeutf8(unsigned char *buf, int nbytes) 108 | { 109 | unsigned long u; 110 | int i, j; 111 | 112 | if (nbytes <= 0) 113 | return INVALID_CHAR; 114 | 115 | if (nbytes == 1) { 116 | if (buf[0] >= 0x80) 117 | return INVALID_CHAR; 118 | return buf[0]; 119 | } 120 | 121 | i = high_ones(buf[0]); 122 | if (i != nbytes) 123 | return INVALID_CHAR; 124 | u = buf[0] & (0xff >> i); 125 | for (j = 1; j < nbytes; ++j) { 126 | if ((buf[j] & 0xC0) != 0x80) 127 | return INVALID_CHAR; 128 | u = (u << 6) | (buf[j] & 0x3f); 129 | } 130 | 131 | /* Conforming UTF-8 cannot contain codes 0xd800–0xdfff (UTF-16 132 | surrogates) as well as 0xfffe and 0xffff. */ 133 | if (u >= 0xD800 && u <= 0xDFFF) 134 | return INVALID_CHAR; 135 | if (u == 0xFFFE || u == 0xFFFF) 136 | return INVALID_CHAR; 137 | 138 | return u; 139 | } 140 | 141 | 142 | /* 143 | * Determine if the contents of an open file form a valid UTF8 byte stream. 144 | * Do this by collecting bytes for a character into a buffer and then 145 | * decode the bytes and re-encode them and compare that they are identical 146 | * to the original bytes. If any step fails, return 0 for error. If EOF 147 | * is reached, return 1 for OK. 148 | */ 149 | static int is_utf8_byte_stream(FILE *file, char *filename, int quiet) { 150 | enum { MAX_UTF8_BYTES = 6 }; 151 | unsigned char buf[MAX_UTF8_BYTES]; 152 | unsigned char buf2[MAX_UTF8_BYTES]; 153 | int nbytes, nbytes2; 154 | int c; 155 | unsigned long code; 156 | unsigned long line, col, byteoff; 157 | 158 | nbytes = 0; 159 | line = 1; 160 | col = 1; 161 | byteoff = 0; 162 | 163 | for (;;) { 164 | c = getc(file); 165 | 166 | if (c == EOF || c < 0x80 || (c & 0xC0) != 0x80) { 167 | /* New char starts, deal with previous one. */ 168 | if (nbytes > 0) { 169 | code = decodeutf8(buf, nbytes); 170 | if (code == INVALID_CHAR) 171 | goto error; 172 | nbytes2 = encodeutf8(code, buf2, 173 | MAX_UTF8_BYTES); 174 | if (nbytes != nbytes2 || 175 | memcmp(buf, buf2, nbytes) != 0) 176 | goto error; 177 | ++col; 178 | } 179 | nbytes = 0; 180 | /* If it's UTF8, start collecting again. */ 181 | if (c != EOF && c >= 0x80) 182 | buf[nbytes++] = c; 183 | } else { 184 | /* This is a continuation byte, append to buffer. */ 185 | if (nbytes == MAX_UTF8_BYTES) 186 | goto error; 187 | buf[nbytes++] = c; 188 | } 189 | 190 | if (c == EOF) 191 | break; 192 | else if (c == '\n') { 193 | ++line; 194 | byteoff = 0; 195 | col = 1; 196 | } else 197 | ++byteoff; 198 | } 199 | 200 | if (nbytes != 0) 201 | goto error; 202 | 203 | return 1; 204 | 205 | error: 206 | if (!quiet) { 207 | printf("%s: line %lu, char %lu, byte offset %lu: " 208 | "invalid UTF-8 code\n", filename, line, col, byteoff); 209 | } 210 | return 0; 211 | } 212 | 213 | 214 | static void usage(const char *program_name) { 215 | printf("Usage: %s [-hq] [--help] [--quiet] [file ...]\n", 216 | program_name); 217 | printf("Check whether input files are valid UTF-8.\n"); 218 | printf("This is version %s.\n", VERSION); 219 | } 220 | 221 | 222 | int main(int argc, char **argv) { 223 | int i, ok; 224 | FILE *file; 225 | 226 | int quiet; 227 | struct option options[] = { 228 | { "help", no_argument, NULL, 'h' }, 229 | { "quiet", no_argument, &quiet, 1 }, 230 | }; 231 | int opt; 232 | 233 | quiet = 0; 234 | 235 | while ((opt = getopt_long(argc, argv, "hq", options, NULL)) != -1) { 236 | switch (opt) { 237 | case 0: 238 | break; 239 | 240 | case 'h': 241 | usage(argv[0]); 242 | exit(0); 243 | break; 244 | 245 | case 'q': 246 | quiet = 1; 247 | break; 248 | 249 | case '?': 250 | exit(EXIT_FAILURE); 251 | 252 | default: 253 | abort(); 254 | } 255 | } 256 | 257 | if (optind == argc) 258 | ok = is_utf8_byte_stream(stdin, "stdin", quiet); 259 | else { 260 | ok = 1; 261 | for (i = optind; i < argc; ++i) { 262 | file = fopen(argv[i], "r"); 263 | if (file == NULL) { 264 | fprintf(stderr, "isutf8: %s: error %d: %s\n", 265 | argv[i], errno, 266 | strerror(errno)); 267 | ok = 0; 268 | } else { 269 | if (! is_utf8_byte_stream(file, argv[i], quiet)) 270 | ok = 0; 271 | (void) fclose(file); 272 | } 273 | } 274 | } 275 | 276 | if (ok) 277 | exit(0); 278 | exit(EXIT_FAILURE); 279 | } 280 | -------------------------------------------------------------------------------- /isutf8.docbook: -------------------------------------------------------------------------------- 1 | 2 | 3 | 22 | 23 | 26 | 27 | 28 | 29 |
30 | liw@iki.fi 31 |
32 | 33 | Lars 34 | Wirzenius 35 | 36 | 2006-02-19 37 | 38 | 39 | 40 | isutf8 41 | 1 42 | 43 | 44 | 45 | isutf8 46 | check whether files are valid UTF-8 47 | 48 | 49 | 50 | 51 | isutf8 52 | 53 | 54 | 55 | 56 | file 57 | 58 | 59 | 60 | 61 | 62 | DESCRIPTION 63 | 64 | isutf8 checks whether files are 65 | syntactically valid UTF-8. Input is either files named on the 66 | command line, or the standard input. Notices about files with 67 | invalid UTF-8 are printed to standard output. 68 | 69 | 70 | 71 | 72 | OPTIONS 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | Print out a help summary. 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | Don't print messages telling which files are 89 | invalid UTF-8, merely indicate it with the exit 90 | status. 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | EXIT STATUS 100 | 101 | If the file is valid UTF-8, the exit status is zero. 102 | If the file is not valid UTF-8, or there is some 103 | error, the exit status is non-zero. 104 | 105 | 106 | 107 | 108 | AUTHOR 109 | Lars Wirzenius 110 | 111 | 112 | 113 | 114 | 115 | 116 | SEE ALSO 117 | 118 | 119 | 120 | utf87 121 | 122 | 123 | 124 | 125 | 126 | -------------------------------------------------------------------------------- /lckdo.c: -------------------------------------------------------------------------------- 1 | /* lckdo.c: run a program with a lock held, 2 | * to prevent multiple processes running in parallel. 3 | * Use just like `nice' or `nohup'. 4 | * Written by Michael Tokarev 5 | * Public domain. 6 | */ 7 | 8 | #define _GNU_SOURCE 9 | #define _BSD_SOURCE 10 | #include 11 | #include 12 | #include 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | 22 | /* compile with -DUSE_FLOCK to use flock() instead of fcntl() */ 23 | 24 | #if !defined(USE_FLOCK) && !defined(F_SETLKW) 25 | # define USE_FLOCK 26 | #endif 27 | 28 | #ifndef __GNUC__ 29 | # ifndef __attribute__ 30 | # define __attribute__(x) 31 | # endif 32 | #endif 33 | 34 | static char *progname; 35 | static void 36 | __attribute__((format(printf,3,4))) 37 | __attribute__((noreturn)) 38 | error(int errnum, int exitcode, const char *fmt, ...) { 39 | va_list ap; 40 | fprintf(stderr, "%s: ", progname); 41 | va_start(ap, fmt); vfprintf(stderr, fmt, ap); va_end(ap); 42 | if (errnum) 43 | fprintf(stderr, ": %s\n", strerror(errnum)); 44 | else 45 | fputs("\n", stderr); 46 | exit(exitcode); 47 | } 48 | 49 | static const char *lckfile; 50 | static int quiet; 51 | 52 | static void sigalarm(int sig) { 53 | if (quiet) 54 | _exit(EX_TEMPFAIL); 55 | error(0, EX_TEMPFAIL, 56 | "lock file `%s' is already locked (timeout waiting)", lckfile); 57 | } 58 | 59 | int main(int argc, char **argv) { 60 | int fd; 61 | int c; 62 | int create = O_CREAT; 63 | int dofork = 1; 64 | int waittime = 0; 65 | int shared = 0; 66 | int test = 0; 67 | int fdn = -1; 68 | #ifndef USE_FLOCK 69 | struct flock fl; 70 | #endif 71 | 72 | if ((progname = strrchr(argv[0], '/')) == NULL) 73 | progname = argv[0]; 74 | else 75 | argv[0] = ++progname; 76 | 77 | if (argc == 1) { 78 | printf( 79 | "%s: execute a program with a lock set.\n" 80 | "Usage: %s [options] lockfile program [arguments]\n" 81 | "where options are:\n" 82 | " -w - if the lock is already held by another process,\n" 83 | " wait for it to complete instead of failing immediately\n" 84 | " -W sec - the same as -w but wait not more than sec seconds\n" 85 | " -e - execute the program directly, no fork/wait\n" 86 | " (keeps extra open file descriptor)\n" 87 | " -E nnn - set the fd# to keep open in -e case (implies -e)\n" 88 | " -n - do not create the lock file if it does not exist\n" 89 | " -q - produce no output if lock is already held\n" 90 | " -s - lock in shared (read) mode\n" 91 | " -x - lock in exclusive (write) mode (default)\n" 92 | " -t - test for lock existence" 93 | #ifndef USE_FLOCK 94 | " (just prints pid if any with -q)\n" 95 | #endif 96 | " (implies -n)\n" 97 | , progname, progname); 98 | return 0; 99 | } 100 | 101 | while ((c = getopt(argc, argv, "+wW:neE:sxtq")) != EOF) { 102 | switch(c) { 103 | case 'w': 104 | if (!waittime) 105 | waittime = -1; 106 | break; 107 | case 'W': 108 | if ((waittime = atoi(optarg)) < 1) 109 | error(0, EX_USAGE, "invalid wait time `%s'", optarg); 110 | break; 111 | case 't': 112 | test = 1; 113 | /* fall thru */ 114 | case 'n': 115 | create = 0; 116 | break; 117 | case 'E': 118 | if ((fdn = atoi(optarg)) < 0 || fdn == 2) 119 | error(0, EX_USAGE, "invalid fd# `%s'", optarg); 120 | /* fall thru */ 121 | case 'e': 122 | dofork = 0; 123 | break; 124 | case 's': 125 | shared = 1; 126 | break; 127 | case 'x': 128 | shared = 0; 129 | break; 130 | case 'q': 131 | quiet = 1; 132 | break; 133 | default: 134 | return EX_USAGE; 135 | } 136 | } 137 | 138 | argc -= optind; argv += optind; 139 | if (!argc || (!test && argc < 2)) 140 | error(0, EX_USAGE, "too few arguments given"); 141 | 142 | lckfile = *argv++; 143 | 144 | #ifdef USE_FLOCK 145 | create |= O_RDONLY; 146 | #else 147 | if (!test) 148 | create |= shared ? O_RDONLY : O_WRONLY; 149 | #endif 150 | fd = open(lckfile, create, 0666); 151 | if (fd < 0) { 152 | if (test && errno == ENOENT) { 153 | if (!quiet) 154 | printf("lockfile `%s' is not locked\n", lckfile); 155 | return 0; 156 | } 157 | error(errno, EX_CANTCREAT, "unable to open `%s'", lckfile); 158 | } 159 | 160 | if (!test && fdn >= 0) { 161 | /* dup it early to comply with stupid POSIX fcntl locking 162 | * semantics */ 163 | if (dup2(fd, fdn) < 0) 164 | error(errno, EX_OSERR, "dup2(%d,%d) failed", fd, fdn); 165 | close(fd); 166 | fd = fdn; 167 | } 168 | 169 | if (test) 170 | waittime = 0; 171 | else if (waittime > 0) { 172 | alarm(waittime); 173 | signal(SIGALRM, sigalarm); 174 | } 175 | #ifdef USE_FLOCK 176 | c = flock(fd, (shared ? LOCK_SH : LOCK_EX) | (waittime ? 0 : LOCK_NB)); 177 | if (test && c < 0 && 178 | (errno == EWOULDBLOCK || errno == EAGAIN || errno == EACCES)) { 179 | if (!quiet) 180 | printf("lockfile `%s' is locked\n", lckfile); 181 | else 182 | printf("locked\n"); 183 | return EX_TEMPFAIL; 184 | } 185 | #else 186 | memset(&fl, 0, sizeof(fl)); 187 | fl.l_type = shared ? F_RDLCK : F_WRLCK; 188 | c = fcntl(fd, test ? F_GETLK : waittime ? F_SETLKW : F_SETLK, &fl); 189 | if (test && c == 0) { 190 | if (fl.l_type == F_UNLCK) { 191 | if (!quiet) 192 | printf("lockfile `%s' is not locked\n", lckfile); 193 | return 0; 194 | } 195 | if (!quiet) 196 | printf("lockfile `%s' is locked by process %d\n", lckfile, fl.l_pid); 197 | else 198 | printf("%d\n", fl.l_pid); 199 | return EX_TEMPFAIL; 200 | } 201 | #endif 202 | if (waittime > 0) 203 | alarm(0); 204 | if (c < 0) { 205 | if (errno != EWOULDBLOCK && errno != EAGAIN && errno != EACCES) 206 | error(errno, EX_OSERR, "unable to lock `%s'", lckfile); 207 | else if (quiet) 208 | return EX_TEMPFAIL; 209 | else 210 | error(0, EX_TEMPFAIL, "lockfile `%s' is already locked", lckfile); 211 | } 212 | 213 | if (dofork) { 214 | pid_t pid; 215 | int flags = fcntl(fd, F_GETFD, 0); 216 | if (flags < 0) 217 | error(errno, EX_OSERR, "fcntl() failed"); 218 | fcntl(fd, F_SETFD, flags | FD_CLOEXEC); 219 | pid = fork(); 220 | if (pid < 0) 221 | error(errno, EX_OSERR, "unable to fork"); 222 | else if (pid) { 223 | if (wait(&c) < 0) 224 | error(errno, EX_OSERR, "wait() failed"); 225 | if (WIFSIGNALED(c)) 226 | error(0, EX_SOFTWARE, "%s: %s", *argv, 227 | strsignal(WTERMSIG(c))); 228 | return WEXITSTATUS(c); 229 | } 230 | } 231 | execvp(*argv, argv); 232 | error(errno, EX_OSERR, "unable to execute %s", *argv); 233 | } 234 | -------------------------------------------------------------------------------- /lckdo.docbook: -------------------------------------------------------------------------------- 1 | 2 | 3 | 9 | 10 | 13 | 14 | 15 | 16 |
17 | mjt@tls.msk.ru 18 |
19 | 20 | Michael 21 | Tokarev 22 | 23 | 2007-08-15 24 | 25 | 26 | 27 | lckdo 28 | 1 29 | 30 | 31 | 32 | lckdo 33 | run a program with a lock held 34 | 35 | 36 | 37 | 38 | lckdo 39 | options 40 | lockfile 41 | program 42 | arguments 43 | 44 | 45 | 46 | 47 | DESCRIPTION 48 | 49 | lckdo runs a program with a lock 50 | held, in order to prevent multiple processes from running in 51 | parallel. Use just like nice or 52 | nohup. 53 | 54 | Now that util-linux contains a similar command 55 | named flock, lckdo is deprecated, 56 | and will be removed from some future version of moreutils. 57 | 58 | 59 | 60 | 61 | 62 | OPTIONS 63 | 64 | 65 | 66 | 67 | 68 | 69 | If the lock is already held by another process, 70 | wait for it to complete instead of failing 71 | immediately. 72 | 73 | 74 | 75 | 76 | 77 | 78 | The same as -w but wait not more than sec 79 | seconds. 80 | 81 | 82 | 83 | 84 | 85 | 86 | Execute the program directly without forking and 87 | waiting (keeps an extra file descriptor open). 88 | 89 | 90 | 91 | 92 | 93 | 94 | Set the file descriptor number to keep open when 95 | exec()ing (implies -e). 96 | 97 | 98 | 99 | 100 | 101 | 102 | Do not create the lock file if it does not 103 | exist. 104 | 105 | 106 | 107 | 108 | 109 | 110 | Produce no output if lock is already held. 111 | 112 | 113 | 114 | 115 | 116 | 117 | Lock in shared (read) mode. 118 | 119 | 120 | 121 | 122 | 123 | 124 | Lock in exclusive (write) mode (default). 125 | 126 | 127 | 128 | 129 | 130 | 131 | Test for lock existence. 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | EXIT STATUS 141 | 142 | If the lock was successfully acquired, the return value is that 143 | of the program invoked by lckdo. If the lock 144 | couldn't be acquired, EX_TEMPFAIL is returned. If there was a problem 145 | opening/creating or locking the lock file, EX_CANTCREAT or EX_OSERR 146 | will be returned. 147 | 148 | 149 | 150 | 151 | AUTHOR 152 | 153 | Michael Tokarev 154 | 155 | 156 | 157 | 158 | -------------------------------------------------------------------------------- /mispipe.c: -------------------------------------------------------------------------------- 1 | /* 2 | * mispipe: written by Nathanael Nerode. 3 | * 4 | * Copyright 2004 Nathanael Nerode. 5 | * 6 | * Licensed under the GPL version 2 or above, and dual-licensed under the 7 | * following license: 8 | * 9 | * Permission is hereby granted, free of charge, to any person obtaining a 10 | * copy of this software and associated documentation files (the "Software"), 11 | * to deal in the Software without restriction, including without limitation 12 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, 13 | * and/or sell copies of the Software, and to permit persons to whom the 14 | * Software is furnished to do so, subject to the following conditions: 15 | * 16 | * The above copyright notice and this permission notice shall be included in 17 | * all copies or substantial portions of the Software. 18 | * 19 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 20 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 21 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL 22 | * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 23 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 24 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 25 | * THE SOFTWARE. 26 | */ 27 | 28 | /* 29 | * Usage: mispipe 30 | * Run | , but return with the exit status of . 31 | * 32 | * This is designed for a very specific purpose: logging. 33 | * "foo | logger -t foo" 34 | * will return with the exit status of logger, not that of foo. 35 | * "mispipe foo 'logger -t foo'" 36 | * will return with the exit status of foo. 37 | */ 38 | 39 | /* 40 | * To do: 41 | * Make this into a fancy, internationalized, option-handling hellbeast. 42 | * (But why bother? It does its job quite well.) 43 | */ 44 | 45 | #include /* errno */ 46 | #include 47 | #include /* pipe(), fork(),... */ 48 | #include /* system() */ 49 | #include /* waitpid(), etc. */ 50 | #include 51 | #include /* va_list, for error() */ 52 | 53 | static const char* progname; /* Stores argv[0] */ 54 | static int filedes[2]; /* Stores pipe file descriptors */ 55 | 56 | /* Subroutine for 'warning' and 'error' which prefixes progname */ 57 | static void warning_prefix(void) { 58 | fputs(progname, stderr); 59 | fputs(": ", stderr); 60 | } 61 | 62 | /* Issue a warning, then die */ 63 | __attribute__(( noreturn, format (printf, 1, 2) )) 64 | static void error(const char* formatstr, ...) { 65 | va_list ap; 66 | va_start(ap, formatstr); 67 | warning_prefix(); 68 | vfprintf(stderr, formatstr, ap); 69 | va_end(ap); 70 | exit(1); 71 | } 72 | 73 | /* Issue a warning, then die, with errno */ 74 | __attribute__(( noreturn )) 75 | static void error_with_errno(const char* message) { 76 | int saved_errno; 77 | saved_errno=errno; 78 | warning_prefix(); 79 | fputs(message, stderr); 80 | fputs(": error number ", stderr); 81 | fprintf(stderr, "%i\n", saved_errno); 82 | exit(1); 83 | } 84 | 85 | /* Convert 'wait'/'system'-style exit status to 'exit'-style exit status */ 86 | __attribute__(( const )) 87 | static int shorten_status(int status_big) { 88 | if (WIFEXITED(status_big)) 89 | return WEXITSTATUS(status_big); 90 | if (WIFSIGNALED(status_big)) 91 | return 128+WTERMSIG(status_big); 92 | if (WIFSTOPPED(status_big)) 93 | return 128+WSTOPSIG(status_big); 94 | #ifdef WCOREDUMP 95 | if (WCOREDUMP(status_big)) 96 | error("Ow, somebody dumped core!"); 97 | #endif 98 | error("shorten_status got an invalid status (?!)"); 99 | } 100 | 101 | /* All the work for command 2. */ 102 | __attribute__(( noreturn )) 103 | static void subprocess2(const char* cmd) { 104 | /* Close the old standard input. */ 105 | if (close(0)) 106 | error_with_errno("Failed (in child) closing standard input"); 107 | /* Make the reading end of the pipe the new standard input. */ 108 | if (dup2(filedes[0], 0) == -1) 109 | error_with_errno("Failed (in child) redefining standard input"); 110 | /* Close the original file descriptor for it */ 111 | if (close(filedes[0])) 112 | error_with_errno("Failed (in child) closing filedes[0]"); 113 | /* Close the other end of the pipe. */ 114 | if (close(filedes[1])) 115 | error_with_errno("Failed (in child) closing filedes[1]"); 116 | /* Do the second command, and throw away the exit status. */ 117 | system(cmd); 118 | /* Close the standard input. */ 119 | if (close(0)) 120 | error_with_errno("Failed (in child) closing standard output " 121 | " (while cleaning up)"); 122 | exit(0); 123 | } 124 | 125 | int main (int argc, const char ** argv) { 126 | int status_big; /* Exit status, 'wait' and 'system' style */ 127 | pid_t child2_pid; 128 | pid_t dead_pid; 129 | 130 | /* Set progname */ 131 | progname = argv[0]; 132 | 133 | /* Verify arguments */ 134 | if (argc != 3) 135 | error("Wrong number of args, aborting\n"); 136 | /* Open a new pipe */ 137 | if (pipe(filedes)) 138 | error_with_errno("pipe() failed"); 139 | 140 | /* Fork to run second command */ 141 | child2_pid = fork(); 142 | if (child2_pid == 0) 143 | subprocess2(argv[2]); 144 | if (child2_pid == -1) 145 | error_with_errno("fork() failed"); 146 | 147 | /* Run first command inline (seriously!) */ 148 | /* Close standard output. */ 149 | if (close(1)) 150 | error_with_errno("Failed closing standard output"); 151 | /* Make the writing end of the pipe the new standard output. */ 152 | if (dup2(filedes[1], 1) == -1) 153 | error_with_errno("Failed redefining standard output"); 154 | /* Close the original file descriptor for it. */ 155 | if (close(filedes[1])) 156 | error_with_errno("Failed closing filedes[1]"); 157 | /* Close the other end of the pipe. */ 158 | if (close(filedes[0])) 159 | error_with_errno("Failed closing filedes[0]"); 160 | /* Do the first command, and (crucially) get the status. */ 161 | status_big = system(argv[1]); 162 | 163 | /* Close standard output. */ 164 | if (close(1)) 165 | error_with_errno("Failed closing standard output (while cleaning up)"); 166 | 167 | /* Wait for the other process to exit. */ 168 | /* We don't care about the status. */ 169 | dead_pid = waitpid(child2_pid, NULL, WUNTRACED); 170 | if (dead_pid == -1) { 171 | error_with_errno("waitpid() failed"); 172 | } 173 | else if (dead_pid != child2_pid) { 174 | error("waitpid(): Who died? %i\n", dead_pid); 175 | } 176 | 177 | /* Return the desired exit status. */ 178 | return shorten_status(status_big); 179 | } 180 | -------------------------------------------------------------------------------- /mispipe.docbook: -------------------------------------------------------------------------------- 1 | 2 | 3 | 22 | 23 | 26 | 27 | 28 | 29 |
30 | neroden@fastmail.fm 31 |
32 | 33 | Nathanael 34 | Nerode 35 | 36 | 2006-09-07 37 | 38 | 39 | 40 | mispipe 41 | 1 42 | 43 | 44 | 45 | mispipe 46 | pipe two commands, returning the exit status of 47 | the first 48 | 49 | 50 | 51 | 52 | mispipe 53 | "command1" 54 | "command2" 55 | 56 | 57 | 58 | 59 | DESCRIPTION 60 | 61 | mispipe pipes two commands 62 | together like the shell does, but unlike piping in the 63 | shell, which returns the exit status of the last command; 64 | when using mispipe, the exit status of the first command 65 | is returned. 66 | 67 | 68 | 69 | Note that some shells, notably bash, 70 | do offer a pipefail option, however, that option does not 71 | behave the same since it makes a failure of any command in 72 | the pipeline be returned, not just the exit status of the 73 | first. 74 | 75 | 76 | 77 | 78 | 79 | EXIT STATUS 80 | 81 | The exit status of the first command. If the process 82 | terminated abnormally (due to a signal), 128 will be added 83 | to its exit status. 84 | 85 | 86 | 87 | 88 | AUTHOR 89 | 90 | Nathanael Nerode 91 | 92 | 93 | 94 | -------------------------------------------------------------------------------- /parallel.c: -------------------------------------------------------------------------------- 1 | /* 2 | * parallel.c - run commands in parallel until you run out of commands 3 | * 4 | * Copyright © 2008 Tollef Fog Heen 5 | * 6 | * This program is free software; you can redistribute it and/or 7 | * modify it under the terms of the GNU General Public License 8 | * version 2 as published by the Free Software Foundation. 9 | * 10 | * This program is distributed in the hope that it will be useful, 11 | * but WITHOUT ANY WARRANTY; without even the implied warranty of 12 | * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 13 | * General Public License for more details. 14 | * 15 | * You should have received a copy of the GNU General Public License 16 | * along with this program; if not, write to the Free Software 17 | * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 18 | * USA 19 | * 20 | */ 21 | 22 | #define _GNU_SOURCE 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include 32 | #include 33 | #include 34 | 35 | #if defined(__FreeBSD_kernel__) 36 | #define WEXITED 0 37 | #endif 38 | 39 | void usage() { 40 | printf("parallel [OPTIONS] command -- arguments\n\tfor each argument, " 41 | "run command with argument, in parallel\n"); 42 | printf("parallel [OPTIONS] -- commands\n\trun specified commands in parallel\n"); 43 | exit(1); 44 | } 45 | 46 | void exec_child(char **command, char **arguments, int replace_cb, int nargs) { 47 | if (fork() != 0) { 48 | return; 49 | } 50 | 51 | if (command[0]) { 52 | char **argv; 53 | int argc = 0; 54 | int i; 55 | char *s; 56 | 57 | while (command[argc] != 0) { 58 | argc++; 59 | } 60 | if (! replace_cb) 61 | argc++; 62 | argv = calloc(sizeof(char*), argc + nargs); 63 | 64 | for (i = 0; i < argc; i++) { 65 | while (replace_cb && (s=strstr(command[i], "{}"))) { 66 | char *buf=malloc(strlen(command[i]) + strlen(arguments[0])); 67 | s[0]='\0'; 68 | sprintf(buf, "%s%s%s", command[i], arguments[0], s+2); 69 | command[i]=buf; 70 | } 71 | argv[i] = command[i]; 72 | } 73 | if (! replace_cb) 74 | memcpy(argv + i - 1, arguments, nargs * sizeof(char *)); 75 | execvp(argv[0], argv); 76 | exit(1); 77 | } 78 | else { 79 | int ret=system(arguments[0]); 80 | if (WIFEXITED(ret)) { 81 | exit(WEXITSTATUS(ret)); 82 | } 83 | else { 84 | exit(1); 85 | } 86 | } 87 | return; 88 | } 89 | 90 | int wait_for_child(int options) { 91 | id_t id_ignored = 0; 92 | siginfo_t infop; 93 | 94 | infop.si_pid = 0; 95 | waitid(P_ALL, id_ignored, &infop, WEXITED | options); 96 | if (infop.si_pid == 0) { 97 | return -1; /* Nothing to wait for */ 98 | } 99 | if (infop.si_code == CLD_EXITED) { 100 | return infop.si_status; 101 | } 102 | return 1; 103 | } 104 | 105 | int main(int argc, char **argv) { 106 | int maxjobs = -1; 107 | int curjobs = 0; 108 | double maxload = -1; 109 | int argsatonce = 1; 110 | int opt; 111 | char **command = calloc(sizeof(char*), argc); 112 | char **arguments = NULL; 113 | int argidx = 0; 114 | int arglen = 0; 115 | int cidx = 0; 116 | int returncode = 0; 117 | int replace_cb = 0; 118 | char *t; 119 | 120 | while ((argv[optind] && strcmp(argv[optind], "--") != 0) && 121 | (opt = getopt(argc, argv, "+hij:l:n:")) != -1) { 122 | switch (opt) { 123 | case 'h': 124 | usage(); 125 | break; 126 | case 'i': 127 | replace_cb = 1; 128 | break; 129 | case 'j': 130 | errno = 0; 131 | maxjobs = strtoul(optarg, &t, 0); 132 | if (errno != 0 || (t-optarg) != strlen(optarg)) { 133 | fprintf(stderr, "option '%s' is not a number\n", 134 | optarg); 135 | exit(2); 136 | } 137 | break; 138 | case 'l': 139 | errno = 0; 140 | maxload = strtod(optarg, &t); 141 | if (errno != 0 || (t-optarg) != strlen(optarg)) { 142 | fprintf(stderr, "option '%s' is not a number\n", 143 | optarg); 144 | exit(2); 145 | } 146 | break; 147 | case 'n': 148 | errno = 0; 149 | argsatonce = strtoul(optarg, &t, 0); 150 | if (errno != 0 || argsatonce < 1 || (t-optarg) != strlen(optarg)) { 151 | fprintf(stderr, "option '%s' is not a positive number\n", 152 | optarg); 153 | exit(2); 154 | } 155 | break; 156 | default: /* ’?’ */ 157 | usage(); 158 | break; 159 | } 160 | } 161 | 162 | if (replace_cb && argsatonce > 1) { 163 | fprintf(stderr, "options -i and -n are incomaptible\n"); 164 | exit(2); 165 | } 166 | 167 | if (maxjobs < 0) { 168 | #ifdef _SC_NPROCESSORS_ONLN 169 | maxjobs = sysconf(_SC_NPROCESSORS_ONLN); 170 | #else 171 | #warning Cannot autodetect number of CPUS on this system: _SC_NPROCESSORS_ONLN not defined. 172 | maxjobs = 1; 173 | #endif 174 | } 175 | 176 | while (optind < argc) { 177 | if (strcmp(argv[optind], "--") == 0) { 178 | int i; 179 | 180 | optind++; 181 | arglen = argc - optind; 182 | arguments = calloc(sizeof(char *), arglen); 183 | if (! arguments) { 184 | exit(1); 185 | } 186 | 187 | for (i = 0; i < arglen; i++) { 188 | arguments[i] = strdup(argv[optind + i]); 189 | } 190 | optind += i; 191 | } 192 | else { 193 | command[cidx] = strdup(argv[optind]); 194 | cidx++; 195 | } 196 | optind++; 197 | } 198 | 199 | if (argsatonce > 1 && ! command[0]) { 200 | fprintf(stderr, "option -n cannot be used without a command\n"); 201 | exit(2); 202 | } 203 | 204 | while (argidx < arglen) { 205 | double load; 206 | 207 | getloadavg(&load, 1); 208 | 209 | if ((maxjobs == 0 || curjobs < maxjobs) && 210 | (maxload < 0 || load < maxload)) { 211 | 212 | if (argsatonce > arglen - argidx) 213 | argsatonce = arglen - argidx; 214 | exec_child(command, arguments + argidx, 215 | replace_cb, argsatonce); 216 | argidx += argsatonce; 217 | curjobs++; 218 | } 219 | 220 | if (maxjobs == 0 || curjobs == maxjobs) { 221 | returncode |= wait_for_child(0); 222 | curjobs--; 223 | } 224 | 225 | if (maxload > 0 && load >= maxload) { 226 | int r; 227 | sleep(1); /* XXX We should have a better 228 | * heurestic than this */ 229 | r = wait_for_child(WNOHANG); 230 | if (r > 0) 231 | returncode |= r; 232 | if (r != -1) 233 | curjobs--; 234 | } 235 | } 236 | while (curjobs > 0) { 237 | returncode |= wait_for_child(0); 238 | curjobs--; 239 | } 240 | 241 | return returncode; 242 | } 243 | 244 | -------------------------------------------------------------------------------- /parallel.docbook: -------------------------------------------------------------------------------- 1 | 2 | 3 | 8 | 9 | 12 | 13 | 14 | 15 |
16 | joey@kitenet.net 17 |
18 | 19 | Joey 20 | Hess 21 | 22 | 2009-07-02 23 | 24 | 25 | 26 | parallel 27 | 1 28 | 29 | 30 | 31 | parallel 32 | run programs in parallel 33 | 34 | 35 | 36 | 37 | parallel 38 | options 39 | command 40 | -- 41 | argument ... 42 | 43 | 44 | parallel 45 | options 46 | -- 47 | command ... 48 | 49 | 50 | 51 | 52 | DESCRIPTION 53 | 54 | parallel runs the specified command, 55 | passing it a single one of the specified arguments. This is 56 | repeated for each argument. Jobs may be run in 57 | parallel. The default is to run one job per CPU. 58 | 59 | If no command is specified before the --, 60 | the commands after it are instead run in parallel. 61 | 62 | 63 | 64 | 65 | OPTIONS 66 | 67 | 68 | 69 | 70 | 71 | 72 | Use to limit the number of jobs 73 | that are run at the same time. 74 | 75 | 76 | 77 | 78 | 79 | 80 | Wait as needed to avoid starting 81 | new jobs when the system's load average 82 | is not below the specified limit. 83 | 84 | 85 | 86 | 87 | 88 | 89 | Normally the command is passed the 90 | argument at the end of its command line. With 91 | this option, any instances of "{}" in 92 | the command are replaced with the argument. 93 | 94 | 95 | 96 | 97 | 98 | 99 | Number of arguments to pass to a 100 | command at a time. Default is 1. 101 | Incompatible with -i 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | EXAMPLE 111 | 112 | 113 | 114 | parallel sh -c "echo hi; sleep 2; echo bye" -- 1 2 3 115 | 116 | 117 | 118 | This runs three subshells that each print a message, delay, 119 | and print another message. If your system has multiple 120 | CPUs, parallel will run some of the jobs in parallel, 121 | which should be clear from the order the messages are 122 | output. 123 | 124 | 125 | 126 | 127 | parallel -j 3 ufraw -o processed -- *.NEF 128 | 129 | 130 | 131 | This runs three ufraw processes at the same time until 132 | all of the NEF files have been processed. 133 | 134 | 135 | 136 | 137 | parallel -j 3 -- ls df "echo hi" 138 | 139 | 140 | 141 | This runs three independent commands in parallel. 142 | 143 | 144 | 145 | 146 | EXIT STATUS 147 | 148 | 149 | Its exit status is the combination of the exit statuses of each 150 | command ran, ORed together. (Thus, if any one command 151 | exits nonzero, parallel as a whole will exit nonzero.) 152 | 153 | 154 | 155 | 156 | AUTHOR 157 | 158 | Tollef Fog Heen 159 | 160 | 161 | 162 | 163 | -------------------------------------------------------------------------------- /pee.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | 6 | /* Licensed under the GPL 7 | * Copyright (c) Miek Gieben, 2006 8 | */ 9 | 10 | /* like tee(1), but then connect to other programs using 11 | * pipes _and_ output to standard output 12 | */ 13 | 14 | int 15 | close_pipes(FILE **p, size_t i) 16 | { 17 | int ret=EXIT_SUCCESS; 18 | size_t j; 19 | for (j = 0; j < i; j++) { 20 | int r = pclose(p[j]); 21 | if (WIFEXITED(r)) 22 | ret |= WEXITSTATUS(r); 23 | else 24 | ret |= 1; 25 | } 26 | return ret; 27 | } 28 | 29 | int 30 | main(int argc, char **argv) { 31 | size_t i, r; 32 | FILE **pipes; 33 | char buf[BUFSIZ]; 34 | 35 | pipes = malloc(((argc - 1) * sizeof *pipes)); 36 | if (!pipes) 37 | exit(EXIT_FAILURE); 38 | 39 | for (i = 1; i < argc; i++) { 40 | pipes[i - 1] = popen(argv[i], "w"); 41 | if (!pipes[i - 1]) { 42 | fprintf(stderr, "Can not open pipe to '%s\'\n", argv[i]); 43 | close_pipes(pipes, i); 44 | 45 | exit(EXIT_FAILURE); 46 | } 47 | } 48 | argc--; 49 | 50 | while(!feof(stdin) && (!ferror(stdin))) { 51 | r = fread(buf, sizeof(char), BUFSIZ, stdin); 52 | for(i = 0; i < argc; i++) { 53 | if (fwrite(buf, sizeof(char), r, pipes[i]) != r) { 54 | fprintf(stderr, "Write error to `%s\'\n", argv[i + 1]); 55 | close_pipes(pipes, i); 56 | exit(EXIT_FAILURE); 57 | } 58 | } 59 | } 60 | exit(close_pipes(pipes, argc)); 61 | } 62 | -------------------------------------------------------------------------------- /pee.docbook: -------------------------------------------------------------------------------- 1 | 2 | 3 | 22 | 23 | 26 | 27 | 28 | 29 |
30 | joey@kitenet.net 31 |
32 | 33 | Joey 34 | Hess 35 | 36 | 2006-03-14 37 | 38 | 39 | 40 | pee 41 | 1 42 | 43 | 44 | 45 | pee 46 | tee standard input to pipes 47 | 48 | 49 | 50 | 51 | pee 52 | 53 | "command" 54 | 55 | 56 | 57 | 58 | 59 | DESCRIPTION 60 | 61 | pee is like tee 62 | but for pipes. Each command is run and fed a copy of the 63 | standard input. The output of all commands is sent to 64 | stdout. 65 | 66 | Note that while this is similar to 67 | tee, a copy of the input is not sent 68 | to stdout, like tee does. If that is desired, use 69 | pee cat ... 70 | 71 | 72 | 73 | SEE ALSO 74 | 75 | 76 | 77 | tee1 78 | 79 | 80 | 81 | 82 | 83 | 84 | AUTHOR 85 | 86 | Miek Gieben 87 | 88 | 89 | 90 | -------------------------------------------------------------------------------- /physmem.c: -------------------------------------------------------------------------------- 1 | /* Calculate the size of physical memory. 2 | 3 | Copyright (C) 2000, 2001, 2003, 2005, 2006 Free Software 4 | Foundation, Inc. 5 | 6 | This program is free software; you can redistribute it and/or modify 7 | it under the terms of the GNU General Public License as published by 8 | the Free Software Foundation; either version 2, or (at your option) 9 | any later version. 10 | 11 | This program is distributed in the hope that it will be useful, 12 | but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 | GNU General Public License for more details. 15 | 16 | You should have received a copy of the GNU General Public License 17 | along with this program; if not, write to the Free Software Foundation, 18 | Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */ 19 | 20 | /* Written by Paul Eggert. */ 21 | 22 | #include 23 | 24 | #if HAVE_SYS_PSTAT_H 25 | # include 26 | #endif 27 | 28 | #if HAVE_SYS_SYSMP_H 29 | # include 30 | #endif 31 | 32 | #if HAVE_SYS_SYSINFO_H && HAVE_MACHINE_HAL_SYSINFO_H 33 | # include 34 | # include 35 | #endif 36 | 37 | #if HAVE_SYS_TABLE_H 38 | # include 39 | #endif 40 | 41 | #include 42 | 43 | #if HAVE_SYS_PARAM_H 44 | # include 45 | #endif 46 | 47 | #if HAVE_SYS_SYSCTL_H 48 | # include 49 | #endif 50 | 51 | #if HAVE_SYS_SYSTEMCFG_H 52 | # include 53 | #endif 54 | 55 | #ifdef _WIN32 56 | # define WIN32_LEAN_AND_MEAN 57 | # include 58 | /* MEMORYSTATUSEX is missing from older windows headers, so define 59 | a local replacement. */ 60 | typedef struct 61 | { 62 | DWORD dwLength; 63 | DWORD dwMemoryLoad; 64 | DWORDLONG ullTotalPhys; 65 | DWORDLONG ullAvailPhys; 66 | DWORDLONG ullTotalPageFile; 67 | DWORDLONG ullAvailPageFile; 68 | DWORDLONG ullTotalVirtual; 69 | DWORDLONG ullAvailVirtual; 70 | DWORDLONG ullAvailExtendedVirtual; 71 | } lMEMORYSTATUSEX; 72 | typedef WINBOOL (WINAPI *PFN_MS_EX) (lMEMORYSTATUSEX*); 73 | #endif 74 | 75 | #define ARRAY_SIZE(a) (sizeof (a) / sizeof ((a)[0])) 76 | 77 | /* Return the total amount of physical memory. */ 78 | double 79 | physmem_total (void) 80 | { 81 | #if defined _SC_PHYS_PAGES && defined _SC_PAGESIZE 82 | { /* This works on linux-gnu, solaris2 and cygwin. */ 83 | double pages = sysconf (_SC_PHYS_PAGES); 84 | double pagesize = sysconf (_SC_PAGESIZE); 85 | if (0 <= pages && 0 <= pagesize) 86 | return pages * pagesize; 87 | } 88 | #endif 89 | 90 | #if HAVE_PSTAT_GETSTATIC 91 | { /* This works on hpux11. */ 92 | struct pst_static pss; 93 | if (0 <= pstat_getstatic (&pss, sizeof pss, 1, 0)) 94 | { 95 | double pages = pss.physical_memory; 96 | double pagesize = pss.page_size; 97 | if (0 <= pages && 0 <= pagesize) 98 | return pages * pagesize; 99 | } 100 | } 101 | #endif 102 | 103 | #if HAVE_SYSMP && defined MP_SAGET && defined MPSA_RMINFO && defined _SC_PAGESIZE 104 | { /* This works on irix6. */ 105 | struct rminfo realmem; 106 | if (sysmp (MP_SAGET, MPSA_RMINFO, &realmem, sizeof realmem) == 0) 107 | { 108 | double pagesize = sysconf (_SC_PAGESIZE); 109 | double pages = realmem.physmem; 110 | if (0 <= pages && 0 <= pagesize) 111 | return pages * pagesize; 112 | } 113 | } 114 | #endif 115 | 116 | #if HAVE_GETSYSINFO && defined GSI_PHYSMEM 117 | { /* This works on Tru64 UNIX V4/5. */ 118 | int physmem; 119 | 120 | if (getsysinfo (GSI_PHYSMEM, (caddr_t) &physmem, sizeof (physmem), 121 | NULL, NULL, NULL) == 1) 122 | { 123 | double kbytes = physmem; 124 | 125 | if (0 <= kbytes) 126 | return kbytes * 1024.0; 127 | } 128 | } 129 | #endif 130 | 131 | #if HAVE_SYSCTL && defined HW_PHYSMEM 132 | { /* This works on *bsd and darwin. */ 133 | unsigned int physmem; 134 | size_t len = sizeof physmem; 135 | static int mib[2] = { CTL_HW, HW_PHYSMEM }; 136 | 137 | if (sysctl (mib, ARRAY_SIZE (mib), &physmem, &len, NULL, 0) == 0 138 | && len == sizeof (physmem)) 139 | return (double) physmem; 140 | } 141 | #endif 142 | 143 | #if HAVE__SYSTEM_CONFIGURATION 144 | /* This works on AIX. */ 145 | return _system_configuration.physmem; 146 | #endif 147 | 148 | #if defined _WIN32 149 | { /* this works on windows */ 150 | PFN_MS_EX pfnex; 151 | HMODULE h = GetModuleHandle ("kernel32.dll"); 152 | 153 | if (!h) 154 | return 0.0; 155 | 156 | /* Use GlobalMemoryStatusEx if available. */ 157 | if ((pfnex = (PFN_MS_EX) GetProcAddress (h, "GlobalMemoryStatusEx"))) 158 | { 159 | lMEMORYSTATUSEX lms_ex; 160 | lms_ex.dwLength = sizeof lms_ex; 161 | if (!pfnex (&lms_ex)) 162 | return 0.0; 163 | return (double) lms_ex.ullTotalPhys; 164 | } 165 | 166 | /* Fall back to GlobalMemoryStatus which is always available. 167 | but returns wrong results for physical memory > 4GB. */ 168 | else 169 | { 170 | MEMORYSTATUS ms; 171 | GlobalMemoryStatus (&ms); 172 | return (double) ms.dwTotalPhys; 173 | } 174 | } 175 | #endif 176 | 177 | /* Guess 64 MB. It's probably an older host, so guess small. */ 178 | return 64 * 1024 * 1024; 179 | } 180 | 181 | /* Return the amount of physical memory available. */ 182 | double 183 | physmem_available (void) 184 | { 185 | #if defined _SC_AVPHYS_PAGES && defined _SC_PAGESIZE 186 | { /* This works on linux-gnu, solaris2 and cygwin. */ 187 | double pages = sysconf (_SC_AVPHYS_PAGES); 188 | double pagesize = sysconf (_SC_PAGESIZE); 189 | if (0 <= pages && 0 <= pagesize) 190 | return pages * pagesize; 191 | } 192 | #endif 193 | 194 | #if HAVE_PSTAT_GETSTATIC && HAVE_PSTAT_GETDYNAMIC 195 | { /* This works on hpux11. */ 196 | struct pst_static pss; 197 | struct pst_dynamic psd; 198 | if (0 <= pstat_getstatic (&pss, sizeof pss, 1, 0) 199 | && 0 <= pstat_getdynamic (&psd, sizeof psd, 1, 0)) 200 | { 201 | double pages = psd.psd_free; 202 | double pagesize = pss.page_size; 203 | if (0 <= pages && 0 <= pagesize) 204 | return pages * pagesize; 205 | } 206 | } 207 | #endif 208 | 209 | #if HAVE_SYSMP && defined MP_SAGET && defined MPSA_RMINFO && defined _SC_PAGESIZE 210 | { /* This works on irix6. */ 211 | struct rminfo realmem; 212 | if (sysmp (MP_SAGET, MPSA_RMINFO, &realmem, sizeof realmem) == 0) 213 | { 214 | double pagesize = sysconf (_SC_PAGESIZE); 215 | double pages = realmem.availrmem; 216 | if (0 <= pages && 0 <= pagesize) 217 | return pages * pagesize; 218 | } 219 | } 220 | #endif 221 | 222 | #if HAVE_TABLE && defined TBL_VMSTATS 223 | { /* This works on Tru64 UNIX V4/5. */ 224 | struct tbl_vmstats vmstats; 225 | 226 | if (table (TBL_VMSTATS, 0, &vmstats, 1, sizeof (vmstats)) == 1) 227 | { 228 | double pages = vmstats.free_count; 229 | double pagesize = vmstats.pagesize; 230 | 231 | if (0 <= pages && 0 <= pagesize) 232 | return pages * pagesize; 233 | } 234 | } 235 | #endif 236 | 237 | #if HAVE_SYSCTL && defined HW_USERMEM 238 | { /* This works on *bsd and darwin. */ 239 | unsigned int usermem; 240 | size_t len = sizeof usermem; 241 | static int mib[2] = { CTL_HW, HW_USERMEM }; 242 | 243 | if (sysctl (mib, ARRAY_SIZE (mib), &usermem, &len, NULL, 0) == 0 244 | && len == sizeof (usermem)) 245 | return (double) usermem; 246 | } 247 | #endif 248 | 249 | #if defined _WIN32 250 | { /* this works on windows */ 251 | PFN_MS_EX pfnex; 252 | HMODULE h = GetModuleHandle ("kernel32.dll"); 253 | 254 | if (!h) 255 | return 0.0; 256 | 257 | /* Use GlobalMemoryStatusEx if available. */ 258 | if ((pfnex = (PFN_MS_EX) GetProcAddress (h, "GlobalMemoryStatusEx"))) 259 | { 260 | lMEMORYSTATUSEX lms_ex; 261 | lms_ex.dwLength = sizeof lms_ex; 262 | if (!pfnex (&lms_ex)) 263 | return 0.0; 264 | return (double) lms_ex.ullAvailPhys; 265 | } 266 | 267 | /* Fall back to GlobalMemoryStatus which is always available. 268 | but returns wrong results for physical memory > 4GB */ 269 | else 270 | { 271 | MEMORYSTATUS ms; 272 | GlobalMemoryStatus (&ms); 273 | return (double) ms.dwAvailPhys; 274 | } 275 | } 276 | #endif 277 | 278 | /* Guess 25% of physical memory. */ 279 | return physmem_total () / 4; 280 | } 281 | 282 | 283 | #if DEBUG 284 | 285 | # include 286 | # include 287 | 288 | int 289 | main (void) 290 | { 291 | printf ("%12.f %12.f\n", physmem_total (), physmem_available ()); 292 | exit (0); 293 | } 294 | 295 | #endif /* DEBUG */ 296 | 297 | /* 298 | Local Variables: 299 | compile-command: "gcc -DDEBUG -g -O -Wall -W physmem.c" 300 | End: 301 | */ 302 | -------------------------------------------------------------------------------- /sponge.c: -------------------------------------------------------------------------------- 1 | /* 2 | * sponge.c - read in all available info from stdin, then output it to 3 | * file named on the command line 4 | * 5 | * Copyright © 2006 Tollef Fog Heen 6 | * 7 | * This program is free software; you can redistribute it and/or 8 | * modify it under the terms of the GNU General Public License 9 | * version 2 as published by the Free Software Foundation. 10 | * 11 | * This program is distributed in the hope that it will be useful, 12 | * but WITHOUT ANY WARRANTY; without even the implied warranty of 13 | * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 14 | * General Public License for more details. 15 | * 16 | * You should have received a copy of the GNU General Public License 17 | * along with this program; if not, write to the Free Software 18 | * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 19 | * USA 20 | * 21 | */ 22 | 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | /* MAX() */ 29 | #include 30 | #include 31 | #include 32 | #include 33 | #include 34 | /* SIZE_MAX */ 35 | #include 36 | #include 37 | 38 | #include "physmem.c" 39 | 40 | #define BUFF_SIZE 8192 41 | #define MIN_SPONGE_SIZE BUFF_SIZE 42 | char *tmpname = NULL; 43 | 44 | void usage() { 45 | printf("sponge : soak up all input from stdin and write it to \n"); 46 | exit(0); 47 | } 48 | 49 | /* all the signal stuff copied from gnu sort */ 50 | 51 | /* The set of signals that are caught. */ 52 | static sigset_t caught_signals; 53 | 54 | /* Critical section status. */ 55 | struct cs_status { 56 | int valid; // was bool 57 | sigset_t sigs; 58 | }; 59 | 60 | /* Enter a critical section. */ 61 | static struct cs_status cs_enter (void) { 62 | struct cs_status status; 63 | status.valid = (sigprocmask(SIG_BLOCK, &caught_signals, &status.sigs) == 0); 64 | return status; 65 | } 66 | 67 | /* Leave a critical section. */ 68 | static void cs_leave (struct cs_status status) { 69 | if (status.valid) { 70 | /* Ignore failure when restoring the signal mask. */ 71 | sigprocmask(SIG_SETMASK, &status.sigs, NULL); 72 | } 73 | } 74 | 75 | static void cleanup () { 76 | if (tmpname) { 77 | unlink(tmpname); 78 | } 79 | } 80 | 81 | static void onexit_cleanup (void) { 82 | struct cs_status cs = cs_enter(); 83 | cleanup(); 84 | cs_leave(cs); 85 | } 86 | 87 | static void sighandler (int sig) { 88 | if (! SA_NOCLDSTOP) 89 | signal(sig, SIG_IGN); 90 | 91 | cleanup(); 92 | 93 | signal(sig, SIG_DFL); 94 | raise(sig); 95 | } 96 | 97 | /* taken from coreutils sort */ 98 | static size_t default_sponge_size (void) { 99 | /* Let MEM be available memory or 1/8 of total memory, whichever 100 | is greater. */ 101 | double avail = physmem_available(); 102 | double total = physmem_total(); 103 | double mem = MAX(avail, total / 8); 104 | struct rlimit rlimit; 105 | 106 | /* Let SIZE be MEM, but no more than the maximum object size or 107 | system resource limits. Avoid the MIN macro here, as it is not 108 | quite right when only one argument is floating point. Don't 109 | bother to check for values like RLIM_INFINITY since in practice 110 | they are not much less than SIZE_MAX. */ 111 | size_t size = SIZE_MAX; 112 | if (mem < size) 113 | size = mem; 114 | if (getrlimit(RLIMIT_DATA, &rlimit) == 0 && rlimit.rlim_cur < size) 115 | size = rlimit.rlim_cur; 116 | #ifdef RLIMIT_AS 117 | if (getrlimit(RLIMIT_AS, &rlimit) == 0 && rlimit.rlim_cur < size) 118 | size = rlimit.rlim_cur; 119 | #endif 120 | 121 | /* Leave a large safety margin for the above limits, as failure can 122 | occur when they are exceeded. */ 123 | size /= 2; 124 | 125 | #ifdef RLIMIT_RSS 126 | /* Leave a 1/16 margin for RSS to leave room for code, stack, etc. 127 | Exceeding RSS is not fatal, but can be quite slow. */ 128 | if (getrlimit(RLIMIT_RSS, &rlimit) == 0 && rlimit.rlim_cur / 16 * 15 < size) 129 | size = rlimit.rlim_cur / 16 * 15; 130 | #endif 131 | 132 | /* Use no less than the minimum. */ 133 | return MAX (size, MIN_SPONGE_SIZE); 134 | } 135 | 136 | void trapsignals (void) { 137 | ssize_t i = 0; 138 | static int const sig[] = { 139 | /* The usual suspects. */ 140 | SIGALRM, SIGHUP, SIGINT, SIGPIPE, SIGQUIT, SIGTERM, 141 | #ifdef SIGPOLL 142 | SIGPOLL, 143 | #endif 144 | #ifdef SIGPROF 145 | SIGPROF, 146 | #endif 147 | #ifdef SIGVTALRM 148 | SIGVTALRM, 149 | #endif 150 | #ifdef SIGXCPU 151 | SIGXCPU, 152 | #endif 153 | #ifdef SIGXFSZ 154 | SIGXFSZ, 155 | #endif 156 | }; 157 | int nsigs = sizeof(sig) / sizeof(sig[0]); 158 | 159 | #if SA_NOCLDSTOP 160 | struct sigaction act; 161 | 162 | sigemptyset(&caught_signals); 163 | for (i = 0; i < nsigs; i++) { 164 | sigaction(sig[i], NULL, &act); 165 | if (act.sa_handler != SIG_IGN) 166 | sigaddset(&caught_signals, sig[i]); 167 | } 168 | 169 | act.sa_handler = sighandler; 170 | act.sa_mask = caught_signals; 171 | act.sa_flags = 0; 172 | 173 | for (i = 0; i < nsigs; i++) 174 | if (sigismember(&caught_signals, sig[i])) 175 | sigaction(sig[i], &act, NULL); 176 | #else 177 | for (i = 0; i < nsigs; i++) 178 | if (signal(sig[i], SIG_IGN) != SIG_IGN) { 179 | signal(sig[i], sighandler); 180 | siginterrupt (sig[i], 1); 181 | } 182 | #endif 183 | } 184 | 185 | static void write_buff_tmp(char* buff, size_t length, FILE *fd) { 186 | if (fwrite(buff, length, 1, fd) < 1) { 187 | perror("error writing buffer to temporary file"); 188 | fclose(fd); 189 | exit(1); 190 | } 191 | } 192 | 193 | static void write_buff_tmp_finish (char* buff, size_t length, FILE *fd) { 194 | if (length) 195 | write_buff_tmp(buff, length, fd); 196 | if (fflush(fd) != 0) { 197 | perror("fflush"); 198 | exit(1); 199 | } 200 | } 201 | 202 | static void write_buff_out (char* buff, size_t length, FILE *fd) { 203 | if (fwrite(buff, length, 1, fd) < 1) { 204 | perror("error writing buffer to output file"); 205 | fclose(fd); 206 | exit(1); 207 | } 208 | } 209 | 210 | static void copy_tmpfile (FILE *tmpfile, FILE *outfile, char *buf, size_t size) { 211 | ssize_t i; 212 | if (lseek(fileno(tmpfile), 0, SEEK_SET)) { 213 | perror("could to seek to start of temporary file"); 214 | fclose(tmpfile); 215 | exit(1); 216 | } 217 | while ((i = read(fileno(tmpfile), buf, size)) > 0) { 218 | write_buff_out(buf, i, outfile); 219 | } 220 | if (i == -1) { 221 | perror("read temporary file"); 222 | fclose(tmpfile); 223 | exit(1); 224 | } 225 | fclose(tmpfile); 226 | fclose(outfile); 227 | } 228 | 229 | FILE *open_tmpfile (void) { 230 | struct cs_status cs; 231 | int tmpfd; 232 | FILE *tmpfile; 233 | mode_t mask; 234 | char *tmpdir; 235 | char const * const template="%s/sponge.XXXXXX"; 236 | 237 | trapsignals(); 238 | cs = cs_enter(); 239 | tmpdir = getenv("TMPDIR"); 240 | if (tmpdir == NULL) 241 | tmpdir = "/tmp"; 242 | /* Subtract 2 for `%s' and add 1 for the trailing NULL. */ 243 | tmpname=malloc(strlen(tmpdir) + strlen(template) - 2 + 1); 244 | if (! tmpname) { 245 | perror("failed to allocate memory"); 246 | exit(1); 247 | } 248 | sprintf(tmpname, template, tmpdir); 249 | mask=umask(077); 250 | tmpfd = mkstemp(tmpname); 251 | umask(mask); 252 | atexit(onexit_cleanup); // solaris on_exit(onexit_cleanup, 0); 253 | cs_leave(cs); 254 | 255 | if (tmpfd < 0) { 256 | perror("mkstemp failed"); 257 | exit(1); 258 | } 259 | tmpfile = fdopen(tmpfd, "w+"); 260 | if (! tmpfile) { 261 | perror("fdopen"); 262 | exit(1); 263 | } 264 | return tmpfile; 265 | } 266 | 267 | int main (int argc, char **argv) { 268 | char *buf, *bufstart, *outname = NULL; 269 | size_t bufsize = BUFF_SIZE; 270 | size_t bufused = 0; 271 | FILE *outfile, *tmpfile = 0; 272 | ssize_t i = 0; 273 | size_t mem_available = default_sponge_size(); 274 | int tmpfile_used=0; 275 | 276 | if (argc > 2 || (argc == 2 && strcmp(argv[1], "-h") == 0)) { 277 | usage(); 278 | } 279 | if (argc == 2) { 280 | outname = argv[1]; 281 | } 282 | 283 | tmpfile = open_tmpfile(); 284 | bufstart = buf = malloc(bufsize); 285 | if (!buf) { 286 | perror("failed to allocate memory"); 287 | exit(1); 288 | } 289 | while ((i = read(0, buf, bufsize - bufused)) > 0) { 290 | bufused = bufused+i; 291 | if (bufused == bufsize) { 292 | if ((bufsize*2) >= mem_available) { 293 | write_buff_tmp(bufstart, bufused, tmpfile); 294 | bufused = 0; 295 | tmpfile_used = 1; 296 | } 297 | else { 298 | bufsize *= 2; 299 | bufstart = realloc(bufstart, bufsize); 300 | if (!bufstart) { 301 | perror("failed to realloc memory"); 302 | exit(1); 303 | } 304 | } 305 | } 306 | buf = bufstart + bufused; 307 | } 308 | if (i < 0) { 309 | perror("failed to read from stdin"); 310 | exit(1); 311 | } 312 | 313 | if (outname) { 314 | mode_t mode; 315 | struct stat statbuf; 316 | int exists = (lstat(outname, &statbuf) == 0); 317 | 318 | write_buff_tmp_finish(bufstart, bufused, tmpfile); 319 | 320 | /* Set temp file mode to match either 321 | * the old file mode, or the default file 322 | * mode for a newly created file. */ 323 | if (exists) { 324 | mode = statbuf.st_mode; 325 | } 326 | else { 327 | mode_t mask = umask(0); 328 | umask(mask); 329 | mode = 0666 & ~mask; 330 | } 331 | if (chmod(tmpname, mode) != 0) { 332 | perror("chmod"); 333 | exit(1); 334 | } 335 | 336 | /* If it's a regular file, or does not yet exist, 337 | * attempt a fast rename of the temp file. */ 338 | if (((exists && 339 | S_ISREG(statbuf.st_mode) && 340 | ! S_ISLNK(statbuf.st_mode) 341 | ) || ! exists) && 342 | rename(tmpname, outname) == 0) { 343 | tmpname=NULL; /* don't try to cleanup tmpname */ 344 | } 345 | else { 346 | /* Fall back to slow copy. */ 347 | outfile = fopen(outname, "w"); 348 | if (!outfile) { 349 | perror("error opening output file"); 350 | exit(1); 351 | } 352 | copy_tmpfile(tmpfile, outfile, bufstart, bufsize); 353 | } 354 | } 355 | else { 356 | if (tmpfile_used) { 357 | write_buff_tmp_finish(bufstart, bufused, tmpfile); 358 | copy_tmpfile(tmpfile, stdout, bufstart, bufsize); 359 | } 360 | else if (bufused) { 361 | /* buffer direct to stdout, no tmpfile */ 362 | write_buff_out(bufstart, bufused, stdout); 363 | } 364 | } 365 | 366 | return 0; 367 | } 368 | -------------------------------------------------------------------------------- /sponge.docbook: -------------------------------------------------------------------------------- 1 | 2 | 3 | 22 | 23 | 26 | 27 | 28 | 29 | 30 | 31 | Joey 32 | Hess 33 | 34 | 2006-02-19 35 | 36 | 37 | 38 | sponge 39 | 1 40 | 41 | 42 | 43 | sponge 44 | soak up standard input and write to a file 45 | 46 | 47 | 48 | 49 | sed '...' file | grep '...' | sponge file 50 | 51 | 52 | 53 | 54 | DESCRIPTION 55 | 56 | sponge reads standard input and 57 | writes it out to the specified file. Unlike a shell 58 | redirect, sponge soaks up all its input before 59 | opening the output file. This allows constricting 60 | pipelines that read from and write to 61 | the same file. 62 | 63 | It also creates the output file 64 | atomically by renaming a temp file into place, 65 | and preserves the permissions of the output file 66 | if it already exists. 67 | If the output file is a special file or symlink, 68 | the data will be written to it. 69 | If no output file is specified, sponge outputs to 70 | stdout. 71 | 72 | 73 | 74 | 75 | AUTHOR 76 | 77 | Colin Watson and Tollef Fog Heen 78 | 79 | 80 | 81 | -------------------------------------------------------------------------------- /ts: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | =head1 NAME 4 | 5 | ts - timestamp input 6 | 7 | =head1 SYNOPSIS 8 | 9 | ts [-r] [format] 10 | 11 | =head1 DESCRIPTION 12 | 13 | ts adds a timestamp to the beginning of each line of input. 14 | 15 | The optional format parameter controls how the timestamp is formatted, 16 | as used by L. The default format is "%b %d %H:%M:%S". In 17 | addition to the regular strftime conversion specifications, "%.S" and "%.s" 18 | are like "%S" and "%s", but provide subsecond resolution 19 | (ie, "30.00001" and "1301682593.00001"). 20 | 21 | If the -r switch is passed, it instead converts existing timestamps in 22 | the input to relative times, such as "15m5s ago". Many common timestamp 23 | formats are supported. Note that the Time::Duration and Date::Parse perl 24 | modules are required for this mode to work. Currently, converting localized 25 | dates is not supported. 26 | 27 | If both -r and a format is passed, the existing timestamps are 28 | converted to the specified format. 29 | 30 | =head1 ENVIRONMENT 31 | 32 | The standard TZ environment variable controls what time zone dates 33 | are assumed to be in, if a timezone is not specified as part of the date. 34 | 35 | =head1 AUTHOR 36 | 37 | Copyright 2006 by Joey Hess 38 | 39 | Licensed under the GNU GPL. 40 | 41 | =cut 42 | 43 | use warnings; 44 | use strict; 45 | use POSIX q{strftime}; 46 | 47 | $|=1; 48 | 49 | my $rel=0; 50 | use Getopt::Long; 51 | GetOptions("r" => \$rel) || die "usage: ts [-r] [format]\n"; 52 | 53 | if ($rel) { 54 | eval q{ 55 | use Date::Parse; 56 | use Time::Duration; 57 | }; 58 | die $@ if $@; 59 | } 60 | 61 | my $use_format=@ARGV; 62 | my $format="%b %d %H:%M:%S"; 63 | $format=shift if @ARGV; 64 | 65 | # For subsecond resolution, Time::HiRes is needed. 66 | my $hires=0; 67 | if ($format=~/\%\.[Ss]/) { 68 | require Time::HiRes; 69 | $hires=1; 70 | } 71 | 72 | while (<>) { 73 | if (! $rel) { 74 | if ($hires) { 75 | my $f=$format; 76 | my ($seconds, $microseconds) = Time::HiRes::gettimeofday(); 77 | my $s=sprintf("%06i", $microseconds); 78 | $f=~s/\%\.([Ss])/%$1.$s/g; 79 | print strftime($f, localtime($seconds)); 80 | } 81 | else { 82 | print strftime($format, localtime); 83 | } 84 | print " ".$_; 85 | } 86 | else { 87 | s{\b( 88 | \d\d[-\s\/]\w\w\w # 21 dec 17:05 89 | (?:\/\d\d+)? # 21 dec/93 17:05 90 | [\s:]\d\d:\d\d # (time part of above) 91 | (?::\d\d)? # (optional seconds) 92 | (?:\s+[+-]\d\d\d\d)? # (optional timezone) 93 | | 94 | \w{3}\s+\d\d\s+\d\d:\d\d:\d\d # syslog form 95 | | 96 | \d\d\d[-:]\d\d[-:]\d\dT\d\d:\d\d:\d\d.\d+ # ISO-8601 97 | | 98 | (?:\w\w\w,?\s+)? # (optional Day) 99 | \d+\s+\w\w\w\s+\d\d+\s+\d\d:\d\d:\d\d 100 | # 16 Jun 94 07:29:35 101 | (?:\s+\w\w\w|\s[+-]\d\d\d\d)? 102 | # (optional timezone) 103 | | 104 | \w\w\w\s+\w\w\w\s+\d\d\s+\d\d:\d\d 105 | # lastlog format 106 | )\b 107 | }{ 108 | $use_format 109 | ? strftime($format, localtime(str2time($1))) 110 | : concise(ago(time - str2time($1), 2)) 111 | }exg; 112 | 113 | print $_; 114 | } 115 | } 116 | -------------------------------------------------------------------------------- /vidir: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | =head1 NAME 4 | 5 | vidir - edit directory 6 | 7 | =head1 SYNOPSIS 8 | 9 | B [--verbose] [directory|file|-] ... 10 | 11 | =head1 DESCRIPTION 12 | 13 | vidir allows editing of the contents of a directory in a text editor. If no 14 | directory is specified, the current directory is edited. 15 | 16 | When editing a directory, each item in the directory will appear on its own 17 | numbered line. These numbers are how vidir keeps track of what items are 18 | changed. Delete lines to remove files from the directory, or 19 | edit filenames to rename files. You can also switch pairs of numbers to 20 | swap filenames. 21 | 22 | Note that if "-" is specified as the directory to edit, it reads a list of 23 | filenames from stdin and displays those for editing. Alternatively, a list 24 | of files can be specified on the command line. 25 | 26 | =head1 OPTIONS 27 | 28 | =over 4 29 | 30 | =item -v, --verbose 31 | 32 | Verbosely display the actions taken by the program. 33 | 34 | =back 35 | 36 | =head1 EXAMPLES 37 | 38 | =over 4 39 | 40 | =item vidir 41 | 42 | =item vidir *.jpeg 43 | 44 | Typical uses. 45 | 46 | =item find | vidir - 47 | 48 | Edit subdirectory contents too. To delete subdirectories, 49 | delete all their contents and the subdirectory itself in the editor. 50 | 51 | =item find -type f | vidir - 52 | 53 | Edit all files under the current directory and subdirectories. 54 | 55 | =back 56 | 57 | =head1 ENVIRONMENT VARIABLES 58 | 59 | =over 4 60 | 61 | =item EDITOR 62 | 63 | Editor to use. 64 | 65 | =item VISUAL 66 | 67 | Also supported to determine what editor to use. 68 | 69 | =back 70 | 71 | =head1 AUTHOR 72 | 73 | Copyright 2006 by Joey Hess 74 | 75 | Licensed under the GNU GPL. 76 | 77 | =cut 78 | 79 | use File::Spec; 80 | use File::Temp; 81 | use Getopt::Long; 82 | 83 | my $error=0; 84 | 85 | my $verbose=0; 86 | if (! GetOptions("verbose|v" => \$verbose)) { 87 | die "Usage: $0 [--verbose] [directory|file|-]\n"; 88 | } 89 | 90 | my @dir; 91 | if (! @ARGV) { 92 | push @ARGV, "." 93 | } 94 | foreach my $item (@ARGV) { 95 | if ($item eq "-") { 96 | push @dir, map { chomp; $_ } ; 97 | close STDIN; 98 | open(STDIN, "/dev/tty") || die "reopen: $!\n"; 99 | } 100 | elsif (-d $item) { 101 | $item =~ s{/?$}{/}; 102 | opendir(DIR, $item) || die "$0: cannot read $item: $!\n"; 103 | push @dir, map { "$item$_" } sort readdir(DIR); 104 | closedir DIR; 105 | } 106 | else { 107 | push @dir, $item; 108 | } 109 | } 110 | 111 | if (grep(/[[:cntrl:]]/, @dir)) { 112 | die "$0: control characters in filenames are not supported\n"; 113 | } 114 | 115 | my $tmp=File::Temp->new(TEMPLATE => "dirXXXXX", DIR => File::Spec->tmpdir); 116 | open (OUT, ">".$tmp->filename) || die "$0: cannot create ".$tmp->filename.": $!\n"; 117 | 118 | my %item; 119 | my $c=0; 120 | foreach (@dir) { 121 | next if /^(.*\/)?\.$/ || /^(.*\/)?\.\.$/; 122 | $item{++$c}=$_; 123 | print OUT "$c\t$_\n"; 124 | } 125 | @dir=(); 126 | close OUT || die "$0: cannot write ".$tmp->filename.": $!\n"; 127 | 128 | my @editor="vi"; 129 | if (-x "/usr/bin/editor") { 130 | @editor="/usr/bin/editor"; 131 | } 132 | if (exists $ENV{EDITOR}) { 133 | @editor=split(' ', $ENV{EDITOR}); 134 | } 135 | if (exists $ENV{VISUAL}) { 136 | @editor=split(' ', $ENV{VISUAL}); 137 | } 138 | $ret=system(@editor, $tmp); 139 | if ($ret != 0) { 140 | die "@editor exited nonzero, aborting\n"; 141 | } 142 | 143 | open (IN, $tmp->filename) || die "$0: cannot read ".$tmp->filename.": $!\n"; 144 | while () { 145 | chomp; 146 | if (/^(\d+)\t{0,1}(.*)/) { 147 | my $num=int($1); 148 | my $name=$2; 149 | if (! exists $item{$num}) { 150 | die "$0: unknown item number $num\n"; 151 | } 152 | elsif ($name ne $item{$num}) { 153 | next unless length $name; 154 | my $src=$item{$num}; 155 | 156 | if (! (-e $src || -l $src) ) { 157 | print STDERR "$0: $src does not exist\n"; 158 | delete $item{$num}; 159 | next; 160 | } 161 | 162 | # deal with swaps 163 | if (-e $name || -l $name) { 164 | my $tmp=$name."~"; 165 | my $c=0; 166 | while (-e $tmp || -l $tmp) { 167 | $c++; 168 | $tmp=$name."~$c"; 169 | } 170 | if (! rename($name, $tmp)) { 171 | print STDERR "$0: failed to rename $name to $tmp: $!\n"; 172 | $error=1; 173 | } 174 | elsif ($verbose) { 175 | print "'$name' -> '$tmp'\n"; 176 | } 177 | foreach my $item (keys %item) { 178 | if ($item{$item} eq $name) { 179 | $item{$item}=$tmp; 180 | } 181 | } 182 | } 183 | 184 | if (! rename($src, $name)) { 185 | print STDERR "$0: failed to rename $src to $name: $!\n"; 186 | $error=1; 187 | } 188 | else { 189 | if (-d $name) { 190 | foreach (values %item) { 191 | s/^\Q$src\E/$name/; 192 | } 193 | } 194 | if ($verbose) { 195 | print "'$src' => '$name'\n"; 196 | } 197 | } 198 | } 199 | delete $item{$num}; 200 | } 201 | elsif (/^\s*$/) { 202 | # skip empty line 203 | } 204 | else { 205 | die "$0: unable to parse line \"$_\", aborting\n"; 206 | } 207 | } 208 | close IN || die "$0: cannot read ".$tmp->filename.": $!\n"; 209 | unlink($tmp.'~') if -e $tmp.'~'; 210 | 211 | sub rm { 212 | my $file = shift; 213 | 214 | if (-d $file && ! -l $file) { 215 | return rmdir $file; 216 | } 217 | else { 218 | return unlink $file; 219 | } 220 | } 221 | 222 | foreach my $item (reverse sort values %item) { 223 | if (! rm($item)) { 224 | print STDERR "$0: failed to remove $item: $!\n"; 225 | $error=1; 226 | } 227 | if ($verbose) { 228 | print "removed '$item'\n"; 229 | } 230 | } 231 | 232 | exit $error; 233 | -------------------------------------------------------------------------------- /vipe: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | =head1 NAME 4 | 5 | vipe - edit pipe 6 | 7 | =head1 SYNOPSIS 8 | 9 | command1 | B [...] | command2 10 | 11 | =head1 DESCRIPTION 12 | 13 | vipe allows you to run your editor in the middle of a unix pipeline and 14 | edit the data that is being piped between programs. Your editor will 15 | have the full data being piped from command1 loaded into it, and when you 16 | save, that data will be piped into command2. 17 | 18 | Any extra arguments are passed as arguments to the editor. 19 | 20 | =head1 ENVIRONMENT VARIABLES 21 | 22 | =over 4 23 | 24 | =item EDITOR 25 | 26 | Editor to use. 27 | 28 | =item VISUAL 29 | 30 | Also supported to determine what editor to use. 31 | 32 | =back 33 | 34 | =head1 AUTHOR 35 | 36 | Copyright 2006 by Joey Hess 37 | 38 | Licensed under the GNU GPL. 39 | 40 | =cut 41 | 42 | use warnings; 43 | use strict; 44 | use File::Temp q{tempfile}; 45 | 46 | $/=undef; 47 | 48 | my ($fh, $tmp)=tempfile(); 49 | die "cannot create tempfile" unless $fh; 50 | print ($fh ) || die "write temp: $!"; 51 | close $fh; 52 | close STDIN; 53 | open(STDIN, "&STDOUT") || die "save stdout: $!"; 55 | close STDOUT; 56 | open(STDOUT, ">/dev/tty") || die "reopen stdout: $!"; 57 | 58 | my @editor="vi"; 59 | if (-x "/usr/bin/editor") { 60 | @editor="/usr/bin/editor"; 61 | } 62 | if (exists $ENV{EDITOR}) { 63 | @editor=split(' ', $ENV{EDITOR}); 64 | } 65 | if (exists $ENV{VISUAL}) { 66 | @editor=split(' ', $ENV{VISUAL}); 67 | } 68 | my $ret=system(@editor, @ARGV, $tmp); 69 | if ($ret != 0) { 70 | die "@editor exited nonzero, aborting\n"; 71 | } 72 | 73 | open (IN, $tmp) || die "$0: cannot read $tmp: $!\n"; 74 | print (OUT ) || die "write failure: $!"; 75 | close IN; 76 | unlink($tmp); 77 | -------------------------------------------------------------------------------- /zrun: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl 2 | 3 | =head1 NAME 4 | 5 | zrun - automatically uncompress arguments to command 6 | 7 | =head1 SYNOPSIS 8 | 9 | zrun command file.gz [...] 10 | 11 | =head1 DESCRIPTION 12 | 13 | Prefixing a shell command with "zrun" causes any compressed files that are 14 | arguments of the command to be transparently uncompressed to temp files 15 | (not pipes) and the uncompressed files fed to the command. 16 | 17 | This is a quick way to run a command that does not itself support 18 | compressed files, without manually uncompressing the files. 19 | 20 | The following compression types are supported: gz bz2 Z xz lzma lzo 21 | 22 | If zrun is linked to some name beginning with z, like zprog, and the link is 23 | executed, this is equivalent to executing "zrun prog". 24 | 25 | =head1 BUGS 26 | 27 | Modifications to the uncompressed temporary file are not fed back into the 28 | input file, so using this as a quick way to make an editor support 29 | compressed files won't work. 30 | 31 | =head1 AUTHOR 32 | 33 | Copyright 2006 by Chung-chieh Shan 34 | 35 | =cut 36 | 37 | use warnings; 38 | use strict; 39 | use IO::Handle; 40 | use File::Spec; 41 | use File::Temp qw{tempfile}; 42 | 43 | my $program; 44 | 45 | if ($0 =~ m{(?:^|/)z([^/]+)$} && $1 ne 'run') { 46 | $program = $1; 47 | if (! @ARGV) { 48 | die "Usage: z$1 \nEquivalent to: zrun $1 \n"; 49 | } 50 | } 51 | else { 52 | $program = shift; 53 | if (! @ARGV) { 54 | die "Usage: zrun \n"; 55 | } 56 | } 57 | 58 | my @argument; 59 | my %child; 60 | foreach my $argument (@ARGV) { 61 | if ($argument =~ m{^(.*/)?([^/]*)\.(gz|Z|bz2|xz|lzo|lzma)$}s) { 62 | my $suffix = "-$2"; 63 | my @preprocess = $3 eq "bz2" ? qw(bzip2 -d -c) : 64 | $3 eq "xz" ? qw(xz -d -c) : 65 | $3 eq "lzo" ? qw(lzop -d -c) : 66 | $3 eq "lzma" ? qw(lzma -d -c) : qw(gzip -d -c); 67 | 68 | my ($fh, $tmpname) = tempfile(SUFFIX => $suffix, 69 | DIR => File::Spec->tmpdir, UNLINK => 1) 70 | or die "zrun: cannot create temporary file: $!\n"; 71 | 72 | if (my $child = fork) { 73 | $child{$child} = $argument; 74 | $argument = $tmpname; 75 | } 76 | elsif (defined $child) { 77 | STDOUT->fdopen($fh, "w"); 78 | exec @preprocess, $argument; 79 | } 80 | else { 81 | die "zrun: cannot fork to handle $argument: $!\n"; 82 | } 83 | } 84 | push @argument, $argument; 85 | } 86 | 87 | while (%child and (my $pid = wait) != -1) { 88 | if (defined(my $argument = delete $child{$pid})) { 89 | if ($? & 255) { 90 | die "zrun: preprocessing for $argument terminated abnormally: $?\n"; 91 | } 92 | elsif (my $code = $? >> 8) { 93 | die "zrun: preprocessing for $argument terminated with code $code\n"; 94 | } 95 | } 96 | } 97 | 98 | my $status = system $program ($program, @argument); 99 | if ($status & 255) { 100 | die "zrun: $program terminated abnormally: $?\n"; 101 | } 102 | else { 103 | my $code = $? >> 8; 104 | exit $code; 105 | } 106 | --------------------------------------------------------------------------------