├── .gitignore
├── GOALS.md
├── CONTRIBUTING.md
├── doc.go
├── README.md
├── psync.go
└── LICENSE.txt


/.gitignore:
--------------------------------------------------------------------------------
1 | psync
2 | *~
3 | 
4 | 


--------------------------------------------------------------------------------
/GOALS.md:
--------------------------------------------------------------------------------
 1 | psync Development goals
 2 | =======================
 3 | 
 4 | Here is how the psync usage might eventally look like.
 5 | 
 6 | None of these options are fixed yet. Plans for new functions might be dropped,
 7 | build in implicitely (without an option), or get another option name. More
 8 | functions and options might appear.
 9 | 
10 | 	psync [-verbose|-quiet] [-dryrun] [-threads <num>] [-owner] [-times] [-create]
11 | 	      [-sync] [-delete] [-update] source destination
12 | 
13 | 	-verbose        - verbose mode, prints the current workload to STDOUT
14 | 	-quiet          - quiet mode, suppress warnings
15 |     -dryrun         - dry run, do not actually copy files or directories
16 | 	-threads <num>  - number of concurrent threads, 1 <= <num> <= 1024, default 16
17 | 	-owner          - preserve ownership (user / group)
18 | 	-times          - preserve timestamps (atime / mtime)
19 | 	-create         - create destination directory, if needed (with standard permissions)
20 | 	-sync           - copy in "sync" mode, e.g. copy only files and directories that
21 | 	                  do not yet exist on the destination side, or exist but differ in
22 | 	                  size or time stamp (mtime).
23 | 	-delete         - in sync mode, delete files and directories that exist on the
24 | 	                  destination side, but not the source side.
25 | 	-update         - in sync mode, do not copy a file when there is a file with the
26 | 	                  same name and newer timestamp (mtime) on the destination side.
27 | 	                  This option has no effect on directories.
28 | 	source          - source directory
29 | 	destination     - destination directory
30 | 


--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
  1 | Contributing to psync
  2 | =====================
  3 | 
  4 | Filing issues
  5 | -------------
  6 | 
  7 | To file a bug or discuss new features, an issue can be created. If you are unsure
  8 | if the behavior you found is a bug, or if the issue is security-related, please
  9 | contact me by e-mail first.
 10 | 
 11 | Please open a new issue for each new problem or proposal. Don't collect multiple
 12 | unrelated topics in one issue.
 13 | 
 14 | When filing a bug, please make sure you include all the information to
 15 | reproduce it:
 16 | 
 17 | 1. What version of psync are you using (version number or commit ID)?
 18 | 2. What version of Go did you use to compile psync (`go version`)?
 19 | 3. What operating system and processor archtecture are you using?
 20 | 4. What filesystems and mount/export options are you using, separate
 21 |    for the source and destination file system?
 22 | 5. What did you do?
 23 | 6. What did you expect?
 24 | 7. What happened?
 25 | 
 26 | Contributing code
 27 | -----------------
 28 | 
 29 | If you believe you can fix a bug or write code for a new feature, feel free to
 30 | contribute. However, before you submit source code or a pull request, please
 31 | consider the following guidelines.
 32 | 
 33 | ### Discuss before submission
 34 | 
 35 | Do not submit code unless the change has been discussed, either publicly in
 36 | the issues, or with me personally. There must be consent on what should be
 37 | done, and at least roughly, how it will be done.
 38 | 
 39 | As an exception, if the code fixes an issue already listed as a TODO in the
 40 | README.md or a `TODO` mark in the source code, it is likely to be accepted
 41 | (if the other guidelines are met).
 42 | 
 43 | ### Fix issues one by one
 44 | 
 45 | Each pull request should fix exactly one issue. If a submission contains fixes
 46 | to several unrelated issues, it will be rejected if one of them is to be
 47 | rejected, even if the others are fine.
 48 | 
 49 | Note that Github always cares about the Git commit history to
 50 | [tell a story](https://github.com/git-guides/git-commit). It looks odd when
 51 | commits are accepted and parts of them get later reverted.
 52 | 
 53 | ### Respect the direction of the project
 54 | 
 55 | Although psync is currently more of a replacement to "cp -r", the goal of the
 56 | project is to be a (very much stripped down) parallel variant of rsync. Patches
 57 | that leave this road are less likely (although not impossible) to be accepted.
 58 | 
 59 | See [GOALS.md](GOALS.md) for a rough plan of the project goals.
 60 | 
 61 | ### Avoid incompatible changes
 62 | 
 63 | Although a 0.x version does not have to be strictly backwards compatible, it
 64 | is annoying for users when new versions of a tool change the behaviour,
 65 | especially when they have to change their scripts. Try to avoid incompatible
 66 | changes, unless the result of the discussion is that they are necessary.
 67 | 
 68 | As a conseqence, don't change the naming or meaning of existing command line
 69 | options. Introduce new options only for features that are expected to stay.
 70 | 
 71 | ### Don't inject new dependencies
 72 | 
 73 | The source code does currently only import packages from the Go standard
 74 | library. Changes that inject dependencies to third party packages must be
 75 | discussed first; I will accept them only if they are really advantageous to
 76 | the project and there is no proper way to implement them without these
 77 | dependencies. The discussion must include a licence compatibility check.
 78 | 
 79 | Patches that require CGO (e.g. calls to C code) or contain assembler code
 80 | will be rejected.
 81 | 
 82 | If you change requires the `unsafe` package, prior discussion is required.
 83 | If unsafe code can't be avoided, it must be put in a very short function
 84 | within an own source code file. The `unsafe` package may only be imported
 85 | there.
 86 | 
 87 | ### Don't lock out older systems
 88 | 
 89 | Please do not submit code that relies on features from very late Go releases,
 90 | unless there are compelling reasons to do so. psync is used for enterprise
 91 | storage migrations. In such environments, stable and long term release
 92 | operating systems can be found. As a rule of thumb, the code should compile
 93 | even with the Go version shipped with Debian stable and the backports to
 94 | Debian oldstable. (At the time of writing, this is Go 1.11).
 95 | 
 96 | This does also mean that the psync projected is not being migrated to
 97 | [Go modules](https://blog.golang.org/using-go-modules) support yet.
 98 | 
 99 | ### Keep the documentation in sync
100 | 
101 | If you code changes require updates to the documentation, keep in mind that
102 | the `README.md` and `go.doc` files must be kept in sync. If the code also
103 | changes the command line interface, check if the `usage()` function in
104 | `psync.go` has to be adapted.
105 | 
106 | 


--------------------------------------------------------------------------------
/doc.go:
--------------------------------------------------------------------------------
  1 | // Copyright 2018-2020 by Harald Weidner <hweidner@gmx.net>. All rights reserved.
  2 | // Use of this source code is governed by the GNU General Public License
  3 | // Version 3 that can be found in the LICENSE.txt file.
  4 | 
  5 | /*
  6 | Parallel Sync - parallel recursive copying of directories
  7 | 
  8 | psync is a tool which copies a directory recursively to another directory.
  9 | Unlike "cp -r", which walks through the files and subdirectories in sequential
 10 | order, psync copies several files concurrently by using threads.
 11 | 
 12 | psync was written to speed up copying of large trees to or from a network
 13 | file system like GlusterFS, Ceph, WebDAV or NFS. Operations on such file
 14 | systems are usually latency bound, especially with small files.
 15 | Parallel execution can help to utilize the bandwidth better and avoid that
 16 | the latencies sum up, as this is the case in sequential operations.
 17 | 
 18 | Currently, psync does only copy directory trees, similar to "cp -r". A "sync"
 19 | mode, similar to "rsync -rl" is planned. See GOALS.md on how psync finally may
 20 | look like. A first version of the sync mode can be found in the branch
 21 | `syncmode`.
 22 | 
 23 | Usage
 24 | 
 25 | psync is invoked as follows:
 26 | 
 27 | 	psync [-verbose|-quiet] [-threads <num>] [-owner] [-times] [-create] source destination
 28 | 
 29 | 	-verbose        - verbose mode, prints the current workload to STDOUT
 30 | 	-quiet          - quiet mode, suppress warnings
 31 | 	-threads <num>  - number of concurrent threads, 1 <= <num> <= 1024, default 16
 32 | 	-owner          - preserve ownership (user / group)
 33 | 	-times          - preserve timestamps (atime / mtime)
 34 | 	-create         - create destination directory, if needed (with standard permissions)
 35 | 	source          - source directory
 36 | 	destination     - destination directory
 37 | 
 38 | Example
 39 | 
 40 | Copy all files and subdirectories from /data/src into /data/dest.
 41 | 
 42 | 	psync -threads 8 /data/src /data/dest
 43 | 
 44 | /data/src and /data/dest must exist and must be directories.
 45 | 
 46 | Why should I use it
 47 | 
 48 | A recursive copy of a directory can be a throughput bound or latency bound
 49 | operation, depending on the size of files and characteristics of the source
 50 | and/or destination file system. When copying between standard file systems on
 51 | two local hard disks, the operation is typically throughput bound, and copying
 52 | in parallel has no performance advantage over copying sequentially. In this
 53 | case, you have a bunch of options, including "cp -r" or "rsync".
 54 | 
 55 | However, when copying from or to network file systems (NFS, CIFS), WAN storage
 56 | (WebDAV, external cloud services), distributed file systems (GlusterFS, CephFS)
 57 | or file systems that live on a DRBD device, the latency for each file access is
 58 | often limiting performance factor. With sequential copies, the operation can
 59 | consume lots of time, although the bandwidth is not saturated. In this case, it
 60 | can make up a significant performance boost if the files are copied in parallel.
 61 | 
 62 | Where psync should not be used
 63 | 
 64 | Parallel copying is typically not so useful when copying between local or
 65 | very fast hard disks. psync can be used there, and with a moderate concurrency
 66 | level like 2..5, it can be slightly faster than a sequential copy.
 67 | 
 68 | Parallel copying should never be used when duplicating directories on the same
 69 | physical hard disk. Even sequential copies suffer from the frequent hard disk head
 70 | movements which are needed to read and write concurrently on/to the same disk.
 71 | Parallel copying even increases the amount of head movements.
 72 | 
 73 | How it works
 74 | 
 75 | psync uses goroutines for copying files in parallel. By default, 16 copy workers
 76 | are spawned as goroutines, but the number can be adjusted with the -threads switch.
 77 | 
 78 | Each worker waits for a directory to be submitted. It then handles all the
 79 | directory entries sequentially. Files are copied one by one to the destination
 80 | directory. When subdirectories are discovered, they are created on the destination
 81 | side. Traversal of the subdirecory is then submitted to other workers and thus done
 82 | in parallel to the current workload.
 83 | 
 84 | Performance values
 85 | 
 86 | Here are some performance values comparing psync to cp and rsync when copying
 87 | a large directory structure with many small files from a local file system to
 88 | an NFS share.
 89 | 
 90 | The NFS server has an AMD E-350 CPU, 8 GB of RAM, a 2TB hard drive (WD Green
 91 | series) running Debian GNU/Linux 10 (Linux kernel 4.19). The NFS export is
 92 | a logical volume on the HDD with ext4 file system. The NFS export options are:
 93 | rw,no_root_squash,async,no_subtree_check.
 94 | 
 95 | The client is a workstation with AMD Ryzen 7 1700 CPU, 64 GB of RAM, running
 96 | Ubuntu 18.04 LTS with HWE stack (Linux kernel 5.3). The data to copy is located
 97 | on a 1TB SATA SSD with XFS, and buffered in memory. The NFS mount options are:
 98 | fstype=nfs,vers=3,soft,intr,async.
 99 | 
100 | The hosts are connected over ethernet with 1 Gbit/s, ping latency is 160µs.
101 | 
102 | The data is an extracted linux kernel source code 4.15.2 tarball, containing
103 | 62273 files and 32 symbolic links in 4377 directories, summing up to 892 MB
104 | (as seen by "du -s"). It is copied from the workstation to the server over NFS.
105 | 
106 | The options for the three commands are selected comparably. They copy the files
107 | and links recursively and preserve permissions, but no ownership or time stamps.
108 | 
109 |     Command                Estimated time  Throughput
110 |     =================================================
111 |     cp -r SRC DEST              1m50,288s   8,09 MB/s
112 |     rsync -rl SRC/ DEST/        3m05,479s   4,81 MB/s
113 |     psync SRC DEST              0m23,398s  38,12 MB/s
114 | 
115 | Limits and TODOs
116 | 
117 | psync currently can only handle directories, regular files, and symbolic links.
118 | Other filesystem entries like devices, sockets or named pipes are silently ignored.
119 | A warning is printed when trying to copy such special files.
120 | 
121 | psync preserves the Unix permissions (rwx) of the copied files and directories,
122 | but has currently no support for preserving other permission bits (suid, sticky).
123 | 
124 | When using the according options, psync tries to preserve the ownership
125 | (user/group) and/or the access and modification time stamps. Preserve ownership
126 | does only work when psync is running under the root user account. Preserving the
127 | time stamps does only work for regular files and directories, not for symbolic
128 | links.
129 | 
130 | psync does currently implement a simple recursive copy, like "cp -r", and not
131 | a versatile sync algorithm like rsync. There is no check wether a file already
132 | exists in the destination, nor its content and timestamps. Existing files on the
133 | destination side are not deleted when they don't exist on the source side.
134 | 
135 | psync is being developed under Linux (Debian, Ubuntu, CentOS). It should work on
136 | other distributions, but this has not been tested. It does currently not compile
137 | for Windows, Darwin (MacOS), NetBSD and FreeBSD (but this should easily be
138 | fixed).
139 | 
140 | License
141 | 
142 | psync is released under the terms of the GNU General Public License, version 3.
143 | 
144 | */
145 | package main
146 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | [![License: GPLv3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
  2 | [![GoDocs](https://godocs.io/github.com/hweidner/psync?status.svg)](https://godocs.io/github.com/hweidner/psync)
  3 | [![Go Reference](https://pkg.go.dev/badge/github.com/hweidner/psync.svg)](https://pkg.go.dev/github.com/hweidner/psync)
  4 | [![Go Report Card](https://goreportcard.com/badge/github.com/hweidner/psync)](https://goreportcard.com/report/github.com/hweidner/psync)
  5 | [![Total alerts](https://img.shields.io/lgtm/alerts/g/hweidner/psync.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/hweidner/psync/alerts/)
  6 | 
  7 | Parallel Sync - parallel recursive copying of directories
  8 | =========================================================
  9 | 
 10 | psync is a tool which copies a directory recursively to another directory.
 11 | Unlike "cp -r", which walks through the files and subdirectories in sequential
 12 | order, psync copies several files concurrently by using threads.
 13 | 
 14 | psync was written to speed up copying of large trees to or from a network
 15 | file system like GlusterFS, Ceph, WebDAV or NFS. Operations on such file
 16 | systems are usually latency bound, especially with small files.
 17 | Parallel execution can help to utilize the bandwidth better and avoid that
 18 | the latencies sum up, as this is the case in sequential operations.
 19 | 
 20 | Currently, psync does only copy directory trees, similar to "cp -r". A "sync"
 21 | mode, similar to "rsync -rl" is planned. See [GOALS.md](GOALS.md) on how psync
 22 | finally may look like. A first version of the sync mode can be found in the
 23 | branch `syncmode`.
 24 | 
 25 | Installation
 26 | ------------
 27 | 
 28 | You need a Go compiler to install psync from source. Follow the instructions on
 29 | the [Go Installation page](https://golang.org/doc/install), or install from your
 30 | operating system distribution (e.g. ``apt install golang`` on Debian/Ubuntu Linux).
 31 | 
 32 | Then install psync with
 33 | 
 34 | 	go get github.com/hweidner/psync
 35 | 
 36 | This command will fetch psync, compile and install it in the directory
 37 | $HOME/go/bin, or $GOPATH/bin if GOPATH is set.
 38 | 
 39 | When working with Go 1.16 or later, you might need to explicitely turn modules
 40 | off by setting the environment variable GO111MODULE=off.
 41 | 
 42 | Usage
 43 | -----
 44 | 
 45 | psync is invoked as follows:
 46 | 
 47 | 	psync [-verbose|-quiet] [-threads <num>] [-owner] [-times] [-create] source destination
 48 | 
 49 | 	-verbose        - verbose mode, prints the current workload to STDOUT
 50 | 	-quiet          - quiet mode, suppress warnings
 51 | 	-threads <num>  - number of concurrent threads, 1 <= <num> <= 1024, default 16
 52 | 	-owner          - preserve ownership (user / group)
 53 | 	-times          - preserve timestamps (atime / mtime)
 54 | 	-create         - create destination directory, if needed (with standard permissions)
 55 | 	source          - source directory
 56 | 	destination     - destination directory
 57 | 
 58 | Example
 59 | -------
 60 | 
 61 | Copy all files and subdirectories from /data/src into /data/dest.
 62 | 
 63 | 	psync -threads 8 /data/src /data/dest
 64 | 
 65 | `/data/src` and `/data/dest` must exist and must be directories.
 66 | 
 67 | Why should I use it
 68 | -------------------
 69 | 
 70 | A recursive copy of a directory can be a throughput bound or latency bound
 71 | operation, depending on the size of files and characteristics of the source
 72 | and/or destination file system. When copying between standard file systems on
 73 | two local hard disks, the operation is typically throughput bound, and copying
 74 | in parallel has no performance advantage over copying sequentially. In this
 75 | case, you have a bunch of options, including "cp -r" or "rsync".
 76 | 
 77 | However, when copying from or to network file systems (NFS, CIFS), WAN storage
 78 | (WebDAV, external cloud services), distributed file systems (GlusterFS, CephFS)
 79 | or file systems that live on a DRBD device, the latency for each file access is
 80 | often limiting performance factor. With sequential copies, the operation can
 81 | consume lots of time, although the bandwidth is not saturated. In this case, it
 82 | can make up a significant performance boost if the files are copied in parallel.
 83 | 
 84 | Where psync should not be used
 85 | ------------------------------
 86 | 
 87 | Parallel copying is typically not so useful when copying between local or
 88 | very fast hard disks. psync can be used there, and with a moderate concurrency
 89 | level like 2..5, it can be slightly faster than a sequential copy.
 90 | 
 91 | Parallel copying should never be used when duplicating directories on the same
 92 | physical hard disk. Even sequential copies suffer from the frequent hard disk head
 93 | movements which are needed to read and write concurrently on/to the same disk.
 94 | Parallel copying even increases the amount of head movements.
 95 | 
 96 | Never use psync when writing to a FAT/VFAT/exFAT file system! Those file systems
 97 | are best written sequentially. Parallel write access will be slower, and leads
 98 | to inefficient data structures and fragmentation. Reading from those file systems
 99 | with psync should be efficient.
100 | 
101 | How it works
102 | ------------
103 | 
104 | psync uses goroutines for copying files in parallel. By default, 16 copy workers
105 | are spawned as goroutines, but the number can be adjusted with the -threads switch.
106 | 
107 | Each worker waits for a directory to be submitted. It then handles all the
108 | directory entries sequentially. Files are copied one by one to the destination
109 | directory. When subdirectories are discovered, they are created on the destination
110 | side. Traversal of the subdirecory is then submitted to other workers and thus done
111 | in parallel to the current workload.
112 | 
113 | Performance values
114 | ------------------
115 | 
116 | Here are some performance values comparing psync to cp and rsync when copying
117 | a large directory structure with many small files from a local file system to
118 | an NFS share.
119 | 
120 | The NFS server has an AMD E-350 CPU, 8 GB of RAM, a 2TB hard drive (WD Green
121 | series) running Debian GNU/Linux 10 (Linux kernel 4.19). The NFS export is
122 | a logical volume on the HDD with ext4 file system. The NFS export options are:
123 | rw,no_root_squash,async,no_subtree_check.
124 | 
125 | The client is a workstation with AMD Ryzen 7 1700 CPU, 64 GB of RAM, running
126 | Ubuntu 18.04 LTS with HWE stack (Linux kernel 5.3). The data to copy is located
127 | on a 1TB SATA SSD with XFS, and buffered in memory. The NFS mount options are:
128 | fstype=nfs,vers=3,soft,intr,async.
129 | 
130 | The hosts are connected over ethernet with 1 Gbit/s, ping latency is 160µs.
131 | 
132 | The data is an extracted linux kernel source code 4.15.2 tarball, containing
133 | 62273 files and 32 symbolic links in 4377 directories, summing up to 892 MB
134 | (as seen by "du -s"). It is copied from the workstation to the server over NFS.
135 | 
136 | The options for the three commands are selected comparably. They copy the files
137 | and links recursively and preserve permissions, but no ownership or time stamps.
138 | 
139 | 	Command                Estimated time  Throughput
140 | 	=================================================
141 |     cp -r SRC DEST              1m50,288s   8,09 MB/s
142 |     rsync -rl SRC/ DEST/        3m05,479s   4,81 MB/s
143 |     psync SRC DEST              0m23,398s  38,12 MB/s
144 | 
145 | Limits and TODOs
146 | ----------------
147 | 
148 | psync currently can only handle directories, regular files, and symbolic links.
149 | Other filesystem entries like devices, sockets or named pipes are silently ignored.
150 | A warning is printed when trying to copy such special files.
151 | 
152 | psync preserves the Unix permissions (rwx) of the copied files and directories,
153 | but has currently no support for preserving other permission bits (suid, sticky).
154 | 
155 | When using the according options, psync tries to preserve the ownership
156 | (user/group) and/or the access and modification time stamps. Preserve ownership
157 | does only work when psync is running under the root user account. Preserving the
158 | time stamps does only work for regular files and directories, not for symbolic
159 | links.
160 | 
161 | psync does currently implement a simple recursive copy, like "cp -r", and not
162 | a versatile sync algorithm like rsync. There is no check wether a file already
163 | exists in the destination, nor its content and timestamps. Existing files on the
164 | destination side are not deleted when they don't exist on the source side.
165 | 
166 | psync is being developed under Linux (Debian, Ubuntu, CentOS). It should work on
167 | other distributions, but this has not been tested. It does currently not compile
168 | for Windows, Darwin (MacOS), NetBSD and FreeBSD (but this should easily be
169 | fixed).
170 | 
171 | Contributing
172 | ------------
173 | 
174 | Please see [CONTRIBUTING.md](CONTRIBUTING.md) on how to contribute to the
175 | development of psync.
176 | 
177 | License
178 | -------
179 | 
180 | psync is released under the terms of the GNU General Public License, version 3.
181 | 


--------------------------------------------------------------------------------
/psync.go:
--------------------------------------------------------------------------------
  1 | // Copyright 2018 by Harald Weidner <hweidner@gmx.net>. All rights reserved.
  2 | // Use of this source code is governed by the GNU General Public License
  3 | // Version 3 that can be found in the LICENSE.txt file.
  4 | 
  5 | package main
  6 | 
  7 | import (
  8 | 	"flag"
  9 | 	"fmt"
 10 | 	"io"
 11 | 	"io/ioutil"
 12 | 	"os"
 13 | 	"sync"
 14 | 	"syscall"
 15 | 	"time"
 16 | )
 17 | 
 18 | // BUFSIZE defines the size of the buffer used for copying. It is currently 64kB.
 19 | const BUFSIZE = 64 * 1024
 20 | 
 21 | // Buffer, Channels and Synchronization
 22 | var (
 23 | 	buffer [][BUFSIZE]byte
 24 | 	dch    = make(chan string, 100) // dispatcher channel - get work into work queue
 25 | 	wch    = make(chan string, 100) // worker channel - get work from work queue to copy thread
 26 | 	wg     sync.WaitGroup           // waitgroup for work queue length
 27 | )
 28 | 
 29 | // Commandline Flags
 30 | var (
 31 | 	threads        uint   // number of threads
 32 | 	src, dest      string // source and destination directory
 33 | 	verbose, quiet bool   // verbose and quiet flags
 34 | 	times, owner   bool   // preserve timestamps and owner flag
 35 | 	create         bool   // create destination directory flag
 36 | )
 37 | 
 38 | func main() {
 39 | 	// parse commandline flags
 40 | 	flags()
 41 | 
 42 | 	// check or create the destination directory
 43 | 	prepareDestDir()
 44 | 
 45 | 	// clear umask, so that it does not interfere with explicite permissions
 46 | 	// used in os.FileOpen()
 47 | 	syscall.Umask(0000)
 48 | 
 49 | 	// initialize buffers
 50 | 	buffer = make([][BUFSIZE]byte, threads)
 51 | 
 52 | 	// Start dispatcher and copy threads
 53 | 	go dispatcher()
 54 | 	for i := uint(0); i < threads; i++ {
 55 | 		go copyDir(i)
 56 | 	}
 57 | 
 58 | 	// start copying top level directory
 59 | 	wg.Add(1)
 60 | 	dch <- ""
 61 | 
 62 | 	// wait for work queue to get empty
 63 | 	wg.Wait()
 64 | }
 65 | 
 66 | // Function flags parses the command line flags and checks them for sanity.
 67 | func flags() {
 68 | 	flag.UintVar(&threads, "threads", 16, "Number of threads to run in parallel")
 69 | 	flag.BoolVar(&verbose, "verbose", false, "Verbose mode")
 70 | 	flag.BoolVar(&quiet, "quiet", false, "Quiet mode")
 71 | 	flag.BoolVar(&times, "times", false, "Preserve time stamps")
 72 | 	flag.BoolVar(&owner, "owner", false, "Preserve user/group ownership (root only)")
 73 | 	flag.BoolVar(&create, "create", false, "Create destination directory, if needed (with standard permissions)")
 74 | 	flag.Parse()
 75 | 
 76 | 	if flag.NArg() != 2 || flag.Arg(0) == "" || flag.Arg(1) == "" || threads > 1024 {
 77 | 		usage()
 78 | 	}
 79 | 
 80 | 	if threads == 0 {
 81 | 		threads = 16
 82 | 	}
 83 | 	src = flag.Arg(0)
 84 | 	dest = flag.Arg(1)
 85 | }
 86 | 
 87 | // Function usage prints a message about how to use psync, and exits.
 88 | func usage() {
 89 | 	fmt.Println("Usage: psync [options] source destination")
 90 | 	flag.Usage()
 91 | 	os.Exit(1)
 92 | }
 93 | 
 94 | // Function prepareDestDir checks for the existence of the destination,
 95 | // or creates it if the flag '-create' is set.
 96 | func prepareDestDir() {
 97 | 	if create {
 98 | 		// create destination directory
 99 | 		err := os.MkdirAll(dest, os.FileMode(0777))
100 | 		if err != nil {
101 | 			fmt.Fprintf(os.Stderr, "ERROR - unable to create destination dir %s: %s\n", dest, err)
102 | 			os.Exit(1)
103 | 		}
104 | 	} else {
105 | 		// test the existence of destination directory prior to syncing
106 | 		stat, err := os.Stat(dest)
107 | 		if os.IsNotExist(err) {
108 | 			fmt.Fprintf(os.Stderr, "ERROR - destination directory %s does not exist: %s.\nUse '-create' to create it.\n", dest, err)
109 | 			os.Exit(1)
110 | 		}
111 | 		if err != nil {
112 | 			fmt.Fprintf(os.Stderr, "ERROR - cannot stat() destination directory %s: %s.\n", dest, err)
113 | 			os.Exit(1)
114 | 		}
115 | 		if !stat.IsDir() {
116 | 			fmt.Fprintf(os.Stderr, "ERROR - destination %s exists, but is not a directory\n", dest)
117 | 			os.Exit(1)
118 | 		}
119 | 	}
120 | }
121 | 
122 | // Function dispatcher maintains a work list of potentially arbitrary size.
123 | // Incoming directories (over the dispather channel) will be forwarded to a
124 | // copy thread through the worker channel, or stored in the work list if no
125 | // copy thread is available. For easier memory handling, the work list is
126 | // treated last-in-first-out.
127 | func dispatcher() {
128 | 	worklist := make([]string, 0, 1000)
129 | 	var dir string
130 | 	for {
131 | 		if len(worklist) == 0 {
132 | 			dir = <-dch
133 | 			worklist = append(worklist, dir)
134 | 		} else {
135 | 			select {
136 | 			case dir = <-dch:
137 | 				worklist = append(worklist, dir)
138 | 			case wch <- worklist[len(worklist)-1]:
139 | 				worklist = worklist[:len(worklist)-1]
140 | 			}
141 | 		}
142 | 	}
143 | }
144 | 
145 | // Function copyDir receives a directory on the worker channel and copies its
146 | // content from src to dest. Files are copied sequentially. If a subdirectory
147 | // is discovered, it is created on the destination side, and then inserted into
148 | // the work queue through the dispatcher channel.
149 | func copyDir(id uint) {
150 | 	for {
151 | 		// read next directory to handle
152 | 		dir := <-wch
153 | 		if verbose {
154 | 			fmt.Printf("[%d] Handling directory %s%s\n", id, src, dir)
155 | 		}
156 | 
157 | 		// read directory content
158 | 		files, err := ioutil.ReadDir(src + dir)
159 | 		if err != nil {
160 | 			if !quiet {
161 | 				fmt.Fprintf(os.Stderr, "WARNING - could not read directory %s: %s\n", src+dir, err)
162 | 			}
163 | 			wg.Done()
164 | 			continue
165 | 		}
166 | 
167 | 		for _, f := range files {
168 | 			fname := f.Name()
169 | 			if fname == "." || fname == ".." {
170 | 				continue
171 | 			}
172 | 
173 | 			if f.IsDir() {
174 | 				// create directory on destination side
175 | 				perm := f.Mode().Perm()
176 | 				err := os.Mkdir(dest+dir+"/"+fname, perm)
177 | 				if err != nil {
178 | 					if !quiet {
179 | 						fmt.Fprintf(os.Stderr, "WARNING - could not create directory %s: %s\n",
180 | 							dest+dir+"/"+fname, err)
181 | 					}
182 | 					continue
183 | 				}
184 | 
185 | 				// submit directory to work queue
186 | 				wg.Add(1)
187 | 				dch <- dir + "/" + fname
188 | 			} else {
189 | 				// copy file sequentially
190 | 				if verbose {
191 | 					fmt.Printf("[%d] Copying %s%s/%s to %s%s/%s\n",
192 | 						id, src, dir, fname, dest, dir, fname)
193 | 				}
194 | 				copyFile(id, dir+"/"+fname, f)
195 | 			}
196 | 		}
197 | 		finfo, err := os.Stat(src + dir)
198 | 		if err != nil {
199 | 			if !quiet {
200 | 				fmt.Fprintf(os.Stderr, "WARNING - could not read fileinfo of directory %s: %s\n",
201 | 					dest+dir, err)
202 | 			}
203 | 		} else {
204 | 			// preserve user and group of the destination directory
205 | 			if owner {
206 | 				preserveOwner(dest+dir, finfo, "directory")
207 | 			}
208 | 			// setting the timestamps of the destination directory
209 | 			if times {
210 | 				preserveTimes(dest+dir, finfo, "directory")
211 | 			}
212 | 		}
213 | 		if verbose {
214 | 			fmt.Printf("[%d] Finished directory %s%s\n", id, src, dir)
215 | 		}
216 | 		wg.Done()
217 | 	}
218 | }
219 | 
220 | // Function copyFile copies a file from the source to the destination directory.
221 | func copyFile(id uint, file string, f os.FileInfo) {
222 | 	mode := f.Mode()
223 | 	switch {
224 | 
225 | 	case mode&os.ModeSymlink != 0: // symbolic link
226 | 		// read link
227 | 		link, err := os.Readlink(src + file)
228 | 		if err != nil {
229 | 			if !quiet {
230 | 				fmt.Fprintf(os.Stderr, "WARNING - link %s disappeared while copying %s\n", src+file, err)
231 | 			}
232 | 			return
233 | 		}
234 | 
235 | 		// write link to destination
236 | 		err = os.Symlink(link, dest+file)
237 | 		if err != nil {
238 | 			if !quiet {
239 | 				fmt.Fprintf(os.Stderr, "WARNING - link %s could not be created: %s\n", dest+file, err)
240 | 			}
241 | 			return
242 | 		}
243 | 
244 | 		// preserve owner of symbolic link
245 | 		if owner {
246 | 			preserveOwner(dest+file, f, "link")
247 | 		}
248 | 		// preserving the timestamps of links seems not be supported in Go
249 | 		// TODO: it should be possible by using the futimesat system call,
250 | 		// see https://github.com/golang/go/issues/3951
251 | 		//if times {
252 | 		//	preserveTimes(dest+file, f, "link")
253 | 		//}
254 | 
255 | 	case mode&(os.ModeDevice|os.ModeNamedPipe|os.ModeSocket) != 0: // special files
256 | 		// TODO: not yet implemented
257 | 		fmt.Fprintf(os.Stderr, "WARNING - %s: syncing of UNIX special files is not implemented yet.\n", src+file)
258 | 
259 | 	default:
260 | 		// copy regular file
261 | 		// open source file for reading
262 | 		rd, err := os.Open(src + file)
263 | 		if err != nil {
264 | 			if !quiet {
265 | 				fmt.Fprintf(os.Stderr, "WARNING - file %s disappeared while copying: %s\n", src+file, err)
266 | 			}
267 | 			return
268 | 		}
269 | 		defer rd.Close()
270 | 
271 | 		// open destination file for writing
272 | 		perm := mode.Perm()
273 | 		wr, err := os.OpenFile(dest+file, os.O_WRONLY|os.O_CREATE, perm)
274 | 		if err != nil {
275 | 			if !quiet {
276 | 				fmt.Fprintf(os.Stderr, "WARNING - file %s could not be created: %s\n", dest+file, err)
277 | 			}
278 | 			return
279 | 		}
280 | 		defer wr.Close()
281 | 
282 | 		// copy data
283 | 		_, err = io.CopyBuffer(wr, rd, buffer[id][:])
284 | 		if err != nil {
285 | 			if !quiet {
286 | 				fmt.Fprintf(os.Stderr, "WARNING - file %s could not be created: %s\n", dest+file, err)
287 | 			}
288 | 			return
289 | 		}
290 | 
291 | 		if owner {
292 | 			preserveOwner(dest+file, f, "file")
293 | 		}
294 | 		if times {
295 | 			preserveTimes(dest+file, f, "file")
296 | 		}
297 | 
298 | 	}
299 | }
300 | 
301 | // Function preserveOwner transfers the ownership information from the source to
302 | // the destination file/directory.
303 | func preserveOwner(name string, f os.FileInfo, ftype string) {
304 | 	if stat, ok := f.Sys().(*syscall.Stat_t); ok {
305 | 		uid := int(stat.Uid)
306 | 		gid := int(stat.Gid)
307 | 
308 | 		var err error
309 | 		if ftype == "link" {
310 | 			err = syscall.Lchown(name, uid, gid)
311 | 		} else {
312 | 			err = os.Chown(name, uid, gid)
313 | 		}
314 | 
315 | 		if err != nil && !quiet {
316 | 			fmt.Fprintf(os.Stderr, "WARNING - could not change ownership of %s %s: %s\n",
317 | 				ftype, name, err)
318 | 		}
319 | 	}
320 | }
321 | 
322 | // Function preserveTimes transfers the access and modification timestamp from
323 | // the source to the destination file/directory.
324 | func preserveTimes(name string, f os.FileInfo, ftype string) {
325 | 	mtime := f.ModTime()
326 | 	atime := mtime
327 | 	if stat, ok := f.Sys().(*syscall.Stat_t); ok {
328 | 		atime = time.Unix(int64(stat.Atim.Sec), int64(stat.Atim.Nsec))
329 | 	}
330 | 	err := os.Chtimes(name, atime, mtime)
331 | 	if err != nil && !quiet {
332 | 		fmt.Fprintf(os.Stderr, "WARNING - could not change timestamps for %s %s: %s\n",
333 | 			ftype, name, err)
334 | 	}
335 | }
336 | 


--------------------------------------------------------------------------------
/LICENSE.txt:
--------------------------------------------------------------------------------
  1 |                     GNU GENERAL PUBLIC LICENSE
  2 |                        Version 3, 29 June 2007
  3 | 
  4 |  Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
  5 |  Everyone is permitted to copy and distribute verbatim copies
  6 |  of this license document, but changing it is not allowed.
  7 | 
  8 |                             Preamble
  9 | 
 10 |   The GNU General Public License is a free, copyleft license for
 11 | software and other kinds of works.
 12 | 
 13 |   The licenses for most software and other practical works are designed
 14 | to take away your freedom to share and change the works.  By contrast,
 15 | the GNU General Public License is intended to guarantee your freedom to
 16 | share and change all versions of a program--to make sure it remains free
 17 | software for all its users.  We, the Free Software Foundation, use the
 18 | GNU General Public License for most of our software; it applies also to
 19 | any other work released this way by its authors.  You can apply it to
 20 | your programs, too.
 21 | 
 22 |   When we speak of free software, we are referring to freedom, not
 23 | price.  Our General Public Licenses are designed to make sure that you
 24 | have the freedom to distribute copies of free software (and charge for
 25 | them if you wish), that you receive source code or can get it if you
 26 | want it, that you can change the software or use pieces of it in new
 27 | free programs, and that you know you can do these things.
 28 | 
 29 |   To protect your rights, we need to prevent others from denying you
 30 | these rights or asking you to surrender the rights.  Therefore, you have
 31 | certain responsibilities if you distribute copies of the software, or if
 32 | you modify it: responsibilities to respect the freedom of others.
 33 | 
 34 |   For example, if you distribute copies of such a program, whether
 35 | gratis or for a fee, you must pass on to the recipients the same
 36 | freedoms that you received.  You must make sure that they, too, receive
 37 | or can get the source code.  And you must show them these terms so they
 38 | know their rights.
 39 | 
 40 |   Developers that use the GNU GPL protect your rights with two steps:
 41 | (1) assert copyright on the software, and (2) offer you this License
 42 | giving you legal permission to copy, distribute and/or modify it.
 43 | 
 44 |   For the developers' and authors' protection, the GPL clearly explains
 45 | that there is no warranty for this free software.  For both users' and
 46 | authors' sake, the GPL requires that modified versions be marked as
 47 | changed, so that their problems will not be attributed erroneously to
 48 | authors of previous versions.
 49 | 
 50 |   Some devices are designed to deny users access to install or run
 51 | modified versions of the software inside them, although the manufacturer
 52 | can do so.  This is fundamentally incompatible with the aim of
 53 | protecting users' freedom to change the software.  The systematic
 54 | pattern of such abuse occurs in the area of products for individuals to
 55 | use, which is precisely where it is most unacceptable.  Therefore, we
 56 | have designed this version of the GPL to prohibit the practice for those
 57 | products.  If such problems arise substantially in other domains, we
 58 | stand ready to extend this provision to those domains in future versions
 59 | of the GPL, as needed to protect the freedom of users.
 60 | 
 61 |   Finally, every program is threatened constantly by software patents.
 62 | States should not allow patents to restrict development and use of
 63 | software on general-purpose computers, but in those that do, we wish to
 64 | avoid the special danger that patents applied to a free program could
 65 | make it effectively proprietary.  To prevent this, the GPL assures that
 66 | patents cannot be used to render the program non-free.
 67 | 
 68 |   The precise terms and conditions for copying, distribution and
 69 | modification follow.
 70 | 
 71 |                        TERMS AND CONDITIONS
 72 | 
 73 |   0. Definitions.
 74 | 
 75 |   "This License" refers to version 3 of the GNU General Public License.
 76 | 
 77 |   "Copyright" also means copyright-like laws that apply to other kinds of
 78 | works, such as semiconductor masks.
 79 | 
 80 |   "The Program" refers to any copyrightable work licensed under this
 81 | License.  Each licensee is addressed as "you".  "Licensees" and
 82 | "recipients" may be individuals or organizations.
 83 | 
 84 |   To "modify" a work means to copy from or adapt all or part of the work
 85 | in a fashion requiring copyright permission, other than the making of an
 86 | exact copy.  The resulting work is called a "modified version" of the
 87 | earlier work or a work "based on" the earlier work.
 88 | 
 89 |   A "covered work" means either the unmodified Program or a work based
 90 | on the Program.
 91 | 
 92 |   To "propagate" a work means to do anything with it that, without
 93 | permission, would make you directly or secondarily liable for
 94 | infringement under applicable copyright law, except executing it on a
 95 | computer or modifying a private copy.  Propagation includes copying,
 96 | distribution (with or without modification), making available to the
 97 | public, and in some countries other activities as well.
 98 | 
 99 |   To "convey" a work means any kind of propagation that enables other
100 | parties to make or receive copies.  Mere interaction with a user through
101 | a computer network, with no transfer of a copy, is not conveying.
102 | 
103 |   An interactive user interface displays "Appropriate Legal Notices"
104 | to the extent that it includes a convenient and prominently visible
105 | feature that (1) displays an appropriate copyright notice, and (2)
106 | tells the user that there is no warranty for the work (except to the
107 | extent that warranties are provided), that licensees may convey the
108 | work under this License, and how to view a copy of this License.  If
109 | the interface presents a list of user commands or options, such as a
110 | menu, a prominent item in the list meets this criterion.
111 | 
112 |   1. Source Code.
113 | 
114 |   The "source code" for a work means the preferred form of the work
115 | for making modifications to it.  "Object code" means any non-source
116 | form of a work.
117 | 
118 |   A "Standard Interface" means an interface that either is an official
119 | standard defined by a recognized standards body, or, in the case of
120 | interfaces specified for a particular programming language, one that
121 | is widely used among developers working in that language.
122 | 
123 |   The "System Libraries" of an executable work include anything, other
124 | than the work as a whole, that (a) is included in the normal form of
125 | packaging a Major Component, but which is not part of that Major
126 | Component, and (b) serves only to enable use of the work with that
127 | Major Component, or to implement a Standard Interface for which an
128 | implementation is available to the public in source code form.  A
129 | "Major Component", in this context, means a major essential component
130 | (kernel, window system, and so on) of the specific operating system
131 | (if any) on which the executable work runs, or a compiler used to
132 | produce the work, or an object code interpreter used to run it.
133 | 
134 |   The "Corresponding Source" for a work in object code form means all
135 | the source code needed to generate, install, and (for an executable
136 | work) run the object code and to modify the work, including scripts to
137 | control those activities.  However, it does not include the work's
138 | System Libraries, or general-purpose tools or generally available free
139 | programs which are used unmodified in performing those activities but
140 | which are not part of the work.  For example, Corresponding Source
141 | includes interface definition files associated with source files for
142 | the work, and the source code for shared libraries and dynamically
143 | linked subprograms that the work is specifically designed to require,
144 | such as by intimate data communication or control flow between those
145 | subprograms and other parts of the work.
146 | 
147 |   The Corresponding Source need not include anything that users
148 | can regenerate automatically from other parts of the Corresponding
149 | Source.
150 | 
151 |   The Corresponding Source for a work in source code form is that
152 | same work.
153 | 
154 |   2. Basic Permissions.
155 | 
156 |   All rights granted under this License are granted for the term of
157 | copyright on the Program, and are irrevocable provided the stated
158 | conditions are met.  This License explicitly affirms your unlimited
159 | permission to run the unmodified Program.  The output from running a
160 | covered work is covered by this License only if the output, given its
161 | content, constitutes a covered work.  This License acknowledges your
162 | rights of fair use or other equivalent, as provided by copyright law.
163 | 
164 |   You may make, run and propagate covered works that you do not
165 | convey, without conditions so long as your license otherwise remains
166 | in force.  You may convey covered works to others for the sole purpose
167 | of having them make modifications exclusively for you, or provide you
168 | with facilities for running those works, provided that you comply with
169 | the terms of this License in conveying all material for which you do
170 | not control copyright.  Those thus making or running the covered works
171 | for you must do so exclusively on your behalf, under your direction
172 | and control, on terms that prohibit them from making any copies of
173 | your copyrighted material outside their relationship with you.
174 | 
175 |   Conveying under any other circumstances is permitted solely under
176 | the conditions stated below.  Sublicensing is not allowed; section 10
177 | makes it unnecessary.
178 | 
179 |   3. Protecting Users' Legal Rights From Anti-Circumvention Law.
180 | 
181 |   No covered work shall be deemed part of an effective technological
182 | measure under any applicable law fulfilling obligations under article
183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or
184 | similar laws prohibiting or restricting circumvention of such
185 | measures.
186 | 
187 |   When you convey a covered work, you waive any legal power to forbid
188 | circumvention of technological measures to the extent such circumvention
189 | is effected by exercising rights under this License with respect to
190 | the covered work, and you disclaim any intention to limit operation or
191 | modification of the work as a means of enforcing, against the work's
192 | users, your or third parties' legal rights to forbid circumvention of
193 | technological measures.
194 | 
195 |   4. Conveying Verbatim Copies.
196 | 
197 |   You may convey verbatim copies of the Program's source code as you
198 | receive it, in any medium, provided that you conspicuously and
199 | appropriately publish on each copy an appropriate copyright notice;
200 | keep intact all notices stating that this License and any
201 | non-permissive terms added in accord with section 7 apply to the code;
202 | keep intact all notices of the absence of any warranty; and give all
203 | recipients a copy of this License along with the Program.
204 | 
205 |   You may charge any price or no price for each copy that you convey,
206 | and you may offer support or warranty protection for a fee.
207 | 
208 |   5. Conveying Modified Source Versions.
209 | 
210 |   You may convey a work based on the Program, or the modifications to
211 | produce it from the Program, in the form of source code under the
212 | terms of section 4, provided that you also meet all of these conditions:
213 | 
214 |     a) The work must carry prominent notices stating that you modified
215 |     it, and giving a relevant date.
216 | 
217 |     b) The work must carry prominent notices stating that it is
218 |     released under this License and any conditions added under section
219 |     7.  This requirement modifies the requirement in section 4 to
220 |     "keep intact all notices".
221 | 
222 |     c) You must license the entire work, as a whole, under this
223 |     License to anyone who comes into possession of a copy.  This
224 |     License will therefore apply, along with any applicable section 7
225 |     additional terms, to the whole of the work, and all its parts,
226 |     regardless of how they are packaged.  This License gives no
227 |     permission to license the work in any other way, but it does not
228 |     invalidate such permission if you have separately received it.
229 | 
230 |     d) If the work has interactive user interfaces, each must display
231 |     Appropriate Legal Notices; however, if the Program has interactive
232 |     interfaces that do not display Appropriate Legal Notices, your
233 |     work need not make them do so.
234 | 
235 |   A compilation of a covered work with other separate and independent
236 | works, which are not by their nature extensions of the covered work,
237 | and which are not combined with it such as to form a larger program,
238 | in or on a volume of a storage or distribution medium, is called an
239 | "aggregate" if the compilation and its resulting copyright are not
240 | used to limit the access or legal rights of the compilation's users
241 | beyond what the individual works permit.  Inclusion of a covered work
242 | in an aggregate does not cause this License to apply to the other
243 | parts of the aggregate.
244 | 
245 |   6. Conveying Non-Source Forms.
246 | 
247 |   You may convey a covered work in object code form under the terms
248 | of sections 4 and 5, provided that you also convey the
249 | machine-readable Corresponding Source under the terms of this License,
250 | in one of these ways:
251 | 
252 |     a) Convey the object code in, or embodied in, a physical product
253 |     (including a physical distribution medium), accompanied by the
254 |     Corresponding Source fixed on a durable physical medium
255 |     customarily used for software interchange.
256 | 
257 |     b) Convey the object code in, or embodied in, a physical product
258 |     (including a physical distribution medium), accompanied by a
259 |     written offer, valid for at least three years and valid for as
260 |     long as you offer spare parts or customer support for that product
261 |     model, to give anyone who possesses the object code either (1) a
262 |     copy of the Corresponding Source for all the software in the
263 |     product that is covered by this License, on a durable physical
264 |     medium customarily used for software interchange, for a price no
265 |     more than your reasonable cost of physically performing this
266 |     conveying of source, or (2) access to copy the
267 |     Corresponding Source from a network server at no charge.
268 | 
269 |     c) Convey individual copies of the object code with a copy of the
270 |     written offer to provide the Corresponding Source.  This
271 |     alternative is allowed only occasionally and noncommercially, and
272 |     only if you received the object code with such an offer, in accord
273 |     with subsection 6b.
274 | 
275 |     d) Convey the object code by offering access from a designated
276 |     place (gratis or for a charge), and offer equivalent access to the
277 |     Corresponding Source in the same way through the same place at no
278 |     further charge.  You need not require recipients to copy the
279 |     Corresponding Source along with the object code.  If the place to
280 |     copy the object code is a network server, the Corresponding Source
281 |     may be on a different server (operated by you or a third party)
282 |     that supports equivalent copying facilities, provided you maintain
283 |     clear directions next to the object code saying where to find the
284 |     Corresponding Source.  Regardless of what server hosts the
285 |     Corresponding Source, you remain obligated to ensure that it is
286 |     available for as long as needed to satisfy these requirements.
287 | 
288 |     e) Convey the object code using peer-to-peer transmission, provided
289 |     you inform other peers where the object code and Corresponding
290 |     Source of the work are being offered to the general public at no
291 |     charge under subsection 6d.
292 | 
293 |   A separable portion of the object code, whose source code is excluded
294 | from the Corresponding Source as a System Library, need not be
295 | included in conveying the object code work.
296 | 
297 |   A "User Product" is either (1) a "consumer product", which means any
298 | tangible personal property which is normally used for personal, family,
299 | or household purposes, or (2) anything designed or sold for incorporation
300 | into a dwelling.  In determining whether a product is a consumer product,
301 | doubtful cases shall be resolved in favor of coverage.  For a particular
302 | product received by a particular user, "normally used" refers to a
303 | typical or common use of that class of product, regardless of the status
304 | of the particular user or of the way in which the particular user
305 | actually uses, or expects or is expected to use, the product.  A product
306 | is a consumer product regardless of whether the product has substantial
307 | commercial, industrial or non-consumer uses, unless such uses represent
308 | the only significant mode of use of the product.
309 | 
310 |   "Installation Information" for a User Product means any methods,
311 | procedures, authorization keys, or other information required to install
312 | and execute modified versions of a covered work in that User Product from
313 | a modified version of its Corresponding Source.  The information must
314 | suffice to ensure that the continued functioning of the modified object
315 | code is in no case prevented or interfered with solely because
316 | modification has been made.
317 | 
318 |   If you convey an object code work under this section in, or with, or
319 | specifically for use in, a User Product, and the conveying occurs as
320 | part of a transaction in which the right of possession and use of the
321 | User Product is transferred to the recipient in perpetuity or for a
322 | fixed term (regardless of how the transaction is characterized), the
323 | Corresponding Source conveyed under this section must be accompanied
324 | by the Installation Information.  But this requirement does not apply
325 | if neither you nor any third party retains the ability to install
326 | modified object code on the User Product (for example, the work has
327 | been installed in ROM).
328 | 
329 |   The requirement to provide Installation Information does not include a
330 | requirement to continue to provide support service, warranty, or updates
331 | for a work that has been modified or installed by the recipient, or for
332 | the User Product in which it has been modified or installed.  Access to a
333 | network may be denied when the modification itself materially and
334 | adversely affects the operation of the network or violates the rules and
335 | protocols for communication across the network.
336 | 
337 |   Corresponding Source conveyed, and Installation Information provided,
338 | in accord with this section must be in a format that is publicly
339 | documented (and with an implementation available to the public in
340 | source code form), and must require no special password or key for
341 | unpacking, reading or copying.
342 | 
343 |   7. Additional Terms.
344 | 
345 |   "Additional permissions" are terms that supplement the terms of this
346 | License by making exceptions from one or more of its conditions.
347 | Additional permissions that are applicable to the entire Program shall
348 | be treated as though they were included in this License, to the extent
349 | that they are valid under applicable law.  If additional permissions
350 | apply only to part of the Program, that part may be used separately
351 | under those permissions, but the entire Program remains governed by
352 | this License without regard to the additional permissions.
353 | 
354 |   When you convey a copy of a covered work, you may at your option
355 | remove any additional permissions from that copy, or from any part of
356 | it.  (Additional permissions may be written to require their own
357 | removal in certain cases when you modify the work.)  You may place
358 | additional permissions on material, added by you to a covered work,
359 | for which you have or can give appropriate copyright permission.
360 | 
361 |   Notwithstanding any other provision of this License, for material you
362 | add to a covered work, you may (if authorized by the copyright holders of
363 | that material) supplement the terms of this License with terms:
364 | 
365 |     a) Disclaiming warranty or limiting liability differently from the
366 |     terms of sections 15 and 16 of this License; or
367 | 
368 |     b) Requiring preservation of specified reasonable legal notices or
369 |     author attributions in that material or in the Appropriate Legal
370 |     Notices displayed by works containing it; or
371 | 
372 |     c) Prohibiting misrepresentation of the origin of that material, or
373 |     requiring that modified versions of such material be marked in
374 |     reasonable ways as different from the original version; or
375 | 
376 |     d) Limiting the use for publicity purposes of names of licensors or
377 |     authors of the material; or
378 | 
379 |     e) Declining to grant rights under trademark law for use of some
380 |     trade names, trademarks, or service marks; or
381 | 
382 |     f) Requiring indemnification of licensors and authors of that
383 |     material by anyone who conveys the material (or modified versions of
384 |     it) with contractual assumptions of liability to the recipient, for
385 |     any liability that these contractual assumptions directly impose on
386 |     those licensors and authors.
387 | 
388 |   All other non-permissive additional terms are considered "further
389 | restrictions" within the meaning of section 10.  If the Program as you
390 | received it, or any part of it, contains a notice stating that it is
391 | governed by this License along with a term that is a further
392 | restriction, you may remove that term.  If a license document contains
393 | a further restriction but permits relicensing or conveying under this
394 | License, you may add to a covered work material governed by the terms
395 | of that license document, provided that the further restriction does
396 | not survive such relicensing or conveying.
397 | 
398 |   If you add terms to a covered work in accord with this section, you
399 | must place, in the relevant source files, a statement of the
400 | additional terms that apply to those files, or a notice indicating
401 | where to find the applicable terms.
402 | 
403 |   Additional terms, permissive or non-permissive, may be stated in the
404 | form of a separately written license, or stated as exceptions;
405 | the above requirements apply either way.
406 | 
407 |   8. Termination.
408 | 
409 |   You may not propagate or modify a covered work except as expressly
410 | provided under this License.  Any attempt otherwise to propagate or
411 | modify it is void, and will automatically terminate your rights under
412 | this License (including any patent licenses granted under the third
413 | paragraph of section 11).
414 | 
415 |   However, if you cease all violation of this License, then your
416 | license from a particular copyright holder is reinstated (a)
417 | provisionally, unless and until the copyright holder explicitly and
418 | finally terminates your license, and (b) permanently, if the copyright
419 | holder fails to notify you of the violation by some reasonable means
420 | prior to 60 days after the cessation.
421 | 
422 |   Moreover, your license from a particular copyright holder is
423 | reinstated permanently if the copyright holder notifies you of the
424 | violation by some reasonable means, this is the first time you have
425 | received notice of violation of this License (for any work) from that
426 | copyright holder, and you cure the violation prior to 30 days after
427 | your receipt of the notice.
428 | 
429 |   Termination of your rights under this section does not terminate the
430 | licenses of parties who have received copies or rights from you under
431 | this License.  If your rights have been terminated and not permanently
432 | reinstated, you do not qualify to receive new licenses for the same
433 | material under section 10.
434 | 
435 |   9. Acceptance Not Required for Having Copies.
436 | 
437 |   You are not required to accept this License in order to receive or
438 | run a copy of the Program.  Ancillary propagation of a covered work
439 | occurring solely as a consequence of using peer-to-peer transmission
440 | to receive a copy likewise does not require acceptance.  However,
441 | nothing other than this License grants you permission to propagate or
442 | modify any covered work.  These actions infringe copyright if you do
443 | not accept this License.  Therefore, by modifying or propagating a
444 | covered work, you indicate your acceptance of this License to do so.
445 | 
446 |   10. Automatic Licensing of Downstream Recipients.
447 | 
448 |   Each time you convey a covered work, the recipient automatically
449 | receives a license from the original licensors, to run, modify and
450 | propagate that work, subject to this License.  You are not responsible
451 | for enforcing compliance by third parties with this License.
452 | 
453 |   An "entity transaction" is a transaction transferring control of an
454 | organization, or substantially all assets of one, or subdividing an
455 | organization, or merging organizations.  If propagation of a covered
456 | work results from an entity transaction, each party to that
457 | transaction who receives a copy of the work also receives whatever
458 | licenses to the work the party's predecessor in interest had or could
459 | give under the previous paragraph, plus a right to possession of the
460 | Corresponding Source of the work from the predecessor in interest, if
461 | the predecessor has it or can get it with reasonable efforts.
462 | 
463 |   You may not impose any further restrictions on the exercise of the
464 | rights granted or affirmed under this License.  For example, you may
465 | not impose a license fee, royalty, or other charge for exercise of
466 | rights granted under this License, and you may not initiate litigation
467 | (including a cross-claim or counterclaim in a lawsuit) alleging that
468 | any patent claim is infringed by making, using, selling, offering for
469 | sale, or importing the Program or any portion of it.
470 | 
471 |   11. Patents.
472 | 
473 |   A "contributor" is a copyright holder who authorizes use under this
474 | License of the Program or a work on which the Program is based.  The
475 | work thus licensed is called the contributor's "contributor version".
476 | 
477 |   A contributor's "essential patent claims" are all patent claims
478 | owned or controlled by the contributor, whether already acquired or
479 | hereafter acquired, that would be infringed by some manner, permitted
480 | by this License, of making, using, or selling its contributor version,
481 | but do not include claims that would be infringed only as a
482 | consequence of further modification of the contributor version.  For
483 | purposes of this definition, "control" includes the right to grant
484 | patent sublicenses in a manner consistent with the requirements of
485 | this License.
486 | 
487 |   Each contributor grants you a non-exclusive, worldwide, royalty-free
488 | patent license under the contributor's essential patent claims, to
489 | make, use, sell, offer for sale, import and otherwise run, modify and
490 | propagate the contents of its contributor version.
491 | 
492 |   In the following three paragraphs, a "patent license" is any express
493 | agreement or commitment, however denominated, not to enforce a patent
494 | (such as an express permission to practice a patent or covenant not to
495 | sue for patent infringement).  To "grant" such a patent license to a
496 | party means to make such an agreement or commitment not to enforce a
497 | patent against the party.
498 | 
499 |   If you convey a covered work, knowingly relying on a patent license,
500 | and the Corresponding Source of the work is not available for anyone
501 | to copy, free of charge and under the terms of this License, through a
502 | publicly available network server or other readily accessible means,
503 | then you must either (1) cause the Corresponding Source to be so
504 | available, or (2) arrange to deprive yourself of the benefit of the
505 | patent license for this particular work, or (3) arrange, in a manner
506 | consistent with the requirements of this License, to extend the patent
507 | license to downstream recipients.  "Knowingly relying" means you have
508 | actual knowledge that, but for the patent license, your conveying the
509 | covered work in a country, or your recipient's use of the covered work
510 | in a country, would infringe one or more identifiable patents in that
511 | country that you have reason to believe are valid.
512 | 
513 |   If, pursuant to or in connection with a single transaction or
514 | arrangement, you convey, or propagate by procuring conveyance of, a
515 | covered work, and grant a patent license to some of the parties
516 | receiving the covered work authorizing them to use, propagate, modify
517 | or convey a specific copy of the covered work, then the patent license
518 | you grant is automatically extended to all recipients of the covered
519 | work and works based on it.
520 | 
521 |   A patent license is "discriminatory" if it does not include within
522 | the scope of its coverage, prohibits the exercise of, or is
523 | conditioned on the non-exercise of one or more of the rights that are
524 | specifically granted under this License.  You may not convey a covered
525 | work if you are a party to an arrangement with a third party that is
526 | in the business of distributing software, under which you make payment
527 | to the third party based on the extent of your activity of conveying
528 | the work, and under which the third party grants, to any of the
529 | parties who would receive the covered work from you, a discriminatory
530 | patent license (a) in connection with copies of the covered work
531 | conveyed by you (or copies made from those copies), or (b) primarily
532 | for and in connection with specific products or compilations that
533 | contain the covered work, unless you entered into that arrangement,
534 | or that patent license was granted, prior to 28 March 2007.
535 | 
536 |   Nothing in this License shall be construed as excluding or limiting
537 | any implied license or other defenses to infringement that may
538 | otherwise be available to you under applicable patent law.
539 | 
540 |   12. No Surrender of Others' Freedom.
541 | 
542 |   If conditions are imposed on you (whether by court order, agreement or
543 | otherwise) that contradict the conditions of this License, they do not
544 | excuse you from the conditions of this License.  If you cannot convey a
545 | covered work so as to satisfy simultaneously your obligations under this
546 | License and any other pertinent obligations, then as a consequence you may
547 | not convey it at all.  For example, if you agree to terms that obligate you
548 | to collect a royalty for further conveying from those to whom you convey
549 | the Program, the only way you could satisfy both those terms and this
550 | License would be to refrain entirely from conveying the Program.
551 | 
552 |   13. Use with the GNU Affero General Public License.
553 | 
554 |   Notwithstanding any other provision of this License, you have
555 | permission to link or combine any covered work with a work licensed
556 | under version 3 of the GNU Affero General Public License into a single
557 | combined work, and to convey the resulting work.  The terms of this
558 | License will continue to apply to the part which is the covered work,
559 | but the special requirements of the GNU Affero General Public License,
560 | section 13, concerning interaction through a network will apply to the
561 | combination as such.
562 | 
563 |   14. Revised Versions of this License.
564 | 
565 |   The Free Software Foundation may publish revised and/or new versions of
566 | the GNU General Public License from time to time.  Such new versions will
567 | be similar in spirit to the present version, but may differ in detail to
568 | address new problems or concerns.
569 | 
570 |   Each version is given a distinguishing version number.  If the
571 | Program specifies that a certain numbered version of the GNU General
572 | Public License "or any later version" applies to it, you have the
573 | option of following the terms and conditions either of that numbered
574 | version or of any later version published by the Free Software
575 | Foundation.  If the Program does not specify a version number of the
576 | GNU General Public License, you may choose any version ever published
577 | by the Free Software Foundation.
578 | 
579 |   If the Program specifies that a proxy can decide which future
580 | versions of the GNU General Public License can be used, that proxy's
581 | public statement of acceptance of a version permanently authorizes you
582 | to choose that version for the Program.
583 | 
584 |   Later license versions may give you additional or different
585 | permissions.  However, no additional obligations are imposed on any
586 | author or copyright holder as a result of your choosing to follow a
587 | later version.
588 | 
589 |   15. Disclaimer of Warranty.
590 | 
591 |   THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
592 | APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
596 | PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
597 | IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
599 | 
600 |   16. Limitation of Liability.
601 | 
602 |   IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
610 | SUCH DAMAGES.
611 | 
612 |   17. Interpretation of Sections 15 and 16.
613 | 
614 |   If the disclaimer of warranty and limitation of liability provided
615 | above cannot be given local legal effect according to their terms,
616 | reviewing courts shall apply local law that most closely approximates
617 | an absolute waiver of all civil liability in connection with the
618 | Program, unless a warranty or assumption of liability accompanies a
619 | copy of the Program in return for a fee.
620 | 
621 |                      END OF TERMS AND CONDITIONS
622 | 
623 |             How to Apply These Terms to Your New Programs
624 | 
625 |   If you develop a new program, and you want it to be of the greatest
626 | possible use to the public, the best way to achieve this is to make it
627 | free software which everyone can redistribute and change under these terms.
628 | 
629 |   To do so, attach the following notices to the program.  It is safest
630 | to attach them to the start of each source file to most effectively
631 | state the exclusion of warranty; and each file should have at least
632 | the "copyright" line and a pointer to where the full notice is found.
633 | 
634 |     <one line to give the program's name and a brief idea of what it does.>
635 |     Copyright (C) <year>  <name of author>
636 | 
637 |     This program is free software: you can redistribute it and/or modify
638 |     it under the terms of the GNU General Public License as published by
639 |     the Free Software Foundation, either version 3 of the License, or
640 |     (at your option) any later version.
641 | 
642 |     This program is distributed in the hope that it will be useful,
643 |     but WITHOUT ANY WARRANTY; without even the implied warranty of
644 |     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
645 |     GNU General Public License for more details.
646 | 
647 |     You should have received a copy of the GNU General Public License
648 |     along with this program.  If not, see <http://www.gnu.org/licenses/>.
649 | 
650 | Also add information on how to contact you by electronic and paper mail.
651 | 
652 |   If the program does terminal interaction, make it output a short
653 | notice like this when it starts in an interactive mode:
654 | 
655 |     <program>  Copyright (C) <year>  <name of author>
656 |     This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
657 |     This is free software, and you are welcome to redistribute it
658 |     under certain conditions; type `show c' for details.
659 | 
660 | The hypothetical commands `show w' and `show c' should show the appropriate
661 | parts of the General Public License.  Of course, your program's commands
662 | might be different; for a GUI interface, you would use an "about box".
663 | 
664 |   You should also get your employer (if you work as a programmer) or school,
665 | if any, to sign a "copyright disclaimer" for the program, if necessary.
666 | For more information on this, and how to apply and follow the GNU GPL, see
667 | <http://www.gnu.org/licenses/>.
668 | 
669 |   The GNU General Public License does not permit incorporating your program
670 | into proprietary programs.  If your program is a subroutine library, you
671 | may consider it more useful to permit linking proprietary applications with
672 | the library.  If this is what you want to do, use the GNU Lesser General
673 | Public License instead of this License.  But first, please read
674 | <http://www.gnu.org/philosophy/why-not-lgpl.html>.
675 | 


--------------------------------------------------------------------------------