├── LICENSE ├── README.md ├── block ├── block_out.h └── t_block_out.cpp ├── qi.prj ├── qi ├── qi-make.py └── qi_build.h ├── rsync ├── rsync_client.cpp ├── rsync_client.h ├── rsync_entry.cpp ├── rsync_entry.h ├── rsync_file.cpp ├── rsync_file.h ├── rsync_io.cpp ├── rsync_io.h ├── rsync_log.cpp ├── rsync_log.h ├── rsync_pathutil.cpp ├── rsync_pathutil.h ├── rsync_socketio.cpp ├── rsync_socketio.h ├── rsync_socketutil.cpp ├── rsync_socketutil.h ├── rsync_sshio.cpp ├── rsync_sshio.h ├── rsync_stream.cpp ├── rsync_stream.h ├── rsync_timeutil.cpp ├── rsync_timeutil.h ├── rsync_util.cpp ├── rsync_util.h ├── t_rsync_client.cpp ├── t_rsync_entry.cpp ├── t_rsync_fileutil.cpp └── t_rsync_stream.cpp └── testutil ├── testutil_assert.h └── testutil_newdeletemonitor.h /LICENSE: -------------------------------------------------------------------------------- 1 | Reciprocal Public License (RPL-1.5) 2 | 3 | Version 1.5, July 15, 2007 4 | 5 | Copyright (C) 2001-2007 6 | Technical Pursuit Inc., 7 | All Rights Reserved. 8 | 9 | 10 | PREAMBLE 11 | 12 | The Reciprocal Public License (RPL) is based on the concept of reciprocity or, 13 | if you prefer, fairness. 14 | 15 | In short, this license grew out of a desire to close loopholes in previous open 16 | source licenses, loopholes that allowed parties to acquire open source software 17 | and derive financial benefit from it without having to release their 18 | improvements or derivatives to the community which enabled them. This occurred 19 | any time an entity did not release their application to a "third party". 20 | 21 | While there is a certain freedom in this model of licensing, it struck the 22 | authors of the RPL as being unfair to the open source community at large and to 23 | the original authors of the works in particular. After all, bug fixes, 24 | extensions, and meaningful and valuable derivatives were not consistently 25 | finding their way back into the community where they could fuel further, and 26 | faster, growth and expansion of the overall open source software base. 27 | 28 | While you should clearly read and understand the entire license, the essence of 29 | the RPL is found in two definitions: "Deploy" and "Required Components". 30 | 31 | Regarding deployment, under the RPL your changes, bug fixes, extensions, etc. 32 | must be made available to the open source community at large when you Deploy in 33 | any form -- either internally or to an outside party. Once you start running 34 | the software you have to start sharing the software. 35 | 36 | Further, under the RPL all components you author including schemas, scripts, 37 | source code, etc. -- regardless of whether they're compiled into a single 38 | binary or used as two halves of client/server application -- must be shared. 39 | You have to share the whole pie, not an isolated slice of it. 40 | 41 | In addition to these goals, the RPL was authored to meet the requirements of 42 | the Open Source Definition as maintained by the Open Source Initiative (OSI). 43 | 44 | The specific terms and conditions of the license are defined in the remainder 45 | of this document. 46 | 47 | 48 | LICENSE TERMS 49 | 50 | 1.0 General; Applicability & Definitions. This Reciprocal Public License 51 | Version 1.5 ("License") applies to any programs or other works as well as any 52 | and all updates or maintenance releases of said programs or works ("Software") 53 | not already covered by this License which the Software copyright holder 54 | ("Licensor") makes available containing a License Notice (hereinafter defined) 55 | from the Licensor specifying or allowing use or distribution under the terms of 56 | this License. As used in this License: 57 | 58 | 1.1 "Contributor" means any person or entity who created or contributed to the 59 | creation of an Extension. 60 | 61 | 1.2 "Deploy" means to use, Serve, sublicense or distribute Licensed Software 62 | other than for Your internal Research and/or Personal Use, and includes 63 | without limitation, any and all internal use or distribution of Licensed 64 | Software within Your business or organization other than for Research and/or 65 | Personal Use, as well as direct or indirect sublicensing or distribution of 66 | Licensed Software by You to any third party in any form or manner. 67 | 68 | 1.3 "Derivative Works" as used in this License is defined under U.S. copyright 69 | law. 70 | 71 | 1.4 "Electronic Distribution Mechanism" means a mechanism generally accepted 72 | in the software development community for the electronic transfer of data such 73 | as download from an FTP server or web site, where such mechanism is publicly 74 | accessible. 75 | 76 | 1.5 "Extensions" means any Modifications, Derivative Works, or Required 77 | Components as those terms are defined in this License. 78 | 79 | 1.6 "License" means this Reciprocal Public License. 80 | 81 | 1.7 "License Notice" means any notice contained in EXHIBIT A. 82 | 83 | 1.8 "Licensed Software" means any Software licensed pursuant to this License. 84 | Licensed Software also includes all previous Extensions from any Contributor 85 | that You receive. 86 | 87 | 1.9 "Licensor" means the copyright holder of any Software previously not 88 | covered by this License who releases the Software under the terms of this 89 | License. 90 | 91 | 1.10 "Modifications" means any additions to or deletions from the substance or 92 | structure of (i) a file or other storage containing Licensed Software, or (ii) 93 | any new file or storage that contains any part of Licensed Software, or (iii) 94 | any file or storage which replaces or otherwise alters the original 95 | functionality of Licensed Software at runtime. 96 | 97 | 1.11 "Personal Use" means use of Licensed Software by an individual solely for 98 | his or her personal, private and non-commercial purposes. An individual's use 99 | of Licensed Software in his or her capacity as an officer, employee, member, 100 | independent contractor or agent of a corporation, business or organization 101 | (commercial or non-commercial) does not qualify as Personal Use. 102 | 103 | 1.12 "Required Components" means any text, programs, scripts, schema, 104 | interface definitions, control files, or other works created by You which are 105 | required by a third party of average skill to successfully install and run 106 | Licensed Software containing Your Modifications, or to install and run Your 107 | Derivative Works. 108 | 109 | 1.13 "Research" means investigation or experimentation for the purpose of 110 | understanding the nature and limits of the Licensed Software and its potential 111 | uses. 112 | 113 | 1.14 "Serve" means to deliver Licensed Software and/or Your Extensions by 114 | means of a computer network to one or more computers for purposes of execution 115 | of Licensed Software and/or Your Extensions. 116 | 117 | 1.15 "Software" means any computer programs or other works as well as any 118 | updates or maintenance releases of those programs or works which are 119 | distributed publicly by Licensor. 120 | 121 | 1.16 "Source Code" means the preferred form for making modifications to the 122 | Licensed Software and/or Your Extensions, including all modules contained 123 | therein, plus any associated text, interface definition files, scripts used to 124 | control compilation and installation of an executable program or other 125 | components required by a third party of average skill to build a running 126 | version of the Licensed Software or Your Extensions. 127 | 128 | 1.17 "User-Visible Attribution Notice" means any notice contained in EXHIBIT B. 129 | 130 | 1.18 "You" or "Your" means an individual or a legal entity exercising rights 131 | under this License. For legal entities, "You" or "Your" includes any entity 132 | which controls, is controlled by, or is under common control with, You, where 133 | "control" means (a) the power, direct or indirect, to cause the direction or 134 | management of such entity, whether by contract or otherwise, or (b) ownership 135 | of fifty percent (50%) or more of the outstanding shares or beneficial 136 | ownership of such entity. 137 | 138 | 2.0 Acceptance Of License. You are not required to accept this License since 139 | you have not signed it, however nothing else grants you permission to use, 140 | copy, distribute, modify, or create derivatives of either the Software or any 141 | Extensions created by a Contributor. These actions are prohibited by law if 142 | you do not accept this License. Therefore, by performing any of these actions 143 | You indicate Your acceptance of this License and Your agreement to be bound by 144 | all its terms and conditions. IF YOU DO NOT AGREE WITH ALL THE TERMS AND 145 | CONDITIONS OF THIS LICENSE DO NOT USE, MODIFY, CREATE DERIVATIVES, OR 146 | DISTRIBUTE THE SOFTWARE. IF IT IS IMPOSSIBLE FOR YOU TO COMPLY WITH ALL THE 147 | TERMS AND CONDITIONS OF THIS LICENSE THEN YOU CAN NOT USE, MODIFY, CREATE 148 | DERIVATIVES, OR DISTRIBUTE THE SOFTWARE. 149 | 150 | 3.0 Grant of License From Licensor. Subject to the terms and conditions of 151 | this License, Licensor hereby grants You a world-wide, royalty-free, non- 152 | exclusive license, subject to Licensor's intellectual property rights, and any 153 | third party intellectual property claims derived from the Licensed Software 154 | under this License, to do the following: 155 | 156 | 3.1 Use, reproduce, modify, display, perform, sublicense and distribute 157 | Licensed Software and Your Extensions in both Source Code form or as an 158 | executable program. 159 | 160 | 3.2 Create Derivative Works (as that term is defined under U.S. copyright law) 161 | of Licensed Software by adding to or deleting from the substance or structure 162 | of said Licensed Software. 163 | 164 | 3.3 Under claims of patents now or hereafter owned or controlled by Licensor, 165 | to make, use, have made, and/or otherwise dispose of Licensed Software or 166 | portions thereof, but solely to the extent that any such claim is necessary to 167 | enable You to make, use, have made, and/or otherwise dispose of Licensed 168 | Software or portions thereof. 169 | 170 | 3.4 Licensor reserves the right to release new versions of the Software with 171 | different features, specifications, capabilities, functions, licensing terms, 172 | general availability or other characteristics. Title, ownership rights, and 173 | intellectual property rights in and to the Licensed Software shall remain in 174 | Licensor and/or its Contributors. 175 | 176 | 4.0 Grant of License From Contributor. By application of the provisions in 177 | Section 6 below, each Contributor hereby grants You a world-wide, royalty- 178 | free, non-exclusive license, subject to said Contributor's intellectual 179 | property rights, and any third party intellectual property claims derived from 180 | the Licensed Software under this License, to do the following: 181 | 182 | 4.1 Use, reproduce, modify, display, perform, sublicense and distribute any 183 | Extensions Deployed by such Contributor or portions thereof, in both Source 184 | Code form or as an executable program, either on an unmodified basis or as 185 | part of Derivative Works. 186 | 187 | 4.2 Under claims of patents now or hereafter owned or controlled by 188 | Contributor, to make, use, have made, and/or otherwise dispose of Extensions 189 | or portions thereof, but solely to the extent that any such claim is necessary 190 | to enable You to make, use, have made, and/or otherwise dispose of 191 | Licensed Software or portions thereof. 192 | 193 | 5.0 Exclusions From License Grant. Nothing in this License shall be deemed to 194 | grant any rights to trademarks, copyrights, patents, trade secrets or any 195 | other intellectual property of Licensor or any Contributor except as expressly 196 | stated herein. Except as expressly stated in Sections 3 and 4, no other patent 197 | rights, express or implied, are granted herein. Your Extensions may require 198 | additional patent licenses from Licensor or Contributors which each may grant 199 | in its sole discretion. No right is granted to the trademarks of Licensor or 200 | any Contributor even if such marks are included in the Licensed Software. 201 | Nothing in this License shall be interpreted to prohibit Licensor from 202 | licensing under different terms from this License any code that Licensor 203 | otherwise would have a right to license. 204 | 205 | 5.1 You expressly acknowledge and agree that although Licensor and each 206 | Contributor grants the licenses to their respective portions of the Licensed 207 | Software set forth herein, no assurances are provided by Licensor or any 208 | Contributor that the Licensed Software does not infringe the patent or other 209 | intellectual property rights of any other entity. Licensor and each 210 | Contributor disclaim any liability to You for claims brought by any other 211 | entity based on infringement of intellectual property rights or otherwise. As 212 | a condition to exercising the rights and licenses granted hereunder, You 213 | hereby assume sole responsibility to secure any other intellectual property 214 | rights needed, if any. For example, if a third party patent license is 215 | required to allow You to distribute the Licensed Software, it is Your 216 | responsibility to acquire that license before distributing the Licensed 217 | Software. 218 | 219 | 6.0 Your Obligations And Grants. In consideration of, and as an express 220 | condition to, the licenses granted to You under this License You hereby agree 221 | that any Modifications, Derivative Works, or Required Components (collectively 222 | Extensions) that You create or to which You contribute are governed by the 223 | terms of this License including, without limitation, Section 4. Any Extensions 224 | that You create or to which You contribute must be Deployed under the terms of 225 | this License or a future version of this License released under Section 7. You 226 | hereby grant to Licensor and all third parties a world-wide, non-exclusive, 227 | royalty-free license under those intellectual property rights You own or 228 | control to use, reproduce, display, perform, modify, create derivatives, 229 | sublicense, and distribute Licensed Software, in any form. Any Extensions You 230 | make and Deploy must have a distinct title so as to readily tell any 231 | subsequent user or Contributor that the Extensions are by You. You must 232 | include a copy of this License or directions on how to obtain a copy with 233 | every copy of the Extensions You distribute. You agree not to offer or impose 234 | any terms on any Source Code or executable version of the Licensed Software, 235 | or its Extensions that alter or restrict the applicable version of this 236 | License or the recipients' rights hereunder. 237 | 238 | 6.1 Availability of Source Code. You must make available, under the terms of 239 | this License, the Source Code of any Extensions that You Deploy, via an 240 | Electronic Distribution Mechanism. The Source Code for any version that You 241 | Deploy must be made available within one (1) month of when you Deploy and must 242 | remain available for no less than twelve (12) months after the date You cease 243 | to Deploy. You are responsible for ensuring that the Source Code to each 244 | version You Deploy remains available even if the Electronic Distribution 245 | Mechanism is maintained by a third party. You may not charge a fee for any 246 | copy of the Source Code distributed under this Section in excess of Your 247 | actual cost of duplication and distribution of said copy. 248 | 249 | 6.2 Description of Modifications. You must cause any Modifications that You 250 | create or to which You contribute to be documented in the Source Code, clearly 251 | describing the additions, changes or deletions You made. You must include a 252 | prominent statement that the Modifications are derived, directly or indirectly, 253 | from the Licensed Software and include the names of the Licensor and any 254 | Contributor to the Licensed Software in (i) the Source Code and (ii) in any 255 | notice displayed by the Licensed Software You distribute or in related 256 | documentation in which You describe the origin or ownership of the Licensed 257 | Software. You may not modify or delete any pre-existing copyright notices, 258 | change notices or License text in the Licensed Software without written 259 | permission of the respective Licensor or Contributor. 260 | 261 | 6.3 Intellectual Property Matters. 262 | 263 | a. Third Party Claims. If You have knowledge that a license to a third party's 264 | intellectual property right is required to exercise the rights granted by this 265 | License, You must include a human-readable file with Your distribution that 266 | describes the claim and the party making the claim in sufficient detail that a 267 | recipient will know whom to contact. 268 | 269 | b. Contributor APIs. If Your Extensions include an application programming 270 | interface ("API") and You have knowledge of patent licenses that are 271 | reasonably necessary to implement that API, You must also include this 272 | information in a human-readable file supplied with Your distribution. 273 | 274 | c. Representations. You represent that, except as disclosed pursuant to 6.3(a) 275 | above, You believe that any Extensions You distribute are Your original 276 | creations and that You have sufficient rights to grant the rights conveyed by 277 | this License. 278 | 279 | 6.4 Required Notices. 280 | 281 | a. License Text. You must duplicate this License or instructions on how to 282 | acquire a copy in any documentation You provide along with the Source Code of 283 | any Extensions You create or to which You contribute, wherever You describe 284 | recipients' rights relating to Licensed Software. 285 | 286 | b. License Notice. You must duplicate any notice contained in EXHIBIT A (the 287 | "License Notice") in each file of the Source Code of any copy You distribute 288 | of the Licensed Software and Your Extensions. If You create an Extension, You 289 | may add Your name as a Contributor to the Source Code and accompanying 290 | documentation along with a description of the contribution. If it is not 291 | possible to put the License Notice in a particular Source Code file due to its 292 | structure, then You must include such License Notice in a location where a 293 | user would be likely to look for such a notice. 294 | 295 | c. Source Code Availability. You must notify the software community of the 296 | availability of Source Code to Your Extensions within one (1) month of the date 297 | You initially Deploy and include in such notification a description of the 298 | Extensions, and instructions on how to acquire the Source Code. Should such 299 | instructions change you must notify the software community of revised 300 | instructions within one (1) month of the date of change. You must provide 301 | notification by posting to appropriate news groups, mailing lists, weblogs, or 302 | other sites where a publicly accessible search engine would reasonably be 303 | expected to index your post in relationship to queries regarding the Licensed 304 | Software and/or Your Extensions. 305 | 306 | d. User-Visible Attribution. You must duplicate any notice contained in 307 | EXHIBIT B (the "User-Visible Attribution Notice") in each user-visible display 308 | of the Licensed Software and Your Extensions which delineates copyright, 309 | ownership, or similar attribution information. If You create an Extension, 310 | You may add Your name as a Contributor, and add Your attribution notice, as an 311 | equally visible and functional element of any User-Visible Attribution Notice 312 | content. To ensure proper attribution, You must also include such User-Visible 313 | Attribution Notice in at least one location in the Software documentation 314 | where a user would be likely to look for such notice. 315 | 316 | 6.5 Additional Terms. You may choose to offer, and charge a fee for, warranty, 317 | support, indemnity or liability obligations to one or more recipients of 318 | Licensed Software. However, You may do so only on Your own behalf, and not on 319 | behalf of the Licensor or any Contributor except as permitted under other 320 | agreements between you and Licensor or Contributor. You must make it clear that 321 | any such warranty, support, indemnity or liability obligation is offered by You 322 | alone, and You hereby agree to indemnify the Licensor and every Contributor for 323 | any liability plus attorney fees, costs, and related expenses due to any such 324 | action or claim incurred by the Licensor or such Contributor as a result of 325 | warranty, support, indemnity or liability terms You offer. 326 | 327 | 6.6 Conflicts With Other Licenses. Where any portion of Your Extensions, by 328 | virtue of being Derivative Works of another product or similar circumstance, 329 | fall under the terms of another license, the terms of that license should be 330 | honored however You must also make Your Extensions available under this 331 | License. If the terms of this License continue to conflict with the terms of 332 | the other license you may write the Licensor for permission to resolve the 333 | conflict in a fashion that remains consistent with the intent of this License. 334 | Such permission will be granted at the sole discretion of the Licensor. 335 | 336 | 7.0 Versions of This License. Licensor may publish from time to time revised 337 | versions of the License. Once Licensed Software has been published under a 338 | particular version of the License, You may always continue to use it under the 339 | terms of that version. You may also choose to use such Licensed Software under 340 | the terms of any subsequent version of the License published by Licensor. No 341 | one other than Licensor has the right to modify the terms applicable to 342 | Licensed Software created under this License. 343 | 344 | 7.1 If You create or use a modified version of this License, which You may do 345 | only in order to apply it to software that is not already Licensed Software 346 | under this License, You must rename Your license so that it is not confusingly 347 | similar to this License, and must make it clear that Your license contains 348 | terms that differ from this License. In so naming Your license, You may not 349 | use any trademark of Licensor or of any Contributor. Should Your modifications 350 | to this License be limited to alteration of a) Section 13.8 solely to modify 351 | the legal Jurisdiction or Venue for disputes, b) EXHIBIT A solely to define 352 | License Notice text, or c) to EXHIBIT B solely to define a User-Visible 353 | Attribution Notice, You may continue to refer to Your License as the 354 | Reciprocal Public License or simply the RPL. 355 | 356 | 8.0 Disclaimer of Warranty. LICENSED SOFTWARE IS PROVIDED UNDER THIS LICENSE 357 | ON AN "AS IS" BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, 358 | INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT THE LICENSED SOFTWARE IS FREE 359 | OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE OR NON-INFRINGING. 360 | FURTHER THERE IS NO WARRANTY MADE AND ALL IMPLIED WARRANTIES ARE DISCLAIMED 361 | THAT THE LICENSED SOFTWARE MEETS OR COMPLIES WITH ANY DESCRIPTION OF 362 | PERFORMANCE OR OPERATION, SAID COMPATIBILITY AND SUITABILITY BEING YOUR 363 | RESPONSIBILITY. LICENSOR DISCLAIMS ANY WARRANTY, IMPLIED OR EXPRESSED, THAT 364 | ANY CONTRIBUTOR'S EXTENSIONS MEET ANY STANDARD OF COMPATIBILITY OR DESCRIPTION 365 | OF PERFORMANCE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE 366 | LICENSED SOFTWARE IS WITH YOU. SHOULD LICENSED SOFTWARE PROVE DEFECTIVE IN ANY 367 | RESPECT, YOU (AND NOT THE LICENSOR OR ANY OTHER CONTRIBUTOR) ASSUME THE COST 368 | OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. UNDER THE TERMS OF THIS 369 | LICENSOR WILL NOT SUPPORT THIS SOFTWARE AND IS UNDER NO OBLIGATION TO ISSUE 370 | UPDATES TO THIS SOFTWARE. LICENSOR HAS NO KNOWLEDGE OF ERRANT CODE OR VIRUS IN 371 | THIS SOFTWARE, BUT DOES NOT WARRANT THAT THE SOFTWARE IS FREE FROM SUCH ERRORS 372 | OR VIRUSES. THIS DISCLAIMER OF WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS 373 | LICENSE. NO USE OF LICENSED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS 374 | DISCLAIMER. 375 | 376 | 9.0 Limitation of Liability. UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, 377 | WHETHER TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL THE 378 | LICENSOR, ANY CONTRIBUTOR, OR ANY DISTRIBUTOR OF LICENSED SOFTWARE, OR ANY 379 | SUPPLIER OF ANY OF SUCH PARTIES, BE LIABLE TO ANY PERSON FOR ANY INDIRECT, 380 | SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY CHARACTER INCLUDING, 381 | WITHOUT LIMITATION, DAMAGES FOR LOSS OF GOODWILL, WORK STOPPAGE, COMPUTER 382 | FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER COMMERCIAL DAMAGES OR LOSSES, 383 | EVEN IF SUCH PARTY SHALL HAVE BEEN INFORMED OF THE POSSIBILITY OF SUCH 384 | DAMAGES. THIS LIMITATION OF LIABILITY SHALL NOT APPLY TO LIABILITY FOR DEATH 385 | OR PERSONAL INJURY RESULTING FROM SUCH PARTY'S NEGLIGENCE TO THE EXTENT 386 | APPLICABLE LAW PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO NOT ALLOW THE 387 | EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THIS 388 | EXCLUSION AND LIMITATION MAY NOT APPLY TO YOU. 389 | 390 | 10.0 High Risk Activities. THE LICENSED SOFTWARE IS NOT FAULT-TOLERANT AND IS 391 | NOT DESIGNED, MANUFACTURED, OR INTENDED FOR USE OR DISTRIBUTION AS ON-LINE 392 | CONTROL EQUIPMENT IN HAZARDOUS ENVIRONMENTS REQUIRING FAIL-SAFE PERFORMANCE, 393 | SUCH AS IN THE OPERATION OF NUCLEAR FACILITIES, AIRCRAFT NAVIGATION OR 394 | COMMUNICATIONS SYSTEMS, AIR TRAFFIC CONTROL, DIRECT LIFE SUPPORT MACHINES, OR 395 | WEAPONS SYSTEMS, IN WHICH THE FAILURE OF THE LICENSED SOFTWARE COULD LEAD 396 | DIRECTLY TO DEATH, PERSONAL INJURY, OR SEVERE PHYSICAL OR ENVIRONMENTAL DAMAGE 397 | ("HIGH RISK ACTIVITIES"). LICENSOR AND CONTRIBUTORS SPECIFICALLY DISCLAIM ANY 398 | EXPRESS OR IMPLIED WARRANTY OF FITNESS FOR HIGH RISK ACTIVITIES. 399 | 400 | 11.0 Responsibility for Claims. As between Licensor and Contributors, each 401 | party is responsible for claims and damages arising, directly or indirectly, 402 | out of its utilization of rights under this License which specifically 403 | disclaims warranties and limits any liability of the Licensor. This paragraph 404 | is to be used in conjunction with and controlled by the Disclaimer Of 405 | Warranties of Section 8, the Limitation Of Damages in Section 9, and the 406 | disclaimer against use for High Risk Activities in Section 10. The Licensor 407 | has thereby disclaimed all warranties and limited any damages that it is or 408 | may be liable for. You agree to work with Licensor and Contributors to 409 | distribute such responsibility on an equitable basis consistent with the terms 410 | of this License including Sections 8, 9, and 10. Nothing herein is intended or 411 | shall be deemed to constitute any admission of liability. 412 | 413 | 12.0 Termination. This License and all rights granted hereunder will terminate 414 | immediately in the event of the circumstances described in Section 13.6 or if 415 | applicable law prohibits or restricts You from fully and or specifically 416 | complying with Sections 3, 4 and/or 6, or prevents the enforceability of any 417 | of those Sections, and You must immediately discontinue any use of Licensed 418 | Software. 419 | 420 | 12.1 Automatic Termination Upon Breach. This License and the rights granted 421 | hereunder will terminate automatically if You fail to comply with the terms 422 | herein and fail to cure such breach within thirty (30) days of becoming aware 423 | of the breach. All sublicenses to the Licensed Software that are properly 424 | granted shall survive any termination of this License. Provisions that, by 425 | their nature, must remain in effect beyond the termination of this License, 426 | shall survive. 427 | 428 | 12.2 Termination Upon Assertion of Patent Infringement. If You initiate 429 | litigation by asserting a patent infringement claim (excluding declaratory 430 | judgment actions) against Licensor or a Contributor (Licensor or Contributor 431 | against whom You file such an action is referred to herein as "Respondent") 432 | alleging that Licensed Software directly or indirectly infringes any patent, 433 | then any and all rights granted by such Respondent to You under Sections 3 or 434 | 4 of this License shall terminate prospectively upon sixty (60) days notice 435 | from Respondent (the "Notice Period") unless within that Notice Period You 436 | either agree in writing (i) to pay Respondent a mutually agreeable reasonably 437 | royalty for Your past or future use of Licensed Software made by such 438 | Respondent, or (ii) withdraw Your litigation claim with respect to Licensed 439 | Software against such Respondent. If within said Notice Period a reasonable 440 | royalty and payment arrangement are not mutually agreed upon in writing by the 441 | parties or the litigation claim is not withdrawn, the rights granted by 442 | Licensor to You under Sections 3 and 4 automatically terminate at the 443 | expiration of said Notice Period. 444 | 445 | 12.3 Reasonable Value of This License. If You assert a patent infringement 446 | claim against Respondent alleging that Licensed Software directly or 447 | indirectly infringes any patent where such claim is resolved (such as by 448 | license or settlement) prior to the initiation of patent infringement 449 | litigation, then the reasonable value of the licenses granted by said 450 | Respondent under Sections 3 and 4 shall be taken into account in determining 451 | the amount or value of any payment or license. 452 | 453 | 12.4 No Retroactive Effect of Termination. In the event of termination under 454 | this Section all end user license agreements (excluding licenses to 455 | distributors and resellers) that have been validly granted by You or any 456 | distributor hereunder prior to termination shall survive termination. 457 | 458 | 13.0 Miscellaneous. 459 | 460 | 13.1 U.S. Government End Users. The Licensed Software is a "commercial item," 461 | as that term is defined in 48 C.F.R. 2.101 (Oct. 1995), consisting of 462 | "commercial computer software" and "commercial computer software 463 | documentation," as such terms are used in 48 C.F.R. 12.212 (Sept. 1995). 464 | Consistent with 48 C.F.R. 12.212 and 48 C.F.R. 227.7202-1 through 227.7202-4 465 | (June 1995), all U.S. Government End Users acquire Licensed Software with only 466 | those rights set forth herein. 467 | 468 | 13.2 Relationship of Parties. This License will not be construed as creating 469 | an agency, partnership, joint venture, or any other form of legal association 470 | between or among You, Licensor, or any Contributor, and You will not represent 471 | to the contrary, whether expressly, by implication, appearance, or otherwise. 472 | 473 | 13.3 Independent Development. Nothing in this License will impair Licensor's 474 | right to acquire, license, develop, subcontract, market, or distribute 475 | technology or products that perform the same or similar functions as, or 476 | otherwise compete with, Extensions that You may develop, produce, market, or 477 | distribute. 478 | 479 | 13.4 Consent To Breach Not Waiver. Failure by Licensor or Contributor to 480 | enforce any provision of this License will not be deemed a waiver of future enforcement 481 | of that or any other provision. 482 | 483 | 13.5 Severability. This License represents the complete agreement concerning 484 | the subject matter hereof. If any provision of this License is held to be 485 | unenforceable, such provision shall be reformed only to the extent necessary 486 | to make it enforceable. 487 | 488 | 13.6 Inability to Comply Due to Statute or Regulation. If it is impossible for 489 | You to comply with any of the terms of this License with respect to some or 490 | all of the Licensed Software due to statute, judicial order, or regulation, 491 | then You cannot use, modify, or distribute the software. 492 | 493 | 13.7 Export Restrictions. You may be restricted with respect to downloading or 494 | otherwise acquiring, exporting, or reexporting the Licensed Software or any 495 | underlying information or technology by United States and other applicable 496 | laws and regulations. By downloading or by otherwise obtaining the Licensed 497 | Software, You are agreeing to be responsible for compliance with all 498 | applicable laws and regulations. 499 | 500 | 13.8 Arbitration, Jurisdiction & Venue. This License shall be governed by 501 | Colorado law provisions (except to the extent applicable law, if any, provides 502 | otherwise), excluding its conflict-of-law provisions. You expressly agree that 503 | any dispute relating to this License shall be submitted to binding arbitration 504 | under the rules then prevailing of the American Arbitration Association. You 505 | further agree that Adams County, Colorado USA is proper venue and grant such 506 | arbitration proceeding jurisdiction as may be appropriate for purposes of 507 | resolving any dispute under this License. Judgement upon any award made in 508 | arbitration may be entered and enforced in any court of competent 509 | jurisdiction. The arbitrator shall award attorney's fees and costs of 510 | arbitration to the prevailing party. Should either party find it necessary to 511 | enforce its arbitration award or seek specific performance of such award in a 512 | civil court of competent jurisdiction, the prevailing party shall be entitled 513 | to reasonable attorney's fees and costs. The application of the United Nations 514 | Convention on Contracts for the International Sale of Goods is expressly 515 | excluded. You and Licensor expressly waive any rights to a jury trial in any 516 | litigation concerning Licensed Software or this License. Any law or regulation 517 | that provides that the language of a contract shall be construed against the 518 | drafter shall not apply to this License. 519 | 520 | 13.9 Entire Agreement. This License constitutes the entire agreement between 521 | the parties with respect to the subject matter hereof. 522 | 523 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # acrosync-library 2 | [Acrosync] is a new cross-platform rsync client for Windows/Max/iOS/Android that we built from scratch, without taking any code from the open source rsync project. 3 | 4 | This is the library behind [Acrosync], which implements a large portion of the client-side of the (undocumented) rsync protocol, including the famous delta sync algorithm. 5 | 6 | [PhotoBackup], our iOS app for uploading photos and videos to computers, uses a slightly modified version of this library. 7 | 8 | ## Features 9 | - Talks the rsync protocol version 29 (rsync 2.6.4+) and version 30 (rsync 3.x.x). 10 | - Written in C++ and builds on Win32, Mac OS X, Linux, iOS, and Android. 11 | - The only dependencies are libssh2 and openssl. 12 | - Can connect to the rsync server either via ssh, or via the rsync daemon protocol. 13 | - For ssh connections, supports both password login and public key authentication (with or without a passphrase). 14 | - Symbolic links are supported. 15 | 16 | ## Build Instructions 17 | 18 | First you'll need to install openssl and libssh2. Assume we're on a linux machine and these two packages are already installed in the default locations. 19 | 20 | Run the following command to build the test programs: 21 | ```sh 22 | $ python qi/qi-make.py test 23 | ``` 24 | You will find a new executable *build-linux/rsync/t_rsync_client* which can download a remote directory using the rsync over ssh protocol: 25 | 26 | ```sh 27 | $ build-linux/rsync/t_rsync_client 28 | ``` 29 | The same build command works for Mac and Windows, but you will need to install openssl and libssh2 to a subdirectory named *install-mac* or *install-win* under the top level directory. 30 | 31 | Here the build system is written with [Qi-Make], a tool that we developed by extending the basic syntax of make. The file *qi/qi_build.h* contains actual rules for building intermediate objects and final test programs. It should be fairly easy to make changes for your own build environments. 32 | 33 | ## License 34 | 35 | This library is licensed under the Reciprocal Public License. If this license does not work for you, a commercial license is available for a one-time fee or on a subscription basis. Contact for licensing details. 36 | 37 | [acrosync]:https://acrosync.com 38 | [PhotoBackup]:https://acrosync.com/photobackup.html 39 | [Qi-Make]:https://code.google.com/p/qi-make/ 40 | -------------------------------------------------------------------------------- /block/t_block_out.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include 4 | 5 | struct UserType1 6 | { 7 | int d_i; 8 | double d_d; 9 | }; 10 | 11 | class UserType2 12 | { 13 | int d_i; 14 | double d_d; 15 | public: 16 | UserType2() {} 17 | ~UserType2() {} 18 | 19 | }; 20 | 21 | class Source 22 | { 23 | public: 24 | block::out out0; 25 | block::out out1; 26 | block::out out2; 27 | block::out out3; 28 | block::out out4; 29 | block::out out5; 30 | block::out out6; 31 | block::out out7; 32 | block::out out8; 33 | 34 | // not connected 35 | block::out out100; 36 | block::out out101; 37 | block::out out102; 38 | block::out out103; 39 | block::out out104; 40 | block::out out105; 41 | 42 | // connected to mulitple inports 43 | 44 | block::out out200; 45 | block::out out201; 46 | block::out out202; 47 | block::out out203; 48 | block::out out204; 49 | block::out out205; 50 | 51 | void Run() 52 | { 53 | out0(); 54 | out1("Hello World"); 55 | out2("Hello World", 1); 56 | out3("Hello World", 1, '2'); 57 | out4("Hello World", 1, '2', 3.0f); 58 | out5("Hello World", 1, '2', 3.0f, 4.0); 59 | out6("Hello World", 1, '2', 3.0f, 4.0, 0); 60 | out7("Hello World", 1, '2', 3.0f, 4.0, 0, UserType1()); 61 | out8("Hello World", 1, '2', 3.0f, 4.0, 0, UserType1(), UserType2()); 62 | 63 | int i = 1; 64 | double d = 2.0; 65 | out100(1, 2.0); 66 | out101(1, 2.0); 67 | out102(1, 2.0); 68 | //out103(1, 2.0); 69 | out104(i, d); 70 | out105(i, d); 71 | 72 | out200(1, 2.0); 73 | out201(1, 2.0); 74 | out202(1, 2.0); 75 | out203(1, 2.0); 76 | out204(i, d); 77 | out205(i, d); 78 | } 79 | }; 80 | 81 | struct Dummy 82 | { 83 | int d_i; 84 | int d_double; 85 | }; 86 | 87 | class Sink : public Dummy 88 | { 89 | private: 90 | mutable int d_numCalls; 91 | public: 92 | 93 | Sink() 94 | : d_numCalls(0) 95 | { 96 | } 97 | 98 | int numCalls() const { return d_numCalls; } 99 | 100 | void in0() 101 | { 102 | ++d_numCalls; 103 | } 104 | void in1(const char*) const 105 | { 106 | ++d_numCalls; 107 | } 108 | void in2(const char*, int) 109 | { 110 | ++d_numCalls; 111 | } 112 | void in3(const char*, int, char) const 113 | { 114 | ++d_numCalls; 115 | } 116 | void in4(const char*, int, char, float) 117 | { 118 | ++d_numCalls; 119 | } 120 | void in5(const char*, int, char, float, double) const 121 | { 122 | ++d_numCalls; 123 | } 124 | void in6(const char*, int, char, float, double, void *) 125 | { 126 | ++d_numCalls; 127 | } 128 | void in7(const char*, int, char, float, double, void *, UserType1) const 129 | { 130 | ++d_numCalls; 131 | } 132 | void in8(const char*, int, char, float, double, void *, UserType1, UserType2) 133 | { 134 | ++d_numCalls; 135 | } 136 | 137 | void in200(int, double) 138 | { 139 | ++d_numCalls; 140 | } 141 | void in201(const int&, double) 142 | { 143 | ++d_numCalls; 144 | } 145 | void in202(int, const double&) 146 | { 147 | ++d_numCalls; 148 | } 149 | int in203(int, double) 150 | { 151 | ++d_numCalls; 152 | return d_numCalls; 153 | } 154 | void in204(int&, double) 155 | { 156 | ++d_numCalls; 157 | } 158 | void in205(int, double&) 159 | { 160 | ++d_numCalls; 161 | } 162 | }; 163 | 164 | int main(int argc, char *argv[]) 165 | { 166 | { 167 | Source source; 168 | Sink sink; 169 | 170 | source.out0.connect(&sink, &Sink::in0); 171 | source.out1.connect(&sink, &Sink::in1); 172 | source.out2.connect(&sink, &Sink::in2); 173 | source.out3.connect(&sink, &Sink::in3); 174 | source.out4.connect(&sink, &Sink::in4); 175 | source.out5.connect(&sink, &Sink::in5); 176 | source.out6.connect(&sink, &Sink::in6); 177 | source.out7.connect(&sink, &Sink::in7); 178 | source.out8.connect(&sink, &Sink::in8); 179 | 180 | source.out200.connect(&sink, &Sink::in200); 181 | source.out200.connect(&sink, &Sink::in200); 182 | source.out201.connect(&sink, &Sink::in201); 183 | source.out201.connect(&sink, &Sink::in201); 184 | source.out202.connect(&sink, &Sink::in202); 185 | source.out202.connect(&sink, &Sink::in202); 186 | source.out203.connect(&sink, &Sink::in203); 187 | source.out203.connect(&sink, &Sink::in203); 188 | source.out204.connect(&sink, &Sink::in204); 189 | source.out204.connect(&sink, &Sink::in204); 190 | source.out205.connect(&sink, &Sink::in205); 191 | source.out205.connect(&sink, &Sink::in205); 192 | 193 | source.Run(); 194 | 195 | printf("Member functions of Sink have been called %d times\n", sink.numCalls()); 196 | } 197 | 198 | return 0; 199 | } 200 | -------------------------------------------------------------------------------- /qi.prj: -------------------------------------------------------------------------------- 1 | [Source Files] 2 | rsync/rsync_client.cpp 3 | rsync/rsync_entry.cpp 4 | rsync/rsync_file.cpp 5 | rsync/rsync_io.cpp 6 | rsync/rsync_log.cpp 7 | rsync/rsync_pathutil.cpp 8 | rsync/rsync_socketutil.cpp 9 | rsync/rsync_sshio.cpp 10 | rsync/rsync_stream.cpp 11 | rsync/rsync_timeutil.cpp 12 | rsync/rsync_util.cpp 13 | rsync/t_rsync_client.cpp 14 | rsync/t_rsync_entry.cpp 15 | rsync/t_rsync_fileutil.cpp 16 | rsync/t_rsync_stream.cpp 17 | [Initialization Code] 18 | [Finalization Code] 19 | [User-Defined Functions] 20 | -------------------------------------------------------------------------------- /qi/qi_build.h: -------------------------------------------------------------------------------- 1 | /*qi:begin 2 | 3 | 4 | OSTYPE = $(platform system) 5 | if $OSTYPE == Windows 6 | build_dir = build-win 7 | ext = .exe 8 | 9 | CXX = cl 10 | CXXFLAGS += /c /I. /I install-win/include /DWIN32 /EHsc /MD 11 | LDFLAGS += /NODEFAULTLIB:libcmtd.lib /LIBPATH:install-win/lib crypt32.lib libeay32.lib libssh2.lib ws2_32.lib advapi32.lib shell32.lib user32.lib gdi32.lib 12 | else 13 | if $OSTYPE == Darwin 14 | build_dir = build-mac 15 | ext = 16 | 17 | CXX = g++ 18 | CXXFLAGS += -g -I. -Iinstall-mac/include -c -D_FILE_OFFSET_BITS=64 -std=c++0x 19 | LDFLAGS += -Linstall-mac/lib -lstdc++ -lssh2 -lcrypto -liconv -lz 20 | else 21 | build_dir = build-linux 22 | ext = 23 | 24 | CXX = g++ 25 | CXXFLAGS += -g -I. -c -D_FILE_OFFSET_BITS=64 -std=c++0x 26 | LDFLAGS += -static-libgcc -lstdc++ -lssh2 -lcrypto 27 | endif 28 | endif 29 | 30 | 31 | base_name = $(match ([^.]+) $source) 32 | base_dir = $(dir $build_dir/$base_name) 33 | $(mkdir $base_dir) 34 | 35 | compile($(join $build_dir $base_name.o)): $source 36 | if $OSTYPE == Windows 37 | $CXX $CXXFLAGS /Fo$compile($source) $source 38 | else 39 | $CXX $CXXFLAGS -o $compile($source) $source 40 | endif 41 | 42 | clean: 43 | rm -f $compile($source) 44 | 45 | ifdef TEST_PROGRAM 46 | 47 | link_depends := $(link_depends $source) 48 | 49 | link($build_dir/$base_name$ext): compile($link_depends) 50 | if $OSTYPE == Windows 51 | $CXX /Fe$link($source) $compile($link_depends) /link $LDFLAGS 52 | else 53 | $CXX -o $link($source) $compile($link_depends) $LDFLAGS 54 | endif 55 | 56 | test: link 57 | ./$link($source) 58 | 59 | clean: 60 | rm -f $link($source) 61 | 62 | endif 63 | 64 | 65 | qi:end*/ 66 | -------------------------------------------------------------------------------- /rsync/rsync_client.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_CLIENT_H 17 | #define INCLUDED_RSYNC_CLIENT_H 18 | 19 | #include 20 | #include 21 | #include 22 | 23 | #include 24 | 25 | #include 26 | #include 27 | #include 28 | 29 | namespace rsync 30 | { 31 | 32 | class IO; 33 | class Entry; 34 | 35 | class Client 36 | { 37 | public: 38 | // Create an rsync client on top of the io channel 'io' that has already been connected to the rsync server. 39 | // 'rsyncCommand' is the path of the rsync executable on the server. 'preferredProtocol' is either 29 or 30. 40 | // 'd_cancelFlagAddress' points to flag whose value change will abort the sync operation immediately. 41 | Client(IO*io, const char *rsyncCommand, int preferredProtocol, int *cancelFlagAddress); 42 | 43 | // Destructor. Does not delete the io channel so it can be reused. 44 | ~Client(); 45 | 46 | // List the remote directory 'path' (not recursively). Each item will be returned by invoking the callback 47 | // 'entryOut' defined below. 48 | bool list(const char *path); 49 | 50 | // Perform a full recursive sync (if 'includedFiles' is 0), or a selective sync (of files specified by 51 | // 'includedFiles'), from the remote directory 'remoteTop' to the local directory 'localTop'. For each file to 52 | // be downloaded, the file will be written to 'temporaryFile' first and then moved to its destination after 53 | // each file transfer. The names of all downloaded and deleted files can be retrieved by calling 54 | // 'getUpdatedFiles()' and 'getDeletedFiles()'. 55 | int download(const char *localTop, const char *remoteTop, const char *temporaryFile, 56 | const std::set *includeFiles = 0); 57 | 58 | // Perform a sync, recursive or non-recursive, from the local directory 'localPath' to the remote directory 59 | // 'remoteTop'. If 'includeFiles' is specified, sync only those files contained in 'includedFiles'. 60 | int upload(const char *localPath, const char *remoteTop, const std::set *includeFiles = 0); 61 | 62 | // Remove the file or directory specified by 'remotePath' on the server. 63 | void remove(const char *remotePath); 64 | 65 | // Create a new directory 'remotePath' on the server. 66 | void mkdir(const char *remotePath); 67 | 68 | // Create a symbolic link with the name 'link' that points to 'remotePath' under the same directory. 69 | void link(const char *remotePath, const char *link); 70 | 71 | // List all modules provided by the rsync daemon server. Used only in rsync daemon mode. 72 | void listModules(); 73 | 74 | // Add a remote directory 'backupPath' to be used as the parameter for the '--link-dest' option. Multiple 75 | // backup paths are allowed. 76 | void addBackupPath(const char *backupPath); 77 | 78 | // Remove all backup paths. 79 | void clearBackupPaths(); 80 | 81 | // Set the download and upload limits. The unit is Kilobytes per second. 82 | void setSpeedLimits(int downloadLimit, int uploadLimit); 83 | 84 | // If 'deletionEnabled' is true, files or dirs that do not exist on the other side will be removed. 85 | void setDeletionEnabled(bool deletionEnabled); 86 | 87 | // Statistics that will be updated while the sync is in progress. 88 | // '*totalBytes': the total bytes of all bytes in the source directory 89 | // '*physicalBytes': the number of bytes that have been transmitted by the network 90 | // '*logicalBytes': the number of bytes that have been synced. This value may be greater than '*physicalBytes' 91 | // when the rdiff algorithm is in effect. 92 | // '*skippedBytes': the bytes of files that do not need to be synced. 93 | void setStatsAddresses(int64_t *totalBytes, int64_t *physicalBytes, int64_t *logicalBytes, int64_t *skippedBytes); 94 | 95 | // Think this as a callback. The function connected to it will be called when the info about an file/dir entry 96 | // is about to be sent (for instance, during the 'list' call) 97 | block::out entryOut; 99 | 100 | // Used to output a status message to indicate the sync progress. 101 | block::out statusOut; 102 | 103 | // Return the files that have been created or modified after the sync is complete. Does not include any 104 | // directories. 105 | const std::vector &getUpdatedFiles(); 106 | 107 | // Return the files that have been deleted by the sync operation. Does not include any directories. 108 | const std::vector &getDeletedFiles(); 109 | 110 | private: 111 | // NOT IMPLEMENTED 112 | Client(const Client&); 113 | Client& operator=(const Client&); 114 | 115 | // Resize the size of 'd_chunk'. 116 | void resizeChunk(int size); 117 | 118 | // Send a series of checksums calcuated from the base file 'oldFile' for the file with the specifed 'index'. 119 | void sendChecksum(int index, const char *oldFile); 120 | 121 | // Send an entry to the remote server. 122 | void sendEntry(Entry *entry, bool isTop, bool noDirContent = false); 123 | 124 | // Send a file to the remote server. 125 | bool sendFile(int index, const char *remotePath, const char *localPath); 126 | 127 | // Receive an entry from the remote server. 128 | bool receiveEntry(std::string *path, bool *isDir, int64_t *size, int64_t *time, uint32_t *mode, 129 | std::string *symlink); 130 | 131 | // Receive a file from the remote server. 132 | bool receiveFile(const char *remotePath, const char *newFile, const char *oldFile, int64_t *fileSize); 133 | 134 | // Start a new rsync session. 135 | void start(const char *remotePath, bool isDownloading, bool recursive, bool isDeleting); 136 | 137 | // Close the rsync session. 138 | void stop(); 139 | 140 | // Read an entry index. 141 | int readIndex(); 142 | 143 | // Write an entry index. 144 | void writeIndex(int index); 145 | 146 | bool d_usingSSH; // whether the remote server is an ssh server or an rsync daemon 147 | IO *d_io; // io channel to the remote server 148 | Stream *d_stream; // data stream on top of 'd_io' 149 | 150 | std::string d_rsyncCommand; // the path to the rsync executable on the server 151 | 152 | int d_downloadLimit; // maximum download speed in kiloBytes/sec 153 | bool d_deletionEnabled; // whether to propagate deletions 154 | 155 | std::vector d_includePatterns; // include patterns 156 | std::vector d_excludePatterns; // exclude patterns 157 | 158 | std::vector d_backupPaths; // paths of previous backups; used by the '--link-desk' option 159 | 160 | int32_t d_protocol; // rsync protocol version 161 | int32_t d_checksumSeed; // the seed for checksum calculation 162 | int64_t d_lastEntryTime; // the modified time of the last entry transmitted 163 | uint32_t d_lastEntryMode; // the file mode of the last entry transmitted 164 | std::string d_lastEntryPath; // the path of the last entry transmitted 165 | 166 | struct Checksum 167 | { 168 | uint32_t d_sum1; // the rolling checksum of each block 169 | uint32_t d_next; // the next pointer for making a hash table of checksum 170 | char d_sum2[16]; // the MD5 checksum of of each block 171 | }; 172 | 173 | int d_numberOfChecksums; // the number of checksums for the current file transfer 174 | 175 | int* d_hashBuckets; // the entries to the checksum hash table 176 | Checksum *d_checksums; // store all the checksum 177 | 178 | char *d_chunk; // a chunk buffer used to send or receive file content 179 | int d_chunkSize; // the size of 'd_chunk' 180 | 181 | int64_t *d_totalBytes; // the total bytes of all bytes in the source directory 182 | int64_t *d_physicalBytes; // the number of bytes that have been transmitted by the network 183 | int64_t *d_logicalBytes; // the number of bytes that have been synced 184 | int64_t *d_skippedBytes; // the number of bytes that are the same on the both sides. 185 | 186 | int64_t d_dummyCounter; // used to initialize the above stats counter if they are not being used. 187 | 188 | std::vector d_deletedFiles; // files deleted by the current sync 189 | std::vector d_updatedFiles; // files created or modified by the current sync 190 | 191 | }; 192 | 193 | 194 | } // namespace rsync 195 | 196 | #endif // INCLUDED_RSYNC_CLIENT_H 197 | -------------------------------------------------------------------------------- /rsync/rsync_entry.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | 20 | #include 21 | #include 22 | 23 | #include 24 | 25 | namespace rsync 26 | { 27 | 28 | bool Entry::compareLocally(const Entry* lhs, const Entry* rhs) 29 | { 30 | // Directories are always arranged before files. If both are of the same kind, compare them 31 | // as strings. 32 | if (lhs->isDirectory()) { 33 | if (rhs->isDirectory()) { 34 | return lhs->d_path < rhs->d_path; 35 | } else { 36 | return false; 37 | } 38 | } else { 39 | if (rhs->isDirectory()) { 40 | return true; 41 | } else { 42 | return lhs->d_path < rhs->d_path; 43 | } 44 | } 45 | } 46 | 47 | bool Entry::compareByLocalName(const std::string &lhs, const std::string &rhs) 48 | { 49 | // Directories are always arranged before files. If both are of the same kind, compare them 50 | // as strings. 51 | if (*lhs.rbegin() == '/') { 52 | if (*rhs.rbegin() == '/') { 53 | return lhs < rhs; 54 | } else { 55 | return false; 56 | } 57 | } else { 58 | if (*rhs.rbegin() == '/') { 59 | return true; 60 | } else { 61 | return lhs < rhs; 62 | } 63 | } 64 | } 65 | 66 | bool Entry::compareGlobally(const Entry* lhs, const Entry* rhs) 67 | { 68 | // If two entries lie within the same directory, compare their local names. Otherwise, compare their directories. 69 | 70 | const unsigned char *p1 = reinterpret_cast(lhs->getPath()); 71 | const unsigned char *p2 = reinterpret_cast(rhs->getPath()); 72 | 73 | while (*p1 == *p2) { 74 | if (*p1 == 0) { 75 | return false; 76 | } 77 | ++p1; 78 | ++p2; 79 | } 80 | 81 | const unsigned char *p3 = p1; 82 | for (; *p3 != 0 && *p3 != '/'; ++p3) {} 83 | const unsigned char *p4 = p2; 84 | for (; *p4 != 0 && *p4 != '/'; ++p4) {} 85 | 86 | if (*p1 < *p2) { 87 | if (*p1 == 0 && *(p1 - 1) == '/') { 88 | return true; 89 | } 90 | if (*p3 == '/' && *p4 != '/') { 91 | return false; 92 | } else { 93 | return true; 94 | } 95 | } else { 96 | if (*p2 == 0 && *(p2 - 1) == '/') { 97 | return false; 98 | } 99 | if (*p4 == '/' && *p3 != '/') { 100 | return true; 101 | } else { 102 | return false; 103 | } 104 | } 105 | } 106 | 107 | void Entry::normalizePath() 108 | { 109 | if (isDirectory() && d_path[d_path.size() - 1] != '/') { 110 | d_path.append("/"); 111 | } 112 | } 113 | 114 | bool Entry::contains(const Entry* other) const 115 | { 116 | return other->d_path.compare(0, d_path.size(), d_path) == 0; 117 | } 118 | 119 | bool Entry::isOlderThan(const Entry &other) const 120 | { 121 | #ifdef WIN32 122 | if (other.d_time > 1426620844) { 123 | return false; 124 | } 125 | #endif 126 | return d_time + 1 < other.d_time; 127 | } 128 | 129 | std::string Entry::getLocalName() const 130 | { 131 | size_t begin = d_path.rfind('/'); 132 | if (begin == d_path.size() - 1) { 133 | begin = d_path.rfind('/', begin - 1); 134 | } 135 | if (begin == std::string::npos) { 136 | return d_path; 137 | } else { 138 | return d_path.substr(begin + 1); 139 | } 140 | } 141 | 142 | } // namespace rsync 143 | -------------------------------------------------------------------------------- /rsync/rsync_entry.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_ENTRY_H 17 | #define INCLUDED_RSYNC_ENTRY_H 18 | 19 | #include 20 | 21 | #include 22 | 23 | #include 24 | 25 | namespace rsync 26 | { 27 | 28 | // This class encapsulates info about a file/dir, such as file name, file size, modified time, and mode. 29 | class Entry 30 | { 31 | public: 32 | 33 | // Constants for file mode 34 | enum { 35 | IS_FILE = 0100000, /* S_IFREG */ 36 | IS_DIR = 0040000, /* S_IFDIR */ 37 | IS_READABLE = 0000400, /* S_IRUSR */ 38 | IS_WRITABLE = 0000200, /* S_IRUSR */ 39 | IS_EXECUTABLE = 0000100, /* S_IRUSR */ 40 | IS_LINK = 0020000, /* S_IFLNK */ 41 | IS_ALL_READABLE = 000444, 42 | IS_ALL_EXECUTABLE = 000111 43 | }; 44 | 45 | // Compare two entries according to their paths, assuming that they are under the same directory. 46 | static bool compareLocally(const Entry*, const Entry*); 47 | 48 | // Compare two entries according to their full paths. 49 | static bool compareGlobally(const Entry*, const Entry*); 50 | 51 | // Another version of 'compareLocally' that takes paths rather than entries. 52 | static bool compareByLocalName(const std::string &lhs, const std::string &rhs); 53 | 54 | // If the entry is a directory, make sure it has the trailing '/'. 55 | void normalizePath(); 56 | 57 | // To determine if 'other' is a file or directory under the directory represented by this entry 58 | bool contains(const Entry* other) const; 59 | 60 | // Retrun 'true' if this entry is older than 'other' 61 | bool isOlderThan(const Entry &other) const; 62 | 63 | // Return the last path component 64 | std::string getLocalName() const; 65 | 66 | // Create a new entry 67 | Entry(const char *path, bool isDirectory, int64_t size, int64_t time, uint32_t mode) 68 | : d_path(path) 69 | , d_size(size) 70 | , d_time(time) 71 | , d_mode(mode) 72 | , d_symlink(0) 73 | , d_data(0) 74 | { 75 | if (isDirectory) { 76 | d_size = 0; 77 | d_mode |= IS_DIR; 78 | } 79 | } 80 | 81 | ~Entry() 82 | { 83 | delete d_symlink; 84 | } 85 | 86 | Entry(const Entry& other) 87 | : d_path(other.d_path) 88 | , d_size(other.d_size) 89 | , d_time(other.d_time) 90 | , d_mode(other.d_mode) 91 | , d_symlink(0) 92 | , d_data(0) 93 | { 94 | } 95 | 96 | 97 | const char *getPath() const 98 | { 99 | return d_path.c_str(); 100 | } 101 | 102 | bool isDirectory() const 103 | { 104 | return (d_mode & IS_DIR) && !(d_mode & IS_FILE); 105 | } 106 | 107 | void setDirectory() 108 | { 109 | d_mode |= IS_DIR; 110 | d_mode &= ~IS_FILE; 111 | } 112 | 113 | int64_t getSize() const 114 | { 115 | return d_size; 116 | } 117 | 118 | int64_t getTime() const 119 | { 120 | return d_time; 121 | } 122 | 123 | bool isReadable() const 124 | { 125 | return d_mode & IS_READABLE; 126 | } 127 | 128 | bool isLink() const 129 | { 130 | return (d_mode & IS_LINK) && (d_mode & IS_FILE); 131 | } 132 | 133 | bool isRegular() const 134 | { 135 | return (d_mode & IS_FILE) && !(d_mode & IS_DIR); 136 | } 137 | 138 | int32_t getMode() const 139 | { 140 | return d_mode; 141 | } 142 | 143 | void addSize(int64_t size) 144 | { 145 | d_size += size; 146 | } 147 | 148 | void setSymlink(const std::string& symlink) 149 | { 150 | delete d_symlink; 151 | d_symlink = new std::string(symlink); 152 | } 153 | 154 | const char *getSymlink() const 155 | { 156 | return d_symlink->c_str(); 157 | } 158 | 159 | int getSymlinkLength() const 160 | { 161 | return static_cast(d_symlink->size()); 162 | } 163 | 164 | void *getData() const 165 | { 166 | return d_data; 167 | } 168 | 169 | void setData(void *data) { 170 | d_data = data; 171 | } 172 | 173 | private: 174 | // NOT IMPLEMENTED 175 | Entry& operator=(const Entry&); 176 | 177 | std::string d_path; // path of the entry (usually relative to the top directory) 178 | int64_t d_size; // file size; could be 0 for a directory 179 | int64_t d_time; // last modified time 180 | uint32_t d_mode; // file node 181 | std::string *d_symlink; // contains the symbolic link; is NULL if the entry is not a symlink 182 | 183 | void *d_data; // A generic field usually used to store a pointer 184 | }; 185 | 186 | } // namespace rsync 187 | #endif //INCLUDED_RSYNC_ENTRY_H 188 | -------------------------------------------------------------------------------- /rsync/rsync_file.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | #include 23 | 24 | #include 25 | #include 26 | #include 27 | #include 28 | 29 | #if defined(WIN32) || defined(__MINGW32__) 30 | 31 | #include 32 | 33 | namespace rsync 34 | { 35 | 36 | File::Handle File::InvalidHandle = INVALID_HANDLE_VALUE; 37 | 38 | File::File() 39 | : d_handle(InvalidHandle) 40 | , d_path("") 41 | { 42 | } 43 | 44 | File::File(const char *fullPath, bool forWrite, bool reportError) 45 | : d_handle(InvalidHandle) 46 | , d_path(fullPath) 47 | { 48 | open(fullPath, forWrite, reportError); 49 | } 50 | 51 | bool File::open(const char *fullPath, bool forWrite, bool reportError) 52 | { 53 | d_path = fullPath; 54 | std::basic_string path = PathUtil::convertToUTF16(fullPath); 55 | d_handle = CreateFileW(path.c_str(), forWrite ? GENERIC_WRITE : GENERIC_READ, 56 | FILE_SHARE_READ, 0, forWrite ? CREATE_ALWAYS : OPEN_EXISTING, 57 | FILE_ATTRIBUTE_NORMAL, 0); 58 | if (d_handle == InvalidHandle) { 59 | if (reportError) { 60 | LOG_ERROR(FILE_OPEN) << "Failed to open '" << fullPath << "': " 61 | << Util::getLastError() << LOG_END 62 | } 63 | return false; 64 | } 65 | return true; 66 | } 67 | 68 | File::~File() 69 | { 70 | this->close(); 71 | } 72 | 73 | 74 | int File::read(char *buffer, int size) 75 | { 76 | if (d_handle == InvalidHandle) { 77 | return 0; 78 | } 79 | DWORD bytes = 0; 80 | if (!ReadFile(d_handle, buffer, size, &bytes, NULL)) { 81 | LOG_ERROR(FILE_READ) << "Failed to read " << size << " bytes from '" << d_path << "': " 82 | << Util::getLastError() << LOG_END 83 | this->close(); 84 | } 85 | return bytes; 86 | } 87 | 88 | int File::write(const char *buffer, int size) 89 | { 90 | if (d_handle == InvalidHandle) { 91 | return 0; 92 | } 93 | DWORD bytes = 0; 94 | if (!WriteFile(d_handle, buffer, size, &bytes, NULL)) { 95 | LOG_ERROR(FILE_WRITE) << "Failed to write " << size << " bytes to '" << d_path << "': " 96 | << Util::getLastError() << LOG_END 97 | this->close(); 98 | } 99 | return bytes; 100 | } 101 | 102 | int64_t File::seek(int64_t offset, int method) 103 | { 104 | if (d_handle == InvalidHandle) { 105 | return false; 106 | } 107 | 108 | int moveMethods[] = { FILE_BEGIN, FILE_CURRENT, FILE_END }; 109 | LONG distanceToMoveHigh = offset >> 32; 110 | 111 | int64_t result = SetFilePointer(d_handle, offset & 0xffffffff, &distanceToMoveHigh, moveMethods[method]); 112 | if (result == INVALID_SET_FILE_POINTER && GetLastError() != NO_ERROR) { 113 | LOG_ERROR(FILE_SEEK) << "Failed to seek " << offset << " bytes by method " << method 114 | << " in '" << d_path << "': " << Util::getLastError() << LOG_END 115 | this->close(); 116 | return 0; 117 | } 118 | return result | (static_cast(distanceToMoveHigh) << 32); 119 | } 120 | 121 | void File::close() 122 | { 123 | if (d_handle == InvalidHandle) { 124 | return; 125 | } 126 | CloseHandle(d_handle); 127 | d_handle = InvalidHandle; 128 | } 129 | 130 | } // close namespace rsync 131 | 132 | #else // for unix/linux/mac 133 | 134 | #include 135 | #include 136 | #include 137 | #include 138 | #include 139 | 140 | namespace rsync 141 | { 142 | 143 | File::Handle File::InvalidHandle = -1; 144 | 145 | File::File() 146 | : d_handle(InvalidHandle) 147 | , d_path("") 148 | { 149 | } 150 | 151 | File::File(const char *fullPath, bool forWrite, bool reportError) 152 | : d_handle(InvalidHandle) 153 | , d_path(fullPath) 154 | { 155 | open(fullPath, forWrite, reportError); 156 | } 157 | 158 | bool File::open(const char *fullPath, bool forWrite, bool reportError) 159 | { 160 | d_path = fullPath; 161 | if (forWrite) { 162 | d_handle = ::open(fullPath, O_WRONLY | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR); 163 | } else { 164 | d_handle = ::open(fullPath, O_RDONLY); 165 | } 166 | 167 | if (d_handle == InvalidHandle && reportError) { 168 | LOG_ERROR(RSYNC_OPEN) << "Failed to open '" << fullPath << "': " << strerror(errno) << LOG_END 169 | } 170 | return d_handle != InvalidHandle; 171 | } 172 | 173 | File::~File() 174 | { 175 | this->close(); 176 | } 177 | 178 | int File::read(char *buffer, int size) 179 | { 180 | if (d_handle == InvalidHandle) { 181 | return 0; 182 | } 183 | int rc = static_cast(::read(d_handle, buffer, size)); 184 | if (rc == -1) { 185 | LOG_ERROR(RSYNC_FILE) << "Error reading from '" << d_path << "': " << strerror(errno) << LOG_END 186 | return 0; 187 | } 188 | return rc; 189 | } 190 | 191 | int File::write(const char *buffer, int size) 192 | { 193 | if (d_handle == InvalidHandle) { 194 | return 0; 195 | } 196 | int bytes = 0; 197 | while (bytes < size) { 198 | int rc = static_cast(::write(d_handle, buffer + bytes, size - bytes)); 199 | if (rc == 0) { 200 | LOG_ERROR(RSYNC_FILE) << "Failed to write to '" << d_path << "'" << LOG_END 201 | return bytes; 202 | } else if (rc == -1) { 203 | LOG_ERROR(RSYNC_FILE) << "Error writing '" << d_path << "': " << strerror(errno) << LOG_END 204 | return bytes; 205 | } 206 | bytes += rc; 207 | } 208 | return bytes; 209 | } 210 | 211 | int64_t File::seek(int64_t offset, int method) 212 | { 213 | if (d_handle == InvalidHandle) { 214 | return -1; 215 | } 216 | #ifdef __APPLE__ 217 | return ::lseek(d_handle, offset, method); 218 | #else 219 | return ::lseek64(d_handle, offset, method); 220 | #endif 221 | } 222 | 223 | void File::close() 224 | { 225 | if (d_handle == InvalidHandle) { 226 | return; 227 | } 228 | ::close(d_handle); 229 | } 230 | 231 | } // close namespace rsync 232 | 233 | #endif // defined(WIN32) || defined(__MINGW32__) 234 | -------------------------------------------------------------------------------- /rsync/rsync_file.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_FILE_H 17 | #define INCLUDED_RSYNC_FILE_H 18 | 19 | #include 20 | 21 | #include 22 | 23 | // This class represents a file object to be opened, read, or written. 24 | namespace rsync 25 | { 26 | 27 | class File 28 | { 29 | public: 30 | 31 | #if defined(WIN32) || defined(__MINGW32__) 32 | typedef void * Handle; 33 | static Handle InvalidHandle; 34 | 35 | enum { SEEK_FROM_BEGIN, SEEK_FROM_CURRENT, SEEK_FROM_END }; 36 | 37 | #else 38 | typedef int Handle; 39 | static Handle InvalidHandle; 40 | 41 | enum { SEEK_FROM_BEGIN = SEEK_SET, 42 | SEEK_FROM_CURRENT = SEEK_CUR, 43 | SEEK_FROM_END = SEEK_END }; 44 | #endif 45 | 46 | // Open a file for read or write. If an error occurs, report it in log if 'reportError' is true. 47 | File(const char *fullPath, bool forWrite = false, bool reportError = false); 48 | 49 | // Default constructor creates an invalid file object. 50 | File(); 51 | 52 | // Close the file. 53 | ~File(); 54 | 55 | // Open a file for read or write. If an error occurs, report it in log if 'reportError' is true. 56 | bool open(const char *fullPath, bool forWrite, bool reportError); 57 | 58 | // Read 'size' bytes from the file into 'buffer'. 59 | int read(char *buffer, int size); 60 | 61 | // Write 'size' bytes from 'buffer' into the file. 62 | int write(const char *buffer, int size); 63 | 64 | // Move the current read/write position. 65 | int64_t seek(int64_t offset, int method); 66 | 67 | // Close the file 68 | void close(); 69 | 70 | // If an error has occurred. 71 | bool isValid() const 72 | { 73 | return d_handle != InvalidHandle; 74 | } 75 | 76 | private: 77 | // NOT IMPLEMENTED 78 | File(const File&); 79 | File operator=(const File&); 80 | 81 | Handle d_handle; 82 | std::string d_path; 83 | }; 84 | 85 | } // close namespace rsync 86 | 87 | #endif // INCLUDED_RSYNC_FILE_H 88 | 89 | -------------------------------------------------------------------------------- /rsync/rsync_io.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | 20 | namespace rsync 21 | { 22 | 23 | IO::IO() 24 | { 25 | } 26 | 27 | IO::~IO() 28 | { 29 | } 30 | 31 | } // namespace rsync 32 | -------------------------------------------------------------------------------- /rsync/rsync_io.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_IO_H 17 | #define INCLUDED_RSYNC_IO_H 18 | 19 | #include 20 | 21 | namespace rsync 22 | { 23 | 24 | // This class defines an interface for sending and receiving data through a network channel (which can be a plain 25 | // socket or an SSH channel) 26 | class IO 27 | { 28 | public: 29 | IO(); 30 | virtual ~IO(); 31 | 32 | // Create a communication channel. 'remoteCommand' is the rsync command to execute needed by an SSH connection. 33 | // '*protocol' gives the rsync protocol version. 34 | virtual void createChannel(const char *remoteCommand, int *protocol) = 0; 35 | 36 | // Close the channel (and this IO) 37 | virtual void closeChannel() = 0; 38 | 39 | // Return the account credentials previously stored in the object. 40 | virtual void getConnectInfo(std::string *username, std::string *password, std::string *module) = 0; 41 | 42 | // These two calls are blocking. 43 | virtual int read(char *buffer, int size) = 0; 44 | virtual int write(const char *buffer, int size) = 0; 45 | 46 | virtual void flush() = 0; 47 | 48 | // If the channel has been closed 49 | virtual bool isClosed() = 0; 50 | 51 | // Check if the channel is accessible. Wait util the specified time elapsed. 52 | virtual bool isReadable(int timeoutInMilliSeconds) = 0; 53 | virtual bool isWritable(int timeoutInMilliSeconds) = 0; 54 | 55 | private: 56 | // NOT IMPLEMENTED 57 | IO(const IO&); 58 | IO& operator=(const IO&); 59 | }; 60 | 61 | } // namespace rsync 62 | #endif //INCLUDED_RSYNC_IO_H 63 | -------------------------------------------------------------------------------- /rsync/rsync_log.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | #include 23 | 24 | namespace rsync 25 | { 26 | 27 | int Log::s_level = Log::Info; 28 | 29 | block::out Log::out; 30 | 31 | void Log::setLevel(int level) 32 | { 33 | s_level = level; 34 | } 35 | 36 | int Log::getLevel() 37 | { 38 | return s_level; 39 | } 40 | 41 | const char *Log::getLiteral(int level) 42 | { 43 | switch (level) { 44 | case Log::Debug: return "DEBUG"; 45 | case Log::Trace: return "TRACE"; 46 | case Log::Info: return "INFO"; 47 | case Log::Warning: return "WARNING"; 48 | case Log::Error: return "ERROR"; 49 | case Log::Fatal: return "FATAL"; 50 | case Log::Assert: return "ASSERT"; 51 | default: return "UNDEFINED"; 52 | } 53 | } 54 | 55 | LogRecord::LogRecord(const char *id, int level) 56 | : d_id(id) 57 | , d_level(level) 58 | , d_message() 59 | { 60 | } 61 | 62 | LogRecord::~LogRecord() throw (Exception) 63 | { 64 | // If a custom logging handler was installed, call that handler 65 | if (Log::out.isConnected()) { 66 | Log::out(d_id, d_level, d_message.c_str()); 67 | return; 68 | } 69 | 70 | // Otherwise send the log message to stdout, and throw an exception for an error. 71 | std::string message = Log::getLiteral(d_level); 72 | message += " "; 73 | 74 | time_t t = ::time(0); 75 | struct tm *timeInfo = localtime(&t); 76 | char buffer[100]; 77 | strftime(buffer, sizeof(buffer), "%m/%d/%y %H:%M:%S", timeInfo); 78 | message += buffer; 79 | message += " "; 80 | message += d_id; 81 | message += " "; 82 | message += d_message; 83 | 84 | ::printf("%s\n", message.c_str()); 85 | if (d_level > Log::Error) { 86 | throw Exception(d_id, d_level, message.c_str()); 87 | } 88 | } 89 | 90 | } // namespace rsync 91 | -------------------------------------------------------------------------------- /rsync/rsync_log.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_LOG_H 17 | #define INCLUDED_RSYNC_LOG_H 18 | 19 | #include 20 | 21 | #include 22 | #include 23 | 24 | namespace rsync { 25 | 26 | // The exception to throw if an error has occurred and the operation can't continue. 27 | class Exception 28 | { 29 | public: 30 | Exception(const char *id, int level, const char *message) 31 | : d_id(id) 32 | , d_level(level) 33 | , d_message(message) 34 | { 35 | } 36 | 37 | Exception(const Exception &other) 38 | : d_id(other.d_id) 39 | , d_level(other.d_level) 40 | , d_message(other.d_message) 41 | { 42 | } 43 | 44 | ~Exception() 45 | { 46 | } 47 | 48 | const char *getID() const 49 | { 50 | return d_id; 51 | } 52 | 53 | int getLevel() const 54 | { 55 | return d_level; 56 | } 57 | 58 | const std::string& getMessage() const 59 | { 60 | return d_message; 61 | } 62 | 63 | private: 64 | // NOT IMPLEMENTED 65 | const Exception& operator=(const Exception&); 66 | 67 | const char *d_id; // id of the log message 68 | int d_level; // severity level 69 | std::string d_message; // the actual log message 70 | }; 71 | 72 | struct Log 73 | { 74 | public: 75 | enum Level { 76 | Debug, 77 | Trace, 78 | Info, 79 | Warning, 80 | Error, 81 | Fatal, 82 | Assert 83 | }; 84 | 85 | // Accessors for 's_level' 86 | static void setLevel(int); 87 | static int getLevel(); 88 | 89 | // Basically the string name of the level 90 | static const char *getLiteral(int level); 91 | 92 | // Any log message with this level or above will be reported. Others will be ignored. 93 | static int s_level; 94 | 95 | // A callback for replacing the default logging handler. 96 | static block::out out; 97 | }; 98 | 99 | // A helper object that will generate a log message on destruction. 100 | class LogRecord 101 | { 102 | public: 103 | 104 | LogRecord(const char *id, int level); 105 | 106 | ~LogRecord() throw (Exception); 107 | 108 | LogRecord& operator<<(const char * message) 109 | { 110 | d_message += message; 111 | return *this; 112 | } 113 | 114 | LogRecord& operator<<(const std::string& message) 115 | { 116 | d_message += message; 117 | return *this; 118 | } 119 | 120 | template LogRecord& operator<<(T n) 121 | { 122 | std::stringstream stream; 123 | stream << n; 124 | d_message += stream.str(); 125 | return *this; 126 | } 127 | 128 | private: 129 | // NOT IMPLEMENTED 130 | LogRecord(const LogRecord&); 131 | LogRecord& operator=(const LogRecord&); 132 | 133 | const char *d_id; 134 | int d_level; 135 | std::string d_message; 136 | }; 137 | 138 | #define RSYNC_LOG(ID, LEVEL) \ 139 | if (LEVEL >= rsync::Log::getLevel()) { \ 140 | rsync::LogRecord(ID, LEVEL) 141 | 142 | // Use these macros for creating log messages. 143 | #define LOG_STATUS(ID) RSYNC_LOG(#ID, rsync::Log::Status) 144 | #define LOG_DEBUG(ID) RSYNC_LOG(#ID, rsync::Log::Debug) 145 | #define LOG_TRACE(ID) RSYNC_LOG(#ID, rsync::Log::Trace) 146 | #define LOG_INFO(ID) RSYNC_LOG(#ID, rsync::Log::Info) 147 | #define LOG_WARNING(ID) RSYNC_LOG(#ID, rsync::Log::Warning) 148 | #define LOG_ERROR(ID) RSYNC_LOG(#ID, rsync::Log::Error) 149 | #define LOG_FATAL(ID) RSYNC_LOG(#ID, rsync::Log::Fatal) 150 | #define LOG_ASSERT(ID) RSYNC_LOG(#ID, rsync::Log::Assert) 151 | 152 | #define LOG_END ""; } 153 | 154 | 155 | } 156 | 157 | #endif // INCLUDED_RSYNC_LOG_H 158 | -------------------------------------------------------------------------------- /rsync/rsync_pathutil.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_PATHUTIL_H 17 | #define INCLUDED_RSYNC_PATHUTIL_H 18 | 19 | #include 20 | 21 | #include 22 | 23 | #include 24 | #include 25 | 26 | namespace rsync 27 | { 28 | 29 | class Entry; 30 | 31 | struct PathUtil 32 | { 33 | 34 | #if defined(WIN32) || defined(__MINGW32__) 35 | 36 | // Convert a UTF8 path to UTF16, adding the "\\?\" prefix. 37 | static std::basic_string convertToUTF16(const char *path); 38 | 39 | // Convert a UTF16 path to UTF8, removing the "\\?\" prefix if there is one. 40 | static std::string convertToUTF8(const wchar_t *path); 41 | #endif 42 | 43 | // Return the file size 44 | static int64_t getSize(const char *fullPath); 45 | 46 | // Common file/dir operations. 47 | static bool exists(const char *fullPath); 48 | static bool remove(const char *fullPath, bool enableLogging = true, bool sendToRecyleBin = false); 49 | static bool rename(const char *from, const char *to, bool enableLogging = true); 50 | static bool isDirectory(const char *fullPath); 51 | static bool createDirectory(const char *fullPath, bool hidden = false); 52 | static bool readSymlink(const char *fullPath, std::string *target); 53 | static bool setModifiedTime(const char *fullPath, int64_t time); 54 | static bool createSymlink(const char *fullPath, const char *target, bool isDir); 55 | static bool setMode(const char *fullPath, uint32_t mode); 56 | 57 | // Concatenate two paths. 58 | static std::string join(const char *top, const char *path); 59 | 60 | // Return the directory part of the path 61 | static std::string getDirectory(const char *fullPath); 62 | 63 | // Return the last component. 64 | static std::string getBase(const char *fullPath); 65 | 66 | // If 'directory' is a prefix of 'path' 67 | static bool isPrefix(const char *directory, const char *path); 68 | 69 | // Create all intermediate directories between 'top' and 'top/dir'. 70 | static bool createIntermediateDirectories(const char *top, const char *dir); 71 | 72 | // List the directory joined by 'top' and 'path'; add file entries to 'fileList', and directory entries to 73 | // 'directoryList'. If 'includedPatterns' is specified, add only those entries that match the patterns. 74 | // Files or directories matching any pattern specified in 'excludePatterns' will not be included. 75 | // 'normalization' is used to indicate to convert paths from UTF8-MAC to UTF8. 76 | static void listDirectory(const char *top, const char *path, std::vector *fileList, 77 | std::vector *directoryList, bool normalization = false); 78 | 79 | // Create a new entry from the specified path 80 | static Entry* createEntry(const char *top, const char *path); 81 | 82 | // Return information about a file/directory at the specified path 83 | static bool getFileInfo(const char *fullPath, bool *isDir, int64_t *size, int64_t *time); 84 | 85 | // Return current working directory 86 | static std::string getCurrentDirectory(); 87 | 88 | // Remove a directory and all its contents, recursively 89 | static void removeDirectoryRecursively(const char *fullPath); 90 | 91 | // Replace '/' or '\' with the specified delimiter 92 | static void standardizePath(std::string *path, char delimiter = '/'); 93 | 94 | }; 95 | 96 | } // close namespace rsync 97 | 98 | #endif // INCLUDED_RSYNC_PATHUTIL_H 99 | 100 | -------------------------------------------------------------------------------- /rsync/rsync_socketio.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | #include 20 | 21 | #include 22 | 23 | #include 24 | 25 | #include 26 | 27 | #include 28 | #include 29 | 30 | 31 | namespace rsync 32 | { 33 | 34 | SocketIO::SocketIO() 35 | : IO() 36 | , d_socket(-1) 37 | , d_serverList() 38 | , d_port(0) 39 | , d_user() 40 | , d_password() 41 | , d_module() 42 | { 43 | } 44 | 45 | SocketIO::~SocketIO() 46 | { 47 | closeChannel(); 48 | } 49 | 50 | void SocketIO::connect(const char *serverList, int port, const char *user, const char *password, const char *module) 51 | { 52 | d_serverList = serverList; 53 | d_port = port; 54 | d_user = user; 55 | d_password = password; 56 | d_module = module; 57 | } 58 | 59 | void SocketIO::getConnectInfo(std::string *username, std::string *password, std::string *module) 60 | { 61 | *username = d_user; 62 | *password = d_password; 63 | *module = d_module; 64 | } 65 | 66 | void SocketIO::setModule(const char *module) 67 | { 68 | d_module = module; 69 | } 70 | 71 | void SocketIO::createChannel(const char *remoteCommand, int *protocol) 72 | { 73 | closeChannel(); 74 | 75 | std::stringstream error; 76 | d_socket = rsync::SocketUtil::create(d_serverList.c_str(), d_port, &error); 77 | if (d_socket == -1) { 78 | LOG_FATAL(SOCKET_CONNECT) << error.str() << LOG_END 79 | return; 80 | } 81 | } 82 | 83 | void SocketIO::closeChannel() 84 | { 85 | if (d_socket != -1) { 86 | SocketUtil::close(d_socket); 87 | d_socket = -1; 88 | } 89 | } 90 | 91 | int SocketIO::read(char *buffer, int size) 92 | { 93 | return SocketUtil::read(d_socket, buffer, size); 94 | } 95 | 96 | int SocketIO::write(const char *buffer, int size) 97 | { 98 | return SocketUtil::write(d_socket, buffer, size); 99 | } 100 | 101 | void SocketIO::flush() 102 | { 103 | } 104 | 105 | bool SocketIO::isClosed() 106 | { 107 | return false; 108 | } 109 | 110 | bool SocketIO::isReadable(int timeoutInMilliSeconds) 111 | { 112 | return SocketUtil::isReadable(d_socket, timeoutInMilliSeconds); 113 | 114 | } 115 | 116 | bool SocketIO::isWritable(int timeoutInMilliSeconds) 117 | { 118 | return SocketUtil::isWritable(d_socket, timeoutInMilliSeconds); 119 | } 120 | 121 | } // namespace rsync 122 | -------------------------------------------------------------------------------- /rsync/rsync_socketio.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_SOCKETIO_H 17 | #define INCLUDED_RSYNC_SOCKETIO_H 18 | 19 | #include 20 | 21 | #include 22 | #include 23 | 24 | #include 25 | 26 | namespace rsync 27 | { 28 | 29 | // Implementation of the IO interface using a socket. 30 | class SocketIO : public IO 31 | { 32 | public: 33 | SocketIO(); 34 | ~SocketIO(); 35 | 36 | void connect(const char *serverList, int port, const char *user, const char *password, const char *module); 37 | void getConnectInfo(std::string *username, std::string *password, std::string *module); 38 | void setModule(const char *module); 39 | 40 | void createChannel(const char *remoteCommand, int *protocol); 41 | void closeChannel(); 42 | 43 | virtual int read(char *buffer, int size); 44 | virtual int write(const char *buffer, int size); 45 | virtual void flush(); 46 | virtual bool isClosed(); 47 | 48 | virtual bool isReadable(int timeoutInMilliSeconds); 49 | virtual bool isWritable(int timeoutInMilliSeconds); 50 | 51 | private: 52 | // NOT IMPLEMENTED 53 | SocketIO(const SocketIO&); 54 | SocketIO& operator=(const SocketIO&); 55 | 56 | int d_socket; 57 | 58 | std::string d_serverList; 59 | int d_port; 60 | std::string d_user; 61 | std::string d_password; 62 | std::string d_module; 63 | 64 | }; 65 | 66 | } // namespace rsync 67 | 68 | #endif // INCLUDED_RSYNC_SOCKETIO_H 69 | -------------------------------------------------------------------------------- /rsync/rsync_socketutil.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #if defined(WIN32) || defined(__MINGW32__) 17 | #define _WIN32_WINNT 0x501 // means windows XP and above 18 | #include 19 | #include 20 | #include 21 | #else 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include 32 | #ifdef __APPLE__ 33 | #include 34 | #endif 35 | #define SOCKADDR sockaddr 36 | #define SOCKET_ERROR -1 37 | #endif 38 | 39 | #include 40 | #include 41 | 42 | #include 43 | 44 | #include 45 | 46 | namespace rsync 47 | { 48 | 49 | void SocketUtil::startup() 50 | { 51 | #if defined(WIN32) || defined(__MINGW32__) 52 | static WSADATA wsaData = { 0 }; 53 | int result = WSAStartup(MAKEWORD(2, 2), &wsaData); 54 | if (result != 0) { 55 | LOG_FATAL(SOCKET_STARTUP) << "WSAStartup failed: " << result << LOG_END 56 | } 57 | #endif // WIN32 58 | } 59 | 60 | void SocketUtil::cleanup() 61 | { 62 | #if defined(WIN32) || defined(__MINGW32__) 63 | WSACleanup(); 64 | #endif // WIN32 65 | } 66 | 67 | int SocketUtil::create(const char *host, int port, std::stringstream *error) 68 | { 69 | struct addrinfo hints; 70 | ::memset(&hints, 0, sizeof(hints)); 71 | hints.ai_family = PF_UNSPEC; 72 | hints.ai_socktype = SOCK_STREAM; 73 | hints.ai_protocol = IPPROTO_TCP; 74 | 75 | struct addrinfo *info = 0; 76 | int result = getaddrinfo(host, "22", &hints, &info); 77 | if (result != 0) { 78 | *error << "Failed to resolve the host '" << host << "': " << result; 79 | return -1; 80 | } 81 | struct addrinfo *ptr = info; 82 | for (; ptr != 0; ptr = ptr->ai_next) { 83 | if (ptr->ai_family == AF_INET || ptr->ai_family == AF_INET6) { 84 | break; 85 | } 86 | } 87 | 88 | if (!ptr) { 89 | freeaddrinfo(info); 90 | *error << "Failed to resolve the host '" << host << "' into an ip address"; 91 | return -1; 92 | } 93 | 94 | sockaddr_storage addr; 95 | sockaddr_in6 *addr_v6 = 0; 96 | int addrlen = ptr->ai_addrlen; 97 | ::memcpy(&addr, ptr->ai_addr, ptr->ai_addrlen); 98 | if (ptr->ai_family == AF_INET) { 99 | sockaddr_in *addr_v4 = (sockaddr_in*)&addr; 100 | addr_v4->sin_port = htons(port); 101 | } else { 102 | addr_v6 = (sockaddr_in6*)&addr; 103 | addr_v6->sin6_port = htons(port); 104 | } 105 | freeaddrinfo(info); 106 | 107 | int sock = socket(addr_v6 ? AF_INET6 : AF_INET, SOCK_STREAM, IPPROTO_TCP); 108 | #if defined(WIN32) || defined(__MINGW32__) 109 | if (sock == static_cast(INVALID_SOCKET)) { 110 | *error << "Failed to create the socket: " << WSAGetLastError(); 111 | return -1; 112 | } 113 | #else 114 | if (sock == -1) { 115 | *error << "Failed to create the socket: " << strerror(errno); 116 | return -1; 117 | } 118 | 119 | #ifdef __APPLE__ 120 | // Ignore SIGPIPE 121 | int value = 1; 122 | setsockopt(sock, SOL_SOCKET, SO_NOSIGPIPE, (void *)&value, sizeof(value)); 123 | #endif 124 | 125 | #endif 126 | 127 | 128 | #if defined(WIN32) || defined(__MINGW32__) 129 | u_long iMode=1; 130 | result = ioctlsocket(sock, FIONBIO, &iMode); 131 | if (result != NO_ERROR) { 132 | *error << "Failed to enable nonblocking mode: " << result; 133 | close(sock); 134 | return -1; 135 | } 136 | #else 137 | fcntl(sock, F_SETFL, O_NONBLOCK); 138 | #endif 139 | 140 | #if defined(WIN32) 141 | int bufferLength = 1024 * 1024; 142 | result = setsockopt(sock, SOL_SOCKET, SO_SNDBUF, (char*)&bufferLength, sizeof(bufferLength)); 143 | if (result == SOCKET_ERROR) { 144 | LOG_ERROR(SOCKET_SNDBUF) << "Failed to change the TCP send buffer size: " << WSAGetLastError() << LOG_END 145 | } 146 | result = setsockopt(sock, SOL_SOCKET, SO_RCVBUF, (char*)&bufferLength, sizeof(bufferLength)); 147 | if (result == SOCKET_ERROR) { 148 | LOG_ERROR(SOCKET_SNDBUF) << "Failed to change the TCP receive buffer size: " << WSAGetLastError() << LOG_END 149 | } 150 | #else 151 | int bufferLength = 128 * 1024; 152 | result = setsockopt(sock, SOL_SOCKET, SO_SNDBUF, (char*)&bufferLength, sizeof(bufferLength)); 153 | if (result < 0) { 154 | LOG_ERROR(SOCKET_SNDBUF) << "Failed to change the TCP send buffer size: " << strerror(errno) << LOG_END 155 | } 156 | result = setsockopt(sock, SOL_SOCKET, SO_RCVBUF, (char*)&bufferLength, sizeof(bufferLength)); 157 | if (result < 0) { 158 | LOG_ERROR(SOCKET_SNDBUF) << "Failed to change the TCP receive buffer size: " << strerror(errno) << LOG_END 159 | } 160 | 161 | /*int flag = 1; 162 | result = setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, (char*)&flag, sizeof(flag)); 163 | if (result < 0) { 164 | LOG_ERROR(SOCKET_NODELAY) << "Failed to set the TCP_NODELAY option: " << strerror(errno) << LOG_END 165 | }*/ 166 | #endif 167 | 168 | result = connect(sock, reinterpret_cast(&addr), addrlen); 169 | 170 | fd_set fdset; 171 | struct timeval tv; 172 | FD_ZERO(&fdset); 173 | FD_SET(sock, &fdset); 174 | tv.tv_sec = 10; 175 | tv.tv_usec = 0; 176 | 177 | result = select(sock + 1, NULL, &fdset, NULL, &tv); 178 | 179 | if (result < 0) { 180 | *error << "Failed to connect to '" << host << ":" << port << "': code " << result; 181 | close(sock); 182 | return -1; 183 | } else if (result == 0) { 184 | *error << "Failed to connect to '" << host << ":" << port << "': connection timeout "; 185 | close(sock); 186 | return -1; 187 | } 188 | 189 | return sock; 190 | } 191 | 192 | void SocketUtil::close(int socket) 193 | { 194 | #if defined(WIN32) || defined(__MINGW32__) 195 | shutdown(socket, SD_BOTH); 196 | #else 197 | ::close(socket); 198 | #endif 199 | } 200 | 201 | #if defined(WIN32) || defined(__MINGW32__) 202 | 203 | int SocketUtil::read(int socket, char *buffer, int size) 204 | { 205 | int rc = ::recv(socket, buffer, size, 0); 206 | if (rc == SOCKET_ERROR) { 207 | if (WSAGetLastError() != WSAEWOULDBLOCK) { 208 | LOG_ERROR(RSYNC_SOCKET) << "Error reading from socket: " << Util::getLastError() << LOG_END 209 | return -1; 210 | } else { 211 | return 0; 212 | } 213 | } else if (rc == 0) { 214 | LOG_ERROR(RSYNC_SOCKET) << "Socket was closed unexpectedly" << LOG_END 215 | return -1; 216 | } 217 | return rc; 218 | } 219 | 220 | int SocketUtil::write(int socket, const char *buffer, int size) 221 | { 222 | int rc = ::send(socket, buffer, size, 0); 223 | if (rc == SOCKET_ERROR) { 224 | if (WSAGetLastError() != WSAEWOULDBLOCK) { 225 | LOG_ERROR(RSYNC_SOCKET) << "Error writing to socket: " << Util::getLastError() << LOG_END 226 | return -1; 227 | } else { 228 | return 0; 229 | } 230 | } else if (rc == 0) { 231 | LOG_ERROR(RSYNC_SOCKET) << "Socket was closed unexpectedly" << LOG_END 232 | return -1; 233 | } 234 | return rc; 235 | } 236 | 237 | #else 238 | 239 | int SocketUtil::read(int socket, char *buffer, int size) 240 | { 241 | int rc = static_cast(::read(socket, buffer, size)); 242 | if (rc == -1) { 243 | if (errno != EAGAIN) { 244 | LOG_ERROR(RSYNC_SOCKET) << "Error reading from socket: " << Util::getLastError() << LOG_END 245 | return -1; 246 | } else { 247 | return 0; 248 | } 249 | } else if (rc == 0) { 250 | LOG_ERROR(RSYNC_SOCKET) << "Socket was closed unexpectedly" << LOG_END 251 | return -1; 252 | } 253 | return rc; 254 | } 255 | 256 | int SocketUtil::write(int socket, const char *buffer, int size) 257 | { 258 | int rc = static_cast(::write(socket, buffer, size)); 259 | if (rc == -1) { 260 | if (errno != EAGAIN) { 261 | LOG_ERROR(RSYNC_SOCKET) << "Error writing to socket: " << Util::getLastError() << LOG_END 262 | return -1; 263 | } else { 264 | return 0; 265 | } 266 | } else if (rc == 0) { 267 | LOG_ERROR(RSYNC_SOCKET) << "Socket was closed unexpectedly" << LOG_END 268 | return -1; 269 | } 270 | return rc; 271 | } 272 | #endif 273 | 274 | bool SocketUtil::isReadable(int socket, int timeoutInMilliSeconds) 275 | { 276 | struct timeval tv; 277 | tv.tv_sec = timeoutInMilliSeconds / 1000; 278 | tv.tv_usec = (timeoutInMilliSeconds % 1000) * 1000; 279 | 280 | fd_set readFDs; 281 | FD_ZERO(&readFDs); 282 | FD_SET(static_cast(socket), &readFDs); 283 | 284 | return select(socket + 1, &readFDs, 0, 0, &tv); 285 | } 286 | 287 | bool SocketUtil::isWritable(int socket, int timeoutInMilliSeconds) 288 | { 289 | struct timeval tv; 290 | tv.tv_sec = timeoutInMilliSeconds / 1000; 291 | tv.tv_usec = (timeoutInMilliSeconds % 1000) * 1000; 292 | 293 | fd_set writeFDs; 294 | FD_ZERO(&writeFDs); 295 | FD_SET(static_cast(socket), &writeFDs); 296 | 297 | return select(socket + 1, 0, &writeFDs, 0, &tv); 298 | } 299 | 300 | 301 | } // close namespace rsync 302 | -------------------------------------------------------------------------------- /rsync/rsync_socketutil.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_SOCKETUTIL_H 17 | #define INCLUDED_RSYNC_SOCKETUTIL_H 18 | 19 | #include 20 | 21 | namespace rsync 22 | { 23 | 24 | struct SocketUtil 25 | { 26 | // Methods for working with sockets. 27 | static void startup(); 28 | static void cleanup(); 29 | 30 | static int create(const char *host, int port, std::stringstream *error); 31 | static void close(int socket); 32 | 33 | static int read(int socket, char *buffer, int size); 34 | static int write(int socket, const char *buffer, int size); 35 | static bool isReadable(int socket, int timeoutInMilliSeconds); 36 | static bool isWritable(int socket, int timeoutInMilliSeconds); 37 | }; 38 | 39 | } // close namespace rsync 40 | 41 | #endif // INCLUDED_RSYNC_SOCKETUTIL_H 42 | -------------------------------------------------------------------------------- /rsync/rsync_sshio.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | #include 20 | #include 21 | #include 22 | 23 | #include 24 | 25 | #include 26 | 27 | #include 28 | 29 | #include 30 | 31 | #include 32 | #include 33 | 34 | bool libssh2_debug_enabled = true; 35 | 36 | extern "C" 37 | void libssh2_debug_handler(LIBSSH2_SESSION* /*session*/, 38 | void* /*context*/, 39 | const char *buffer, 40 | size_t /*length*/) 41 | { 42 | if (libssh2_debug_enabled) { 43 | LOG_INFO(LIBSSH2_DEBUG) << buffer << LOG_END 44 | } 45 | } 46 | 47 | namespace rsync 48 | { 49 | 50 | namespace { 51 | 52 | const char *g_password = 0; 53 | 54 | void keyboardCallback(const char *name, int nameLength, 55 | const char *instruction, int instructionLength, int numPrompts, 56 | const LIBSSH2_USERAUTH_KBDINT_PROMPT *prompts, 57 | LIBSSH2_USERAUTH_KBDINT_RESPONSE *responses, 58 | void **abstract) 59 | { 60 | (void)name; 61 | (void)nameLength; 62 | (void)instruction; 63 | (void)instructionLength; 64 | if (numPrompts == 1) { 65 | responses[0].text = ::strdup(g_password); 66 | responses[0].length = ::strlen(g_password); 67 | } 68 | (void)prompts; 69 | (void)abstract; 70 | } 71 | 72 | } // unnamed namesapce 73 | 74 | bool SSHIO::startup() 75 | { 76 | return 0 == libssh2_init(0); 77 | } 78 | 79 | void SSHIO::cleanup() 80 | { 81 | libssh2_exit(); 82 | } 83 | 84 | SSHIO::SSHIO() 85 | : IO() 86 | , d_socket(0) 87 | , d_session(0) 88 | , d_channel(0) 89 | { 90 | } 91 | 92 | SSHIO::~SSHIO() 93 | { 94 | closeSession(); 95 | } 96 | 97 | void SSHIO::connect(const char *serverList, int port, const char *user, const char *password, const char *keyFile, const char *hostKey) 98 | { 99 | closeSession(); 100 | 101 | std::vector servers; 102 | std::string serverString = serverList; 103 | 104 | Util::tokenize(serverString, &servers, ";, \t"); 105 | 106 | if (servers.size() == 0) { 107 | LOG_FATAL(SSH_INIT) << "No server specified" << LOG_END 108 | } 109 | 110 | unsigned int i; 111 | if (hostKey) { 112 | // Only use servers[0]; 113 | i = 0; 114 | std::stringstream error; 115 | d_socket = SocketUtil::create(servers[i].c_str(), port, &error); 116 | if (d_socket == -1) { 117 | LOG_FATAL(SOCKET_CONNECT) << error.str() << LOG_END 118 | } 119 | } else { 120 | for (i = 0; i < servers.size(); ++i) { 121 | std::stringstream error; 122 | d_socket = SocketUtil::create(servers[i].c_str(), port, &error); 123 | if (d_socket == -1) { 124 | if (i == servers.size() - 1) { 125 | LOG_FATAL(SOCKET_CONNECT) << error.str() << LOG_END 126 | } else { 127 | LOG_ERROR(SOCKET_CONNECT) << error.str() << LOG_END 128 | 129 | } 130 | } else { 131 | break; 132 | } 133 | } 134 | } 135 | 136 | d_session = libssh2_session_init(); 137 | if (!d_session) { 138 | LOG_FATAL(SSH_INIT) << "Failed to create a libssh2 session" << LOG_END 139 | } 140 | 141 | //libssh2_trace_sethandler(d_session, 0, libssh2_debug_handler); 142 | //libssh2_trace(d_session, LIBSSH2_TRACE_TRANS /*| LIBSSH2_TRACE_CONN */| LIBSSH2_TRACE_ERROR | LIBSSH2_TRACE_SOCKET); 143 | 144 | libssh2_session_set_blocking(d_session, 1); 145 | 146 | libssh2_session_set_timeout(d_session, 100000); 147 | 148 | int rc = libssh2_session_startup(d_session, d_socket); 149 | if (rc != 0) { 150 | LOG_FATAL(SSH_START) << "Failed to establish an SSH session to '" << servers[i] << ":" << port << "': " << getLastError() << LOG_END 151 | return; 152 | } 153 | 154 | if (hostKeyOut.isConnected() || hostKey) { 155 | const char *fingerPrint = libssh2_hostkey_hash(d_session, LIBSSH2_HOSTKEY_HASH_SHA1); 156 | std::string hash; 157 | const char *hex = "0123456789ABCDEF"; 158 | 159 | for (unsigned int j = 0; j < 20; ++j) { 160 | unsigned char c = fingerPrint[j]; 161 | if (j != 0) { 162 | hash += ":"; 163 | } 164 | hash += hex[c / 16]; 165 | hash += hex[c % 16]; 166 | } 167 | 168 | if (hostKeyOut.isConnected()) { 169 | if (!hostKeyOut(servers[i].c_str(), hash.c_str())) { 170 | LOG_FATAL(SSH_HOSTKEY) << "New host key for server '" << servers[i] << "' is not accepted" << LOG_END 171 | } 172 | } else { 173 | if (hash != hostKey) { 174 | LOG_FATAL(SSH_HOSTKEY) << hash << " : new host key for server '" << servers[i] << ";" << LOG_END; 175 | return; 176 | } 177 | } 178 | 179 | } 180 | 181 | const char *methods = libssh2_userauth_list(d_session, user, ::strlen(user)); 182 | 183 | if (methods == NULL) { 184 | LOG_FATAL(SSH_AUTH) << "User '" << user << "' is not allowed to log in to '" 185 | << servers[i] << "'" << LOG_END 186 | return; 187 | } 188 | 189 | if (keyFile != 0 && *keyFile != 0) { 190 | if (::strstr(methods, "publickey") == NULL) { 191 | LOG_FATAL(SSH_AUTH) << "Public key authorization is not permitted by the server" << LOG_END 192 | return; 193 | } 194 | 195 | rc = libssh2_userauth_publickey_fromfile(d_session, user, NULL, keyFile, password); 196 | 197 | if (rc != 0) { 198 | LOG_FATAL(SSH_AUTH) << "Authentication by public key failed: " << getLastError() << LOG_END 199 | return; 200 | } 201 | } else if (::strstr(methods, "password") != NULL) { 202 | rc = libssh2_userauth_password(d_session, user, password); 203 | if (rc != 0) { 204 | LOG_FATAL(SSH_AUTH) << "Invalid username or password: " << getLastError() << LOG_END 205 | return; 206 | } 207 | } else if (::strstr(methods, "keyboard-interactive") != NULL) { 208 | g_password = password; 209 | rc = libssh2_userauth_keyboard_interactive(d_session, user, &keyboardCallback); 210 | if (rc != 0) { 211 | LOG_FATAL(SSH_AUTH) << "Invalid username/password: " << getLastError() << LOG_END 212 | return; 213 | } 214 | } else { 215 | LOG_FATAL(SSH_AUTH) << "Authentication by password not supported" << LOG_END 216 | return; 217 | } 218 | 219 | libssh2_keepalive_config (d_session, 1, 5); 220 | } 221 | 222 | void SSHIO::getConnectInfo(std::string * /*username*/, std::string * /*password*/, std::string * /*module*/) 223 | { 224 | LOG_FATAL(SSH_NOT_IMPLEMENTED) << "This 'getConnectInfo' method is never supposed to be called" << LOG_END 225 | } 226 | 227 | void SSHIO::createChannel(const char *remoteCommand, int * /* protocol */) 228 | { 229 | closeChannel(); 230 | 231 | libssh2_session_set_blocking(d_session, 1); 232 | 233 | if (!d_session) { 234 | LOG_FATAL(SSH_NO_SESSION) << "No SSH session available" << LOG_END 235 | } 236 | 237 | d_channel = libssh2_channel_open_session(d_session); 238 | if (!d_channel) { 239 | LOG_FATAL(SSH_CREATE) << "Failed to create a new ssh channel" << LOG_END 240 | } 241 | 242 | int rc = libssh2_channel_exec(d_channel, remoteCommand); 243 | if (rc != 0) { 244 | LOG_FATAL(SSH_EXEC) << "Failed to execute the remote command '" << remoteCommand 245 | << "': " << getLastError() << LOG_END 246 | } 247 | 248 | libssh2_session_set_blocking(d_session, 0); 249 | d_recentWrites.clear(); 250 | } 251 | 252 | void SSHIO::closeChannel() 253 | { 254 | if (d_channel) { 255 | libssh2_channel_close(d_channel); 256 | libssh2_channel_free(d_channel); 257 | d_channel = 0; 258 | } 259 | } 260 | 261 | void SSHIO::closeSession() 262 | { 263 | int rc; 264 | if (d_session) { 265 | closeChannel(); 266 | 267 | int64_t startTime = TimeUtil::getTimeOfDay() / 1000000; 268 | while ((rc = libssh2_session_disconnect(d_session, "work done")) == LIBSSH2_ERROR_EAGAIN) { 269 | if (TimeUtil::getTimeOfDay() / 1000000 - startTime > 10) { 270 | break; 271 | } 272 | } 273 | 274 | startTime = TimeUtil::getTimeOfDay() / 1000000; 275 | while ((rc = libssh2_session_free(d_session)) == LIBSSH2_ERROR_EAGAIN) { 276 | if (TimeUtil::getTimeOfDay() / 1000000 - startTime > 10) { 277 | break; 278 | } 279 | } 280 | d_session = 0; 281 | } 282 | 283 | if (d_socket) { 284 | SocketUtil::close(d_socket); 285 | d_socket = 0; 286 | } 287 | } 288 | 289 | int SSHIO::read(char *buffer, int size) 290 | { 291 | // Make sure the receive window is large 292 | const unsigned long minWindowSize = 128 * 1024; 293 | unsigned long windowSize = libssh2_channel_window_read_ex(d_channel, NULL, NULL); 294 | if (windowSize < minWindowSize) { 295 | libssh2_channel_receive_window_adjust2(d_channel, minWindowSize * 2, 0, 0); 296 | } 297 | 298 | int rc = libssh2_channel_read(d_channel, buffer, size); 299 | if (rc > 0) { 300 | return rc; 301 | } 302 | 303 | checkError(rc); 304 | return 0; 305 | } 306 | 307 | int SSHIO::write(const char *buffer, int size) 308 | { 309 | int rc = libssh2_channel_write(d_channel, buffer, size); 310 | if (rc > 0) { 311 | // Save rc and the first 4 bytes for dump() in case an error is encountered. 312 | uint64_t value = rc; 313 | value <<= 32; 314 | ::memcpy(&value, buffer, (size < 4) ? size : 4); 315 | if (d_recentWrites.size() > 1000) { 316 | d_recentWrites.pop_front(); 317 | } 318 | d_recentWrites.push_back(value); 319 | return rc; 320 | } 321 | 322 | checkError(rc); 323 | return 0; 324 | } 325 | 326 | void SSHIO::flush() 327 | { 328 | libssh2_channel_flush(d_channel); 329 | } 330 | 331 | void SSHIO::dump() 332 | { 333 | const char *hex = "0123456789ABCDEF0"; 334 | for (std::list::iterator iter = d_recentWrites.begin(); iter != d_recentWrites.end(); ++iter) { 335 | uint64_t value = *iter; 336 | int rc = value >> 32; 337 | 338 | char data[9]; 339 | int n = 4; 340 | if (rc < n) { 341 | n = rc; 342 | } 343 | unsigned char *buffer = reinterpret_cast(&value); 344 | for (int i = 0; i < n ; ++i) { 345 | unsigned char c = buffer[i]; 346 | data[2 * i] = hex[c / 16]; 347 | data[2 * i + 1] = hex[c % 16]; 348 | } 349 | data[2 * n] = 0; 350 | if (n == 4 && buffer[3] == 0x07) { 351 | int length = (unsigned char)buffer[0] + 256 * (unsigned char)buffer[1] + 65536 * (unsigned char)buffer[2]; 352 | LOG_INFO(SSHIO_DUMP) << "[Debug] Wrote " << rc << " bytes: " << data << "(" << length << ")" << LOG_END 353 | } else { 354 | LOG_INFO(SSHIO_DUMP) << "[Debug] Wrote " << rc << " bytes: " << data << LOG_END 355 | } 356 | } 357 | } 358 | 359 | bool SSHIO::isReadable(int timeoutInMilliSeconds) 360 | { 361 | return SocketUtil::isReadable(d_socket, timeoutInMilliSeconds); 362 | } 363 | 364 | bool SSHIO::isWritable(int timeoutInMilliSeconds) 365 | { 366 | return SocketUtil::isWritable(d_socket, timeoutInMilliSeconds); 367 | } 368 | 369 | void SSHIO::checkError(int rc) 370 | { 371 | if (rc == LIBSSH2_ERROR_EAGAIN) { 372 | return; 373 | } 374 | 375 | if (rc == 0) { 376 | // Get STDERR messages if there are any. 377 | char buffer[256]; 378 | std::string message; 379 | while ((rc = libssh2_channel_read_stderr(d_channel, buffer, sizeof(buffer) - 1)) > 0) { 380 | buffer[rc] = 0; 381 | message += buffer; 382 | } 383 | if (message.size()) { 384 | std::string line; 385 | std::istringstream iss(message); 386 | while (getline(iss, line, '\n')) { 387 | LOG_INFO(SSH_STDERR) << line.c_str() << LOG_END 388 | } 389 | LOG_FATAL(SSH_CLOSED) << "The ssh channel has been unexpectedly closed" << LOG_END 390 | } 391 | return; 392 | } 393 | 394 | switch (rc) { 395 | case LIBSSH2_ERROR_ALLOC: 396 | LOG_FATAL(SSH_ALLOC) << "Allocation failed during an SSH operation" << LOG_END 397 | break; 398 | case LIBSSH2_ERROR_SOCKET_SEND: 399 | LOG_FATAL(SSH_SOCK) << "Unable to send data over the SSH channel" << LOG_END 400 | break; 401 | case LIBSSH2_ERROR_CHANNEL_CLOSED: 402 | LOG_FATAL(SSH_CLOSED) << "The ssh channel has been closed" << LOG_END 403 | break; 404 | case LIBSSH2_ERROR_CHANNEL_EOF_SENT: 405 | LOG_FATAL(SSH_EOF) << "The ssh channel has been requested to be closed" << LOG_END 406 | break; 407 | default: 408 | LOG_FATAL(SSH_ERROR) << "SSH error: " << getLastError() << LOG_END 409 | break; 410 | } 411 | return; 412 | } 413 | 414 | bool SSHIO::isClosed() 415 | { 416 | // EOF must be explicitly checked to detect channel closing. 417 | if (libssh2_channel_eof(d_channel)) { 418 | LOG_FATAL(SSH_EOF) << "The ssh channel has been closed" << LOG_END 419 | return true; 420 | } 421 | 422 | int status = libssh2_channel_get_exit_status(d_channel); 423 | if (status) { 424 | LOG_FATAL(SSH_EXIT) << "The remote program has exited with status '" << status << "'" << LOG_END 425 | return true; 426 | } 427 | 428 | char *signal = 0; 429 | libssh2_channel_get_exit_signal(d_channel, &signal, 0, 0, 0, 0, 0); 430 | if (signal != 0) { 431 | closeChannel(); 432 | LOG_FATAL(SSH_SIGNAL) << "The channel was closed by signal '" << signal << "'" << LOG_END 433 | return true; 434 | } 435 | return false; 436 | } 437 | 438 | std::string SSHIO::getLastError() 439 | { 440 | char *message; 441 | int length; 442 | 443 | if (libssh2_session_last_error(d_session, &message, &length, 0)) { 444 | return std::string(message, length); 445 | } else { 446 | return std::string(""); 447 | } 448 | } 449 | 450 | } // namespace rsync 451 | -------------------------------------------------------------------------------- /rsync/rsync_sshio.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_SSHIO_H 17 | #define INCLUDED_RSYNC_SSHIO_H 18 | 19 | #include 20 | 21 | #include 22 | 23 | #include 24 | #include 25 | 26 | #include 27 | 28 | struct _LIBSSH2_SESSION; 29 | struct _LIBSSH2_CHANNEL; 30 | 31 | namespace rsync 32 | { 33 | 34 | // Implementation of the IO interface on top of an SSH session. 35 | class SSHIO : public IO 36 | { 37 | public: 38 | SSHIO(); 39 | ~SSHIO(); 40 | 41 | static bool startup(); 42 | static void cleanup(); 43 | 44 | void connect(const char *serverList, int port, const char *user, const char *password, const char *keyFile, const char *hostKey); 45 | void getConnectInfo(std::string *username, std::string *password, std::string *module); 46 | 47 | void createChannel(const char *remoteCommand, int *protocol); 48 | void closeChannel(); 49 | 50 | void closeSession(); 51 | 52 | virtual int read(char *buffer, int size); 53 | virtual int write(const char *buffer, int size); 54 | virtual void flush(); 55 | virtual bool isClosed(); 56 | virtual bool isReadable(int timeoutInMilliSeconds); 57 | virtual bool isWritable(int timeoutInMilliSeconds); 58 | 59 | bool isConnected() const { 60 | return d_session != 0; 61 | } 62 | 63 | std::string getLastError(); 64 | 65 | void dump(); 66 | 67 | block::out hostKeyOut; 68 | 69 | private: 70 | // NOT IMPLEMENTED 71 | SSHIO(const SSHIO&); 72 | SSHIO& operator=(const SSHIO&); 73 | 74 | bool isReadable(); 75 | void checkError(int rc); 76 | 77 | int d_socket; 78 | _LIBSSH2_SESSION *d_session; 79 | _LIBSSH2_CHANNEL *d_channel; 80 | 81 | std::list d_recentWrites; 82 | }; 83 | 84 | } // namespace rsync 85 | 86 | #endif // INCLUDED_RSYNC_SSHIO_H 87 | -------------------------------------------------------------------------------- /rsync/rsync_stream.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | #include 23 | 24 | #include 25 | #include 26 | #include 27 | 28 | 29 | #include 30 | #include 31 | #include 32 | #include 33 | 34 | #include 35 | 36 | namespace rsync 37 | { 38 | 39 | namespace 40 | { 41 | 42 | // Return the length of the variable-length integer indicated by 'c' 43 | int getVariableLength(uint8_t c) 44 | { 45 | for (int i = 0; i <= 6; ++i) { 46 | if ((c & 0x80) == 0) { 47 | return i; 48 | } 49 | c = c << 1; 50 | } 51 | return 6; 52 | } 53 | 54 | // Default read/write buffer size 55 | int g_BufferSize = 64000; 56 | 57 | // Message flags. MSG_DATA is followed by data that will be put in the read/write buffer. Messages with 58 | // other flags are processed differently. 59 | enum { MSG_BASE = 7 }; 60 | enum { 61 | MSG_DATA = 0, 62 | MSG_ERROR_XFER = 1, 63 | MSG_INFO = 2, 64 | MSG_ERROR = 3, 65 | MSG_WARNING = 4, 66 | MSG_IO_ERROR = 22, 67 | MSG_NOOP = 42, 68 | MSG_SUCCESS = 100, 69 | MSG_DELETED = 101, 70 | MSG_NO_SEND = 102 71 | }; 72 | 73 | // Replace new line characters by spaces. 74 | void stripNewLine(char *message) 75 | { 76 | for (char *i = message; *i; ++i) { 77 | if (*i == '\n') { 78 | *i = ' '; 79 | } 80 | } 81 | } 82 | 83 | // Encode a string in base64. 84 | std::string base64_encode(char *data, int length) 85 | { 86 | BIO *b64 = BIO_new(BIO_f_base64()); 87 | BIO *bmem = BIO_new(BIO_s_mem()); 88 | b64 = BIO_push(b64, bmem); 89 | BIO_write(b64, data, length); 90 | BIO_flush(b64); 91 | 92 | BUF_MEM *bptr = NULL; 93 | BIO_get_mem_ptr(b64, &bptr); 94 | 95 | std::vector buff(bptr->length, 0); 96 | 97 | memcpy(&buff.front(), bptr->data, bptr->length-1); 98 | buff[bptr->length-1] = '\0'; 99 | BIO_free_all(b64); 100 | 101 | return std::string(buff.data()); 102 | } 103 | 104 | } // unnamed namespace 105 | 106 | Stream::Stream(IO* io, int *cancelFlag) 107 | : d_io(io) 108 | , d_cancelFlag(cancelFlag) 109 | , d_isReadBuffered(false) 110 | , d_isWriteBuffered(false) 111 | , d_isWriteMultiplexed(false) 112 | , d_readDataLength(0) 113 | , d_writeBuffer(new char[g_BufferSize]) 114 | , d_writeBufferSize(g_BufferSize) 115 | , d_writeBufferPosition(0) 116 | , d_readBuffer(new char[g_BufferSize]) 117 | , d_readBufferSize(g_BufferSize) 118 | , d_readBufferPosition(0) 119 | , d_readBufferEnd(0) 120 | , d_flushStart(0) 121 | , d_automaticFlush(true) 122 | , d_lastPositiveIndexRead(-1) 123 | , d_lastNegativeIndexRead(1) 124 | , d_lastPositiveIndexWritten(-1) 125 | , d_lastNegativeIndexWritten(1) 126 | , d_blockedTime(0) 127 | , d_uploadLimit(0) 128 | , d_uploadBuckets() 129 | , d_bucketStartTime(0) 130 | , d_deletedFiles() 131 | , d_messageFlag(-1) 132 | { 133 | } 134 | 135 | Stream::~Stream() 136 | { 137 | delete [] d_writeBuffer; 138 | delete [] d_readBuffer; 139 | } 140 | 141 | void Stream::reset() 142 | { 143 | d_isReadBuffered = false; 144 | d_isWriteBuffered = false; 145 | d_isWriteMultiplexed = false; 146 | d_readDataLength = 0; 147 | d_readBufferPosition = 0; 148 | d_writeBufferPosition = 0; 149 | d_readBufferEnd = 0; 150 | d_automaticFlush = true; 151 | d_flushStart = 0; 152 | d_lastPositiveIndexRead = -1; 153 | d_lastNegativeIndexRead = 1; 154 | d_lastPositiveIndexWritten = -1; 155 | d_lastNegativeIndexWritten = 1; 156 | } 157 | 158 | int Stream::read(char *buffer, int size) 159 | { 160 | // Read directly from the io if the buffer is not being used. 161 | if (!d_isReadBuffered) { 162 | return readAll(buffer, size); 163 | } 164 | 165 | int bytes = 0; 166 | 167 | // Read from the read buffer if there is anything. 168 | if (d_readBufferEnd > d_readBufferPosition) { 169 | int readLength = size; 170 | if (readLength > d_readBufferEnd - d_readBufferPosition) { 171 | readLength = d_readBufferEnd - d_readBufferPosition; 172 | } 173 | ::memcpy(buffer, d_readBuffer + d_readBufferPosition, readLength); 174 | bytes += readLength; 175 | d_readBufferPosition += readLength; 176 | if (d_readBufferPosition >= d_readBufferEnd) { 177 | d_readBufferPosition = 0; 178 | d_readBufferEnd = 0; 179 | } 180 | } 181 | 182 | // The read buffer is exhausted. Try to read from the io, which is multiplexed. 183 | while (bytes < size) { 184 | d_blockedTime = 0; 185 | // 'd_readDataLength == 0' means that we've finished reading the current chunk. 186 | // Need to get a new chunk 187 | while (d_readDataLength <= 0) { 188 | uint32_t flag; 189 | // Read the message flag. If the message flag is MSG_DATA then d_readDataLength will indicate 190 | // how many bytes of data are available. 191 | if (readMessageFlag(&flag)) { 192 | readMessageContent(flag); 193 | d_blockedTime = 0; 194 | } else { 195 | // Check timeout 196 | timedWait(true, "read"); 197 | } 198 | } 199 | int n = size - bytes; 200 | if (d_readDataLength < n) { 201 | n = d_readDataLength; 202 | } 203 | // Now we can read from the io directly. 204 | bytes += readAll(buffer + bytes, n); 205 | d_readDataLength -= n; 206 | } 207 | return size; 208 | } 209 | 210 | int Stream::write(const char *buffer, int size) 211 | { 212 | // Write directly to the io is the buffer mode isn't on. 213 | if (!d_isWriteBuffered) { 214 | return writeAll(buffer, size); 215 | } 216 | 217 | // A non-zero 'd_flushStart' means we started but haven't finished flushing. In this case a new write is prohibited. 218 | if (d_flushStart) { 219 | LOG_FATAL(RSYNC_WRITE) << "Attempt to write while a previous flush operation is incomplete" << LOG_END 220 | return 0; 221 | } 222 | 223 | if (d_automaticFlush) { 224 | if (size < d_writeBufferSize - d_writeBufferPosition) { 225 | ::memcpy(d_writeBuffer + d_writeBufferPosition, buffer, size); 226 | d_writeBufferPosition += size; 227 | return size; 228 | } 229 | // Not sufficient space in the write buffer. Flush it. 230 | flushWriteBuffer(buffer, size); 231 | } else { 232 | // We can't flush. Grow the write buffer if needed. 233 | if (size > d_writeBufferSize - d_writeBufferPosition) { 234 | d_writeBufferSize = d_writeBufferSize * 2; 235 | if (size + d_writeBufferPosition > d_writeBufferSize) { 236 | d_writeBufferSize = (size + d_writeBufferPosition) * 2; 237 | } 238 | char *newBuffer = new char[d_writeBufferSize]; 239 | ::memcpy(newBuffer, d_writeBuffer, d_writeBufferPosition); 240 | delete [] d_writeBuffer; 241 | d_writeBuffer = newBuffer; 242 | } 243 | ::memcpy(d_writeBuffer + d_writeBufferPosition, buffer, size); 244 | d_writeBufferPosition += size; 245 | } 246 | 247 | return size; 248 | } 249 | 250 | void Stream::enableBuffer() 251 | { 252 | d_isReadBuffered = true; 253 | d_isWriteBuffered = true; 254 | } 255 | 256 | void Stream::enableWriteMultiplex() 257 | { 258 | d_isWriteMultiplexed = true; 259 | } 260 | 261 | bool Stream::readMessageFlag(uint32_t *flag) 262 | { 263 | // If there is some data available for reading, we can't read the message flag. Instead, we should read the 264 | // rest of the current chunk into the read buffer. 265 | if (d_readDataLength) { 266 | if (d_readBufferEnd + d_readDataLength > d_readBufferSize) { 267 | // Grow the read buffer if needed. 268 | int newSize = d_readBufferSize * 2; 269 | if (newSize < d_readBufferEnd + d_readDataLength) { 270 | newSize = (d_readBufferEnd + d_readDataLength) * 2; 271 | } 272 | char *newBuffer = new char[newSize]; 273 | ::memcpy(newBuffer, d_readBuffer, d_readBufferEnd); 274 | delete [] d_readBuffer; 275 | d_readBuffer = newBuffer; 276 | d_readBufferSize = newSize; 277 | } 278 | 279 | int bytes = d_io->read(d_readBuffer + d_readBufferEnd, d_readDataLength); 280 | d_readBufferEnd += bytes; 281 | d_readDataLength -= bytes; 282 | return false; 283 | } 284 | 285 | // If nothing is available, return immediately; otherwise make sure the flag is read completely. 286 | int bytes = 0; 287 | bytes = d_io->read(reinterpret_cast(flag), sizeof(*flag)); 288 | 289 | if (bytes == 0) { 290 | return false; 291 | } 292 | 293 | if (bytes < static_cast(sizeof(*flag))) { 294 | readAll(reinterpret_cast(flag) + bytes, sizeof(*flag) - bytes); 295 | } 296 | 297 | return true; 298 | } 299 | 300 | void Stream::readMessageContent(uint32_t flag) 301 | { 302 | int length = flag & 0xffffff; 303 | flag = (flag >> 24) - MSG_BASE; 304 | if (flag == MSG_DATA) { 305 | d_readDataLength = length; 306 | return; 307 | } 308 | 309 | d_messageFlag = flag; 310 | 311 | char *message = new char[length + 1]; 312 | message[length] = 0; 313 | switch (flag) { 314 | case MSG_NOOP: 315 | if (length != 0) { 316 | LOG_FATAL(RSYNC_MSG) << "Invalid noop message" << LOG_END 317 | } 318 | break; 319 | case MSG_IO_ERROR: 320 | if (length != 4) { 321 | LOG_FATAL(RSYNC_MSG) << "Invalid io error message" << LOG_END 322 | } 323 | readAll(message, length); 324 | LOG_FATAL(RSYNC_IOERROR) << "Remote I/O error" << LOG_END 325 | break; 326 | case MSG_SUCCESS: 327 | if (length != 4) { 328 | LOG_FATAL(RSYNC_MSG) << "Invalid success message" << LOG_END 329 | } 330 | readAll(message, length); 331 | LOG_INFO(RSYNC_SUCESS) << "Remote operator successful" << LOG_END 332 | break; 333 | case MSG_NO_SEND: 334 | if (length != 4) { 335 | LOG_FATAL(RSYNC_MSG) << "Invalid no_send message:" << flag << LOG_END 336 | } 337 | readAll(message, length); 338 | LOG_INFO(RSYNC_NOSEND) << "Remote no_send message received" << LOG_END 339 | break; 340 | case MSG_INFO: 341 | readAll(message, length); 342 | stripNewLine(message); 343 | LOG_DEBUG(RSYNC_INFO) << "INFO: " << message << LOG_END 344 | break; 345 | case MSG_ERROR: 346 | case MSG_ERROR_XFER: 347 | readAll(message, length); 348 | stripNewLine(message); 349 | LOG_FATAL(RSYNC_ERROR) << message << LOG_END 350 | break; 351 | case MSG_WARNING: 352 | readAll(message, length); 353 | stripNewLine(message); 354 | LOG_FATAL(RSYNC_WARNING) << message << LOG_END 355 | break; 356 | case MSG_DELETED: 357 | readAll(message, length); 358 | stripNewLine(message); 359 | LOG_DEBUG(RSYNC_DELETE) << message << " has been deleted" << LOG_END 360 | d_deletedFiles.push_back(message); 361 | break; 362 | default: 363 | LOG_FATAL(RSYNC_FLAG) << "Unexpected flag: " << flag << " (" 364 | << d_messageFlag << ":" 365 | << d_isReadBuffered << ":" 366 | << d_isWriteBuffered << ":" 367 | << d_isWriteMultiplexed << ":" 368 | << d_readDataLength << ":" 369 | << d_writeBufferSize << ":" 370 | << d_writeBufferPosition << ":" 371 | << d_readBufferSize << ":" 372 | << d_readBufferPosition << ":" 373 | << d_readBufferEnd << ":" 374 | << d_flushStart << ":" 375 | << d_automaticFlush << ")" 376 | << LOG_END 377 | break; 378 | } 379 | d_messageFlag = -1; 380 | return; 381 | } 382 | 383 | bool Stream::isDataAvailable() 384 | { 385 | if (d_readDataLength > 0) { 386 | return true; 387 | } 388 | 389 | uint32_t flag; 390 | if (readMessageFlag(&flag)) { 391 | readMessageContent(flag); 392 | } 393 | return d_readDataLength > 0; 394 | } 395 | 396 | // Log into the rsync daemon. Not used in the SSH mode. 397 | // 398 | // TODO: This method probably shouldn't belong to this class. Find a better place. 399 | void Stream::login(const char *remoteCommand, int *protocol, std::vector *moduleListing) 400 | { 401 | std::string username; 402 | std::string password; 403 | std::string module; 404 | 405 | d_io->getConnectInfo(&username, &password, &module); 406 | 407 | std::stringstream output; 408 | output << "@RSYNCD: " << *protocol << ".0\n"; 409 | writeLine(output.str(), true); 410 | writeLine(std::string(module), true); 411 | 412 | std::string line = readLine(); 413 | if (line.compare(0, 9, "@RSYNCD: ") != 0) { 414 | LOG_FATAL(RSYNCD_GREETING) << "Greeting not received from the daemon" << LOG_END 415 | return; 416 | } 417 | 418 | int remoteProtocol = atoi(line.c_str() + 9); 419 | if (remoteProtocol >= 31) { 420 | *protocol = 30; 421 | } else if (remoteProtocol == 29 || remoteProtocol == 30) { 422 | *protocol = remoteProtocol; 423 | } else if (remoteProtocol > 0) { 424 | LOG_FATAL(RSYNCD_PROTOCOL) << "Incompatible protocol " << remoteProtocol << LOG_END 425 | return; 426 | } else { 427 | LOG_FATAL(RSYNCD_GREETING) << "Greeting not received from the daemon" << LOG_END 428 | return; 429 | } 430 | 431 | while (true) { 432 | line = readLine(); 433 | if (line.compare(0, 18, "@RSYNCD: AUTHREQD ") == 0) { 434 | Util::md_struct mdContext; 435 | Util::md_init(*protocol, &mdContext); 436 | Util::md_update(*protocol, &mdContext, password.c_str(), password.size()); 437 | std::string challenge = line.substr(18); 438 | Util::md_update(*protocol, &mdContext, challenge.c_str(), challenge.size()); 439 | char digest[16]; 440 | Util::md_final(*protocol, &mdContext, digest); 441 | std::string response = base64_encode(digest, sizeof(digest)); 442 | while (response.back() == '=') { 443 | response.erase(response.size() - 1); 444 | } 445 | output.str(""); 446 | output.clear(); 447 | output << username << " " << response; 448 | writeLine(output.str(), true); 449 | } else if (line.compare(0, 11, "@RSYNCD: OK") == 0) { 450 | 451 | // We need to break up the remote command to extract command line arguments. 452 | std::string command(remoteCommand); 453 | size_t begin = 0; 454 | std::vector args; 455 | 456 | while(begin != std::string::npos) { 457 | if (command[begin] == '"' && command.find_first_of('"', begin + 1)) { 458 | size_t end = command.find_first_of('"', begin + 1); 459 | args.push_back(command.substr(begin + 1, end - begin - 1)); 460 | begin = command.find_first_not_of(" ", end + 1); 461 | } else { 462 | size_t end = command.find_first_of(' ', begin + 1); 463 | if (end != std::string::npos) { 464 | args.push_back(command.substr(begin, end - begin)); 465 | } else { 466 | args.push_back(command.substr(begin)); 467 | } 468 | begin = command.find_first_not_of(" ", end); 469 | } 470 | } 471 | 472 | // Starting from 1 as the rsync executable path is not needed. 473 | for (unsigned int i = 1; i < args.size(); ++i) { 474 | writeLine(args[i], *protocol == 29); 475 | } 476 | writeLine(std::string(""), *protocol == 29); 477 | 478 | return; 479 | } else if (line.compare(0, 8, "@ERROR: ") == 0) { 480 | LOG_FATAL(RSYNCD_ERROR) << "Failed to establish the connection: " << line.c_str() + 8 << LOG_END 481 | return; 482 | } else if (line.compare(0, 13, "@RSYNCD: EXIT") == 0) { 483 | return; 484 | } else { 485 | if (moduleListing) { 486 | moduleListing->push_back(line); 487 | } 488 | } 489 | } 490 | } 491 | 492 | void Stream::flushWriteBuffer(const char *additionalData, int additionalDataLength) 493 | { 494 | if (!d_isWriteBuffered || d_writeBufferPosition == 0) { 495 | return; 496 | } 497 | 498 | if (d_flushStart > 0) { 499 | LOG_FATAL(RSYNC_WRITE) << "Attempt to flush while a previous flush operation is incomplete" << LOG_END 500 | } 501 | 502 | int length = d_writeBufferPosition + additionalDataLength; 503 | if (d_isWriteMultiplexed) { 504 | uint32_t flag = ((MSG_DATA + MSG_BASE) << 24) | (length & 0xffffff); 505 | writeAll(reinterpret_cast(&flag), sizeof(flag)); 506 | } 507 | writeAll(d_writeBuffer, d_writeBufferPosition); 508 | d_writeBufferPosition = 0; 509 | if (additionalDataLength) { 510 | writeAll(additionalData, additionalDataLength); 511 | } 512 | d_writeBufferPosition = 0; 513 | d_flushStart = 0; 514 | } 515 | 516 | bool Stream::tryFlushWriteBuffer() 517 | { 518 | if (d_writeBufferPosition == 0) { 519 | return true; 520 | } 521 | 522 | // 'd_flushStart' must take into account the size of flag (which is 4). 523 | 524 | int bytes = 0; 525 | int flagLength = d_isWriteMultiplexed ? 4 : 0; 526 | if (d_flushStart < flagLength) { 527 | uint32_t flag = ((MSG_DATA + MSG_BASE) << 24) | (d_writeBufferPosition & 0xffffff); 528 | bytes = d_io->write(reinterpret_cast(&flag) + d_flushStart, sizeof(flag) - d_flushStart); 529 | } else { 530 | bytes = d_io->write(d_writeBuffer + d_flushStart - flagLength, d_writeBufferPosition - (d_flushStart - flagLength)); 531 | } 532 | 533 | d_flushStart += bytes; 534 | 535 | if (d_flushStart >= d_writeBufferPosition + flagLength) { 536 | d_writeBufferPosition = 0; 537 | d_flushStart = 0; 538 | return true; 539 | } else { 540 | return false; 541 | } 542 | } 543 | 544 | void Stream::disableAutomaticFlush() 545 | { 546 | if (d_writeBufferPosition > 0) { 547 | flushWriteBuffer(); 548 | } 549 | d_automaticFlush = false; 550 | } 551 | 552 | int Stream::readAll(char *buffer, int size) 553 | { 554 | int bytes = 0; 555 | d_blockedTime = 0; 556 | 557 | while (bytes < size) { 558 | int rc = d_io->read(buffer + bytes, size - bytes); 559 | bytes += rc; 560 | if (rc == 0) { 561 | timedWait(true, "readAll"); 562 | } else { 563 | d_blockedTime = 0; 564 | } 565 | } 566 | 567 | //LOG_INFO(STREAM_DEBUG) << "ReadALL: " << toHex(buffer, size) << LOG_END 568 | 569 | return size; 570 | } 571 | 572 | int Stream::writeAll(const char *buffer, int size) 573 | { 574 | //LOG_INFO(STREAM_DEBUG) << "Write: " << toHex(buffer, size) << LOG_END 575 | 576 | int bytes = 0; 577 | d_blockedTime = 0; 578 | 579 | if (d_uploadLimit > 0) { 580 | // Calculate the current upload speed. Introduct a delay if we're sending too fast. 581 | int n = 5; 582 | int64_t now = TimeUtil::getTimeOfDay(); 583 | if (d_bucketStartTime == 0) { 584 | d_bucketStartTime = now; 585 | d_uploadBuckets.push_back(0); 586 | } else { 587 | while (now - d_bucketStartTime >= n * 1000000) { 588 | d_uploadBuckets.erase(d_uploadBuckets.begin()); 589 | d_uploadBuckets.push_back(0); 590 | d_bucketStartTime += 1000000; 591 | } 592 | while (now - d_bucketStartTime >= d_uploadBuckets.size() * 1000000) { 593 | d_uploadBuckets.push_back(0); 594 | } 595 | } 596 | 597 | double sum = 0.0; 598 | for (unsigned int i = 0; i < d_uploadBuckets.size(); ++i) { 599 | sum += d_uploadBuckets[i]; 600 | } 601 | sum += size / 1024; 602 | 603 | if (sum / d_uploadBuckets.size() > d_uploadLimit) { 604 | 605 | // sum + size / 1024 606 | // --------------------- == uploadLimit 607 | // now + delay - start 608 | 609 | int delay = ((sum / d_uploadLimit) - (now - d_bucketStartTime) / 1000000.0) * 1000.0; 610 | if (delay > 0) { 611 | if (delay > 1000) { 612 | delay = 1000; 613 | } 614 | TimeUtil::sleep(delay); 615 | } 616 | } 617 | 618 | d_uploadBuckets[d_uploadBuckets.size() - 1] += size / 1024; 619 | 620 | } 621 | 622 | while (bytes < size) { 623 | int rc = d_io->write(buffer + bytes, size - bytes); 624 | bytes += rc; 625 | 626 | if (rc == 0) { 627 | timedWait(false, "writeAll"); 628 | 629 | uint32_t flag; 630 | if (readMessageFlag(&flag)) { 631 | readMessageContent(flag); 632 | } 633 | } else { 634 | d_blockedTime = 0; 635 | } 636 | } 637 | 638 | return size; 639 | } 640 | 641 | void Stream::flush() 642 | { 643 | d_io->flush(); 644 | } 645 | 646 | void Stream::timedWait(bool isReading, const char *location) 647 | { 648 | checkCancelFlag(); 649 | 650 | const int maximumTimeout = 600; // in seconds 651 | bool result; 652 | if (isReading) { 653 | result = d_io->isReadable(100); // milliseconds 654 | } else { 655 | result = d_io->isWritable(100); // milliseconds 656 | } 657 | 658 | if (result) { 659 | d_blockedTime = 0; 660 | } else { 661 | if (d_blockedTime == 0) { 662 | d_blockedTime = ::time(0); 663 | } else if ((::time(0) - d_blockedTime) > maximumTimeout / 2) { 664 | d_io->flush(); 665 | if ((::time(0) - d_blockedTime) > maximumTimeout) { 666 | if (!d_io->isClosed()) { 667 | LOG_FATAL(STREAM_TIMEOUT) << "The rsync channel has been blocked for more than " 668 | << maximumTimeout << " seconds (" 669 | << location << ":" 670 | << d_messageFlag << ":" 671 | << d_isReadBuffered << ":" 672 | << d_isWriteBuffered << ":" 673 | << d_isWriteMultiplexed << ":" 674 | << d_readDataLength << ":" 675 | << d_writeBufferSize << ":" 676 | << d_writeBufferPosition << ":" 677 | << d_readBufferSize << ":" 678 | << d_readBufferPosition << ":" 679 | << d_readBufferEnd << ":" 680 | << d_flushStart << ":" 681 | << d_automaticFlush << ")" 682 | << LOG_END 683 | } 684 | } 685 | } 686 | } 687 | } 688 | 689 | void Stream::checkCancelFlag() const 690 | { 691 | if (d_cancelFlag && *d_cancelFlag) { 692 | LOG_FATAL(RSYNC_CANCEL) << "The operation was cancelled by user" << LOG_END 693 | } 694 | } 695 | 696 | void Stream::setUploadLimit(int uploadLimit) 697 | { 698 | d_uploadLimit = uploadLimit; 699 | } 700 | 701 | std::string Stream::toHex(const char *buffer, int size) 702 | { 703 | const char *hex = "O123456789ABCDEF"; 704 | std::string str; 705 | for (int i = 0; i < size; ++i) { 706 | unsigned char c = buffer[i]; 707 | str += hex[c / 16]; 708 | str += hex[c % 16]; 709 | } 710 | return str; 711 | } 712 | 713 | uint8_t Stream::readUInt8() 714 | { 715 | uint8_t i; 716 | read(reinterpret_cast(&i), 1); 717 | return i; 718 | } 719 | 720 | uint16_t Stream::readUInt16() 721 | { 722 | uint16_t i; 723 | read(reinterpret_cast(&i), 2); 724 | return i; 725 | } 726 | 727 | int32_t Stream::readInt32() 728 | { 729 | uint32_t i; 730 | read(reinterpret_cast(&i), 4); 731 | return i; 732 | } 733 | 734 | int64_t Stream::readInt64() 735 | { 736 | int32_t i = readInt32(); 737 | 738 | if (i != -1) { 739 | return i; 740 | } 741 | 742 | uint32_t low = readInt32(); 743 | uint32_t high = readInt32(); 744 | return static_cast(low) | (static_cast(high) << 32); 745 | } 746 | 747 | int32_t Stream::readVariableInt32() 748 | { 749 | uint8_t firstByte = readUInt8(); 750 | int length = getVariableLength(firstByte); 751 | if (length == 0) { 752 | return firstByte; 753 | } else if (length == 4) { 754 | int32_t i; 755 | read(reinterpret_cast(&i), sizeof(i)); 756 | return i; 757 | } else if (length > 0 && length < 4) { 758 | int32_t i = 0; 759 | char *buffer = reinterpret_cast(&i); 760 | read(buffer, length); 761 | uint8_t mask = (1 << (8 - length)) - 1; 762 | buffer[length] = firstByte & mask; 763 | return i; 764 | } else { 765 | LOG_FATAL(STREAM_INT32) << "32 bit interger overflow: " 766 | << toHex(reinterpret_cast(&firstByte), 1) 767 | << LOG_END 768 | return 0; 769 | } 770 | } 771 | 772 | int64_t Stream::readVariableInt64(int minimumBytes) 773 | { 774 | char bytes[9]; 775 | ::memset(bytes, 0, sizeof(bytes)); 776 | read(bytes, minimumBytes); 777 | int extra = getVariableLength(bytes[0]); 778 | if (extra + minimumBytes > 9) { 779 | LOG_FATAL(STREAM_INT64) << "64 bit interger overflow: " 780 | << toHex(bytes, minimumBytes) 781 | << LOG_END 782 | } 783 | if (!extra) { 784 | bytes[minimumBytes] = bytes[0]; 785 | } else { 786 | read(bytes + minimumBytes, extra); 787 | if (extra + minimumBytes < 9) { 788 | char mask = (1 << (8 - extra)) - 1; 789 | bytes[extra + minimumBytes] = bytes[0] & mask; 790 | } 791 | } 792 | ::memcpy(bytes, bytes + 1, 8); 793 | return *reinterpret_cast(bytes); 794 | } 795 | 796 | int32_t Stream::readIndex() 797 | { 798 | bool isPositive; 799 | int32_t lastIndex; 800 | int32_t index; 801 | uint8_t b0 = readUInt8(); 802 | if (b0 == 0xff) { 803 | b0 = readUInt8(); 804 | isPositive = false; 805 | lastIndex = d_lastNegativeIndexRead; 806 | } else if (b0 == 0) { 807 | return INDEX_DONE; 808 | } else { 809 | isPositive = true; 810 | lastIndex = d_lastPositiveIndexRead; 811 | } 812 | 813 | if (b0 == 0xfe) { 814 | b0 = readUInt8(); 815 | uint8_t b1 = readUInt8(); 816 | if (b0 & 0x80) { 817 | int32_t b2 = readUInt8(); 818 | int32_t b3 = readUInt8(); 819 | index = b1 + (b2 << 8) + (b3 << 16) + (int32_t(b0 & 0x7f) << 24); 820 | } else { 821 | index = (int32_t(b0) << 8) + b1 + lastIndex; 822 | } 823 | } else { 824 | index = b0 + lastIndex; 825 | } 826 | 827 | if (isPositive) { 828 | d_lastPositiveIndexRead = index; 829 | } else { 830 | d_lastNegativeIndexRead = index; 831 | index = -index; 832 | } 833 | return index; 834 | } 835 | 836 | void Stream::writeUInt8(uint8_t i) 837 | { 838 | write(reinterpret_cast(&i), 1); 839 | } 840 | 841 | void Stream::writeUInt16(uint16_t i) 842 | { 843 | write(reinterpret_cast(&i), 2); 844 | } 845 | 846 | void Stream::writeInt32(int32_t i) 847 | { 848 | write(reinterpret_cast(&i), 4); 849 | } 850 | 851 | void Stream::writeInt64(int64_t i) 852 | { 853 | if (i <= 0x7fffffff && i >= 0) { 854 | writeInt32(static_cast(i)); 855 | return; 856 | } 857 | 858 | writeInt32(-1); 859 | writeInt32(i & 0xffffffff); 860 | writeInt32(i >> 32); 861 | } 862 | 863 | void Stream::writeVariableInt32(int32_t i) 864 | { 865 | int count = 3; 866 | uint8_t bytes[5] = { 0 }; 867 | ::memcpy(bytes + 1, &i, sizeof(i)); 868 | while (count > 0 && bytes[count + 1] == 0) { 869 | --count; 870 | } 871 | uint8_t mask = 1 << (7 - count); 872 | if (bytes[count + 1] >= mask) { 873 | ++count; 874 | bytes[0] = ~(mask - 1); 875 | } else if (count > 0) { 876 | bytes[0] = bytes[count + 1] | (~(mask * 2 - 1)); 877 | } else { 878 | bytes[0] = bytes[count + 1]; 879 | } 880 | write(reinterpret_cast(bytes), count + 1); 881 | } 882 | 883 | void Stream::writeVariableInt64(int64_t i, int minimumBytes) 884 | { 885 | int count = 7; 886 | uint8_t bytes[9] = { 0 }; 887 | ::memcpy(bytes + 1, &i, sizeof(i)); 888 | while (count >= minimumBytes && bytes[count + 1] == 0) { 889 | --count; 890 | } 891 | uint8_t mask = 1 << (6 - count + minimumBytes); 892 | if (bytes[count + 1] >= mask) { 893 | ++count; 894 | bytes[0] = ~(mask - 1); 895 | } else if (count >= minimumBytes) { 896 | bytes[0] = bytes[count + 1] | (~(mask * 2 - 1)); 897 | } else { 898 | bytes[0] = bytes[count + 1]; 899 | } 900 | write(reinterpret_cast(bytes), count + 1); 901 | } 902 | 903 | void Stream::writeIndex(int32_t index) 904 | { 905 | int32_t diff; 906 | char bytes[6]; 907 | char *ptr = bytes; 908 | if (index >= 0) { 909 | diff = index - d_lastPositiveIndexWritten; 910 | d_lastPositiveIndexWritten = index; 911 | } else if (index == INDEX_DONE) { 912 | writeUInt8(0); 913 | return; 914 | } else { 915 | *ptr++ = 0xffu; 916 | index = -index; 917 | diff = index - d_lastNegativeIndexWritten; 918 | d_lastNegativeIndexWritten = index; 919 | } 920 | 921 | if (diff > 0 && diff < 0xfe) { 922 | *ptr++ = diff; 923 | } else if (diff < 0 || diff > 0x7fff) { 924 | *ptr++ = 0xfeu; 925 | *ptr++ = (index >> 24) | 0x80; 926 | *ptr++ = index & 0xff; 927 | *ptr++ = index >> 8; 928 | *ptr++ = index >> 16; 929 | } else { 930 | *ptr++ = 0xfeu; 931 | *ptr++ = diff >> 8; 932 | *ptr++ = diff & 0xff; 933 | } 934 | write(bytes, ptr - bytes); 935 | } 936 | 937 | std::string Stream::readLine() 938 | { 939 | std::string line; 940 | while (true) { 941 | char c; 942 | 943 | while (readAll(&c, 1) != 1) { 944 | } 945 | 946 | if (c == 0 || c == '\n') { 947 | return line; 948 | } 949 | if (c != '\r') { 950 | line.push_back(c); 951 | } 952 | } 953 | } 954 | 955 | void Stream::writeLine(std::string line, bool appendNewLine) 956 | { 957 | if (appendNewLine && (!line.size() || line[line.size() - 1] != '\n')) { 958 | line.push_back('\n'); 959 | } 960 | int size = line.size(); 961 | if (!appendNewLine) { 962 | ++size; 963 | } 964 | writeAll(line.c_str(), size); 965 | } 966 | 967 | } // namespace rsync 968 | -------------------------------------------------------------------------------- /rsync/rsync_stream.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_STREAM_H 17 | #define INCLUDED_RSYNC_STREAM_H 18 | 19 | #include 20 | 21 | #include 22 | #include 23 | 24 | #include 25 | #include 26 | 27 | namespace rsync 28 | { 29 | 30 | 31 | // IMPORTANT NOTE: 32 | // To save developement time, this class only works on little-endian machines, as the rsync protocol transmits 33 | // integers in little-endian format. Conversion must be added if you want to port this class to big-endian machines. 34 | // 35 | // Blocking model: 36 | // 37 | // - all reads and writes are all blocking 38 | // - writeAll will try to fill the read buffer when it can't write 39 | // - read will read from the read buffer first, and then from the stream until d_readDataLength is 0 40 | // - write will write to the write buffer first. If d_automaticFlush is on, it will call flushWriteBuffer when the 41 | // buffer is full. Otherwise, it will never block and tryFlushWriteBuffer must be called frequently. 42 | // - flushWriteBuffer is blocking 43 | // - tryFlushWriteBuffer is non-blocking 44 | // 45 | // Potential deadlock scenario: 46 | // 47 | // read is trying to complete, but the remote side has nothing to send because 48 | // data is still in the local write buffer and not yet flushed. This can only happen 49 | // in download mode (in upload mode the remote side starts the stream). So make sure 50 | // not to call read() unless the write buffer has been flushed. 51 | 52 | 53 | class Stream 54 | { 55 | public: 56 | 57 | // Create a stream on top of 'io'. If 'cancelFlag' is provide, whenever '*cancelFlag' becomes non-zero, any 58 | // operation, even a blocking one, will be terminated immediately. 59 | Stream(IO *io, int *cancelFlag = 0); 60 | ~Stream(); 61 | 62 | // Reset the steam to the intial state. 63 | void reset(); 64 | 65 | enum {INDEX_DONE = -1 }; 66 | 67 | // Read 'size' bytes to 'buffer'. Return 'size' until all bytes have been read. Note that a write can be 68 | // buffered or unbuffered 69 | int read(char *buffer, int size); 70 | 71 | // Wrtie 'size' bytes from 'buffer'. Return 'size' until all bytes have been written. Note that a write can be 72 | // buffered or unbuffered 73 | int write(const char *buffer, int size); 74 | 75 | // Send out the content in the write buffer. 76 | void flush(); 77 | 78 | // Check if '*d_cancelFlag' has been set. 79 | void checkCancelFlag() const; 80 | 81 | // A special method for loggin into an rsync daemon. 82 | void login(const char *remoteCommand, int *protocol, std::vector *modules); 83 | 84 | // Turn on read and write buffering. 85 | void enableBuffer(); 86 | 87 | // Turn on write multiplexing, which means it can send not just data, but also other kinds of messages, like 88 | // logging messages. Multiplexing is always on for reads. 89 | void enableWriteMultiplex(); 90 | 91 | // Flush the write buffer. Can take additional data for convenience. 92 | void flushWriteBuffer(const char *additionalData = 0, int additionalLength = 0); 93 | 94 | // Attempt to flush the write buffer. Once the flush starts it won't return until done. 95 | bool tryFlushWriteBuffer(); 96 | 97 | // Normally when the write buffer is full it will flush by itself. This method is to disable this behavior. 98 | void disableAutomaticFlush(); 99 | 100 | // If there is any data available for reading. 101 | bool isDataAvailable(); 102 | 103 | // Limit how fast to send out data. 104 | void setUploadLimit(int uploadLimit); 105 | 106 | // Read a unsigned 8-bit integer 107 | uint8_t readUInt8(); 108 | 109 | // Methods for receiving integers or strings. 110 | uint16_t readUInt16(); 111 | int32_t readInt32(); 112 | int64_t readInt64(); 113 | int32_t readVariableInt32(); // in variable-length integer format 114 | int64_t readVariableInt64(int minimumBytes); // in variable-length integer format 115 | int32_t readIndex(); 116 | std::string readLine(); 117 | 118 | // Methods for sending integers or strings 119 | void writeUInt8(uint8_t); 120 | void writeUInt16(uint16_t); 121 | void writeInt32(int32_t); 122 | void writeInt64(int64_t); 123 | void writeVariableInt32(int32_t); // in variable-length integer format 124 | void writeVariableInt64(int64_t, int minimumBytes); // in variable-length integer format 125 | void writeIndex(int32_t index); 126 | void writeLine(std::string line, bool appendNewLine); 127 | 128 | // Files deleted on the remote server are sent as DELETION messages. 129 | const std::vector &getDeletedFiles() const 130 | { 131 | return d_deletedFiles; 132 | } 133 | 134 | private: 135 | // NOT IMPLEMENTED 136 | Stream(const Stream&); 137 | Stream& operator=(const Stream&); 138 | 139 | // Read 'size' bytes into 'buffer'. Will throw an exception on timeout. 140 | int readAll(char *buffer, int size); 141 | 142 | // Write 'size' bytes from 'buffer'. Will throw an exception on timeout. May also attempt to fill the read buffer 143 | // if there is any data. 144 | int writeAll(const char *buffer, int size); 145 | 146 | // Check if the operation timeouts. 147 | void timedWait(bool isReading, const char *location); 148 | 149 | // Read the message flag (which indicates whether it is a data message or other kinds). This is a non-blocking call. 150 | bool readMessageFlag(uint32_t *flag); 151 | 152 | // Read the message content. This is a blocking call. 153 | void readMessageContent(uint32_t flag); 154 | 155 | // Convert to a hexical string for easy display. 156 | std::string toHex(const char *buffer, int size); 157 | 158 | IO *d_io; // The io channel 159 | 160 | int *d_cancelFlag; // The pointer to the cancellation flag 161 | 162 | bool d_isReadBuffered; // If the read buffer is being used 163 | bool d_isWriteBuffered; // If the write buffer is being used 164 | 165 | bool d_isWriteMultiplexed; // If writing is multiplexed 166 | 167 | int d_readDataLength; // How many bytes are available for read after the data flag 168 | 169 | char *d_writeBuffer; // All writes will go to this buffer first in the buffered mode 170 | int d_writeBufferSize; // Size of the write buffer 171 | int d_writeBufferPosition; // The start of empty space in the writer buffer 172 | 173 | char *d_readBuffer; // This buffer is used to store data received during write operations 174 | int d_readBufferSize; // Size of the read buffer 175 | int d_readBufferPosition; // The start of new data in the read buffer 176 | int d_readBufferEnd; // The end of data in the read buffer 177 | 178 | int d_flushStart; // Data before this position have been flushed 179 | bool d_automaticFlush; // If the write buffer will be automatically flushed 180 | 181 | int d_lastPositiveIndexRead; // There variables are used by readIndex() and writeIndex(); 182 | int d_lastNegativeIndexRead; 183 | int d_lastPositiveIndexWritten; 184 | int d_lastNegativeIndexWritten; 185 | 186 | time_t d_blockedTime; // The first moment when read/write becomes blcoked 187 | 188 | int d_uploadLimit; // How fast to limit sending 189 | std::vector d_uploadBuckets; // For speed limiting; each bucket contains the number of bytes sent 190 | // during a period of 1 second 191 | int64_t d_bucketStartTime; // The time of the earliest counting bucket 192 | 193 | std::vector d_deletedFiles; // Stores files deleted on the remote server 194 | 195 | int d_messageFlag; // The message flag; currently only used for tracing 196 | }; 197 | 198 | } // namespace rsync 199 | 200 | #endif // INCLUDED_RSYNC_STREAM_H 201 | -------------------------------------------------------------------------------- /rsync/rsync_timeutil.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | 20 | 21 | #if defined(WIN32) || defined(__MINGW32__) 22 | 23 | #include 24 | 25 | namespace rsync 26 | { 27 | 28 | int64_t TimeUtil::getUnixTime(uint32_t windowsTimeHigh, uint32_t windowsTimeLow) 29 | { 30 | return ((uint64_t(windowsTimeHigh) << 32) | windowsTimeLow) / 10000000 - 11644473600ll; 31 | } 32 | 33 | void TimeUtil::getWindowsTime(int64_t unixTime, uint32_t *windowsTimeHigh, uint32_t *windowsTimeLow) 34 | { 35 | unixTime += 11644473600ll; 36 | unixTime *= 10000000; 37 | *windowsTimeHigh = unixTime >> 32; 38 | *windowsTimeLow = unixTime & 0xffffffff; 39 | } 40 | 41 | int64_t TimeUtil::getTimeOfDay() 42 | { 43 | FILETIME windowTime; 44 | GetSystemTimeAsFileTime(&windowTime); 45 | 46 | uint64_t unixTime = ((uint64_t(windowTime.dwHighDateTime) << 32) | windowTime.dwLowDateTime) - 116444736000000000ULL; 47 | unixTime /= 10; 48 | return unixTime; 49 | } 50 | 51 | void TimeUtil::sleep(int milliseconds) 52 | { 53 | Sleep(milliseconds); 54 | } 55 | 56 | } // close namespace rsync 57 | 58 | #else // non-windows 59 | 60 | #include 61 | #include 62 | 63 | namespace rsync { 64 | 65 | int64_t TimeUtil::getTimeOfDay() 66 | { 67 | struct timeval t; 68 | gettimeofday(&t, 0); 69 | int64_t value = static_cast(t.tv_sec); 70 | value = value * 1000000 + t.tv_usec; 71 | return value; 72 | } 73 | 74 | void TimeUtil::sleep(int milliseconds) 75 | { 76 | struct timespec delay; 77 | delay.tv_sec = milliseconds / 1000; 78 | delay.tv_nsec = 1000000 * (milliseconds % 1000); 79 | nanosleep(&delay, 0); 80 | } 81 | 82 | } // close namesapce rsync 83 | 84 | #endif 85 | 86 | 87 | 88 | -------------------------------------------------------------------------------- /rsync/rsync_timeutil.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_TIMEUTIL_H 17 | #define INCLUDED_RSYNC_TIMEUTIL_H 18 | 19 | #include 20 | 21 | #include 22 | 23 | namespace rsync 24 | { 25 | 26 | struct TimeUtil 27 | { 28 | #if defined(WIN32) || defined(__MINGW32__) 29 | // Convert a Windows time into a Unix time. 30 | static int64_t getUnixTime(uint32_t windowsTimeHigh, uint32_t windowsTimeLow); 31 | 32 | // Convert a Unix time to a Windows time. 33 | static void getWindowsTime(int64_t unixTime, uint32_t *windowsTimeHigh, uint32_t *windowsTimeLow); 34 | #endif 35 | 36 | // Return current time in microseconds. 37 | static int64_t getTimeOfDay(); 38 | 39 | // Sleep for the specified time. 40 | static void sleep(int milliseconds); 41 | }; 42 | 43 | } // namespace rsync 44 | #endif //INCLUDED_RSYNC_TIMEUTIL_H 45 | -------------------------------------------------------------------------------- /rsync/rsync_util.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | #if defined(WIN32) || defined(__MINGW32__) 23 | #include 24 | #else 25 | #include 26 | #include 27 | #endif 28 | 29 | #include 30 | 31 | namespace rsync 32 | { 33 | 34 | void Util::md_init(int protocol, md_struct *context) 35 | { 36 | if (protocol >= 30) { 37 | MD5_Init(&context->md5); 38 | } else { 39 | MD4_Init(&context->md4); 40 | } 41 | } 42 | 43 | void Util::md_update(int protocol, md_struct *context, const char *data, int size) 44 | { 45 | if (protocol >= 30) { 46 | MD5_Update(&context->md5, data, size); 47 | } else { 48 | MD4_Update(&context->md4, data, size); 49 | } 50 | 51 | } 52 | 53 | void Util::md_final(int protocol, md_struct *context, char digest[16]) 54 | { 55 | if (protocol >= 30) { 56 | MD5_Final(reinterpret_cast(digest), &context->md5); 57 | } else { 58 | MD4_Final(reinterpret_cast(digest), &context->md4); 59 | } 60 | 61 | } 62 | 63 | void Util::startup() 64 | { 65 | rsync::SocketUtil::startup(); 66 | rsync::SSHIO::startup(); 67 | } 68 | 69 | void Util::cleanup() 70 | { 71 | rsync::SocketUtil::cleanup(); 72 | rsync::SSHIO::cleanup(); 73 | } 74 | 75 | std::string Util::getLastError() 76 | { 77 | #if defined(WIN32) || defined(__MINGW32__) 78 | LPVOID lpMsgBuf; 79 | FormatMessage( 80 | FORMAT_MESSAGE_ALLOCATE_BUFFER | 81 | FORMAT_MESSAGE_FROM_SYSTEM | 82 | FORMAT_MESSAGE_IGNORE_INSERTS, 83 | NULL, 84 | GetLastError(), 85 | MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), 86 | (LPTSTR) &lpMsgBuf, 87 | 0, NULL ); 88 | std::string message((char*)lpMsgBuf); 89 | LocalFree(lpMsgBuf); 90 | return message; 91 | #else 92 | return strerror(errno); 93 | #endif 94 | } 95 | 96 | void Util::tokenize(const std::string line, std::vector *results, const char *delimiters) 97 | { 98 | size_t begin = 0; 99 | while(begin != std::string::npos) { 100 | size_t end = line.find_first_of(delimiters, begin); 101 | if (end == std::string::npos) { 102 | std::string token = line.substr(begin, end); 103 | if (token.size()) { 104 | results->push_back(token); 105 | } 106 | return; 107 | } 108 | if (end != begin) { 109 | results->push_back(line.substr(begin, end - begin)); 110 | } 111 | begin = line.find_first_not_of(delimiters, end); 112 | } 113 | } 114 | 115 | #if defined(WIN32) || defined(__MINGW32__) 116 | 117 | bool Util::isProcessElevated() 118 | { 119 | bool isElevated = false; 120 | HANDLE hToken = NULL; 121 | if (OpenProcessToken(GetCurrentProcess( ), TOKEN_QUERY, &hToken)) { 122 | TOKEN_ELEVATION elevation; 123 | DWORD cbSize = sizeof(TOKEN_ELEVATION); 124 | if(GetTokenInformation(hToken, TokenElevation, &elevation, sizeof(elevation), &cbSize)) { 125 | isElevated = elevation.TokenIsElevated; 126 | } 127 | } 128 | if (hToken) { 129 | CloseHandle(hToken); 130 | } 131 | return isElevated; 132 | } 133 | 134 | bool elevateProcess() 135 | { 136 | // Elevate the process. 137 | wchar_t szPath[MAX_PATH]; 138 | if (!GetModuleFileNameW(NULL, szPath, ARRAYSIZE(szPath))) { 139 | return false; 140 | } 141 | 142 | // Launch itself as administrator. 143 | SHELLEXECUTEINFOW sei = { sizeof(sei) }; 144 | sei.lpVerb = L"runas"; 145 | sei.lpFile = szPath; 146 | sei.hwnd = NULL; 147 | sei.nShow = SW_NORMAL; 148 | 149 | if (!ShellExecuteExW(&sei)) { 150 | DWORD dwError = GetLastError(); 151 | if (dwError == ERROR_CANCELLED) { 152 | return false; 153 | } 154 | } 155 | return true; 156 | } 157 | #endif 158 | 159 | } // close namesapce rsysnc 160 | 161 | -------------------------------------------------------------------------------- /rsync/rsync_util.h: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #ifndef INCLUDED_RSYNC_UTIL_H 17 | #define INCLUDED_RSYNC_UTIL_H 18 | 19 | #include 20 | 21 | #include 22 | #include 23 | 24 | #include 25 | #include 26 | 27 | #include 28 | 29 | namespace rsync 30 | { 31 | 32 | struct Util 33 | { 34 | // A helpfer class for deleting entries in an entry list. 35 | class EntryListReleaser 36 | { 37 | public: 38 | EntryListReleaser(std::vector *entryList): 39 | d_entryList(entryList) 40 | { 41 | } 42 | 43 | ~EntryListReleaser() 44 | { 45 | for (unsigned int i = 0; i < d_entryList->size(); ++i) { 46 | delete (*d_entryList)[i]; 47 | } 48 | } 49 | private: 50 | // NOT IMPLEMENTED 51 | EntryListReleaser(const EntryListReleaser&); 52 | EntryListReleaser& operator=(const EntryListReleaser&); 53 | 54 | std::vector *d_entryList; 55 | }; 56 | 57 | // For calculating MD4 or MD5 checksums based on the rsync protocol version 58 | union md_struct 59 | { 60 | MD4_CTX md4; 61 | MD5_CTX md5; 62 | }; 63 | static void md_init(int protocol, md_struct *context); 64 | static void md_update(int protocol, md_struct *context, const char *data, int size); 65 | static void md_final(int protocol, md_struct *context, char digest[16]); 66 | 67 | // For proper intialization and shutdown of dependency libraries. 68 | static void startup(); 69 | static void cleanup(); 70 | 71 | // Return the last error in a readable format. 72 | static std::string getLastError(); 73 | 74 | // Break up a string separated by the delimiters. 75 | static void tokenize(const std::string line, std::vector *results, const char *delimiters); 76 | 77 | #if defined(WIN32) || defined(__MINGW32__) 78 | 79 | // If the process has elevated priviledges. 80 | static bool isProcessElevated(); 81 | 82 | // Elevate the process to have a higher priviledge level. 83 | static bool elevateProcess(); 84 | #endif 85 | }; 86 | 87 | } 88 | 89 | #endif //INCLUDED_RSYNC_UTIL_H 90 | -------------------------------------------------------------------------------- /rsync/t_rsync_client.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #include 26 | #include 27 | 28 | #include 29 | #include 30 | 31 | #include 32 | #include 33 | #include 34 | #include 35 | #include 36 | 37 | #include 38 | #include 39 | 40 | //qi: TEST_PROGRAM = 1 41 | //qi: LDFLAGS += -lssh2 -lssl -lcrypto -lz 42 | #include 43 | 44 | int g_cancelFlag = 0; 45 | 46 | #if defined(WIN32) || defined(__MINGW32__) 47 | 48 | #include 49 | BOOL CtrlHandler( DWORD fdwCtrlType ) 50 | { 51 | if (fdwCtrlType == CTRL_C_EVENT) { 52 | g_cancelFlag = 1; 53 | return TRUE; 54 | } 55 | return FALSE; 56 | } 57 | 58 | #endif 59 | 60 | using namespace rsync; 61 | 62 | int main(int argc, char *argv[]) 63 | { 64 | TESTUTIL_INIT_RAND 65 | 66 | if (argc < 7) { 67 | ::printf("Usage: %s action server user password remote_dir local_dir\n", argv[0]); 68 | ::printf(" where action is either 'download' or 'upload'\n"); 69 | return 0; 70 | } 71 | 72 | #if defined(WIN32) || defined(__MINGW32__) 73 | SetConsoleCtrlHandler((PHANDLER_ROUTINE)CtrlHandler, TRUE); 74 | #endif 75 | 76 | SocketUtil::startup(); 77 | 78 | int rc = libssh2_init(0); 79 | if (rc != 0) { 80 | LOG_ERROR(LIBSSH2_INIT) << "libssh2 initialization failed: " << rc << LOG_END 81 | return 0; 82 | } 83 | 84 | const char *action = argv[1]; 85 | const char *server = argv[2]; 86 | const char *user = argv[3]; 87 | const char *password = argv[4]; 88 | 89 | // Make sure remoteDir ends with '/' to indicate it is a directory 90 | std::string remoteDir = argv[5]; 91 | if (remoteDir.back() != '/') { 92 | remoteDir = remoteDir + "/"; 93 | } 94 | 95 | std::string localDir = PathUtil::join(PathUtil::getCurrentDirectory().c_str(), argv[6]); 96 | std::string temporaryFile = PathUtil::join(PathUtil::getCurrentDirectory().c_str(), "acrosync.part"); 97 | 98 | Log::setLevel(Log::Debug); 99 | 100 | try { 101 | SSHIO sshio; 102 | sshio.connect(server, 22, user, password, 0, 0); 103 | 104 | Client client(&sshio, "rsync", 32, &g_cancelFlag); 105 | 106 | if (::strcasecmp(action, "download") == 0) { 107 | client.download(localDir.c_str(), remoteDir.c_str(), temporaryFile.c_str(), 0); 108 | } else if (::strcasecmp(action, "upload") == 0) { 109 | client.upload(localDir.c_str(), remoteDir.c_str()); 110 | } else { 111 | printf("Invalid action: %s\n", action); 112 | } 113 | } catch (Exception &e) { 114 | LOG_ERROR(RSYNC_ERROR) << "Sync failed: " << e.getMessage() << LOG_END 115 | } 116 | 117 | libssh2_exit(); 118 | SocketUtil::cleanup(); 119 | 120 | return 0; 121 | } 122 | -------------------------------------------------------------------------------- /rsync/t_rsync_entry.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | 18 | #include 19 | #include 20 | 21 | #include 22 | #include 23 | 24 | #include 25 | 26 | 27 | //qi: TEST_PROGRAM = 1 28 | #include 29 | 30 | using namespace rsync; 31 | 32 | int main(int /* argc */, char ** /* argv */) 33 | { 34 | TESTUTIL_INIT_RAND; 35 | 36 | const char *unsortedFiles[] = { 37 | "x", 38 | "d/c/f", 39 | "d/", 40 | "d/e", 41 | "d/c/", 42 | "d/c ", 43 | "ad/", 44 | "ad/ef", 45 | "b", 46 | "f", 47 | }; 48 | 49 | const char *sortedFiles[] = { 50 | "b", 51 | "f", 52 | "x", 53 | "ad/", 54 | "ad/ef", 55 | "d/", 56 | "d/c ", 57 | "d/e", 58 | "d/c/", 59 | "d/c/f", 60 | }; 61 | 62 | std::vector list; 63 | for (size_t i = 0; i < sizeof(unsortedFiles) / sizeof(unsortedFiles[0]); ++i) { 64 | Entry *entry = 65 | new Entry(unsortedFiles[i], 66 | unsortedFiles[i][::strlen(unsortedFiles[i]) - 1] == '/', 0, 0, 0); 67 | list.push_back(entry); 68 | } 69 | 70 | std::sort(list.begin(), list.end(), Entry::compareGlobally); 71 | 72 | for (size_t i = 0; i < list.size(); ++i) { 73 | //printf("%s\n", list[i]->getPath()); 74 | ASSERT(::strcmp(sortedFiles[i], list[i]->getPath()) == 0); 75 | } 76 | 77 | for (size_t i = 0; i < list.size(); ++i) { 78 | delete list[i]; 79 | } 80 | return ASSERT_COUNT; 81 | } 82 | -------------------------------------------------------------------------------- /rsync/t_rsync_fileutil.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | #include 17 | #include 18 | 19 | #include 20 | 21 | #include 22 | #include 23 | 24 | #include 25 | 26 | //qi: TEST_PROGRAM = 1 27 | #include 28 | 29 | #include 30 | 31 | using namespace rsync; 32 | 33 | void createFile(const char *top, const char *name, const char *content) 34 | { 35 | File f(PathUtil::join(top, name).c_str(), true); 36 | ASSERT(f.isValid()); 37 | int bytes = f.write(content, ::strlen(content)); 38 | ASSERT(bytes > 0); 39 | ASSERT(bytes == ::strlen(content)); 40 | } 41 | 42 | std::string getFileChecksum(const char *file) 43 | { 44 | char buffer[32 * 1024]; 45 | 46 | MD5_CTX md5; 47 | MD5_Init(&md5); 48 | 49 | File f(file, false); 50 | int bytes = 0; 51 | while ((bytes = f.read(buffer, sizeof buffer)) > 0) { 52 | MD5_Update(&md5, buffer, bytes); 53 | } 54 | 55 | unsigned char digest[16]; 56 | MD5_Final(digest, &md5); 57 | 58 | return std::string(reinterpret_cast(&digest[0]), sizeof(digest)); 59 | } 60 | 61 | 62 | int main(int /* argc */, char ** /* argv */) 63 | { 64 | TESTUTIL_INIT_RAND; 65 | 66 | std::string top = PathUtil::getCurrentDirectory(); 67 | top = PathUtil::join(top.c_str(), "test_dir"); 68 | 69 | PathUtil::removeDirectoryRecursively(top.c_str()); 70 | 71 | ASSERT(PathUtil::createDirectory(top.c_str())); 72 | 73 | ASSERT(PathUtil::createIntermediateDirectories(top.c_str(), "test_dir1/test_dir2/test_dir3")); 74 | ASSERT(PathUtil::createDirectory(PathUtil::join(top.c_str(), "test_dir1/test_dir2/test_dir3").c_str())); 75 | 76 | ASSERT(PathUtil::createDirectory(PathUtil::join(top.c_str(), "test_dir4").c_str())); 77 | 78 | const char *content = "this is just a test"; 79 | createFile(top.c_str(), "test_file1", content); 80 | 81 | std::string checksum = getFileChecksum(PathUtil::join(top.c_str(), "test_file1").c_str()); 82 | 83 | char buffer[256]; 84 | File f(PathUtil::join(top.c_str(), "test_file1").c_str()); 85 | ASSERT(f.isValid()); 86 | ASSERT(f.read(buffer, 4) == 4); 87 | ASSERT(::strncmp(buffer, "this", 4) == 0); 88 | ASSERT(f.read(buffer, 4) == 4); 89 | ASSERT(::strncmp(buffer, " is ", 4) == 0); 90 | ASSERT(f.seek(File::SEEK_FROM_BEGIN, 0) == 0); 91 | ASSERT(f.read(buffer, ::strlen(content)) == ::strlen(content)); 92 | ASSERT(::strncmp(buffer, content, ::strlen(content)) == 0); 93 | ASSERT(f.read(buffer, ::strlen(content)) == 0); 94 | f.close(); 95 | 96 | createFile(top.c_str(), "test_file2", "this is another test"); 97 | 98 | // Call these two 'file' but they are acutally directories 99 | // to verify the sorting result. 100 | ASSERT(PathUtil::createDirectory(PathUtil::join(top.c_str(), "test_file4").c_str())); 101 | ASSERT(PathUtil::createDirectory(PathUtil::join(top.c_str(), "test_file3").c_str())); 102 | 103 | std::vector files, directories; 104 | 105 | PathUtil::listDirectory(top.c_str(), "", &files, &directories); 106 | 107 | ASSERT(files.size() == 2); 108 | ASSERT(::strcmp(files[0]->getPath(), "test_file1") == 0); 109 | ASSERT(::strcmp(files[1]->getPath(), "test_file2") == 0); 110 | 111 | ASSERT(directories.size() == 4); 112 | ASSERT(::strcmp(directories[0]->getPath(), "test_file4/") == 0); 113 | ASSERT(::strcmp(directories[1]->getPath(), "test_file3/") == 0); 114 | ASSERT(::strcmp(directories[2]->getPath(), "test_dir4/") == 0); 115 | ASSERT(::strcmp(directories[3]->getPath(), "test_dir1/") == 0); 116 | 117 | for (unsigned int i = 0; i < files.size(); ++i) { 118 | delete files[i]; 119 | } 120 | for (unsigned int i = 0; i < directories.size(); ++i) { 121 | delete directories[i]; 122 | } 123 | PathUtil::removeDirectoryRecursively(top.c_str()); 124 | return ASSERT_COUNT; 125 | } 126 | -------------------------------------------------------------------------------- /rsync/t_rsync_stream.cpp: -------------------------------------------------------------------------------- 1 | // Copyright (C) 2015 Acrosync LLC 2 | // 3 | // Unless explicitly acquired and licensed from Licensor under another 4 | // license, the contents of this file are subject to the Reciprocal Public 5 | // License ("RPL") Version 1.5, or subsequent versions as allowed by the RPL, 6 | // and You may not copy or use this file in either source code or executable 7 | // form, except in compliance with the terms and conditions of the RPL. 8 | // 9 | // All software distributed under the RPL is provided strictly on an "AS 10 | // IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, AND 11 | // LICENSOR HEREBY DISCLAIMS ALL SUCH WARRANTIES, INCLUDING WITHOUT 12 | // LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 13 | // PURPOSE, QUIET ENJOYMENT, OR NON-INFRINGEMENT. See the RPL for specific 14 | // language governing rights and limitations under the RPL. 15 | 16 | 17 | #include 18 | 19 | #include 20 | #include 21 | #include 22 | 23 | #include 24 | #include 25 | 26 | //qi: TEST_PROGRAM = 1 27 | #include 28 | 29 | 30 | using namespace rsync; 31 | 32 | class StringIO : public IO 33 | { 34 | public: 35 | StringIO(std::string &data) 36 | : IO() 37 | , d_data(data) 38 | , d_position(0) 39 | { 40 | } 41 | 42 | virtual ~StringIO() 43 | { 44 | } 45 | 46 | virtual int read(char *buffer, int size) 47 | { 48 | int bytes = size; 49 | if (d_position + size > int(d_data.size())) { 50 | bytes = d_data.size() - d_position; 51 | } 52 | ::memcpy(buffer, d_data.c_str() + d_position, size); 53 | d_position += bytes; 54 | return bytes; 55 | } 56 | 57 | virtual int write(const char *buffer, int size) 58 | { 59 | d_data += std::string(buffer, size); 60 | return size; 61 | } 62 | 63 | virtual bool isReadable(int /* timeoutInMilliSeconds */) 64 | { 65 | return true; 66 | } 67 | 68 | virtual bool isWritable(int /* timeoutInMilliSeconds */) 69 | { 70 | return true; 71 | } 72 | 73 | virtual bool isClosed() 74 | { 75 | return false; 76 | } 77 | 78 | void reset() 79 | { 80 | d_position = 0; 81 | } 82 | 83 | void createChannel(const char*, int*) {} 84 | 85 | void closeChannel() {} 86 | 87 | void getConnectInfo(std::string*, std::string*, std::string*) {} 88 | 89 | void flush() {} 90 | 91 | 92 | private: 93 | // NOT IMPLEMENTED 94 | StringIO(const StringIO&); 95 | StringIO& operator=(const StringIO&); 96 | 97 | std::string& d_data; 98 | int d_position; 99 | }; 100 | 101 | void testReadWriteVariableInt32() 102 | { 103 | int32_t DATA[] = { 104 | 0, 1, 127, 128, 129, 105 | 0xffff, 0x10000, 0x10001, 106 | 0xffffff, 0x1000000, 0x1000001, 107 | 0x7ffffffe, 0x7fffffff, 108 | }; 109 | 110 | std::string data; 111 | StringIO stringIO(data); 112 | Stream stream(&stringIO); 113 | for (unsigned int i = 0; i < sizeof(DATA) / sizeof(DATA[0]); ++i) { 114 | stream.writeVariableInt32(DATA[i]); 115 | } 116 | 117 | stringIO.reset(); 118 | 119 | for (unsigned int i = 0; i < sizeof(DATA) / sizeof(DATA[0]); ++i) { 120 | int32_t n = stream.readVariableInt32(); 121 | ASSERT(n == DATA[i]); 122 | } 123 | } 124 | 125 | void testReadWriteInt64() 126 | { 127 | int64_t DATA[] = { 128 | 0, 1, 127, 128, 129, 0xffff, 0x10000, 0x10001, 129 | 0xffffff, 0x1000000, 0x1000001, 130 | 0xffffffffll, 0x100000000ll, 0x100000001ll, 131 | 0xffffffffffll, 0x10000000000ll, 0x10000000001ll, 132 | 0xffffffffffffll, 0x1000000000000ll, 0x1000000000001ll, 133 | 0xffffffffffffffll, 0x100000000000000ll, 0x100000000000001ll, 134 | 0x7fffffffffffffffll, 135 | }; 136 | 137 | std::string data; 138 | StringIO stringIO(data); 139 | Stream stream(&stringIO); 140 | for (unsigned int i = 0; i < sizeof(DATA) / sizeof(DATA[0]); ++i) { 141 | stream.writeInt64(DATA[i]); 142 | } 143 | 144 | stringIO.reset(); 145 | 146 | for (unsigned int i = 0; i < sizeof(DATA) / sizeof(DATA[0]); ++i) { 147 | int64_t n = stream.readInt64(); 148 | ASSERT(n == DATA[i]); 149 | } 150 | } 151 | 152 | void testReadWriteVariableInt64() 153 | { 154 | int64_t DATA[] = { 155 | 0, 1, 127, 128, 129, 0xffff, 0x10000, 0x10001, 156 | 0xffffff, 0x1000000, 0x1000001, 157 | 0xffffffffll, 0x100000000ll, 0x100000001ll, 158 | 0xffffffffffll, 0x10000000000ll, 0x10000000001ll, 159 | 0xffffffffffffll, 0x1000000000000ll, 0x1000000000001ll, 160 | 0xffffffffffffffll, 0x100000000000000ll, 0x100000000000001ll, 161 | 0x7fffffffffffffffll, 162 | }; 163 | 164 | std::string data; 165 | StringIO stringIO(data); 166 | Stream stream(&stringIO); 167 | for (unsigned int i = 0; i < sizeof(DATA) / sizeof(DATA[0]); ++i) { 168 | stream.writeVariableInt64(DATA[i], 3); 169 | } 170 | 171 | stringIO.reset(); 172 | 173 | for (unsigned int i = 0; i < sizeof(DATA) / sizeof(DATA[0]); ++i) { 174 | int64_t n = stream.readVariableInt64(3); 175 | ASSERT(n == DATA[i]); 176 | } 177 | } 178 | 179 | void testReadWriteIndex() 180 | { 181 | int32_t DATA[] = { 182 | 0, 1, 2, 4, -1, 127, -127, 183 | 0xffff, -0xffff, -0x10000, -0x20000, 184 | 0xffff, 0x10000, 0x20000, 0x100000, 0x200000, 0x1000000, 185 | 20000, 1000, 100, 50, 25 186 | }; 187 | 188 | std::string data; 189 | StringIO stringIO(data); 190 | Stream stream(&stringIO); 191 | for (unsigned int i = 0; i < sizeof(DATA) / sizeof(DATA[0]); ++i) { 192 | stream.writeIndex(DATA[i]); 193 | } 194 | 195 | stringIO.reset(); 196 | 197 | for (unsigned int i = 0; i < sizeof(DATA) / sizeof(DATA[0]); ++i) { 198 | int32_t n = stream.readIndex(); 199 | ASSERT(n == DATA[i]); 200 | } 201 | } 202 | int main(int /* argc */, char ** /* argv */) 203 | { 204 | testReadWriteInt64(); 205 | testReadWriteVariableInt32(); 206 | testReadWriteVariableInt64(); 207 | testReadWriteIndex(); 208 | return 0; 209 | } 210 | -------------------------------------------------------------------------------- /testutil/testutil_assert.h: -------------------------------------------------------------------------------- 1 | #ifndef TESTUTIL_ASSERT 2 | #define TESTUTIL_ASSERT 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | //============================================================================ 9 | // Standard Test Macros 10 | //============================================================================ 11 | 12 | unsigned int testutil_seed = 0; 13 | static int ASSERT_COUNT = 0; 14 | static void TESTUTIL_ASSERT_FUNCT(int c, const char *msg, const char *file, int line) 15 | { 16 | if (c) { 17 | std::cout << "[" << testutil_seed << "] Error " << file 18 | << "(" << line << "): " << msg 19 | << " (failed)" << std::endl; 20 | ++ASSERT_COUNT; 21 | } 22 | } 23 | #define ASSERT(X) { TESTUTIL_ASSERT_FUNCT(!(X), #X, __FILE__, __LINE__); } 24 | 25 | #define TESTUTIL_INIT_RAND \ 26 | { \ 27 | testutil_seed = static_cast(std::time(0)); \ 28 | std::srand(testutil_seed); \ 29 | } 30 | 31 | #endif // TESTUTIL_ASSERT 32 | -------------------------------------------------------------------------------- /testutil/testutil_newdeletemonitor.h: -------------------------------------------------------------------------------- 1 | #ifndef TESTUTIL_NEWDELETEMONITOR 2 | #define TESTUTIL_NEWDELETEMONITOR 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | //============================================================================ 9 | // New/Delete Overriding for Checking Memory Leak 10 | //============================================================================ 11 | 12 | namespace { 13 | 14 | static int BLOCKS_IN_USE = -1; 15 | 16 | void checkMemoryOnExit() 17 | { 18 | if (BLOCKS_IN_USE) { 19 | std::fprintf(stderr, "Memory Leak: %d block(s) not released\n", BLOCKS_IN_USE); 20 | } 21 | } 22 | 23 | void *allocateMemory(std::size_t size) 24 | { 25 | if (BLOCKS_IN_USE == -1) { 26 | BLOCKS_IN_USE = 0; 27 | std::atexit(&checkMemoryOnExit); 28 | } 29 | ++BLOCKS_IN_USE; 30 | void *ptr = std::malloc(size); 31 | if (!ptr) { 32 | throw std::bad_alloc(); 33 | } 34 | return ptr; 35 | } 36 | 37 | void releaseMemory(void *ptr) 38 | { 39 | --BLOCKS_IN_USE; 40 | std::free(ptr); 41 | } 42 | 43 | } // unnamed namespace 44 | 45 | void *operator new(std::size_t size) 46 | { 47 | return allocateMemory(size); 48 | } 49 | 50 | void *operator new[] (std::size_t size) 51 | { 52 | return allocateMemory(size); 53 | } 54 | 55 | void operator delete(void *ptr) noexcept 56 | { 57 | releaseMemory(ptr); 58 | } 59 | 60 | void operator delete[](void *ptr) noexcept 61 | { 62 | releaseMemory(ptr); 63 | } 64 | 65 | #endif // TESTUTIL_NEWDELETEMONITOR 66 | --------------------------------------------------------------------------------