├── CHANGELOG.md
├── CONTRIBUTING.md
├── .travis.yml
├── README.md
├── LICENSE
├── getSnapshot
└── putSnapshot


/CHANGELOG.md:
--------------------------------------------------------------------------------
 1 | # Change Log
 2 | Notable features, fixes, and changes (breaking and non-breaking) between releases will be documented in this file.  [Semantic Versioning](http://semver.org) shall be used for version numbering in this project.
 3 | 
 4 | ## [v1.0.1](https://github.com/AppliedInfrastructure/cassandra-snapshot-tools/tree/v1.0.1) (2016-09-22)
 5 | 
 6 | **New Features**
 7 | 
 8 | * Added basic Travis tests for Cassandra 2.0, 2.1, 2.2, 3.0, and 3.7
 9 | 
10 | **Fixes**
11 | 
12 | * Fixed errors packaging/restoring from Cassandra 3.0 and 3.7
13 | * Cleaned up putSnapshot output when restoring to Cassandra 2.2
14 | 
15 | **Changes**
16 | 
17 | 
18 | ## [v1.0.0](https://github.com/AppliedInfrastructure/cassandra-snapshot-tools/tree/v1.0.0) (2016-09-15)
19 | 
20 | **New Features**
21 | 
22 | * Initial commit of getSnapshot and putSnapshot utilities
23 | * Added basic documentation via README.md
24 | * Added contribution guidelines and change log
25 | 
26 | **Fixes**
27 | 
28 | **Changes**
29 | 


--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
 1 | # How to Contribute
 2 | The goal of Cassandra Snapshot Tools is to provide a simple and intuitive way to leverage Cassandra snapshots that "just works".  I hope you find this true of the project today and in future releases.
 3 | 
 4 | Everyone's contributions are welcome.  In the current form, Cassandra Snapshot Tools is a set of Bash shell scripts that leverage standard Cassandra utilities like `nodetool`, `cqlsh`, and `sstableloader`, but should always aim to be simple, reliable, and intuitive.  Please keep this in mind as you submit enhancement requests or pull requests, but don't hesitate to submit an issue if something doesn't work in a simple, reliable, or intuitive way.
 5 | 
 6 | ### Defect Tracking
 7 | Please review the [Issue Tracker](https://github.com/AppliedInfrastructure/cassandra-snapshot-tools/issues) and open an issue if a similar one doesn't already exist.
 8 | 
 9 | ### Pull Requests Welcome
10 | If something doesn't quite work the way you'd like, and you'd like to contribute a fix:
11 | 
12 | 1. [Open an Issue](https://github.com/AppliedInfrastructure/cassandra-snapshot-tools/issues) that describes the defect/enhancement.
13 | 2. [Fork the Repository](https://github.com/AppliedInfrastructure/cassandra-snapshot-tools/fork_select)
14 | 3. Create a branch (`git checkout -b <your branch>`)
15 | 4. Commit your changes (`git commit -am "<descriptive change message>"`)
16 | 5. Push your changes to the branch (`git push origin <your branch>`)
17 | 6. [Create a Pull Request](https://github.com/AppliedInfrastructure/cassandra-snapshot-tools/pulls) to start the review/discuss/merge process.
18 | 


--------------------------------------------------------------------------------
/.travis.yml:
--------------------------------------------------------------------------------
 1 | language: bash
 2 | 
 3 | sudo: required
 4 | 
 5 | services:
 6 |   - docker
 7 | 
 8 | env:
 9 |   - CASSANDRA_VER=2.0
10 |   - CASSANDRA_VER=2.1
11 |   - CASSANDRA_VER=2.2
12 |   - CASSANDRA_VER=3.0
13 |   - CASSANDRA_VER=3.7
14 | 
15 | before_install:
16 |   - docker pull cassandra:${CASSANDRA_VER}
17 |   - docker run --name c01n01 -d cassandra:${CASSANDRA_VER}
18 |   - docker run --name c01n02 -e CASSANDRA_SEEDS="$( docker inspect --format='{{ .NetworkSettings.IPAddress }}' c01n01 )" -d cassandra:${CASSANDRA_VER}
19 |   - docker run --name c02n01 -d cassandra:${CASSANDRA_VER}
20 |   - docker ps -a
21 | 
22 | install:
23 |   - docker cp ./getSnapshot c01n01:/root/
24 |   - docker cp ./putSnapshot c01n01:/root/
25 |   - docker cp ./putSnapshot c02n01:/root/
26 |   - docker cp ./test/testData_SimpleStrategy.cql c01n01:/root/
27 |   - docker cp ./test/testData_NetworkTopologyStrategy.cql c01n01:/root/
28 |   - sleep 60
29 |   - docker exec c01n01 bash -c "nodetool info"
30 |   - docker exec c01n01 bash -c "cqlsh -f /root/testData_SimpleStrategy.cql"
31 |   - docker exec c01n01 bash -c "cqlsh -f /root/testData_NetworkTopologyStrategy.cql"
32 | 
33 | script:
34 |   - docker exec c01n01 bash -c "cqlsh <<< 'DESCRIBE KEYSPACES'"
35 |   - docker exec c01n01 bash -c "cqlsh <<< 'SELECT count(*) FROM test_keyspace01_topology.playlists;'"
36 |   - docker exec c01n01 bash -c "/root/getSnapshot -k test_keyspace01_topology"
37 |   - docker exec c01n01 bash -c "/root/putSnapshot -f *.tar.gz -k test_keyspace02_topology"
38 |   - docker exec c01n01 bash -c "cqlsh <<< 'DESCRIBE KEYSPACES'"
39 |   - docker exec c01n01 bash -c "cqlsh <<< 'SELECT count(*) FROM test_keyspace02_topology.playlists;'"
40 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # Cassandra Snapshot Tools [![Build Status](https://travis-ci.org/AppliedInfrastructure/cassandra-snapshot-tools.svg?branch=master)](https://travis-ci.org/AppliedInfrastructure/cassandra-snapshot-tools)
 2 | A handy couple of scripts that make creating, moving, and restoring SSTable snapshots between clusters easy.  Some common use cases include:
 3 | * Copying keyspaces between clusters (QA environments, for example)
 4 | * Staging an upgrade (application, Cassandra, both) using a copy of live data
 5 | * Seeding data in a test environment
 6 | * Snapshotting a keyspace for offline analytics
 7 | * Backup and restore
 8 | * Generally moving a keyspace from one Cassandra cluster to another
 9 | 
10 | Cassandra Snapshot Tools currently includes two BASH shell scripts, `getSnapshot` and `putSnapshot`, which leverage standard Cassandra utilities like `nodetool`, `cqlsh`, and `sstableloader` to simplify the process of creating snapshots and moving them between hosts.  A convenient compressed tar archive is generated by `getSnapshot`, and includes all of the SSTable snapshot files, metadata, and schema information necessary to restore the keyspace into another Cassandra cluster (or the same cluster).  The `putSnapshot` tool is then used to alter various attributes of the original snapshot, and restore into the destination Cassandra cluster.
11 | 
12 | ### Features
13 | * Automatically generates a keyspace snapshot and packages into an easy-to-move archive
14 | * Copies snapshots out of the Cassandra data directory, allowing space to be reclaimed using `nodetool clearsnapshot`
15 | * Archive a previously created snapshot (e.g. created manually using `nodetool snapshot`)
16 | * Schedulable using Cron (e.g. for scheduled backups)
17 | * Change keyspace name, datacenter name and replication factor on restore
18 | * Restore to local or remote clusters, either privately hosted or through hosted services like [Datascale.io](http://datascale.io)
19 | * Easy to use with sane defaults
20 | * Most Cassandra versions supported (tested against Cassandra 2.0, 2.1, 2.2, 3.0, and 3.7)
21 | 
22 | ### Usage
23 | #### getSnapshot
24 | 
25 |     Usage: ./getSnapshot -h
26 |            ./getSnapshot -k <keyspace name> [-s <snapshot name>] [-y <cassandra.yaml file>] [--no-timestamp]
27 |         -h,--help                          Print usage and exit
28 |         -v,--version                       Print version information and exit
29 |         -k,--keyspace <keyspace name>      REQUIRED: The name of the keyspace to snapshot
30 |         -s,--snapshot <snapshot name>      The name of an existing snapshot to package
31 |         -y,--yaml <cassandra.yaml file>    Alternate cassandra.yaml file
32 |         --no-timestamp                     Don't include a timestamp in the resulting filename
33 | 
34 | #### putSnapshot
35 | 
36 |     Usage: ./putSnapshot -h
37 |            ./putSnapshot -f <snapshot file> [-n <node address>] [-k <new ks name>] [-d <new dc name>] [-r <new rf>] [-y <cassandra.yaml file>]
38 |         -h,--help                          Print usage and exit
39 |         -v,--version                       Print version information and exit
40 |         -f,--file <snapshot file>          REQUIRED: The snapshot file name (created using the
41 |                                            getSnapshot utility
42 |         -n,--node <node address>           Destination Cassandra node IP (defaults to the local
43 |                                            Cassandra IP if run on a Cassandra node, otherwise
44 |                                            required in order to connect to Cassandra.  Will take
45 |                                            precedence if provided and run on a Cassandra node
46 |         -k,--keyspace <new ks name>        Override the destination keyspace name (defaults to
47 |                                            the source keyspace name)
48 |         -d,--datacenter <new dc name>      Override the destination datacenter name (defaults
49 |                                            to the sourcen datacenter name)
50 |         -r,--replication <new rf>          Override the destination replication factor (defaults
51 |                                            to source replication factor)
52 |         -y,--yaml <cassandra.yaml file>    Alternate cassandra.yaml file
53 | 
54 | ### Basic Examples
55 | 1. Copy a keyspace to the same Cassandra cluster using a different keyspace name:
56 | 
57 |     ```sh
58 |     $ getSnapshot -k <keyspace name>
59 |     $ putSnapshot -f <snapshot package file> -k <new keyspace name>
60 |     ```
61 | 2. Copy a keyspace to a remote Cassandra cluster using the same keyspace name:
62 | 
63 |     ```sh
64 |     $ getSnapshot -k <keyspace name>
65 |     $ putSnapshot -f <snapshot package file> -n <destination node IP>
66 |     ```
67 | 3. Copy a keyspace from a Cassandra cluster to a remote cluster using a different keyspace name and replication factor:
68 | 
69 |     ```sh
70 |     $ getSnapshot -k <keyspace name>
71 |     $ putSnapshot -f <snapshot package file> -n <destination node IP> -k <new keyspace name> -r 1
72 |     ```
73 | 4. Copy a snapshot previously created using `nodetool snapshot` to a new keyspace:
74 | 
75 |     ```sh
76 |     $ nodetool snapshot <keyspace name> -t <custom snapshot name>
77 |     $ getSnapshot -k <keyspace name> -s <custom snapshot name>
78 |     $ putSnapshot -f <snapshot package file> -k <new keyspace name>
79 |     ```
80 | 
81 | ### Caveats
82 | * Currently supports snapshots/restores between clusters running similar versions of Cassandra.
83 | * Partitioner configuration (e.g. RandomPartitioner, Murmur3Partitioner, etc.) must be the same between source and destination Cassandra clusters.
84 | * Local access to the source Cassandra node is required (to create/collect snapshot files).  Fairly open network access is required to the destination Cassandra node (to create the schema and load SSTables).
85 | 
86 | ### Contributing
87 | All contributions welcome!  Please see [How to Contribute](CONTRIBUTING.md).
88 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
  1 |                                  Apache License
  2 |                            Version 2.0, January 2004
  3 |                         http://www.apache.org/licenses/
  4 | 
  5 |    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
  6 | 
  7 |    1. Definitions.
  8 | 
  9 |       "License" shall mean the terms and conditions for use, reproduction,
 10 |       and distribution as defined by Sections 1 through 9 of this document.
 11 | 
 12 |       "Licensor" shall mean the copyright owner or entity authorized by
 13 |       the copyright owner that is granting the License.
 14 | 
 15 |       "Legal Entity" shall mean the union of the acting entity and all
 16 |       other entities that control, are controlled by, or are under common
 17 |       control with that entity. For the purposes of this definition,
 18 |       "control" means (i) the power, direct or indirect, to cause the
 19 |       direction or management of such entity, whether by contract or
 20 |       otherwise, or (ii) ownership of fifty percent (50%) or more of the
 21 |       outstanding shares, or (iii) beneficial ownership of such entity.
 22 | 
 23 |       "You" (or "Your") shall mean an individual or Legal Entity
 24 |       exercising permissions granted by this License.
 25 | 
 26 |       "Source" form shall mean the preferred form for making modifications,
 27 |       including but not limited to software source code, documentation
 28 |       source, and configuration files.
 29 | 
 30 |       "Object" form shall mean any form resulting from mechanical
 31 |       transformation or translation of a Source form, including but
 32 |       not limited to compiled object code, generated documentation,
 33 |       and conversions to other media types.
 34 | 
 35 |       "Work" shall mean the work of authorship, whether in Source or
 36 |       Object form, made available under the License, as indicated by a
 37 |       copyright notice that is included in or attached to the work
 38 |       (an example is provided in the Appendix below).
 39 | 
 40 |       "Derivative Works" shall mean any work, whether in Source or Object
 41 |       form, that is based on (or derived from) the Work and for which the
 42 |       editorial revisions, annotations, elaborations, or other modifications
 43 |       represent, as a whole, an original work of authorship. For the purposes
 44 |       of this License, Derivative Works shall not include works that remain
 45 |       separable from, or merely link (or bind by name) to the interfaces of,
 46 |       the Work and Derivative Works thereof.
 47 | 
 48 |       "Contribution" shall mean any work of authorship, including
 49 |       the original version of the Work and any modifications or additions
 50 |       to that Work or Derivative Works thereof, that is intentionally
 51 |       submitted to Licensor for inclusion in the Work by the copyright owner
 52 |       or by an individual or Legal Entity authorized to submit on behalf of
 53 |       the copyright owner. For the purposes of this definition, "submitted"
 54 |       means any form of electronic, verbal, or written communication sent
 55 |       to the Licensor or its representatives, including but not limited to
 56 |       communication on electronic mailing lists, source code control systems,
 57 |       and issue tracking systems that are managed by, or on behalf of, the
 58 |       Licensor for the purpose of discussing and improving the Work, but
 59 |       excluding communication that is conspicuously marked or otherwise
 60 |       designated in writing by the copyright owner as "Not a Contribution."
 61 | 
 62 |       "Contributor" shall mean Licensor and any individual or Legal Entity
 63 |       on behalf of whom a Contribution has been received by Licensor and
 64 |       subsequently incorporated within the Work.
 65 | 
 66 |    2. Grant of Copyright License. Subject to the terms and conditions of
 67 |       this License, each Contributor hereby grants to You a perpetual,
 68 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 69 |       copyright license to reproduce, prepare Derivative Works of,
 70 |       publicly display, publicly perform, sublicense, and distribute the
 71 |       Work and such Derivative Works in Source or Object form.
 72 | 
 73 |    3. Grant of Patent License. Subject to the terms and conditions of
 74 |       this License, each Contributor hereby grants to You a perpetual,
 75 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 76 |       (except as stated in this section) patent license to make, have made,
 77 |       use, offer to sell, sell, import, and otherwise transfer the Work,
 78 |       where such license applies only to those patent claims licensable
 79 |       by such Contributor that are necessarily infringed by their
 80 |       Contribution(s) alone or by combination of their Contribution(s)
 81 |       with the Work to which such Contribution(s) was submitted. If You
 82 |       institute patent litigation against any entity (including a
 83 |       cross-claim or counterclaim in a lawsuit) alleging that the Work
 84 |       or a Contribution incorporated within the Work constitutes direct
 85 |       or contributory patent infringement, then any patent licenses
 86 |       granted to You under this License for that Work shall terminate
 87 |       as of the date such litigation is filed.
 88 | 
 89 |    4. Redistribution. You may reproduce and distribute copies of the
 90 |       Work or Derivative Works thereof in any medium, with or without
 91 |       modifications, and in Source or Object form, provided that You
 92 |       meet the following conditions:
 93 | 
 94 |       (a) You must give any other recipients of the Work or
 95 |           Derivative Works a copy of this License; and
 96 | 
 97 |       (b) You must cause any modified files to carry prominent notices
 98 |           stating that You changed the files; and
 99 | 
100 |       (c) You must retain, in the Source form of any Derivative Works
101 |           that You distribute, all copyright, patent, trademark, and
102 |           attribution notices from the Source form of the Work,
103 |           excluding those notices that do not pertain to any part of
104 |           the Derivative Works; and
105 | 
106 |       (d) If the Work includes a "NOTICE" text file as part of its
107 |           distribution, then any Derivative Works that You distribute must
108 |           include a readable copy of the attribution notices contained
109 |           within such NOTICE file, excluding those notices that do not
110 |           pertain to any part of the Derivative Works, in at least one
111 |           of the following places: within a NOTICE text file distributed
112 |           as part of the Derivative Works; within the Source form or
113 |           documentation, if provided along with the Derivative Works; or,
114 |           within a display generated by the Derivative Works, if and
115 |           wherever such third-party notices normally appear. The contents
116 |           of the NOTICE file are for informational purposes only and
117 |           do not modify the License. You may add Your own attribution
118 |           notices within Derivative Works that You distribute, alongside
119 |           or as an addendum to the NOTICE text from the Work, provided
120 |           that such additional attribution notices cannot be construed
121 |           as modifying the License.
122 | 
123 |       You may add Your own copyright statement to Your modifications and
124 |       may provide additional or different license terms and conditions
125 |       for use, reproduction, or distribution of Your modifications, or
126 |       for any such Derivative Works as a whole, provided Your use,
127 |       reproduction, and distribution of the Work otherwise complies with
128 |       the conditions stated in this License.
129 | 
130 |    5. Submission of Contributions. Unless You explicitly state otherwise,
131 |       any Contribution intentionally submitted for inclusion in the Work
132 |       by You to the Licensor shall be under the terms and conditions of
133 |       this License, without any additional terms or conditions.
134 |       Notwithstanding the above, nothing herein shall supersede or modify
135 |       the terms of any separate license agreement you may have executed
136 |       with Licensor regarding such Contributions.
137 | 
138 |    6. Trademarks. This License does not grant permission to use the trade
139 |       names, trademarks, service marks, or product names of the Licensor,
140 |       except as required for reasonable and customary use in describing the
141 |       origin of the Work and reproducing the content of the NOTICE file.
142 | 
143 |    7. Disclaimer of Warranty. Unless required by applicable law or
144 |       agreed to in writing, Licensor provides the Work (and each
145 |       Contributor provides its Contributions) on an "AS IS" BASIS,
146 |       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147 |       implied, including, without limitation, any warranties or conditions
148 |       of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149 |       PARTICULAR PURPOSE. You are solely responsible for determining the
150 |       appropriateness of using or redistributing the Work and assume any
151 |       risks associated with Your exercise of permissions under this License.
152 | 
153 |    8. Limitation of Liability. In no event and under no legal theory,
154 |       whether in tort (including negligence), contract, or otherwise,
155 |       unless required by applicable law (such as deliberate and grossly
156 |       negligent acts) or agreed to in writing, shall any Contributor be
157 |       liable to You for damages, including any direct, indirect, special,
158 |       incidental, or consequential damages of any character arising as a
159 |       result of this License or out of the use or inability to use the
160 |       Work (including but not limited to damages for loss of goodwill,
161 |       work stoppage, computer failure or malfunction, or any and all
162 |       other commercial damages or losses), even if such Contributor
163 |       has been advised of the possibility of such damages.
164 | 
165 |    9. Accepting Warranty or Additional Liability. While redistributing
166 |       the Work or Derivative Works thereof, You may choose to offer,
167 |       and charge a fee for, acceptance of support, warranty, indemnity,
168 |       or other liability obligations and/or rights consistent with this
169 |       License. However, in accepting such obligations, You may act only
170 |       on Your own behalf and on Your sole responsibility, not on behalf
171 |       of any other Contributor, and only if You agree to indemnify,
172 |       defend, and hold each Contributor harmless for any liability
173 |       incurred by, or claims asserted against, such Contributor by reason
174 |       of your accepting any such warranty or additional liability.
175 | 
176 |    END OF TERMS AND CONDITIONS
177 | 


--------------------------------------------------------------------------------
/getSnapshot:
--------------------------------------------------------------------------------
  1 | #!/bin/bash
  2 | # Name: getSnapshot
  3 | # Description: Packages a keyspace snapshot to be restored to the same, or a
  4 | #              different Cassandra cluster.  Must be executed on a running
  5 | #              Cassandra node, have access to the cassandra.yaml file, and be
  6 | #              able to read the data file location(s).
  7 | #
  8 | #              A valid keyspace name is all that is required for getSnapshot
  9 | #              to run.  The script will run attempt to find cassandra.yaml in
 10 | #              the standard locations (for both DSE and ASF) or a supplied
 11 | #              location, determine the data file directory, and look for the
 12 | #              requested keyspace among all configured data file directories.
 13 | #              If a previous snapshot is not specified, a new snapshot is
 14 | #              created using "nodetool snapshot <keyspace>".  The snapshot is
 15 | #              packaged in a compressed TAR file with a copy of the schema.
 16 | #
 17 | # Copyright 2016 Applied Infrastructure, LLC
 18 | #
 19 | # Licensed under the Apache License, Version 2.0 (the "License");
 20 | # you may not use this file except in compliance with the License.
 21 | # You may obtain a copy of the License at
 22 | #
 23 | #     http://www.apache.org/licenses/LICENSE-2.0
 24 | #
 25 | # Unless required by applicable law or agreed to in writing, software
 26 | # distributed under the License is distributed on an "AS IS" BASIS,
 27 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 28 | # See the License for the specific language governing permissions and
 29 | # limitations under the License.
 30 | 
 31 | 
 32 | # Configuration
 33 | # -------------
 34 |     PROGNAME="getSnapshot"
 35 |     PROGVER="1.0.1"
 36 |     ASFCFG="/etc/cassandra"
 37 |     DSECFG="/etc/dse/cassandra"
 38 |     DUMPDIR="$( pwd )/${PROGNAME}.tmp${RANDOM}"
 39 |     CLITMPFILE="${DUMPDIR}/cqlschema"
 40 |     CASIP="127.0.0.1"
 41 |     JMXIP="127.0.0.1"
 42 |     HOSTNAME="$( hostname )"
 43 |     SNAPCREATE=false
 44 |     KEYSPFILE="cassandra.keyspace"
 45 |     SNAPSFILE="cassandra.snapshot"
 46 |     HOSTSFILE="cassandra.hostname"
 47 |     DATESFILE="cassandra.snapdate"
 48 |     APPENDTIMESTAMP="yes"
 49 | 
 50 | # Functions
 51 | # ---------
 52 |     function check_dependencies() {
 53 |         # Function to iterate through a list of required executables to ensure
 54 |         # they are installed and executable by the current user.
 55 |         DEPS="awk basename cp cqlsh date dirname echo find "
 56 |         DEPS+="getopt grep hostname mkdir rm sed tail tar "
 57 |         for bin in $DEPS; do
 58 |             $( which $bin >/dev/null 2>&1 ) || NOTFOUND+="$bin "
 59 |         done
 60 | 
 61 |         if [ ! -z "$NOTFOUND" ]; then
 62 |             printf "Error finding required executables: ${NOTFOUND}\n" >&2
 63 |             exit 1
 64 |         fi
 65 |     }
 66 | 
 67 |     function parse_yaml() {
 68 |         # Basic (as in imperfect) parsing of a given YAML file.  Parameters
 69 |         # are stored as environment variables.
 70 |         local prefix=$2
 71 |         local s
 72 |         local w
 73 |         local fs
 74 |         s='[[:space:]]*'
 75 |         w='[a-zA-Z0-9_]*'
 76 |         fs="$(echo @|tr @ '\034')"
 77 |         sed -ne "s|^\($s\)\($w\)$s:$s\"\(.*\)\"$s\$|\1$fs\2$fs\3|p" \
 78 |             -e "s|^\($s\)\($w\)$s[:-]$s\(.*\)$s\$|\1$fs\2$fs\3|p" "$1" |
 79 |         awk -F"$fs" '{
 80 |           indent = length($1)/2;
 81 |           if (length($2) == 0) { conj[indent]="+";} else {conj[indent]="";}
 82 |           vname[indent] = $2;
 83 |           for (i in vname) {if (i > indent) {delete vname[i]}}
 84 |           if (length($3) > 0) {
 85 |             vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
 86 |             printf("%s%s%s%s=(\"%s\")\n", "'"$prefix"'",vn, $2, conj[indent-1],$3);
 87 |           }
 88 |         }' | sed 's/_=/+=/g'
 89 |     }
 90 | 
 91 |     function usage() {
 92 |         printf "Usage: $0 -h\n"
 93 |         printf "       $0 -k <keyspace name> [-s <snapshot name>] [-y <cassandra.yaml file>] [--no-timestamp]\n"
 94 |         printf "    -h,--help                          Print usage and exit\n"
 95 |         printf "    -v,--version                       Print version information and exit\n"
 96 |         printf "    -k,--keyspace <keyspace name>      REQUIRED: The name of the keyspace to snapshot\n"
 97 |         printf "    -s,--snapshot <snapshot name>      The name of an existing snapshot to package\n"
 98 |         printf "    -y,--yaml <cassandra.yaml file>    Alternate cassandra.yaml file\n"
 99 |         printf "    --no-timestamp                     Don't include a timestamp in the resulting filename\n"
100 |         exit 0
101 |     }
102 | 
103 |     function version() {
104 |         printf "$PROGNAME version $PROGVER\n"
105 |         printf "Cassandra snapshot packaging utility\n\n"
106 |         printf "Copyright 2016 Applied Infrastructure, LLC\n\n"
107 |         printf "Licensed under the Apache License, Version 2.0 (the \"License\");\n"
108 |         printf "you may not use this file except in compliance with the License.\n"
109 |         printf "You may obtain a copy of the License at\n\n"
110 |         printf "    http://www.apache.org/licenses/LICENSE-2.0\n\n"
111 |         printf "Unless required by applicable law or agreed to in writing, software\n"
112 |         printf "distributed under the License is distributed on an \"AS IS\" BASIS,\n"
113 |         printf "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n"
114 |         printf "See the License for the specific language governing permissions and\n"
115 |         printf "limitations under the License.\n"
116 |         exit 0
117 |     }
118 | 
119 | # Validate Input/Environment
120 | # --------------------------
121 |     # Great sample getopt implementation by Cosimo Streppone
122 |     # https://gist.github.com/cosimo/3760587#file-parse-options-sh
123 |     SHORT='hvk:s:y:'
124 |     LONG='help,version,keyspace:,snapshot:,yaml:,no-timestamp'
125 |     OPTS=$( getopt -o $SHORT --long $LONG -n "$0" -- "$@" )
126 | 
127 |     if [ $? -gt 0 ]; then
128 |         # Exit early if argument parsing failed
129 |         printf "Error parsing command arguments\n" >&2
130 |         exit 1
131 |     fi
132 | 
133 |     eval set -- "$OPTS"
134 |     while true; do
135 |         case "$1" in
136 |             -h|--help) usage;;
137 |             -v|--version) version;;
138 |             -k|--keyspace) KEYSPACE="$2"; shift 2;;
139 |             -s|--snapshot) SNAPSHOT="$2"; shift 2;;
140 |             -y|--yaml) INPYAML="$2"; shift 2;;
141 |             --no-timestamp) APPENDTIMESTAMP="no"; shift;;
142 |             --) shift; break;;
143 |             *) printf "Error processing command arguments\n" >&2; exit 1;;
144 |         esac
145 |     done
146 | 
147 |     # Verify required binaries at this point
148 |     check_dependencies
149 | 
150 |     # Only KEYSPACE is absolutely required
151 |     if [ "$KEYSPACE" == "" ]; then
152 |         printf "You must provide a keyspace name\n"
153 |         exit 1
154 |     fi
155 | 
156 |     # Need write access to local directory to create dump file
157 |     if [ ! -w $( pwd ) ]; then
158 |         printf "You must have write access to the current directory $( pwd )\n"
159 |         exit 1
160 |     fi
161 | 
162 |     # Attempt to locate data directory and keyspace files
163 |     YAMLLIST="${INPYAML:-$( find "$DSECFG" "$ASFCFG" -type f -name cassandra.yaml 2>/dev/null ) }"
164 | 
165 |     for yaml in $YAMLLIST; do
166 |         if [ -r "$yaml" ]; then
167 |             eval $( parse_yaml "$yaml" )
168 |             # Search each data directory in the YAML
169 |             for directory in ${data_file_directories_[@]}; do
170 |                 if [ -d "$directory/$KEYSPACE" ]; then
171 |                     # Use the YAML that references the keyspace
172 |                     DATADIR="$directory"
173 |                     YAMLFILE="$yaml"
174 |                     break
175 |                 fi
176 |                 # Used only when the keyspace can't be found
177 |                 TESTED="$TESTED $directory"
178 |             done
179 |         fi
180 |     done
181 | 
182 |     if [ -z "$TESTED" ] && [ -z "$DATADIR" ]; then
183 |         printf "No data directories, or no cassandra.yaml file found\n" >&2
184 |         exit 1
185 |     elif [ -z "$DATADIR" ] || [ -z "$YAMLFILE" ]; then
186 |         printf "Keyspace data directory could not be found in:\n"
187 |         for dir in $TESTED; do
188 |             printf "    $dir/$KEYSPACE\n"
189 |         done
190 |         exit 1
191 |     fi
192 | 
193 | # Preparation
194 | # -----------
195 |     eval $( parse_yaml "$YAMLFILE" )
196 |     TIMESTAMP=$( date +"%Y%m%d%H%M%S" )
197 |     DATESTRING=$( date )
198 | 
199 |     # Create temporary working directory.  Yes, deliberately avoiding mktemp
200 |     if [ ! -d "$DUMPDIR" ] && [ ! -e "$DUMPDIR" ]; then
201 |         mkdir -p "$DUMPDIR"
202 |     else
203 |         printf "Error creating temporary directory $DUMPDIR"
204 |         exit 1
205 |     fi
206 | 
207 |     # Write temp command file for Cassandra CLI
208 |     printf "desc keyspace $KEYSPACE;\n" > $CLITMPFILE
209 | 
210 |     # Get local Cassandra listen address.  Should be loaded via the selected
211 |     # cassandra.yaml file above.
212 |     if [ -z $listen_address ]; then
213 |         CASIP=$( hostname )
214 |     elif [ "$listen_address" == "0.0.0.0" ]; then
215 |         CASIP=127.0.0.1
216 |     else
217 |         CASIP=$listen_address
218 |     fi
219 | 
220 |     # Get local Cassandra JMX address
221 |     # Cheating for now - this is *usually* right, but may be set to a real IP
222 |     # in cassandra-env.sh in some environments.
223 |     JMXIP=127.0.0.1
224 | 
225 | # Create/Pull Snapshot
226 | # --------------------
227 |     if [ -z "$SNAPSHOT" ]; then
228 |         # Create a new snapshot if a snapshot name was not provided
229 |         printf "No snapshot name provided, creating new snapshot\n"
230 | 
231 |         nodetool -h $JMXIP flush -- $KEYSPACE
232 |         OUTPUT=$( nodetool -h $JMXIP snapshot $KEYSPACE 2>&1 )
233 |         SNAPSHOT=$( grep -Eo '[0-9]{10}[0-9]+' <<< "$OUTPUT" | tail -1 )
234 | 
235 |         # Check if the snapshot process failed
236 |         if [ -z "$SNAPSHOT" ]; then
237 |             printf "Problem creating snapshot for keyspace $KEYSPACE\n\n"
238 |             printf "$OUTPUT\n"
239 |             [ "$DUMPDIR" != "/" ] && rm -rf "$DUMPDIR"
240 |             exit 1
241 |         fi
242 |     else
243 |         # If a snapshot name was provided, check if it exists
244 |         SEARCH=$( find "${DATADIR}/${KEYSPACE}" -type d -name "${SNAPSHOT}" )
245 | 
246 |         if [ -z "$SEARCH" ]; then
247 |             printf "No snapshots found with name ${SNAPSHOT}\n"
248 |             [ "$DUMPDIR" != "/" ] && rm -rf "$DUMPDIR"
249 |             exit 1
250 |         else
251 |             printf "Using provided snapshot name ${SNAPSHOT}\n"
252 |         fi
253 |     fi
254 | 
255 |     # Pull new/existing snapshot
256 |     SNAPDIR="snapshots/$SNAPSHOT"
257 |     SCHEMA="schema-$KEYSPACE-$TIMESTAMP.cdl"
258 | 
259 |     for dir in $( find "$DATADIR" -regex ".*/$SNAPDIR/[^\.]*.db" ); do
260 |         NEWDIR=$( sed "s|${DATADIR}||" <<< $( dirname $dir ) | \
261 |                   awk -F / '{print "/"$2"/"$3}' )
262 | 
263 |         mkdir -p "$DUMPDIR/$NEWDIR"
264 |         cp $dir "$DUMPDIR/$NEWDIR/"
265 |     done
266 | 
267 | # Backup the schema and create tar archive
268 | # ----------------------------------------
269 |     printf "$KEYSPACE" > "$DUMPDIR/$KEYSPFILE"
270 |     printf "$SNAPSHOT" > "$DUMPDIR/$SNAPSFILE"
271 |     printf "$HOSTNAME" > "$DUMPDIR/$HOSTSFILE"
272 |     printf "$DATESTRING" > "$DUMPDIR/$DATESFILE"
273 |     cqlsh $CASIP -k $KEYSPACE -f $CLITMPFILE | tail -n +2 > "$DUMPDIR/$SCHEMA"
274 |     RC=$?
275 | 
276 |     if [ $? -gt 0 ] && [ ! -s "$DUMPDIR/$SCHEMA" ]; then
277 |         printf "Schema backup failed\n"
278 |         [ "$DUMPDIR" != "/" ] && rm -rf "$DUMPDIR"
279 |         exit 1
280 |     else
281 |         # Include the timestamp in the filename or not (i.e. --no-timestamp)
282 |         [ "$APPENDTIMESTAMP" == "no" ] && FILENAME="$KEYSPACE.tar.gz" \
283 |                                        || FILENAME="$KEYSPACE-$TIMESTAMP.tar.gz"
284 | 
285 |         tar --directory "$DUMPDIR" \
286 |             -zcvf $FILENAME \
287 |                   $KEYSPACE \
288 |                   $SCHEMA \
289 |                   $KEYSPFILE \
290 |                   $SNAPSFILE \
291 |                   $HOSTSFILE \
292 |                   $DATESFILE >/dev/null 2>&1
293 |         RC=$?
294 | 
295 |         if [ $RC -gt 0 ]; then
296 |             printf "Error generating tar archive\n"
297 |             [ "$DUMPDIR" != "/" ] && rm -rf "$DUMPDIR"
298 |             exit 1
299 |         else
300 |             printf "Successfully created snapshot package\n"
301 |             [ "$DUMPDIR" != "/" ] && rm -rf "$DUMPDIR"
302 |             exit 0
303 |         fi
304 |     fi
305 | 
306 | # Fin.
307 | 


--------------------------------------------------------------------------------
/putSnapshot:
--------------------------------------------------------------------------------
  1 | #!/bin/bash
  2 | # Name: putSnapshot
  3 | # Description: Accepts a snapshot package file prepared by the getSnapshot utility
  4 | #              and imports its contents to a new keyspace.  This utility was
  5 | #              intended to be run locally on a Cassandra node but should also run
  6 | #              fine on a remote host having the appropriate Cassandra utilities.
  7 | #              A new keyspace name may be specified, which will perform the
  8 | #              operations necessary to create the new keyspace and import the
  9 | #              data stored in the snapshot package.  If a new keyspace name is
 10 | #              not specified, the original keyspace name (from the original
 11 | #              server and stored in the snapshot package) will be used.  An error
 12 | #              will be thrown in the event of a keyspace name conflict.
 13 | #
 14 | # Copyright 2016 Applied Infrastructure, LLC
 15 | #
 16 | # Licensed under the Apache License, Version 2.0 (the "License");
 17 | # you may not use this file except in compliance with the License.
 18 | # You may obtain a copy of the License at
 19 | #
 20 | #     http://www.apache.org/licenses/LICENSE-2.0
 21 | #
 22 | # Unless required by applicable law or agreed to in writing, software
 23 | # distributed under the License is distributed on an "AS IS" BASIS,
 24 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 25 | # See the License for the specific language governing permissions and
 26 | # limitations under the License.
 27 | 
 28 | # Configuration
 29 | # -------------
 30 |     PROGNAME="putSnapshot"
 31 |     PROGVER="1.0.1"
 32 |     ASFCFG="/etc/cassandra"
 33 |     DSECFG="/etc/dse/cassandra"
 34 |     TEMPDIR="$( pwd )/${PROGNAME}.tmp${RANDOM}"
 35 |     CLITMPFILE="${TEMPDIR}/cqlkeyspace"
 36 |     KEYSPFILE="cassandra.keyspace"
 37 |     SNAPSFILE="cassandra.snapshot"
 38 |     HOSTSFILE="cassandra.hostname"
 39 |     DATESFILE="cassandra.snapdate"
 40 | 
 41 | # Functions
 42 | # ---------
 43 |     function check_dependencies() {
 44 |         # Function to iterate through a list of required executables to ensure
 45 |         # they are installed and executable by the current user.
 46 |         DEPS="awk cat cqlsh cut echo find getopt grep hostname "
 47 |         DEPS+="mkdir rm sed sstableloader tar tr "
 48 |         for bin in $DEPS; do
 49 |             $( which $bin >/dev/null 2>&1 ) || NOTFOUND+="$bin "
 50 |         done
 51 | 
 52 |         if [ ! -z "$NOTFOUND" ]; then
 53 |             printf "Error finding required executables: ${NOTFOUND}\n" >&2
 54 |             exit 1
 55 |         fi
 56 |     }
 57 | 
 58 |     function parse_yaml() {
 59 |         # Basic (as in imperfect) parsing of a given YAML file.  Parameters
 60 |         # are stored as environment variables.
 61 |         local prefix=$2
 62 |         local s
 63 |         local w
 64 |         local fs
 65 |         s='[[:space:]]*'
 66 |         w='[a-zA-Z0-9_]*'
 67 |         fs="$(echo @|tr @ '\034')"
 68 |         sed -ne "s|^\($s\)\($w\)$s:$s\"\(.*\)\"$s\$|\1$fs\2$fs\3|p" \
 69 |             -e "s|^\($s\)\($w\)$s[:-]$s\(.*\)$s\$|\1$fs\2$fs\3|p" "$1" |
 70 |         awk -F"$fs" '{
 71 |           indent = length($1)/2;
 72 |           if (length($2) == 0) { conj[indent]="+";} else {conj[indent]="";}
 73 |           vname[indent] = $2;
 74 |           for (i in vname) {if (i > indent) {delete vname[i]}}
 75 |           if (length($3) > 0) {
 76 |             vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
 77 |             printf("%s%s%s%s=(\"%s\")\n", "'"$prefix"'",vn, $2, conj[indent-1],$3);
 78 |           }
 79 |         }' | sed 's/_=/+=/g'
 80 |     }
 81 | 
 82 |     function usage() {
 83 |         printf "Usage: $0 -h\n"
 84 |         printf "       $0 -f <snapshot file> [-n <node address>] [-k <new ks name>] [-d <new dc name>] [-r <new rf>] [-y <cassandra.yaml file>]\n"
 85 |         printf "    -h,--help                          Print usage and exit\n"
 86 |         printf "    -v,--version                       Print version information and exit\n"
 87 |         printf "    -f,--file <snapshot file>          REQUIRED: The snapshot file name (created using the\n"
 88 |         printf "                                       getSnapshot utility)\n"
 89 |         printf "    -n,--node <node address>           Destination Cassandra node IP (defaults to the local\n"
 90 |         printf "                                       Cassandra IP if run on a Cassandra node, otherwise\n"
 91 |         printf "                                       required in order to connect to Cassandra.  Will take\n"
 92 |         printf "                                       precedence if provided and run on a Cassandra node\n"
 93 |         printf "    -k,--keyspace <new ks name>        Override the destination keyspace name (defaults to\n"
 94 |         printf "                                       the source keyspace name)\n"
 95 |         printf "    -d,--datacenter <new dc name>      Override the destination datacenter name (defaults\n"
 96 |         printf "                                       to the sourcen datacenter name)\n"
 97 |         printf "    -r,--replication <new rf>          Override the destination replication factor (defaults\n"
 98 |         printf "                                       to source replication factor)\n"
 99 |         printf "    -t,--table-keyspace-name <boolean> Replace the original keyspace string if it appears in\n"
100 |         printf "                                       a table name (defaults to true)\n"
101 |         printf "    -y,--yaml <cassandra.yaml file>    Alternate cassandra.yaml file\n"
102 |         exit 0
103 |     }
104 | 
105 |     function version() {
106 |         printf "$PROGNAME version $PROGVER\n"
107 |         printf "Cassandra snapshot loader utility\n\n"
108 |         printf "Copyright 2016 Applied Infrastructure, LLC\n\n"
109 |         printf "Licensed under the Apache License, Version 2.0 (the \"License\");\n"
110 |         printf "you may not use this file except in compliance with the License.\n"
111 |         printf "You may obtain a copy of the License at\n\n"
112 |         printf "    http://www.apache.org/licenses/LICENSE-2.0\n\n"
113 |         printf "Unless required by applicable law or agreed to in writing, software\n"
114 |         printf "distributed under the License is distributed on an \"AS IS\" BASIS,\n"
115 |         printf "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n"
116 |         printf "See the License for the specific language governing permissions and\n"
117 |         printf "limitations under the License.\n"
118 |         exit 0
119 |     }
120 | 
121 | # Validate Input/Environment
122 | # --------------------------
123 |     # Great sample getopt implementation by Cosimo Streppone
124 |     # https://gist.github.com/cosimo/3760587#file-parse-options-sh
125 |     SHORT='hvd:f:n:k:r:t:y:'
126 |     LONG='help,version,datacenter:,file:,node:,keyspace:,replication:,table-keyspace-name:,yaml:'
127 |     OPTS=$( getopt -o $SHORT --long $LONG -n "$0" -- "$@" )
128 | 
129 |     if [ $? -gt 0 ]; then
130 |         # Exit early if argument parsing failed
131 |         printf "Error parsing command arguments\n" >&2
132 |         exit 1
133 |     fi
134 | 
135 |     eval set -- "$OPTS"
136 |     while true; do
137 |         case "$1" in
138 |             -h|--help) usage;;
139 |             -v|--version) version;;
140 |             -f|--file) SNAPPKG="$2"; shift 2;;
141 |             -n|--node) IPINPUT="$2"; shift 2;;
142 |             -k|--keyspace) INPKEYSPACE="$2"; shift 2;;
143 |             -d|--datacenter) DATACENTER="$2"; shift 2;;
144 |             -r|--replication) RFACTOR="$2"; shift 2;;
145 |             -y|--yaml) INPYAML="$2"; shift 2;;
146 |             -t|--table-keyspace-name) TABLE_KEYSPACE_NAME="$2"; shift 2;;
147 |             --) shift; break;;
148 |             *) printf "Error processing command arguments\n" >&2; exit 1;;
149 |         esac
150 |     done
151 | 
152 |     # Verify required binaries at this point
153 |     check_dependencies
154 | 
155 |     # Only a snapshot file is required
156 |     if [ ! -r "$SNAPPKG" ]; then
157 |         printf "You must provide the location of a snapshot package\n"
158 |         exit 1
159 |     fi
160 | 
161 |     # Need write access to local directory to create dump file
162 |     if [ ! -w $( pwd ) ]; then
163 |         printf "You must have write access to the current directory $( pwd )\n"
164 |         exit 1
165 |     fi
166 | 
167 |     # Attempt to locate a local Cassandra install and YAML file
168 |     YAMLLIST="${INPYAML:-$( find "$DSECFG" "$ASFCFG" -type f -name cassandra.yaml 2>/dev/null ) }"
169 | 
170 |     for yaml in $YAMLLIST; do
171 |         if [ -r "$yaml" ]; then
172 |             # Cassandra YAML found - load it (assume a local Cassandra)
173 |             eval $( parse_yaml "$yaml" )
174 |             YAMLFILE="$yaml"
175 | 
176 |             if [ -z $listen_address ]; then
177 |                 CASIP=$( hostname )
178 |             elif [ "$listen_address" == "0.0.0.0" ]; then
179 |                 CASIP=127.0.0.1
180 |             else
181 |                 CASIP=$listen_address
182 |             fi
183 |             break
184 |         fi
185 |     done
186 | 
187 |     # Determine IP to use to connect to Cassandra.  If an IP is provided via
188 |     # -n,--node argument, prefer it.  If not, and a Cassandra IP cannot be
189 |     # discovered via YAML (i.e. this is not a Cassandra node), then return
190 |     # an error and exit.
191 |     if [ ! -z $IPINPUT ]; then
192 |         CASIP="$IPINPUT"
193 |     elif [ -z $CASIP ]; then
194 |         printf "Cassandra IP not provided and not discoverable locally\n"
195 |         exit 1
196 |     fi
197 | 
198 |     # Check if a new keyspace name is provided, and validate input
199 |     if [ -z $INPKEYSPACE ]; then
200 |         printf "New keyspace name not provided, using original keyspace name\n"
201 |     elif [[ ! "$INPKEYSPACE" =~ ^[_a-zA-Z0-9]*$ ]]; then
202 |         printf "Cassandra keyspace names can only contain alpha-numerics and underscore (_)\n"
203 |         exit 1
204 |     else
205 |         KEYSPACE="$INPKEYSPACE"
206 |         printf "New keyspace name $KEYSPACE to be used\n"
207 |     fi
208 | 
209 |     # Let the user know which datacenter and replication factor values being used
210 |     if [ -z $DATACENTER ]; then
211 |         printf "New datacenter name not provided, using original datacenter name\n"
212 |     fi
213 |     if [ -z $RFACTOR ]; then
214 |         printf "New replication factor not provided, using original replication factor\n"
215 |     fi
216 | 
217 | # Preparation
218 | # -----------
219 |     # Remove local temp directory
220 |     [ "$TEMPDIR" != "/" ] && rm -rf "$TEMPDIR"
221 | 
222 |     # Verify/Extract Snapshot Package
223 |     tar -tvf "$SNAPPKG" 2>&1 | grep "$KEYSPFILE" 2>&1 >/dev/null
224 |     RC=$?
225 | 
226 |     if [ $RC -gt 0 ]; then
227 |         printf "\nSnapshot package $SNAPPKG appears invalid or corrupt\n"
228 |         exit 1
229 |     else
230 |         # Create temporary working directory.  Yes, deliberately avoiding mktemp
231 |         if [ ! -d "$TEMPDIR" ] && [ ! -e "$TEMPDIR" ]; then
232 |             mkdir -p "$TEMPDIR"
233 |         else
234 |             printf "\nError creating temporary directory $TEMPDIR"
235 |             exit 1
236 |         fi
237 | 
238 |         # Extract snapshot package
239 |         tar -xf "$SNAPPKG" --directory "$TEMPDIR"
240 |     fi
241 | 
242 | # Prepare Snapshot
243 | # ----------------
244 |     FILEKSNAME=$( cat "${TEMPDIR}/${KEYSPFILE}" )
245 |     FILEHOSTNAME=$( cat "${TEMPDIR}/${HOSTSFILE}" )
246 |     FILESNAPDATE=$( cat "${TEMPDIR}/${DATESFILE}" )
247 |     SCHEMAFILE=$( ls "${TEMPDIR}"/schema-${FILEKSNAME}-*.cdl 2>/dev/null )
248 | 
249 |     # Place schema on single line and extract replication setup
250 |     FILEREPL=$( sed ':a;N;$!ba;s/\n/ /g' "$SCHEMAFILE" | \
251 |                 grep -Eo 'replication ?= ?{[^}]*}' | \
252 |                 tr -dc "[:alnum:],:=_" | \
253 |                 cut -d, -f 2 )
254 |     FILEDCNAME=$( cut -d: -f 1 <<< ${FILEREPL} )
255 |     FILERFACTOR=$( cut -d: -f 2 <<< ${FILEREPL} )
256 | 
257 |     if [ ! -z $KEYSPACE ]; then
258 |         # Update keyspace names in snapshot
259 |         sed -i 's/\s'$FILEKSNAME'\s/ '$KEYSPACE' /g' "$SCHEMAFILE"
260 |         sed -i 's/\s'$FILEKSNAME\\.'/ '$KEYSPACE\.'/g' "$SCHEMAFILE"
261 |         mv "${TEMPDIR}/${FILEKSNAME}" "${TEMPDIR}/${KEYSPACE}"
262 |         if [ "x${TABLE_KEYSPACE_NAME}" != "xfalse" ]; then
263 |             for dbfile in $( cd "${TEMPDIR}/${KEYSPACE}" && find . -type f ); do
264 |                 if grep "${FILEKSNAME}" <<< "${dbfile}" >/dev/null; then
265 |                     echo mv "$dbfile" "${dbfile//$FILEKSNAME/$KEYSPACE}"
266 |                 fi
267 |             done
268 |         fi
269 |         NEWKSNAME=$KEYSPACE
270 |     else
271 |         NEWKSNAME=$FILEKSNAME
272 |     fi
273 | 
274 |     if [ ! -z $DATACENTER ]; then
275 |         # Update datacenter name in snapshot
276 |         sed -i 's/'$FILEDCNAME'/'$DATACENTER'/g' "$SCHEMAFILE"
277 |         DCNAME=$DATACENTER
278 |     else
279 |         DCNAME=$FILEDCNAME
280 |     fi
281 | 
282 |     if [ ! -z $RFACTOR ]; then
283 |         # Update replication factor in snapshot
284 |         sed -i 's/\(\d039'$DCNAME'\d039[^:]*:[^\d039]*\d039\)'$FILERFACTOR'\(\d039\)/\1'$RFACTOR'\2/' "$SCHEMAFILE"
285 |         NEWRFACTOR="$RFACTOR"
286 |     else
287 |         NEWRFACTOR="$FILERFACTOR"
288 |     fi
289 | 
290 | # Load Snapshot
291 | # -------------
292 |     # Check for keyspace name conflict
293 |     while true; do
294 |         echo "describe keyspace $NEWKSNAME;" > "$CLITMPFILE"
295 | 
296 |         # Use CQL version in environment, if available
297 |         [ -z $CQLVER ] && CQLSHVER="" || CQLSHVER="--cqlversion=$CQLVER"
298 |         OUTPUT=$( cqlsh -f $CLITMPFILE $CQLSHVER $CASIP 2>&1 )
299 |         RC=$?
300 | 
301 |         if [ $RC -eq 0 ] && grep -qi 'CREATE KEYSPACE '$NEWKSNAME' ' <<< $OUTPUT; then
302 |             printf "ERROR: Keyspace name $NEWKSNAME conflicts with existing keyspace name\n"
303 |             [ "$TEMPDIR" != "/" ] && rm -rf "$TEMPDIR"
304 |             exit 1
305 |         elif grep -qi 'version .\?[.0-9]*.\? is not' <<< $OUTPUT; then
306 |             ERRORCQLVER=$( grep -o 'version .\?[0-9.]*.\? is not' <<< $OUTPUT | tr -dc ' 0-9.' )
307 |             SUPPORTED=($( grep -Eo 'upported( versions)?: .*[ 0-9.,]*' <<< $OUTPUT | tr -dc ' 0-9.' ))
308 |             CQLVER=${SUPPORTED[$((${#SUPPORTED[@]}-1))]}
309 |             printf "Default CQL version $ERRORCQLVER not supported by Cassandra at ${CASIP}.\n"
310 |             printf "Reported versions are ${SUPPORTED[@]}.  Attempting with ${CQLVER}.\n"
311 |             continue
312 |         elif grep -qi 'connection error\|not connect' <<< $OUTPUT; then
313 |             printf "ERROR: Unable to connect to Cassandra at ${CASIP}.\n"
314 |             [ "$TEMPDIR" != "/" ] && rm -rf "$TEMPDIR"
315 |             exit 1
316 |         elif [ $RC -eq 1 ]; then
317 |             printf "ERROR: Error executing cqlsh command:\n ${OUTPUT}\n"
318 |             [ "$TEMPDIR" != "/" ] && rm -rf "$TEMPDIR"
319 |             exit 1
320 |         else
321 |             printf "Performing Import:\n"
322 |             printf "    Original cassandra host: $FILEHOSTNAME\n"
323 |             printf "    Original keyspace name: $FILEKSNAME\n"
324 |             printf "    Original snapshot date: $FILESNAPDATE\n\n"
325 |             printf "    Importing schema into keyspace $NEWKSNAME\n"
326 |             printf "       Using datacenter $DCNAME and replication factor $NEWRFACTOR\n"
327 | 
328 |             # Wait - give the user a chance to panic and CTRL-C out
329 |             sleep 5
330 | 
331 |             # Create schema for the new keyspace
332 |             cqlsh -f "$SCHEMAFILE" $CQLSHVER $CASIP
333 | 
334 |             printf "    Loading snapshot into keyspace $NEWKSNAME\n"
335 |             for columnfamily in `ls "${TEMPDIR}/${NEWKSNAME}"`; do
336 |                 sstableloader -d $CASIP "${TEMPDIR}/${NEWKSNAME}/${columnfamily}"
337 |             done
338 | 
339 |             printf "\n\nImport operation complete - check output for errors\n"
340 |             [ "$TEMPDIR" != "/" ] && rm -rf "$TEMPDIR"
341 |             exit 0
342 |         fi
343 |     done
344 | 
345 | # Fin.
346 | 


--------------------------------------------------------------------------------