├── README.md
├── github-to-s3
├── install.sh
└── s3-to-github


/README.md:
--------------------------------------------------------------------------------
 1 | # github-s3
 2 | 
 3 | At Hyper, we make lots of things. Many of those things are open source, but
 4 | there are also things that are closed source. We have a limited amount of private
 5 | repositories on GitHub, though, so once we haven't needed them for a while we
 6 | archive them to AWS S3.
 7 | 
 8 | These are shell scripts that make it really easy to archive and restore repositories
 9 | between GitHub and AWS S3.
10 | 
11 | ## Usage
12 | 
13 | Clone the repository from GitHub, compress it and upload it to AWS S3:
14 | 
15 | ```zsh
16 | $ github-to-s3 hyperoslo/old-repository
17 | ```
18 | 
19 | Download the repository from AWS S3, uncompress it and push it to a new private
20 | repository on GitHub:
21 | 
22 | ```zsh
23 | $ s3-to-github hyperoslo/old-repository
24 | ```
25 | 
26 | ## Installation
27 | 
28 | ```
29 | $ curl -s --location http://raw.github.com/hyperoslo/github-s3/master/install.sh | sh
30 | ```
31 | 
32 | ## Configuration
33 | 
34 | You'll need to set the following environment variables to use these scripts:
35 | 
36 | * `GITHUB_S3_BUCKET`: The name of the bucket you want to archive to.
37 | * `AWS_DEFAULT_REGION`: The AWS region your bucket is in.
38 | * `GITHUB_S3_GITHUB_TOKEN`: A GitHub access token with access to create and
39 |   delete private repositories.
40 | 
41 | ## Hyper loves you
42 | 
43 | [Hyper] made this. We're a bunch of folks who love building things. You should
44 | [tweet us] if you can't get it to work. In fact, you should tweet us anyway.
45 | If you're using github-s3, we probably want to [hire you].
46 | 
47 | [Hyper]: https://github.com/hyperoslo
48 | [tweet us]: http://twitter.com/hyperoslo
49 | [hire you]: http://www.hyper.no/jobs/engineers
50 | 


--------------------------------------------------------------------------------
/github-to-s3:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/env bash
 2 | 
 3 | set -e
 4 | 
 5 | if [ -z $GITHUB_S3_BUCKET ]; then
 6 |   echo 'Error: $GITHUB_S3_BUCKET must be set to the bucket you wish to archive the repository to'
 7 |   return
 8 | fi
 9 | 
10 | GITHUB_PATH=$1
11 | NAME=$(echo $GITHUB_PATH | sed 's/\//-/')
12 | ARCHIVE_PATH=/tmp/$NAME.tar.gz
13 | REPOSITORY_PATH=/tmp/$NAME
14 | 
15 | rm -rf $REPOSITORY_PATH
16 | git clone --mirror git@github.com:$1 $REPOSITORY_PATH
17 | 
18 | rm -rf $ARCHIVE_PATH
19 | tar -C /tmp -zcf $ARCHIVE_PATH $NAME
20 | 
21 | aws s3 mv $ARCHIVE_PATH s3://$GITHUB_S3_BUCKET/$NAME.tar.gz
22 | 
23 | curl -X DELETE -H "Authorization: token $GITHUB_S3_GITHUB_TOKEN" https://api.github.com/repos/$1
24 | rm -rf $REPOSITORY_PATH $ARCHIVE_PATH
25 | 


--------------------------------------------------------------------------------
/install.sh:
--------------------------------------------------------------------------------
 1 | #!/bin/bash
 2 | 
 3 | echo "
 4 |      ______ __  __ __     __       ________
 5 |     / ___(_) /_/ // /_ __/ /  ____/ __/_  /
 6 |    / (_ / / __/ _  / // / _ \/___/\ \_/_ < 
 7 |    \___/_/\__/_//_/\_,_/_.__/   /___/____/ 
 8 | "
 9 | 
10 | echo "Downloading scripts to /usr/local/bin..."
11 | curl --progress-bar --location "https://raw.github.com/hyperoslo/github-s3/master/github-to-s3" > /usr/local/bin/github-to-s3
12 | curl --progress-bar --location "https://raw.github.com/hyperoslo/github-s3/master/s3-to-github" > /usr/local/bin/s3-to-github
13 | 
14 | chmod +x /usr/local/bin/github-to-s3
15 | chmod +x /usr/local/bin/s3-to-github
16 | 
17 | echo "Done."
18 | 


--------------------------------------------------------------------------------
/s3-to-github:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/env bash
 2 | 
 3 | set -e
 4 | 
 5 | if [ -z $GITHUB_S3_BUCKET ]; then
 6 |   echo 'Error: $GITHUB_S3_BUCKET must be set to the bucket you wish to restore the repository from'
 7 |   return
 8 | fi
 9 | 
10 | GITHUB_PATH=$1
11 | GITHUB_ORGANIZATION_NAME=$(echo $GITHUB_PATH | sed -E 's/^(.+)\/(.+)/\1/')
12 | GITHUB_REPOSITORY_NAME=$(echo $GITHUB_PATH | sed -E 's/^(.+)\/(.+)/\2/')
13 | NAME=$(echo $GITHUB_PATH | sed 's/\//-/')
14 | ARCHIVE_PATH=/tmp/$NAME.tar.gz
15 | REPOSITORY_PATH=/tmp/$NAME
16 | 
17 | aws s3 cp s3://$GITHUB_S3_BUCKET/$NAME.tar.gz /tmp
18 | 
19 | rm -rf $REPOSITORY_PATH
20 | tar -C /tmp -xf $ARCHIVE_PATH
21 | 
22 | curl -s -X POST -H "Authorization: token $GITHUB_S3_GITHUB_TOKEN" \
23 |   https://api.github.com/orgs/$GITHUB_ORGANIZATION_NAME/repos \
24 |   -d "{\"name\":\"$GITHUB_REPOSITORY_NAME\", \"private\": true}" > /dev/null
25 | 
26 | git -C $REPOSITORY_PATH push --mirror
27 | rm -rf $REPOSITORY_PATH $ARCHIVE_PATH
28 | 


--------------------------------------------------------------------------------