├── .gitignore ├── .travis.yml ├── LICENSE ├── Makefile ├── README.md ├── backends └── local │ ├── local.go │ └── local_test.go ├── crypto └── nacl │ ├── nacl.go │ └── nacl_test.go ├── git ├── handler │ ├── encoder-decoder.go │ ├── git_suite_test.go │ ├── handler.go │ ├── handler_test.go │ └── integration_test.go ├── merger │ ├── packfile_merger.go │ └── packfile_merger_test.go └── repo │ ├── json_repo.go │ ├── repo.go │ └── repo_test.go ├── main.go └── main_test.go /.gitignore: -------------------------------------------------------------------------------- 1 | *.zip 2 | git-cr 3 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: go 2 | 3 | go: 4 | - 1.2 5 | - 1.4 6 | - tip 7 | 8 | install: go get -t ./... 9 | script: go test -v ./... 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Lucas Clemente 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: release 2 | 3 | release: 4 | GOOS=darwin go build 5 | zip git-cr.osx.zip git-cr 6 | GOOS=linux go build 7 | zip git-cr.linux.zip git-cr 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 🔒 git-cr — Client side encryption for git 2 | 3 | [![Build Status](https://travis-ci.org/lucas-clemente/git-cr.svg?branch=master)](https://travis-ci.org/lucas-clemente/git-cr) 4 | 5 | ## What it does 6 | 7 | git-cr is a git remote that encrypts all data in a repo (including metadata) client-side. You can still use all of git's feature, including efficient deltas. 8 | 9 | Currently git-cr stores your data in encrypted form in a local directory (e.g. in Dropbox, Google Drive, …), but a remote backend might be added soon. 10 | 11 | ## What's new about git-cr 12 | 13 | There are some [tools](https://github.com/shadowhand/git-encrypt) and [tutorials](https://gist.github.com/shadowhand/873637) on how to encrypt single files stored in git. git-cr is different: it encrypts the whole repo, including metadata such as file names, branch names, commit messages. You also don't loose as many git features (e.g. awesome compression and efficient pushes / pulls). 14 | 15 | ## Instructions 16 | 17 | ### Installation 18 | 19 | Installation using go: 20 | 21 | ```shell 22 | go get github.com/lucas-clemente/git-cr 23 | ``` 24 | 25 | Alternatively (if you don't have go), you can download a current release from [github](https://github.com/lucas-clemente/git-cr/releases) and move it somewhere into your `$PATH`. 26 | 27 | ### Cloning 28 | 29 | To clone an existing repo: 30 | 31 | ```shell 32 | git cr clone /path/to/git-cr/repo nacl:MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI= my-clone 33 | ``` 34 | 35 | ### Pushing 36 | 37 | ```shell 38 | git cr add crypto /path/to/git-cr/repo nacl:MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI= 39 | git push crypto master 40 | ``` 41 | 42 | ### Encryption 43 | 44 | The secret for NaCl is a 32 byte base64 encoded string. You can generate a new secret using 45 | 46 | ```shell 47 | echo -n nacl:; head -c32 /dev/urandom |base64 48 | ``` 49 | 50 | ### Everything else 51 | 52 | Just use git! 53 | 54 | ## How it works 55 | 56 | git-cr uses a git feature called [external remotes](http://git-scm.com/docs/git-remote-ext): 57 | 58 | ```shell 59 | $ git remote -v 60 | crypto ext::git cr %G run /path/to/remote nacl:MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI= (fetch) 61 | crypto ext::git cr %G run /path/to/remote nacl:MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI= (push) 62 | ``` 63 | 64 | Any git operation that needs the remote (e.g. pull, push, clone) then starts git-cr as a child process and uses pipes to talk the git protocol. 65 | 66 | git-cr manages two things, refs (i.e. branch names) and packfiles (i.e. your data), in numbered _revisions_. Each push creates a new revision. These revisions are never visible to git in any way! 67 | 68 | When pushing, git first sends the ref updates that git-cr uses to create a new revision. Then git sends the diffs as a so-called _thin packfile_, that git-cr encrypts and stores. 69 | 70 | When pulling, git and git-cr first work out the current state of the local git repo. git-cr calculates the minimum set of previously stored packfiles it needs to send (i.e. all packfiles since the last revision the client completely has). Then it decrypts these packfiles, merges them into one and sends it to git. 71 | 72 | ## Is it secure? 73 | 74 | I'm not a cryptographer and git-cr was never audited by anyone. So you probably shouldn't trust it for anything critical. 75 | 76 | git-cr uses the backend to store whole files only. Files can either be git packfiles, or a manifest file containing the git refs for each revision. Each file is encrypted using [NaCl's](http://nacl.cr.yp.to) authenticated encryption `crypto_secretbox`. The key is static and part of the repository URL, while the nonce is generated (using `crypto/rand`) per file and stored prepended to the ciphertext. 77 | 78 | The source code for this can be found [here](crypto/nacl/nacl.go). Check it out! 79 | 80 | What git-cr does not hide: 81 | 82 | - The size of your deltas (be aware of oracle attacks). 83 | - The dates when you push. 84 | 85 | Currently the encryption key is stored in plain text on disk and is visible during some commands, see [#5](https://github.com/lucas-clemente/git-cr/issues/5). 86 | 87 | ## License 88 | 89 | [MIT](LICENSE) of course. 90 | -------------------------------------------------------------------------------- /backends/local/local.go: -------------------------------------------------------------------------------- 1 | package local 2 | 3 | import ( 4 | "io" 5 | "os" 6 | 7 | "github.com/lucas-clemente/git-cr/git/repo" 8 | ) 9 | 10 | type localBackend struct { 11 | path string 12 | } 13 | 14 | // NewLocalBackend returns a backend that stores data in the given path 15 | func NewLocalBackend(path string) (repo.Backend, error) { 16 | if err := os.MkdirAll(path, 0755); err != nil { 17 | return nil, err 18 | } 19 | return &localBackend{path: path}, nil 20 | } 21 | 22 | func (b *localBackend) ReadBlob(name string) (io.ReadCloser, error) { 23 | f, err := os.Open(b.path + "/" + name) 24 | if os.IsNotExist(err) { 25 | return nil, repo.ErrNotFound 26 | } 27 | return f, err 28 | } 29 | 30 | func (b *localBackend) WriteBlob(name string, r io.Reader) error { 31 | f, err := os.Create(b.path + "/" + name) 32 | if err != nil { 33 | return err 34 | } 35 | defer f.Close() 36 | _, err = io.Copy(f, r) 37 | return err 38 | } 39 | -------------------------------------------------------------------------------- /backends/local/local_test.go: -------------------------------------------------------------------------------- 1 | package local_test 2 | 3 | import ( 4 | "bytes" 5 | "io/ioutil" 6 | "os" 7 | "testing" 8 | 9 | "github.com/lucas-clemente/git-cr/backends/local" 10 | "github.com/lucas-clemente/git-cr/git/repo" 11 | 12 | . "github.com/onsi/ginkgo" 13 | . "github.com/onsi/gomega" 14 | ) 15 | 16 | func TestLocalRepo(t *testing.T) { 17 | RegisterFailHandler(Fail) 18 | RunSpecs(t, "Local Backend Suite") 19 | } 20 | 21 | var _ = Describe("Local Backend", func() { 22 | var ( 23 | tmpDir string 24 | backend repo.Backend 25 | ) 26 | 27 | BeforeEach(func() { 28 | var err error 29 | 30 | tmpDir, err = ioutil.TempDir("", "io.clemente.git-cr.test") 31 | Ω(err).ShouldNot(HaveOccurred()) 32 | 33 | backend, err = local.NewLocalBackend(tmpDir) 34 | Ω(err).ShouldNot(HaveOccurred()) 35 | }) 36 | 37 | AfterEach(func() { 38 | os.RemoveAll(tmpDir) 39 | }) 40 | 41 | It("reads", func() { 42 | err := ioutil.WriteFile(tmpDir+"/foo", []byte("bar"), 0644) 43 | Ω(err).ShouldNot(HaveOccurred()) 44 | r, err := backend.ReadBlob("foo") 45 | Ω(err).ShouldNot(HaveOccurred()) 46 | data, err := ioutil.ReadAll(r) 47 | Ω(err).ShouldNot(HaveOccurred()) 48 | Ω(data).Should(Equal([]byte("bar"))) 49 | }) 50 | 51 | It("writes", func() { 52 | err := backend.WriteBlob("foo", bytes.NewBufferString("bar")) 53 | Ω(err).ShouldNot(HaveOccurred()) 54 | data, err := ioutil.ReadFile(tmpDir + "/foo") 55 | Ω(err).ShouldNot(HaveOccurred()) 56 | Ω(data).Should(Equal([]byte("bar"))) 57 | }) 58 | }) 59 | -------------------------------------------------------------------------------- /crypto/nacl/nacl.go: -------------------------------------------------------------------------------- 1 | package nacl 2 | 3 | import ( 4 | "bytes" 5 | "crypto/rand" 6 | "errors" 7 | "io" 8 | "io/ioutil" 9 | 10 | "github.com/lucas-clemente/git-cr/git/repo" 11 | 12 | "golang.org/x/crypto/nacl/secretbox" 13 | ) 14 | 15 | type naclBackend struct { 16 | backend repo.Backend 17 | key [32]byte 18 | } 19 | 20 | // NewNaClBackend returns a repo.Backend implementation that encrypts data using nacl 21 | func NewNaClBackend(backend repo.Backend, key [32]byte) repo.Backend { 22 | return &naclBackend{ 23 | backend: backend, 24 | key: key, 25 | } 26 | } 27 | 28 | func (r *naclBackend) ReadBlob(name string) (io.ReadCloser, error) { 29 | encryptedRdr, err := r.backend.ReadBlob(name + ".nacl") 30 | if err != nil { 31 | return nil, err 32 | } 33 | defer encryptedRdr.Close() 34 | 35 | data, err := ioutil.ReadAll(encryptedRdr) 36 | if err != nil { 37 | return nil, err 38 | } 39 | 40 | if len(data) < 24 { 41 | return nil, errors.New("encrypted message is too short") 42 | } 43 | var nonce [24]byte 44 | copy(nonce[:], data) 45 | data = data[24:] 46 | 47 | out, ok := secretbox.Open([]byte{}, data, &nonce, &r.key) 48 | if !ok { 49 | return nil, errors.New("error verifying encrypted data") 50 | } 51 | return ioutil.NopCloser(bytes.NewBuffer(out)), nil 52 | } 53 | 54 | func (r *naclBackend) WriteBlob(name string, rdr io.Reader) error { 55 | data, err := ioutil.ReadAll(rdr) 56 | if err != nil { 57 | return err 58 | } 59 | nonce := makeNonce() 60 | out := secretbox.Seal(nonce[:], data, nonce, &r.key) 61 | return r.backend.WriteBlob(name+".nacl", bytes.NewBuffer(out)) 62 | } 63 | 64 | func makeNonce() *[24]byte { 65 | var nonce [24]byte 66 | _, err := rand.Read(nonce[:]) 67 | if err != nil { 68 | panic(err) 69 | } 70 | return &nonce 71 | } 72 | -------------------------------------------------------------------------------- /crypto/nacl/nacl_test.go: -------------------------------------------------------------------------------- 1 | package nacl_test 2 | 3 | import ( 4 | "bytes" 5 | "io" 6 | "io/ioutil" 7 | "testing" 8 | 9 | "github.com/lucas-clemente/git-cr/crypto/nacl" 10 | "github.com/lucas-clemente/git-cr/git/repo" 11 | 12 | . "github.com/onsi/ginkgo" 13 | . "github.com/onsi/gomega" 14 | ) 15 | 16 | func TestNaCl(t *testing.T) { 17 | RegisterFailHandler(Fail) 18 | RunSpecs(t, "NaCl Suite") 19 | } 20 | 21 | type fixtureBackend map[string][]byte 22 | 23 | func (f fixtureBackend) ReadBlob(name string) (io.ReadCloser, error) { 24 | return ioutil.NopCloser(bytes.NewBuffer(f[name])), nil 25 | } 26 | 27 | func (f fixtureBackend) WriteBlob(name string, r io.Reader) error { 28 | data, err := ioutil.ReadAll(r) 29 | if err != nil { 30 | return err 31 | } 32 | f[name] = data 33 | return nil 34 | } 35 | 36 | var _ = Describe("NaCl", func() { 37 | var ( 38 | naclBackend repo.Backend 39 | backend fixtureBackend 40 | key [32]byte 41 | err error 42 | ) 43 | 44 | BeforeEach(func() { 45 | copy(key[:], "Forty-two, said Deep Thought, with infinite majesty and calm.") 46 | Ω(err).ShouldNot(HaveOccurred()) 47 | backend = fixtureBackend{} 48 | naclBackend = nacl.NewNaClBackend(backend, key) 49 | }) 50 | 51 | It("writes data", func() { 52 | err := naclBackend.WriteBlob("foo", bytes.NewBufferString("foobar")) 53 | Ω(err).ShouldNot(HaveOccurred()) 54 | Ω(backend["foo.nacl"]).ShouldNot(HaveLen(0)) 55 | }) 56 | 57 | It("reads data", func() { 58 | backend["foo.nacl"] = []byte{0x5d, 0x10, 0x39, 0x1c, 0x77, 0x2, 0xb, 0x26, 0x7e, 0xa6, 0x58, 0x52, 0xb9, 0x18, 0x55, 0x40, 0xb, 0x1, 0xd2, 0xc0, 0x40, 0xc9, 0xb3, 0xec, 0x27, 0x95, 0x9d, 0xf8, 0x17, 0x4b, 0xc7, 0xbb, 0xbb, 0x7, 0x31, 0x64, 0x66, 0xc9, 0xb9, 0xf8, 0x81, 0xdc, 0xef, 0xd, 0x6d, 0x56} 59 | rdr, err := naclBackend.ReadBlob("foo") 60 | Ω(err).ShouldNot(HaveOccurred()) 61 | data, err := ioutil.ReadAll(rdr) 62 | Ω(err).ShouldNot(HaveOccurred()) 63 | Ω(data).Should(Equal([]byte("foobar"))) 64 | }) 65 | }) 66 | -------------------------------------------------------------------------------- /git/handler/encoder-decoder.go: -------------------------------------------------------------------------------- 1 | package handler 2 | 3 | // Encoder is used for sending data via pkt-line 4 | // Sending a `nil` does an ACK. 5 | type Encoder interface { 6 | Encode([]byte) error 7 | } 8 | 9 | // Decoder is used to decode data using pkt-line 10 | // ACKs are decoded as `nil`. 11 | type Decoder interface { 12 | Decode(*[]byte) error 13 | 14 | // Read bypasses the pkt-line decoding, used when receiving packfiles 15 | Read(p []byte) (n int, err error) 16 | } 17 | -------------------------------------------------------------------------------- /git/handler/git_suite_test.go: -------------------------------------------------------------------------------- 1 | package handler_test 2 | 3 | import ( 4 | "bytes" 5 | "encoding/base64" 6 | "io" 7 | "io/ioutil" 8 | 9 | "github.com/lucas-clemente/git-cr/git/repo" 10 | . "github.com/onsi/ginkgo" 11 | . "github.com/onsi/gomega" 12 | 13 | "testing" 14 | ) 15 | 16 | func TestGitHandler(t *testing.T) { 17 | RegisterFailHandler(Fail) 18 | RunSpecs(t, "Git Handler Suite") 19 | } 20 | 21 | // A FixtureRepo for tests 22 | type FixtureRepo struct { 23 | Revisions []repo.Revision 24 | Packfiles [][]byte 25 | } 26 | 27 | var _ repo.Repo = &FixtureRepo{} 28 | 29 | // NewFixtureRepo makes a new fixture repo 30 | func NewFixtureRepo() *FixtureRepo { 31 | return &FixtureRepo{} 32 | } 33 | 34 | // GetRevisions implements repo.Repo 35 | func (r *FixtureRepo) GetRevisions() ([]repo.Revision, error) { 36 | return r.Revisions, nil 37 | } 38 | 39 | // SaveNewRevision implements repo.Repo 40 | func (r *FixtureRepo) SaveNewRevision(rev repo.Revision, packfile io.Reader) error { 41 | r.Revisions = append(r.Revisions, rev) 42 | data, err := ioutil.ReadAll(packfile) 43 | if err != nil { 44 | return err 45 | } 46 | r.Packfiles = append(r.Packfiles, data) 47 | return nil 48 | } 49 | 50 | // ReadPackfile implements repo.Repo 51 | func (r *FixtureRepo) ReadPackfile(toRev int) (io.ReadCloser, error) { 52 | return ioutil.NopCloser(bytes.NewBuffer(r.Packfiles[toRev])), nil 53 | } 54 | 55 | // SaveNewRevisionB64 adds a base64-encoded packfile to the repo 56 | func (r *FixtureRepo) SaveNewRevisionB64(rev repo.Revision, b64 string) { 57 | pack, err := base64.StdEncoding.DecodeString(b64) 58 | if err != nil { 59 | panic("invalid base64 in FixtureRepo.AddPackfile") 60 | } 61 | if err := r.SaveNewRevision(rev, bytes.NewBuffer(pack)); err != nil { 62 | panic(err) 63 | } 64 | } 65 | -------------------------------------------------------------------------------- /git/handler/handler.go: -------------------------------------------------------------------------------- 1 | package handler 2 | 3 | import ( 4 | "bytes" 5 | "errors" 6 | "io" 7 | "io/ioutil" 8 | "strings" 9 | 10 | "github.com/lucas-clemente/git-cr/git/repo" 11 | "github.com/lucas-clemente/git-cr/git/merger" 12 | ) 13 | 14 | const pullCapabilities = "multi_ack_detailed side-band-64k thin-pack" 15 | const pushCapabilities = "delete-refs ofs-delta" 16 | 17 | var ( 18 | // ErrorInvalidHandshake occurs if the client presents an invalid handshake 19 | ErrorInvalidHandshake = errors.New("invalid handshake from client") 20 | // ErrorInvalidWantLine occurs if the client sends an invalid want line 21 | ErrorInvalidWantLine = errors.New("invalid `want` line sent by client") 22 | // ErrorInvalidHaveLine occurs if the client sends an invalid have line 23 | ErrorInvalidHaveLine = errors.New("invalid `have` line sent by client") 24 | // ErrorInvalidPushRefsLine occurs if the client sends an invalid line during ref update 25 | ErrorInvalidPushRefsLine = errors.New("invalid line sent by client during ref update") 26 | // ErrorNoHead occurs if a repo has no HEAD 27 | ErrorNoHead = errors.New("no HEAD in repo") 28 | ) 29 | 30 | // A GitOperation can either be a pull or push 31 | type GitOperation int 32 | 33 | const ( 34 | // GitPull is a pull 35 | GitPull GitOperation = iota + 1 36 | // GitPush is a push 37 | GitPush 38 | ) 39 | 40 | // GitRequestHandler handles the git protocol 41 | type GitRequestHandler struct { 42 | out Encoder 43 | in Decoder 44 | 45 | repo repo.Repo 46 | } 47 | 48 | // A RefUpdate is a delta for a git reference 49 | type RefUpdate struct { 50 | Name, OldID, NewID string 51 | } 52 | 53 | // NewGitRequestHandler makes a handler for the git protocol 54 | func NewGitRequestHandler(out Encoder, in Decoder, repo repo.Repo) *GitRequestHandler { 55 | return &GitRequestHandler{ 56 | out: out, 57 | in: in, 58 | repo: repo, 59 | } 60 | } 61 | 62 | // ServeRequest handles a single git request 63 | func (h *GitRequestHandler) ServeRequest() error { 64 | op, err := h.ReceiveHandshake() 65 | if err != nil { 66 | return err 67 | } 68 | 69 | revisions, err := h.repo.GetRevisions() 70 | if err != nil { 71 | return err 72 | } 73 | 74 | currentRevIndex := len(revisions) - 1 75 | var currentRev repo.Revision 76 | if currentRevIndex == -1 { 77 | currentRev = repo.Revision{} 78 | } else { 79 | currentRev = revisions[currentRevIndex] 80 | } 81 | 82 | if err := h.SendRefs(currentRev, op); err != nil { 83 | return err 84 | } 85 | 86 | // TODO(lucas): Split up into two functions 87 | if op == GitPull { 88 | wants, err := h.ReceivePullWants() 89 | if err != nil { 90 | return err 91 | } 92 | 93 | if len(wants) == 0 { 94 | return nil 95 | } 96 | 97 | fromRev, err := h.NegotiatePullPackfile(revisions) 98 | if err != nil { 99 | return err 100 | } 101 | 102 | packfiles := [][]byte{} 103 | for i := fromRev; i <= currentRevIndex; i++ { 104 | rdr, err := h.repo.ReadPackfile(i) 105 | if err != nil { 106 | return err 107 | } 108 | packfile, err := ioutil.ReadAll(rdr) 109 | if err != nil { 110 | return err 111 | } 112 | packfiles = append(packfiles, packfile) 113 | } 114 | 115 | packfile, err := merger.MergePackfiles(packfiles) 116 | if err != nil { 117 | return err 118 | } 119 | 120 | if err := h.SendPackfile(ioutil.NopCloser(bytes.NewBuffer(packfile))); err != nil { 121 | return err 122 | } 123 | } else if op == GitPush { 124 | refUpdates, err := h.ReceivePushRefs() 125 | if err != nil { 126 | return err 127 | } 128 | 129 | if len(refUpdates) == 0 { 130 | return nil 131 | } 132 | 133 | newRevision := repo.Revision{} 134 | for k, v := range currentRev { 135 | newRevision[k] = v 136 | } 137 | 138 | for _, update := range refUpdates { 139 | if update.Name == "refs/heads/master" && update.NewID != "" { 140 | newRevision["HEAD"] = update.NewID 141 | } 142 | if update.NewID == "" { 143 | delete(newRevision, update.Name) 144 | } else { 145 | newRevision[update.Name] = update.NewID 146 | } 147 | } 148 | 149 | // Read packfile 150 | packfile, err := ioutil.ReadAll(h.in) 151 | if err != nil { 152 | return err 153 | } 154 | if len(packfile) == 0 { 155 | packfile = []byte{'P', 'A', 'C', 'K', 0, 0, 0, 2, 0, 0, 0, 0, 0x02, 0x9d, 0x08, 0x82, 0x3b, 0xd8, 0xa8, 0xea, 0xb5, 0x10, 0xad, 0x6a, 0xc7, 0x5c, 0x82, 0x3c, 0xfd, 0x3e, 0xd3, 0x1e} 156 | } 157 | 158 | if err = h.repo.SaveNewRevision(newRevision, ioutil.NopCloser(bytes.NewBuffer(packfile))); err != nil { 159 | return err 160 | } 161 | } else { 162 | panic("unexpected git op") 163 | } 164 | 165 | return nil 166 | } 167 | 168 | // ReceiveHandshake reads repo and host info from the client 169 | func (h *GitRequestHandler) ReceiveHandshake() (GitOperation, error) { 170 | // format: "git-[upload|receive]-pack repo-name\0host=host-name" 171 | var handshake []byte 172 | 173 | if err := h.in.Decode(&handshake); err != nil { 174 | return 0, err 175 | } 176 | 177 | if bytes.HasPrefix(handshake, []byte("git-upload-pack ")) { 178 | return GitPull, nil 179 | } else if bytes.HasPrefix(handshake, []byte("git-receive-pack ")) { 180 | return GitPush, nil 181 | } 182 | return 0, ErrorInvalidHandshake 183 | } 184 | 185 | // SendRefs sends the given references to the client 186 | func (h *GitRequestHandler) SendRefs(refs map[string]string, op GitOperation) error { 187 | if len(refs) == 0 { 188 | return h.out.Encode(nil) 189 | } 190 | 191 | var caps string 192 | if op == GitPull { 193 | caps = pullCapabilities 194 | } else { 195 | caps = pushCapabilities 196 | } 197 | 198 | head, ok := refs["HEAD"] 199 | if !ok { 200 | return ErrorNoHead 201 | } 202 | if err := h.out.Encode([]byte(head + " HEAD\000" + caps)); err != nil { 203 | return err 204 | } 205 | 206 | for name, sha1 := range refs { 207 | if name == "HEAD" { 208 | continue 209 | } 210 | if err := h.out.Encode([]byte(sha1 + " " + name)); err != nil { 211 | return err 212 | } 213 | } 214 | 215 | return h.out.Encode(nil) 216 | } 217 | 218 | // ReceivePullWants receives the requested refs from the client 219 | func (h *GitRequestHandler) ReceivePullWants() ([]string, error) { 220 | refs := []string{} 221 | var line []byte 222 | for { 223 | if err := h.in.Decode(&line); err != nil { 224 | return nil, err 225 | } 226 | 227 | if line == nil { 228 | break 229 | } 230 | 231 | if !bytes.HasPrefix(line, []byte("want ")) { 232 | return nil, ErrorInvalidWantLine 233 | } 234 | 235 | if len(line) < 45 { 236 | return nil, ErrorInvalidWantLine 237 | } 238 | refs = append(refs, string(line[5:45])) 239 | } 240 | return refs, nil 241 | } 242 | 243 | // NegotiatePullPackfile receives the client's haves and uses the repo 244 | // to calculate the deltas that should be sent to the client 245 | func (h *GitRequestHandler) NegotiatePullPackfile(revisions []repo.Revision) (int, error) { 246 | // multi_ack_detailed implementation 247 | var line []byte 248 | 249 | // Each time we receive a have from a client, we remove it from all revisions. 250 | // Once we have an empty revision, we return that to be useed as a base. 251 | // revisionCommits has commits in reverse order. 252 | revisionCommits := make([]map[string]struct{}, len(revisions)) 253 | for i, r := range revisions { 254 | m := make(map[string]struct{}) 255 | for _, sha := range r { 256 | m[sha] = struct{}{} 257 | } 258 | revisionCommits[len(revisions)-i-1] = m 259 | } 260 | 261 | lastCommon := "" 262 | 263 | result := -1 264 | 265 | for { 266 | if err := h.in.Decode(&line); err != nil { 267 | return 0, err 268 | } 269 | 270 | if line == nil { 271 | h.out.Encode([]byte("NAK")) 272 | continue 273 | } 274 | 275 | if bytes.HasPrefix(line, []byte("done")) { 276 | if len(lastCommon) == 0 { 277 | h.out.Encode([]byte("NAK")) 278 | } else { 279 | h.out.Encode([]byte("ACK " + lastCommon)) 280 | } 281 | break 282 | } 283 | 284 | if !bytes.HasPrefix(line, []byte("have ")) { 285 | return 0, ErrorInvalidHaveLine 286 | } 287 | 288 | if len(line) < 45 { 289 | return 0, ErrorInvalidHaveLine 290 | } 291 | have := string(line[5:45]) 292 | 293 | common := false 294 | 295 | // Remove have from all revisions 296 | for i, commits := range revisionCommits { 297 | oldLen := len(commits) 298 | delete(commits, have) 299 | newLen := len(commits) 300 | 301 | if newLen == 0 { 302 | result = len(revisions) - i - 1 303 | break 304 | } else if newLen != oldLen { 305 | common = true 306 | } 307 | } 308 | 309 | if result != -1 { 310 | h.out.Encode([]byte("ACK " + have + " ready")) 311 | lastCommon = have 312 | } else if common { 313 | h.out.Encode([]byte("ACK " + have + " common")) 314 | lastCommon = have 315 | } 316 | } 317 | 318 | if result == -1 { 319 | // From the beginning 320 | return 0, nil 321 | } 322 | 323 | return result, nil 324 | } 325 | 326 | // SendPackfile sends a packfile using the side-band-64k encoding 327 | func (h *GitRequestHandler) SendPackfile(r io.Reader) error { 328 | for { 329 | line := make([]byte, 65519) 330 | line[0] = 1 331 | n, err := r.Read(line[1:]) 332 | if n != 0 { 333 | h.out.Encode(line[0 : n+1]) 334 | } 335 | if err != nil { 336 | if err == io.EOF { 337 | break 338 | } 339 | return err 340 | } 341 | } 342 | return h.out.Encode(nil) 343 | } 344 | 345 | // ReceivePushRefs receives the references to be updates in a push from the client 346 | func (h *GitRequestHandler) ReceivePushRefs() ([]RefUpdate, error) { 347 | var line []byte 348 | refs := []RefUpdate{} 349 | for { 350 | if err := h.in.Decode(&line); err != nil { 351 | return nil, err 352 | } 353 | 354 | if line == nil { 355 | break 356 | } 357 | 358 | parts := bytes.Split(line, []byte(" ")) 359 | if len(parts) != 3 { 360 | return nil, ErrorInvalidPushRefsLine 361 | } 362 | 363 | name := string(parts[2]) 364 | if name[len(name)-1] == 0 { 365 | name = name[0 : len(name)-1] 366 | } 367 | name = strings.TrimSpace(name) 368 | oldID := string(parts[0]) 369 | if isNullID(oldID) { 370 | oldID = "" 371 | } 372 | newID := string(parts[1]) 373 | if isNullID(newID) { 374 | newID = "" 375 | } 376 | 377 | refs = append(refs, RefUpdate{Name: name, OldID: oldID, NewID: newID}) 378 | } 379 | return refs, nil 380 | } 381 | 382 | func isNullID(id string) bool { 383 | for _, c := range id { 384 | if c != '0' { 385 | return false 386 | } 387 | } 388 | return true 389 | } 390 | -------------------------------------------------------------------------------- /git/handler/handler_test.go: -------------------------------------------------------------------------------- 1 | package handler_test 2 | 3 | import ( 4 | "bytes" 5 | "io" 6 | "math/rand" 7 | 8 | "github.com/lucas-clemente/git-cr/git/handler" 9 | "github.com/lucas-clemente/git-cr/git/repo" 10 | 11 | . "github.com/onsi/ginkgo" 12 | . "github.com/onsi/gomega" 13 | ) 14 | 15 | type sampleDecoder struct { 16 | data [][]byte 17 | } 18 | 19 | func (d *sampleDecoder) Decode(b *[]byte) error { 20 | if len(d.data) == 0 { 21 | return io.EOF 22 | } 23 | *b = d.data[0] 24 | d.data = d.data[1:] 25 | return nil 26 | } 27 | 28 | func (d *sampleDecoder) Read(p []byte) (int, error) { 29 | panic("not implemented") 30 | } 31 | 32 | func (d *sampleDecoder) setData(data ...[]byte) { 33 | d.data = data 34 | } 35 | 36 | type sampleEncoder struct { 37 | data [][]byte 38 | } 39 | 40 | func (d *sampleEncoder) Encode(b []byte) error { 41 | d.data = append(d.data, b) 42 | return nil 43 | } 44 | 45 | var _ = Describe("git server", func() { 46 | var ( 47 | decoder *sampleDecoder 48 | encoder *sampleEncoder 49 | fixtureRepo *FixtureRepo 50 | gitHandler *handler.GitRequestHandler 51 | ) 52 | 53 | BeforeEach(func() { 54 | decoder = &sampleDecoder{} 55 | encoder = &sampleEncoder{data: [][]byte{}} 56 | fixtureRepo = NewFixtureRepo() 57 | gitHandler = handler.NewGitRequestHandler(encoder, decoder, fixtureRepo) 58 | }) 59 | 60 | Context("decoding client handshake", func() { 61 | It("handles pulls", func() { 62 | decoder.setData([]byte("git-upload-pack foo\000host=bar")) 63 | op, err := gitHandler.ReceiveHandshake() 64 | Ω(err).ShouldNot(HaveOccurred()) 65 | Ω(op).Should(Equal(handler.GitPull)) 66 | }) 67 | 68 | It("handles pushes", func() { 69 | decoder.setData([]byte("git-receive-pack foo\000host=bar")) 70 | op, err := gitHandler.ReceiveHandshake() 71 | Ω(err).ShouldNot(HaveOccurred()) 72 | Ω(op).Should(Equal(handler.GitPush)) 73 | }) 74 | }) 75 | 76 | Context("sending refs", func() { 77 | It("sends reflist for pull", func() { 78 | refs := map[string]string{"HEAD": "bar", "foo": "bar"} 79 | Ω(gitHandler.SendRefs(refs, handler.GitPull)).ShouldNot(HaveOccurred()) 80 | Ω(encoder.data).Should(HaveLen(3)) 81 | Ω(encoder.data[0]).Should(Equal([]byte("bar HEAD\000multi_ack_detailed side-band-64k thin-pack"))) 82 | Ω(encoder.data[1]).Should(Equal([]byte("bar foo"))) 83 | Ω(encoder.data[2]).Should(BeNil()) 84 | }) 85 | 86 | It("sends reflist for push", func() { 87 | refs := map[string]string{"HEAD": "bar", "foo": "bar"} 88 | Ω(gitHandler.SendRefs(refs, handler.GitPush)).ShouldNot(HaveOccurred()) 89 | Ω(encoder.data).Should(HaveLen(3)) 90 | Ω(encoder.data[0]).Should(Equal([]byte("bar HEAD\000delete-refs ofs-delta"))) 91 | Ω(encoder.data[1]).Should(Equal([]byte("bar foo"))) 92 | Ω(encoder.data[2]).Should(BeNil()) 93 | }) 94 | }) 95 | 96 | Context("reading pull wants", func() { 97 | It("receives wants", func() { 98 | decoder.setData( 99 | []byte("want 30f79bec32243c31dd91a05c0ad7b80f1e301aea\n"), 100 | []byte("want f1d2d2f924e986ac86fdf7b36c94bcdf32beec15\n"), 101 | nil, 102 | ) 103 | wants, err := gitHandler.ReceivePullWants() 104 | Ω(err).ShouldNot(HaveOccurred()) 105 | Ω(wants).Should(HaveLen(2)) 106 | Ω(wants[0]).Should(Equal("30f79bec32243c31dd91a05c0ad7b80f1e301aea")) 107 | Ω(wants[1]).Should(Equal("f1d2d2f924e986ac86fdf7b36c94bcdf32beec15")) 108 | }) 109 | 110 | It("handles client capabilities", func() { 111 | decoder.setData( 112 | []byte("want 30f79bec32243c31dd91a05c0ad7b80f1e301aea foobar\n"), 113 | nil, 114 | ) 115 | wants, err := gitHandler.ReceivePullWants() 116 | Ω(err).ShouldNot(HaveOccurred()) 117 | Ω(wants).Should(HaveLen(1)) 118 | Ω(wants[0]).Should(Equal("30f79bec32243c31dd91a05c0ad7b80f1e301aea")) 119 | }) 120 | 121 | It("receives empty wants", func() { 122 | decoder.setData(nil) 123 | wants, err := gitHandler.ReceivePullWants() 124 | Ω(err).ShouldNot(HaveOccurred()) 125 | Ω(wants).Should(HaveLen(0)) 126 | }) 127 | }) 128 | 129 | Context("negotiating packfiles", func() { 130 | It("handles full deltas", func() { 131 | revisions := []repo.Revision{ 132 | repo.Revision{ 133 | "refs/heads/master": "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 134 | }, 135 | } 136 | decoder.setData( 137 | []byte("have 30f79bec32243c31dd91a05c0ad7b80f1e301aea\n"), 138 | []byte("done\n"), 139 | ) 140 | i, err := gitHandler.NegotiatePullPackfile(revisions) 141 | Ω(err).ShouldNot(HaveOccurred()) 142 | Ω(encoder.data).Should(HaveLen(1)) 143 | Ω(encoder.data[0]).Should(Equal([]byte("NAK"))) 144 | Ω(i).Should(Equal(0)) 145 | }) 146 | 147 | It("handles intermediate flushes", func() { 148 | revisions := []repo.Revision{ 149 | repo.Revision{ 150 | "refs/heads/master": "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 151 | }, 152 | } 153 | decoder.setData( 154 | []byte("have 30f79bec32243c31dd91a05c0ad7b80f1e301aea\n"), 155 | nil, 156 | []byte("have 30f79bec32243c31dd91a05c0ad7b80f1e301aea\n"), 157 | []byte("done\n"), 158 | ) 159 | i, err := gitHandler.NegotiatePullPackfile(revisions) 160 | Ω(err).ShouldNot(HaveOccurred()) 161 | Ω(encoder.data).Should(HaveLen(2)) 162 | Ω(encoder.data[0]).Should(Equal([]byte("NAK"))) 163 | Ω(encoder.data[1]).Should(Equal([]byte("NAK"))) 164 | Ω(i).Should(Equal(0)) 165 | }) 166 | 167 | It("handles single have with delta", func() { 168 | revisions := []repo.Revision{ 169 | repo.Revision{ 170 | "refs/heads/master": "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 171 | }, 172 | } 173 | decoder.setData( 174 | []byte("have f1d2d2f924e986ac86fdf7b36c94bcdf32beec15"), 175 | []byte("done"), 176 | ) 177 | i, err := gitHandler.NegotiatePullPackfile(revisions) 178 | Ω(err).ShouldNot(HaveOccurred()) 179 | Ω(encoder.data).Should(HaveLen(2)) 180 | Ω(encoder.data[0]).Should(Equal([]byte("ACK f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 ready"))) 181 | Ω(encoder.data[1]).Should(Equal([]byte("ACK f1d2d2f924e986ac86fdf7b36c94bcdf32beec15"))) 182 | Ω(i).Should(Equal(0)) 183 | }) 184 | 185 | It("handles single have with delta and followup haves", func() { 186 | revisions := []repo.Revision{ 187 | repo.Revision{ 188 | "refs/heads/master": "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 189 | }, 190 | } 191 | decoder.setData( 192 | []byte("have f1d2d2f924e986ac86fdf7b36c94bcdf32beec15"), 193 | []byte("have e242ed3bffccdf271b7fbaf34ed72d089537b42f"), 194 | []byte("done"), 195 | ) 196 | i, err := gitHandler.NegotiatePullPackfile(revisions) 197 | Ω(err).ShouldNot(HaveOccurred()) 198 | Ω(encoder.data).Should(HaveLen(3)) 199 | Ω(encoder.data[0]).Should(Equal([]byte("ACK f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 ready"))) 200 | Ω(encoder.data[1]).Should(Equal([]byte("ACK e242ed3bffccdf271b7fbaf34ed72d089537b42f ready"))) 201 | Ω(encoder.data[2]).Should(Equal([]byte("ACK e242ed3bffccdf271b7fbaf34ed72d089537b42f"))) 202 | Ω(i).Should(Equal(0)) 203 | }) 204 | 205 | It("handles single have with multiple revisions", func() { 206 | revisions := []repo.Revision{ 207 | repo.Revision{ 208 | "refs/heads/master": "103ad77dc08d41c0b7490967903ac276c2b5cfce", 209 | }, 210 | repo.Revision{ 211 | "refs/heads/master": "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 212 | }, 213 | repo.Revision{ 214 | "refs/heads/master": "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 215 | "refs/heads/foobar": "30f79bec32243c31dd91a05c0ad7b80f1e301aea", 216 | }, 217 | } 218 | decoder.setData( 219 | []byte("have f1d2d2f924e986ac86fdf7b36c94bcdf32beec15"), 220 | []byte("done"), 221 | ) 222 | i, err := gitHandler.NegotiatePullPackfile(revisions) 223 | Ω(err).ShouldNot(HaveOccurred()) 224 | Ω(encoder.data).Should(HaveLen(2)) 225 | Ω(encoder.data[0]).Should(Equal([]byte("ACK f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 ready"))) 226 | Ω(encoder.data[1]).Should(Equal([]byte("ACK f1d2d2f924e986ac86fdf7b36c94bcdf32beec15"))) 227 | Ω(i).Should(Equal(1)) 228 | }) 229 | 230 | It("handles multiple haves with multiple revisions", func() { 231 | revisions := []repo.Revision{ 232 | repo.Revision{ 233 | "refs/heads/master": "103ad77dc08d41c0b7490967903ac276c2b5cfce", 234 | }, 235 | repo.Revision{ 236 | "refs/heads/master": "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 237 | "refs/heads/foobar": "d54852cea1ae42ee83c244b23190b03245b62a27", 238 | }, 239 | repo.Revision{ 240 | "refs/heads/master": "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 241 | "refs/heads/foobar": "30f79bec32243c31dd91a05c0ad7b80f1e301aea", 242 | }, 243 | } 244 | decoder.setData( 245 | []byte("have f1d2d2f924e986ac86fdf7b36c94bcdf32beec15"), 246 | []byte("have d54852cea1ae42ee83c244b23190b03245b62a27"), 247 | []byte("done"), 248 | ) 249 | i, err := gitHandler.NegotiatePullPackfile(revisions) 250 | Ω(err).ShouldNot(HaveOccurred()) 251 | Ω(encoder.data).Should(HaveLen(3)) 252 | Ω(encoder.data[0]).Should(Equal([]byte("ACK f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 common"))) 253 | Ω(encoder.data[1]).Should(Equal([]byte("ACK d54852cea1ae42ee83c244b23190b03245b62a27 ready"))) 254 | Ω(encoder.data[2]).Should(Equal([]byte("ACK d54852cea1ae42ee83c244b23190b03245b62a27"))) 255 | Ω(i).Should(Equal(1)) 256 | }) 257 | }) 258 | 259 | Context("sending packfiles", func() { 260 | It("sends short packfiles", func() { 261 | pack := bytes.NewBufferString("foobar") 262 | err := gitHandler.SendPackfile(pack) 263 | Ω(err).ShouldNot(HaveOccurred()) 264 | Ω(encoder.data).Should(HaveLen(2)) 265 | Ω(encoder.data[0]).Should(Equal([]byte("\001foobar"))) 266 | Ω(encoder.data[1]).Should(BeNil()) 267 | }) 268 | 269 | It("sends long packfiles", func() { 270 | data := make([]byte, 65518+1) 271 | src := rand.NewSource(42) 272 | for i := range data { 273 | data[i] = byte(src.Int63()) 274 | } 275 | err := gitHandler.SendPackfile(bytes.NewBuffer(data)) 276 | Ω(err).ShouldNot(HaveOccurred()) 277 | Ω(encoder.data).Should(HaveLen(3)) 278 | Ω(encoder.data[0][0]).Should(Equal(byte(1))) 279 | Ω(encoder.data[0][1:]).Should(HaveLen(65518)) 280 | Ω(bytes.Equal(encoder.data[0][1:], data[0:65518])).Should(BeTrue()) 281 | Ω(encoder.data[1][0]).Should(Equal(byte(1))) 282 | Ω(encoder.data[1][1]).Should(Equal(data[65518])) 283 | Ω(encoder.data[2]).Should(BeNil()) 284 | }) 285 | }) 286 | 287 | Context("receiving push refs", func() { 288 | It("receives creates", func() { 289 | decoder.setData([]byte("0000000000000000000000000000000000000000 f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 refs/heads/master\n"), nil) 290 | refs, err := gitHandler.ReceivePushRefs() 291 | Ω(err).ShouldNot(HaveOccurred()) 292 | Ω(refs).Should(Equal([]handler.RefUpdate{handler.RefUpdate{ 293 | Name: "refs/heads/master", 294 | OldID: "", 295 | NewID: "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 296 | }})) 297 | }) 298 | 299 | It("receives with trailing NUL", func() { 300 | decoder.setData([]byte("0000000000000000000000000000000000000000 f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 refs/heads/master\000"), nil) 301 | refs, err := gitHandler.ReceivePushRefs() 302 | Ω(err).ShouldNot(HaveOccurred()) 303 | Ω(refs).Should(Equal([]handler.RefUpdate{handler.RefUpdate{ 304 | Name: "refs/heads/master", 305 | OldID: "", 306 | NewID: "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 307 | }})) 308 | }) 309 | 310 | It("receives updates", func() { 311 | decoder.setData([]byte("30f79bec32243c31dd91a05c0ad7b80f1e301aea f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 refs/heads/master\n"), nil) 312 | refs, err := gitHandler.ReceivePushRefs() 313 | Ω(err).ShouldNot(HaveOccurred()) 314 | Ω(refs).Should(Equal([]handler.RefUpdate{handler.RefUpdate{ 315 | Name: "refs/heads/master", 316 | OldID: "30f79bec32243c31dd91a05c0ad7b80f1e301aea", 317 | NewID: "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 318 | }})) 319 | }) 320 | 321 | It("receives deletes", func() { 322 | decoder.setData([]byte("f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 0000000000000000000000000000000000000000 refs/heads/master\n"), nil) 323 | refs, err := gitHandler.ReceivePushRefs() 324 | Ω(err).ShouldNot(HaveOccurred()) 325 | Ω(refs).Should(Equal([]handler.RefUpdate{handler.RefUpdate{ 326 | Name: "refs/heads/master", 327 | OldID: "f1d2d2f924e986ac86fdf7b36c94bcdf32beec15", 328 | NewID: "", 329 | }})) 330 | }) 331 | }) 332 | }) 333 | -------------------------------------------------------------------------------- /git/handler/integration_test.go: -------------------------------------------------------------------------------- 1 | package handler_test 2 | 3 | import ( 4 | "fmt" 5 | "io" 6 | "io/ioutil" 7 | "net" 8 | "os" 9 | "os/exec" 10 | "strings" 11 | "sync" 12 | 13 | "github.com/bargez/pktline" 14 | "github.com/lucas-clemente/git-cr/git/handler" 15 | "github.com/lucas-clemente/git-cr/git/repo" 16 | . "github.com/onsi/ginkgo" 17 | . "github.com/onsi/gomega" 18 | ) 19 | 20 | type pktlineDecoderWrapper struct { 21 | *pktline.Decoder 22 | io.Reader 23 | } 24 | 25 | func fillRepo(b *FixtureRepo) { 26 | b.SaveNewRevisionB64( 27 | repo.Revision{ 28 | "HEAD": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 29 | "refs/heads/master": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 30 | }, 31 | "UEFDSwAAAAIAAAADlwt4nJ3MQQrCMBBA0X1OMXtBJk7SdEBEcOslJmGCgaSFdnp/ET2By7f43zZVmAS5RC46a/Y55lBnDhE9kk6pVs4klL2ok8Ne6wbPo8gOj65DF1O49o/v5edzW2/gAxEnShzghBdEV9Yxmpn+V7u2NGvS4btxb5cEOSI0eJxLSiziAgADnQFArwF4nDM0MDAzMVFIy89nCBc7Fdl++mdt9lZPhX3L1t5T0W1/BgCtgg0ijmEEgEsIHYPJopDmNYTk3nR5stM=", 32 | ) 33 | } 34 | 35 | func runCommandInDir(dir, command string, args ...string) { 36 | cmd := exec.Command(command, args...) 37 | cmd.Dir = dir 38 | err := cmd.Run() 39 | Ω(err).ShouldNot(HaveOccurred()) 40 | } 41 | 42 | func configGit(dir string) { 43 | runCommandInDir(dir, "git", "config", "user.name", "test") 44 | runCommandInDir(dir, "git", "config", "user.email", "test@example.com") 45 | } 46 | 47 | var _ = Describe("integration with git", func() { 48 | var ( 49 | tempDir string 50 | fixtureRepo *FixtureRepo 51 | server *handler.GitRequestHandler 52 | listener net.Listener 53 | port string 54 | mutex sync.Mutex 55 | ) 56 | 57 | BeforeEach(func() { 58 | var err error 59 | 60 | mutex = sync.Mutex{} 61 | 62 | tempDir, err = ioutil.TempDir("", "io.clemente.git-cr.test") 63 | Ω(err).ShouldNot(HaveOccurred()) 64 | 65 | fixtureRepo = NewFixtureRepo() 66 | 67 | listener, err = net.Listen("tcp", "localhost:0") 68 | Ω(err).ShouldNot(HaveOccurred()) 69 | port = strings.Split(listener.Addr().String(), ":")[1] 70 | 71 | go func() { 72 | defer GinkgoRecover() 73 | 74 | for { 75 | conn, err := listener.Accept() 76 | if err != nil { 77 | return 78 | } 79 | defer conn.Close() 80 | 81 | mutex.Lock() 82 | 83 | encoder := pktline.NewEncoder(conn) 84 | decoder := &pktlineDecoderWrapper{Decoder: pktline.NewDecoder(conn), Reader: conn} 85 | 86 | server = handler.NewGitRequestHandler(encoder, decoder, fixtureRepo) 87 | err = server.ServeRequest() 88 | if err != nil { 89 | fmt.Println("error in integration test: ", err.Error()) 90 | } 91 | Ω(err).ShouldNot(HaveOccurred()) 92 | conn.Close() 93 | 94 | mutex.Unlock() 95 | } 96 | }() 97 | 98 | }) 99 | 100 | AfterEach(func() { 101 | listener.Close() 102 | mutex.Lock() 103 | mutex.Unlock() 104 | os.RemoveAll(tempDir) 105 | }) 106 | 107 | Context("cloning", func() { 108 | It("clones using git", func() { 109 | fillRepo(fixtureRepo) 110 | runCommandInDir(tempDir, "git", "clone", "git://localhost:"+port+"/fixtureRepo", ".") 111 | contents, err := ioutil.ReadFile(tempDir + "/foo") 112 | Ω(err).ShouldNot(HaveOccurred()) 113 | Ω(contents).Should(Equal([]byte("bar\n"))) 114 | 115 | mutex.Lock() 116 | mutex.Unlock() 117 | }) 118 | 119 | It("clones multiple references", func() { 120 | fillRepo(fixtureRepo) 121 | fixtureRepo.SaveNewRevisionB64( 122 | repo.Revision{ 123 | "HEAD": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 124 | "refs/heads/master": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 125 | "refs/heads/foobar": "226b4f2fd9f8ca09f9abe37612c06fe4527694f5", 126 | }, 127 | "UEFDSwAAAAIAAAADnAp4nJ3LwQrCMAwA0Hu/IndB0qZpEUQEr/uJtKY6WC1s2f+LsC/w+A7PVlVoyNJCpiKUmrLPSVlFCsVLSl44FXqGLOhkt/dYYdqrbPBYtOvHFK7Lz/d6+DyPG/hInMlTjnDCgOjq6H020/+269vLfQEVLTSCMHicAwAAAAABoAJ4nDM0MDAzMVEoSS0uYXg299HsTRevOXt3a64rj7px6ElP8EQA1EMPGJoJJjoehuEy+kV9XYBCyAkBMpTu", 128 | ) 129 | runCommandInDir(tempDir, "git", "clone", "git://localhost:"+port+"/fixtureRepo", ".") 130 | }) 131 | }) 132 | 133 | Context("pulling", func() { 134 | BeforeEach(func() { 135 | fillRepo(fixtureRepo) 136 | runCommandInDir(tempDir, "git", "clone", "git://localhost:"+port+"/fixtureRepo", ".") 137 | }) 138 | 139 | It("pulls updates", func() { 140 | fixtureRepo.SaveNewRevisionB64( 141 | repo.Revision{ 142 | "HEAD": "1a6d946069d483225913cf3b8ba8eae4c894c322", 143 | "refs/heads/master": "1a6d946069d483225913cf3b8ba8eae4c894c322", 144 | }, 145 | "UEFDSwAAAAIAAAADlgx4nJXLSwrCMBRG4XlWkbkgSe5NbgpS3Eoef1QwtrQRXL51CU7O4MA3NkDnmqgFT0CSBhIGI0RhmeBCCb5Mk2cbWa1pw2voFjmbKiQ+l2xDrU7YER8oNSuUgNxKq0Gl97gvmx7Yh778esUn9fWJc1n6rC0TG0suOn0yzhh13P4YA38Q1feb+gIlsDr0M3icS0qsAgACZQE+rwF4nDM0MDAzMVFIy89nsJ9qkZYUaGwfv1Tygdym9MuFp+ZUAACUGAuBskz7fFz81Do1iG8hcUrj/ncK63Q=", 146 | ) 147 | runCommandInDir(tempDir, "git", "pull") 148 | contents, err := ioutil.ReadFile(tempDir + "/foo") 149 | Ω(err).ShouldNot(HaveOccurred()) 150 | Ω(contents).Should(Equal([]byte("baz"))) 151 | }) 152 | 153 | It("pulls nothing", func() { 154 | runCommandInDir(tempDir, "git", "pull") 155 | }) 156 | 157 | It("lists remote refs", func() { 158 | cmd := exec.Command("git", "ls-remote") 159 | cmd.Dir = tempDir 160 | out, err := cmd.CombinedOutput() 161 | Ω(err).ShouldNot(HaveOccurred()) 162 | Ω(out).Should(ContainSubstring("refs/heads/master")) 163 | Ω(out).Should(ContainSubstring("HEAD")) 164 | Ω(out).Should(ContainSubstring("f84b0d7375bcb16dd2742344e6af173aeebfcfd6")) 165 | }) 166 | }) 167 | 168 | Context("pushing changes", func() { 169 | BeforeEach(func() { 170 | fillRepo(fixtureRepo) 171 | runCommandInDir(tempDir, "git", "clone", "git://localhost:"+port+"/fixtureRepo", ".") 172 | }) 173 | 174 | It("pushes updates", func() { 175 | // Modify file 176 | err := ioutil.WriteFile(tempDir+"/foo", []byte("baz"), 0644) 177 | Ω(err).ShouldNot(HaveOccurred()) 178 | // Add 179 | runCommandInDir(tempDir, "git", "add", "foo") 180 | // Settings 181 | configGit(tempDir) 182 | // Commit 183 | cmd := exec.Command("git", "commit", "--message=msg") 184 | cmd.Dir = tempDir 185 | cmd.Env = []string{ 186 | "GIT_COMMITTER_DATE=Thu Jun 11 11:01:22 2015 +0200", 187 | "GIT_AUTHOR_DATE=Thu Jun 11 11:01:22 2015 +0200", 188 | } 189 | err = cmd.Run() 190 | Ω(err).ShouldNot(HaveOccurred()) 191 | // Push 192 | runCommandInDir(tempDir, "git", "push") 193 | // Verify 194 | mutex.Lock() 195 | mutex.Unlock() 196 | Ω(fixtureRepo.Revisions).Should(HaveLen(2)) 197 | Ω(fixtureRepo.Packfiles[1]).ShouldNot(HaveLen(0)) 198 | Ω(fixtureRepo.Revisions[1]).Should(Equal(repo.Revision{ 199 | "refs/heads/master": "1a6d946069d483225913cf3b8ba8eae4c894c322", 200 | "HEAD": "1a6d946069d483225913cf3b8ba8eae4c894c322", 201 | })) 202 | }) 203 | 204 | It("pushes new branches", func() { 205 | // Push 206 | runCommandInDir(tempDir, "git", "push", "origin", "master:foobar") 207 | // Verify 208 | mutex.Lock() 209 | mutex.Unlock() 210 | 211 | Ω(fixtureRepo.Revisions).Should(HaveLen(2)) 212 | Ω(fixtureRepo.Packfiles[1]).ShouldNot(HaveLen(0)) 213 | Ω(fixtureRepo.Revisions[1]).Should(Equal(repo.Revision{ 214 | "refs/heads/master": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 215 | "refs/heads/foobar": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 216 | "HEAD": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 217 | })) 218 | }) 219 | 220 | It("pushes deletes", func() { 221 | // Push 222 | runCommandInDir(tempDir, "git", "push", "origin", "master:foobar") 223 | runCommandInDir(tempDir, "git", "push", "origin", ":foobar") 224 | // Verify 225 | mutex.Lock() 226 | mutex.Unlock() 227 | 228 | Ω(fixtureRepo.Revisions).Should(HaveLen(3)) 229 | Ω(fixtureRepo.Packfiles[2]).ShouldNot(HaveLen(0)) 230 | Ω(fixtureRepo.Revisions[2]).Should(Equal(repo.Revision{ 231 | "HEAD": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 232 | "refs/heads/master": "f84b0d7375bcb16dd2742344e6af173aeebfcfd6", 233 | })) 234 | 235 | // Clone again 236 | workingDir2, err := ioutil.TempDir("", "io.clemente.git-cr.test") 237 | Ω(err).ShouldNot(HaveOccurred()) 238 | defer os.RemoveAll(workingDir2) 239 | 240 | cmd := exec.Command("git", "clone", "git://localhost:"+port+"/fixtureRepo", workingDir2) 241 | err = cmd.Run() 242 | Ω(err).ShouldNot(HaveOccurred()) 243 | 244 | cmd = exec.Command("git", "branch") 245 | cmd.Dir = workingDir2 246 | Ω(cmd.CombinedOutput()).Should(Equal([]byte("* master\n"))) 247 | }) 248 | 249 | It("pushes empty updates", func() { 250 | runCommandInDir(tempDir, "git", "push", "origin") 251 | Ω(fixtureRepo.Revisions).Should(HaveLen(1)) 252 | }) 253 | 254 | It("pushes multiple refs at once", func() { 255 | configGit(tempDir) 256 | 257 | err := ioutil.WriteFile(tempDir+"/foo", []byte("baz"), 0644) 258 | Ω(err).ShouldNot(HaveOccurred()) 259 | runCommandInDir(tempDir, "git", "add", "foo") 260 | runCommandInDir(tempDir, "git", "commit", "-m", "msg") 261 | 262 | runCommandInDir(tempDir, "git", "checkout", "-b", "foobar", "HEAD^") 263 | 264 | err = ioutil.WriteFile(tempDir+"/bar", []byte("baz"), 0644) 265 | Ω(err).ShouldNot(HaveOccurred()) 266 | runCommandInDir(tempDir, "git", "add", "bar") 267 | runCommandInDir(tempDir, "git", "commit", "-m", "msg2") 268 | 269 | runCommandInDir(tempDir, "git", "push", "--all") 270 | 271 | mutex.Lock() 272 | mutex.Unlock() 273 | 274 | Ω(fixtureRepo.Revisions).Should(HaveLen(2)) 275 | Ω(fixtureRepo.Packfiles[1]).ShouldNot(HaveLen(0)) 276 | Ω(fixtureRepo.Revisions[1]).Should(HaveKey("refs/heads/master")) 277 | Ω(fixtureRepo.Revisions[1]).Should(HaveKey("refs/heads/foobar")) 278 | 279 | workingDir2, err := ioutil.TempDir("", "io.clemente.git-cr.test") 280 | Ω(err).ShouldNot(HaveOccurred()) 281 | defer os.RemoveAll(workingDir2) 282 | 283 | cmd := exec.Command("git", "clone", "git://localhost:"+port+"/fixtureRepo", workingDir2) 284 | err = cmd.Run() 285 | Ω(err).ShouldNot(HaveOccurred()) 286 | }) 287 | }) 288 | 289 | Context("pushing into empty fixtureRepos", func() { 290 | It("works", func() { 291 | runCommandInDir(tempDir, "git", "init") 292 | configGit(tempDir) 293 | 294 | err := ioutil.WriteFile(tempDir+"/foo", []byte("foobar"), 0644) 295 | Ω(err).ShouldNot(HaveOccurred()) 296 | 297 | runCommandInDir(tempDir, "git", "add", "foo") 298 | runCommandInDir(tempDir, "git", "commit", "-m", "test") 299 | runCommandInDir(tempDir, "git", "remote", "add", "origin", "git://localhost:"+port+"/fixtureRepo") 300 | runCommandInDir(tempDir, "git", "push", "origin", "master") 301 | 302 | mutex.Lock() 303 | mutex.Unlock() 304 | 305 | // Clone into second dir 306 | tempDir2, err := ioutil.TempDir("", "io.clemente.git-cr.test") 307 | Ω(err).ShouldNot(HaveOccurred()) 308 | defer os.RemoveAll(tempDir2) 309 | 310 | err = exec.Command("git", "clone", "git://localhost:"+port+"/fixtureRepo", tempDir2).Run() 311 | Ω(err).ShouldNot(HaveOccurred()) 312 | contents, err := ioutil.ReadFile(tempDir2 + "/foo") 313 | Ω(err).ShouldNot(HaveOccurred()) 314 | Ω(contents).Should(Equal([]byte("foobar"))) 315 | }) 316 | }) 317 | }) 318 | -------------------------------------------------------------------------------- /git/merger/packfile_merger.go: -------------------------------------------------------------------------------- 1 | package merger 2 | 3 | import ( 4 | "bytes" 5 | "crypto/sha1" 6 | "encoding/binary" 7 | ) 8 | 9 | // MergePackfiles merges two packfiles 10 | func MergePackfiles(packfiles [][]byte) ([]byte, error) { 11 | buf := new(bytes.Buffer) 12 | 13 | buf.WriteString("PACK") 14 | // Version 2 15 | buf.Write([]byte{0, 0, 0, 2}) 16 | // Leave object count empty, will be filled later 17 | buf.Write([]byte{0, 0, 0, 0}) 18 | 19 | var count uint32 20 | 21 | for _, pack := range packfiles { 22 | count += binary.BigEndian.Uint32(pack[8:12]) 23 | buf.Write(pack[12 : len(pack)-sha1.Size]) 24 | } 25 | 26 | data := buf.Bytes() 27 | // Write object count 28 | binary.BigEndian.PutUint32(data[8:12], count) 29 | // Write checksum 30 | hash := sha1.New() 31 | hash.Write(data) 32 | data = hash.Sum(data) 33 | return data, nil 34 | } 35 | -------------------------------------------------------------------------------- /git/merger/packfile_merger_test.go: -------------------------------------------------------------------------------- 1 | package merger_test 2 | 3 | import ( 4 | "encoding/base64" 5 | "io/ioutil" 6 | "os" 7 | "os/exec" 8 | "testing" 9 | 10 | "github.com/lucas-clemente/git-cr/git/merger" 11 | . "github.com/onsi/ginkgo" 12 | . "github.com/onsi/gomega" 13 | ) 14 | 15 | func TestPackfileMerger(t *testing.T) { 16 | RegisterFailHandler(Fail) 17 | RunSpecs(t, "Packfile Merger Suite") 18 | } 19 | 20 | var _ = Describe("PackfileMerger", func() { 21 | var ( 22 | tempDir string 23 | packfile1, packfile2 []byte 24 | ) 25 | 26 | BeforeEach(func() { 27 | var err error 28 | 29 | tempDir, err = ioutil.TempDir("", "io.clemente.git-cr.test.pack") 30 | Ω(err).ShouldNot(HaveOccurred()) 31 | 32 | packfile1, err = base64.StdEncoding.DecodeString("UEFDSwAAAAIAAAADlwt4nJ3MQQrCMBBA0X1OMXtBJk7SdEBEcOslJmGCgaSFdnp/ET2By7f43zZVmAS5RC46a/Y55lBnDhE9kk6pVs4klL2ok8Ne6wbPo8gOj65DF1O49o/v5edzW2/gAxEnShzghBdEV9Yxmpn+V7u2NGvS4btxb5cEOSI0eJxLSiziAgADnQFArwF4nDM0MDAzMVFIy89nCBc7Fdl++mdt9lZPhX3L1t5T0W1/BgCtgg0ijmEEgEsIHYPJopDmNYTk3nR5stM=") 33 | Ω(err).ShouldNot(HaveOccurred()) 34 | packfile2, err = base64.StdEncoding.DecodeString("UEFDSwAAAAIAAAADlgx4nJXLSwrCMBRG4XlWkbkgSe5NbgpS3Eoef1QwtrQRXL51CU7O4MA3NkDnmqgFT0CSBhIGI0RhmeBCCb5Mk2cbWa1pw2voFjmbKiQ+l2xDrU7YER8oNSuUgNxKq0Gl97gvmx7Yh778esUn9fWJc1n6rC0TG0suOn0yzhh13P4YA38Q1feb+gIlsDr0M3icS0qsAgACZQE+rwF4nDM0MDAzMVFIy89nsJ9qkZYUaGwfv1Tygdym9MuFp+ZUAACUGAuBskz7fFz81Do1iG8hcUrj/ncK63Q=") 35 | Ω(err).ShouldNot(HaveOccurred()) 36 | }) 37 | 38 | AfterEach(func() { 39 | os.RemoveAll(tempDir) 40 | }) 41 | 42 | It("reads original packfiles", func() { 43 | p, err := merger.MergePackfiles([][]byte{packfile1}) 44 | Ω(err).ShouldNot(HaveOccurred()) 45 | Ω(p).Should(Equal(packfile1)) 46 | }) 47 | 48 | It("reads merged packfiles", func() { 49 | pack, err := merger.MergePackfiles([][]byte{packfile1, packfile2}) 50 | Ω(err).ShouldNot(HaveOccurred()) 51 | 52 | Ω(pack[0:4]).Should(Equal([]byte("PACK"))) 53 | Ω(pack[4:8]).Should(Equal([]byte{0, 0, 0, 2})) 54 | Ω(pack[8:12]).Should(Equal([]byte{0, 0, 0, 6})) 55 | 56 | err = ioutil.WriteFile(tempDir+"/pack.pack", pack, 0644) 57 | Ω(err).ShouldNot(HaveOccurred()) 58 | 59 | err = exec.Command("git", "index-pack", "--strict", tempDir+"/pack.pack").Run() 60 | Ω(err).ShouldNot(HaveOccurred()) 61 | 62 | out, err := exec.Command("git", "verify-pack", "-v", tempDir+"/pack.pack").CombinedOutput() 63 | Ω(err).ShouldNot(HaveOccurred()) 64 | Ω(string(out)).Should(ContainSubstring("f84b0d7375bcb16dd2742344e6af173aeebfcfd6")) 65 | Ω(string(out)).Should(ContainSubstring("5716ca5987cbf97d6bb54920bea6adde242d87e6")) 66 | Ω(string(out)).Should(ContainSubstring("6a09c59ce8eb1b5b4f89450103e67ff9b3a3b1ae")) 67 | Ω(string(out)).Should(ContainSubstring("1a6d946069d483225913cf3b8ba8eae4c894c322")) 68 | Ω(string(out)).Should(ContainSubstring("3f9538666251333f5fa519e01eb267d371ca9c78")) 69 | Ω(string(out)).Should(ContainSubstring("bda3f653eea7fe374e4e687479e26c65c9954184")) 70 | }) 71 | }) 72 | -------------------------------------------------------------------------------- /git/repo/json_repo.go: -------------------------------------------------------------------------------- 1 | package repo 2 | 3 | import ( 4 | "bytes" 5 | "encoding/json" 6 | "io" 7 | "strconv" 8 | ) 9 | 10 | type jsonRepo struct { 11 | backend Backend 12 | } 13 | 14 | // NewJSONRepo returns a Repo implementation that stores revisions as json 15 | func NewJSONRepo(backend Backend) Repo { 16 | return &jsonRepo{backend: backend} 17 | } 18 | 19 | func (r *jsonRepo) GetRevisions() ([]Revision, error) { 20 | rdr, err := r.backend.ReadBlob("revisions.json") 21 | if err != nil { 22 | if err == ErrNotFound { 23 | return []Revision{}, nil 24 | } 25 | return nil, err 26 | } 27 | defer rdr.Close() 28 | 29 | var revisions []Revision 30 | if err := json.NewDecoder(rdr).Decode(&revisions); err != nil { 31 | return nil, err 32 | } 33 | return revisions, nil 34 | } 35 | 36 | func (r *jsonRepo) SaveNewRevision(rev Revision, packfile io.Reader) error { 37 | revisions, err := r.GetRevisions() 38 | if err != nil { 39 | return err 40 | } 41 | revisions = append(revisions, rev) 42 | 43 | // Write revisions 44 | revisionsJSON, err := json.Marshal(revisions) 45 | if err != nil { 46 | return err 47 | } 48 | if err := r.backend.WriteBlob("revisions.json", bytes.NewBuffer(revisionsJSON)); err != nil { 49 | return err 50 | } 51 | 52 | // Write pack 53 | if err := r.backend.WriteBlob(strconv.Itoa(len(revisions)-1)+".pack", packfile); err != nil { 54 | return err 55 | } 56 | 57 | return nil 58 | } 59 | 60 | func (r *jsonRepo) ReadPackfile(toRev int) (io.ReadCloser, error) { 61 | return r.backend.ReadBlob(strconv.Itoa(toRev) + ".pack") 62 | } 63 | -------------------------------------------------------------------------------- /git/repo/repo.go: -------------------------------------------------------------------------------- 1 | package repo 2 | 3 | import ( 4 | "errors" 5 | "io" 6 | ) 7 | 8 | // A Revision is a version of the server's state 9 | type Revision map[string]string 10 | 11 | // A Repo for git data 12 | type Repo interface { 13 | // GetRevisions should return all revisions in chronological order 14 | GetRevisions() ([]Revision, error) 15 | 16 | SaveNewRevision(rev Revision, packfile io.Reader) error 17 | 18 | ReadPackfile(toRev int) (io.ReadCloser, error) 19 | } 20 | 21 | // ErrNotFound should be returned by Backend.ReadBlob if a blob was not found. 22 | var ErrNotFound = errors.New("not found") 23 | 24 | // A Backend for a crypto repo 25 | type Backend interface { 26 | ReadBlob(name string) (io.ReadCloser, error) 27 | WriteBlob(name string, r io.Reader) error 28 | } 29 | -------------------------------------------------------------------------------- /git/repo/repo_test.go: -------------------------------------------------------------------------------- 1 | package repo_test 2 | 3 | import ( 4 | "bytes" 5 | "io" 6 | "io/ioutil" 7 | "testing" 8 | 9 | "github.com/lucas-clemente/git-cr/git/repo" 10 | 11 | . "github.com/onsi/ginkgo" 12 | . "github.com/onsi/gomega" 13 | ) 14 | 15 | func TestLocalRepo(t *testing.T) { 16 | RegisterFailHandler(Fail) 17 | RunSpecs(t, "JSON Repo Suite") 18 | } 19 | 20 | type fixtureBackend map[string][]byte 21 | 22 | func (f fixtureBackend) ReadBlob(name string) (io.ReadCloser, error) { 23 | return ioutil.NopCloser(bytes.NewBuffer(f[name])), nil 24 | } 25 | 26 | func (f fixtureBackend) WriteBlob(name string, r io.Reader) error { 27 | data, err := ioutil.ReadAll(r) 28 | if err != nil { 29 | return err 30 | } 31 | f[name] = data 32 | return nil 33 | } 34 | 35 | var _ = Describe("JSON Repo", func() { 36 | var ( 37 | backend fixtureBackend 38 | jsonRepo repo.Repo 39 | ) 40 | 41 | BeforeEach(func() { 42 | backend = fixtureBackend{} 43 | jsonRepo = repo.NewJSONRepo(backend) 44 | }) 45 | 46 | It("reads packfiles", func() { 47 | backend["42.pack"] = []byte("foo") 48 | r, err := jsonRepo.ReadPackfile(42) 49 | Ω(err).ShouldNot(HaveOccurred()) 50 | data, err := ioutil.ReadAll(r) 51 | Ω(err).ShouldNot(HaveOccurred()) 52 | Ω(data).Should(Equal([]byte("foo"))) 53 | }) 54 | 55 | It("reads revisions", func() { 56 | backend["revisions.json"] = []byte(`[{"refs/heads/master":"foobar"}]`) 57 | refs, err := jsonRepo.GetRevisions() 58 | Ω(err).ShouldNot(HaveOccurred()) 59 | Ω(refs).Should(Equal([]repo.Revision{{"refs/heads/master": "foobar"}})) 60 | }) 61 | 62 | It("saves new revisions", func() { 63 | backend["revisions.json"] = []byte(`[{"refs/heads/master":"foobar"}]`) 64 | err := jsonRepo.SaveNewRevision(repo.Revision{"refs/heads/master": "foobaz"}, bytes.NewBufferString("bar")) 65 | Ω(err).ShouldNot(HaveOccurred()) 66 | Ω(backend["revisions.json"]).Should(Equal([]byte(`[{"refs/heads/master":"foobar"},{"refs/heads/master":"foobaz"}]`))) 67 | Ω(backend["1.pack"]).Should(Equal([]byte("bar"))) 68 | }) 69 | }) 70 | -------------------------------------------------------------------------------- /main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "encoding/base64" 5 | "fmt" 6 | "io" 7 | "net/url" 8 | "os" 9 | "os/exec" 10 | "strings" 11 | 12 | "github.com/bargez/pktline" 13 | "github.com/codegangsta/cli" 14 | "github.com/lucas-clemente/git-cr/backends/local" 15 | "github.com/lucas-clemente/git-cr/crypto/nacl" 16 | "github.com/lucas-clemente/git-cr/git/repo" 17 | "github.com/lucas-clemente/git-cr/git/handler" 18 | ) 19 | 20 | func main() { 21 | app := cli.NewApp() 22 | app.Name = "git cr" 23 | app.Usage = "Encrypted git remote" 24 | app.Version = "0.1.0" 25 | app.Commands = []cli.Command{ 26 | { 27 | Name: "add", 28 | Usage: "Setup a crypto remote in the current repo", 29 | Action: add, 30 | }, 31 | { 32 | Name: "run", 33 | Usage: "Run the git server (should not be called manually)", 34 | Action: run, 35 | }, 36 | { 37 | Name: "clone", 38 | Usage: "Clone from a crypto remote", 39 | Action: clone, 40 | }, 41 | } 42 | app.Run(os.Args) 43 | } 44 | 45 | func add(c *cli.Context) { 46 | if len(c.Args()) != 3 { 47 | fmt.Println("usage: git cr add ") 48 | os.Exit(1) 49 | } 50 | remoteName := c.Args()[0] 51 | remoteURL := c.Args()[1] 52 | encryptionSettings := c.Args()[2] 53 | cmd := exec.Command("git", "remote", "add", remoteName, buildRemote(remoteURL, encryptionSettings)) 54 | out, err := cmd.CombinedOutput() 55 | if err != nil { 56 | fmt.Printf("git errored: %v\n%s", err, out) 57 | os.Exit(1) 58 | } 59 | } 60 | 61 | type pktlineDecoderWrapper struct { 62 | *pktline.Decoder 63 | io.Reader 64 | } 65 | 66 | func run(c *cli.Context) { 67 | if len(c.Args()) != 2 { 68 | fmt.Println("don't run this manually, checkout git cr help :)") 69 | os.Exit(1) 70 | } 71 | 72 | repoURLString := c.Args()[0] 73 | encryptionSettings := c.Args()[1] 74 | 75 | repoURL, err := url.Parse(repoURLString) 76 | if err != nil { 77 | fmt.Fprintf(os.Stderr, "an error occured while parsing the URL:\n%v\n", err) 78 | os.Exit(1) 79 | } 80 | 81 | // Load repo 82 | 83 | backend, err := local.NewLocalBackend(repoURL.Path) 84 | if err != nil { 85 | fmt.Fprintf(os.Stderr, "an error occured while initing the repo:\n%v\n", err) 86 | os.Exit(1) 87 | } 88 | 89 | // Wrap in encryption 90 | 91 | if encryptionSettings == "none" { 92 | // Do nothing 93 | } else if strings.HasPrefix(encryptionSettings, "nacl:") { 94 | secretB64 := strings.TrimPrefix(encryptionSettings, "nacl:") 95 | secret, err := base64.StdEncoding.DecodeString(secretB64) 96 | if err != nil || len(secret) != 32 { 97 | fmt.Fprintf(os.Stderr, "the nacl secret should be 32 bytes in base64") 98 | os.Exit(1) 99 | } 100 | 101 | secretArray := [32]byte{} 102 | copy(secretArray[:], secret) 103 | backend = nacl.NewNaClBackend(backend, secretArray) 104 | } else { 105 | fmt.Fprintf(os.Stderr, "the encryption settings are invalid") 106 | os.Exit(1) 107 | } 108 | 109 | // Setup repo 110 | 111 | repo := repo.NewJSONRepo(backend) 112 | 113 | // Handle request 114 | 115 | encoder := pktline.NewEncoder(os.Stdout) 116 | decoder := &pktlineDecoderWrapper{Decoder: pktline.NewDecoder(os.Stdin), Reader: os.Stdin} 117 | 118 | server := handler.NewGitRequestHandler(encoder, decoder, repo) 119 | if err := server.ServeRequest(); err != nil { 120 | fmt.Fprintf(os.Stderr, "an error occured while serving git:\n%v\n", err) 121 | } 122 | } 123 | 124 | func clone(c *cli.Context) { 125 | if len(c.Args()) < 2 { 126 | fmt.Println("usage: git cr clone [destination]") 127 | os.Exit(1) 128 | } 129 | remoteURL := c.Args()[0] 130 | encryptionSettings := c.Args()[1] 131 | 132 | cloneArgs := []string{"clone", buildRemote(remoteURL, encryptionSettings)} 133 | cloneArgs = append(cloneArgs, c.Args()[2:]...) 134 | cmd := exec.Command("git", cloneArgs...) 135 | out, err := cmd.CombinedOutput() 136 | if err != nil { 137 | fmt.Printf("git errored: %v\n%s", err, out) 138 | os.Exit(1) 139 | } 140 | } 141 | 142 | func buildRemote(url, encryptionSettings string) string { 143 | return "ext::git cr %G run " + url + " " + encryptionSettings 144 | } 145 | -------------------------------------------------------------------------------- /main_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "encoding/base64" 5 | "io/ioutil" 6 | "os" 7 | "os/exec" 8 | "path/filepath" 9 | 10 | . "github.com/onsi/ginkgo" 11 | . "github.com/onsi/gomega" 12 | "github.com/onsi/gomega/gexec" 13 | 14 | "testing" 15 | ) 16 | 17 | func TestGitCr(t *testing.T) { 18 | RegisterFailHandler(Fail) 19 | RunSpecs(t, "Main Suite") 20 | } 21 | 22 | func runCommandInDir(dir, command string, args ...string) { 23 | cmd := exec.Command(command, args...) 24 | cmd.Dir = dir 25 | err := cmd.Run() 26 | Ω(err).ShouldNot(HaveOccurred()) 27 | } 28 | 29 | func configGit(dir string) { 30 | runCommandInDir(dir, "git", "config", "user.name", "test") 31 | runCommandInDir(dir, "git", "config", "user.email", "test@example.com") 32 | } 33 | 34 | var _ = Describe("Main", func() { 35 | var ( 36 | workingDir string 37 | remoteDir string 38 | pathToGitCR string 39 | folderOfGitCR string 40 | encryptionSettings string 41 | ) 42 | 43 | BeforeSuite(func() { 44 | var err error 45 | pathToGitCR, err = gexec.Build("github.com/lucas-clemente/git-cr") 46 | Ω(err).ShouldNot(HaveOccurred()) 47 | folderOfGitCR = filepath.Dir(pathToGitCR) 48 | }) 49 | 50 | AfterSuite(func() { 51 | gexec.CleanupBuildArtifacts() 52 | }) 53 | 54 | BeforeEach(func() { 55 | var err error 56 | workingDir, err = ioutil.TempDir("", "io.clemente.git-cr.test") 57 | Ω(err).ShouldNot(HaveOccurred()) 58 | remoteDir, err = ioutil.TempDir("", "io.clemente.git-cr.test") 59 | Ω(err).ShouldNot(HaveOccurred()) 60 | 61 | }) 62 | 63 | remoteURL := func() string { 64 | return "ext::" + pathToGitCR + " %G run " + "file://" + remoteDir + " " + encryptionSettings 65 | } 66 | 67 | AfterEach(func() { 68 | os.RemoveAll(workingDir) 69 | os.RemoveAll(remoteDir) 70 | }) 71 | 72 | sharedTests := func() { 73 | It("adds remotes", func() { 74 | cmd := exec.Command("git", "init", workingDir) 75 | err := cmd.Run() 76 | Ω(err).ShouldNot(HaveOccurred()) 77 | 78 | runCommandInDir(workingDir, pathToGitCR, "add", "origin", remoteDir, encryptionSettings) 79 | 80 | cmd = exec.Command("git", "remote", "-v") 81 | cmd.Dir = workingDir 82 | output, err := cmd.CombinedOutput() 83 | Ω(err).ShouldNot(HaveOccurred()) 84 | Ω(output).Should(ContainSubstring("origin\text::git cr %G run " + remoteDir + " " + encryptionSettings)) 85 | }) 86 | 87 | It("pushes updates", func() { 88 | runCommandInDir(workingDir, "git", "init") 89 | configGit(workingDir) 90 | 91 | runCommandInDir(workingDir, "git", "remote", "add", "origin", remoteURL()) 92 | 93 | err := ioutil.WriteFile(workingDir+"/foo", []byte("foobar"), 0644) 94 | Ω(err).ShouldNot(HaveOccurred()) 95 | 96 | runCommandInDir(workingDir, "git", "add", "foo") 97 | runCommandInDir(workingDir, "git", "commit", "-m", "test") 98 | runCommandInDir(workingDir, "git", "push", "origin", "master") 99 | }) 100 | 101 | It("pushes new branches", func() { 102 | runCommandInDir(workingDir, "git", "init") 103 | configGit(workingDir) 104 | 105 | runCommandInDir(workingDir, "git", "remote", "add", "origin", remoteURL()) 106 | 107 | err := ioutil.WriteFile(workingDir+"/foo", []byte("foobar"), 0644) 108 | Ω(err).ShouldNot(HaveOccurred()) 109 | 110 | runCommandInDir(workingDir, "git", "add", "foo") 111 | runCommandInDir(workingDir, "git", "commit", "-m", "test") 112 | runCommandInDir(workingDir, "git", "push", "origin", "master") 113 | runCommandInDir(workingDir, "git", "push", "origin", "master:foobar") 114 | }) 115 | 116 | It("force-pushes and clones", func() { 117 | runCommandInDir(workingDir, "git", "init") 118 | configGit(workingDir) 119 | 120 | err := ioutil.WriteFile(workingDir+"/foo", []byte("foobar"), 0644) 121 | Ω(err).ShouldNot(HaveOccurred()) 122 | runCommandInDir(workingDir, "git", "add", "foo") 123 | runCommandInDir(workingDir, "git", "commit", "-m", "test") 124 | 125 | err = ioutil.WriteFile(workingDir+"/bar", []byte("foobaz"), 0644) 126 | Ω(err).ShouldNot(HaveOccurred()) 127 | runCommandInDir(workingDir, "git", "add", "bar") 128 | runCommandInDir(workingDir, "git", "commit", "-m", "test2") 129 | runCommandInDir(workingDir, "git", "remote", "add", "origin", remoteURL()) 130 | runCommandInDir(workingDir, "git", "push", "origin", "master") 131 | runCommandInDir(workingDir, "git", "reset", "--hard", "HEAD^") 132 | runCommandInDir(workingDir, "git", "push", "-f", "origin", "master") 133 | 134 | // Now try cloning 135 | 136 | workingDir2, err := ioutil.TempDir("", "io.clemente.git-cr.test") 137 | Ω(err).ShouldNot(HaveOccurred()) 138 | defer os.RemoveAll(workingDir2) 139 | 140 | cmd := exec.Command("git", "clone", remoteURL(), workingDir2) 141 | err = cmd.Run() 142 | Ω(err).ShouldNot(HaveOccurred()) 143 | 144 | contents, err := ioutil.ReadFile(workingDir2 + "/foo") 145 | Ω(err).ShouldNot(HaveOccurred()) 146 | Ω(contents).Should(Equal([]byte("foobar"))) 147 | _, err = ioutil.ReadFile(workingDir2 + "/bar") 148 | Ω(os.IsNotExist(err)).Should(BeTrue()) 149 | }) 150 | 151 | It("pushes multiple times and clones", func() { 152 | runCommandInDir(workingDir, "git", "init") 153 | configGit(workingDir) 154 | 155 | err := ioutil.WriteFile(workingDir+"/foo", []byte("foobar"), 0644) 156 | Ω(err).ShouldNot(HaveOccurred()) 157 | runCommandInDir(workingDir, "git", "add", "foo") 158 | runCommandInDir(workingDir, "git", "commit", "-m", "test") 159 | runCommandInDir(workingDir, "git", "remote", "add", "origin", remoteURL()) 160 | runCommandInDir(workingDir, "git", "push", "origin", "master") 161 | 162 | err = ioutil.WriteFile(workingDir+"/bar", []byte("foobaz"), 0644) 163 | Ω(err).ShouldNot(HaveOccurred()) 164 | runCommandInDir(workingDir, "git", "add", "bar") 165 | runCommandInDir(workingDir, "git", "commit", "-m", "test2") 166 | runCommandInDir(workingDir, "git", "push", "origin", "master") 167 | 168 | // Now try cloning 169 | 170 | workingDir2, err := ioutil.TempDir("", "io.clemente.git-cr.test") 171 | Ω(err).ShouldNot(HaveOccurred()) 172 | defer os.RemoveAll(workingDir2) 173 | 174 | cmd := exec.Command("git", "clone", remoteURL(), workingDir2) 175 | err = cmd.Run() 176 | Ω(err).ShouldNot(HaveOccurred()) 177 | 178 | contents, err := ioutil.ReadFile(workingDir2 + "/foo") 179 | Ω(err).ShouldNot(HaveOccurred()) 180 | Ω(contents).Should(Equal([]byte("foobar"))) 181 | contents, err = ioutil.ReadFile(workingDir2 + "/bar") 182 | Ω(err).ShouldNot(HaveOccurred()) 183 | Ω(contents).Should(Equal([]byte("foobaz"))) 184 | }) 185 | } 186 | 187 | Context("without encryption", func() { 188 | BeforeEach(func() { 189 | encryptionSettings = "none" 190 | }) 191 | 192 | sharedTests() 193 | 194 | It("clones fixtures", func() { 195 | err := ioutil.WriteFile(remoteDir+"/revisions.json", []byte(`[{"HEAD":"f84b0d7375bcb16dd2742344e6af173aeebfcfd6","refs/heads/master":"f84b0d7375bcb16dd2742344e6af173aeebfcfd6"}]`), 0644) 196 | Ω(err).ShouldNot(HaveOccurred()) 197 | 198 | pack, err := base64.StdEncoding.DecodeString("UEFDSwAAAAIAAAADlwt4nJ3MQQrCMBBA0X1OMXtBJk7SdEBEcOslJmGCgaSFdnp/ET2By7f43zZVmAS5RC46a/Y55lBnDhE9kk6pVs4klL2ok8Ne6wbPo8gOj65DF1O49o/v5edzW2/gAxEnShzghBdEV9Yxmpn+V7u2NGvS4btxb5cEOSI0eJxLSiziAgADnQFArwF4nDM0MDAzMVFIy89nCBc7Fdl++mdt9lZPhX3L1t5T0W1/BgCtgg0ijmEEgEsIHYPJopDmNYTk3nR5stM=") 199 | Ω(err).ShouldNot(HaveOccurred()) 200 | err = ioutil.WriteFile(remoteDir+"/0.pack", pack, 0644) 201 | Ω(err).ShouldNot(HaveOccurred()) 202 | 203 | cmd := exec.Command(pathToGitCR, "clone", remoteDir, encryptionSettings, workingDir) 204 | // git-cr needs to be in the PATH to be discovered by the ext:: remote 205 | cmd.Env = []string{"PATH=" + folderOfGitCR + ":" + os.Getenv("PATH")} 206 | err = cmd.Run() 207 | Ω(err).ShouldNot(HaveOccurred()) 208 | 209 | data, err := ioutil.ReadFile(workingDir + "/foo") 210 | Ω(err).ShouldNot(HaveOccurred()) 211 | Ω(data).Should(Equal([]byte("bar\n"))) 212 | }) 213 | }) 214 | 215 | Context("with nacl encryption", func() { 216 | BeforeEach(func() { 217 | encryptionSettings = "nacl:MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=" 218 | }) 219 | 220 | sharedTests() 221 | }) 222 | }) 223 | --------------------------------------------------------------------------------