├── .gitignore ├── Gemfile ├── LICENSE ├── README.md ├── Rakefile ├── TODO ├── git_store.gemspec ├── lib ├── git_store.rb └── git_store │ ├── blob.rb │ ├── commit.rb │ ├── diff.rb │ ├── handlers.rb │ ├── pack.rb │ ├── tag.rb │ ├── tree.rb │ └── user.rb └── test ├── bare_store_spec.rb ├── benchmark.rb ├── commit_spec.rb ├── git_store_spec.rb ├── helper.rb ├── tree_spec.rb └── user_spec.rb /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | doc/* 3 | pkg/* 4 | test/repo 5 | html/* 6 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | gem 'rake' 2 | gem 'rspec' 3 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2008 Matthias Georgi 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to 5 | deal in the Software without restriction, including without limitation the 6 | rights to use, copy, modify, merge, publish, distribute, sublicense, and/or 7 | sell copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL 16 | THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 17 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 18 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 19 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Git Store - using Git as versioned data store in Ruby 2 | ===================================================== 3 | 4 | GitStore implements a versioned data store based on the revision 5 | management system [Git][1]. You can store object hierarchies as nested 6 | hashes, which will be mapped on the directory structure of a git 7 | repository. Basically GitStore checks out the repository into a 8 | in-memory representation, which can be modified and finally committed. 9 | 10 | GitStore supports transactions, so that updates to the store either 11 | fail or succeed completely. 12 | 13 | ### Installation 14 | 15 | GitStore can be installed as gem easily: 16 | 17 | $ gem sources -a http://gems.github.com 18 | $ sudo gem install georgi-git_store 19 | 20 | ### Usage Example 21 | 22 | First thing you should do, is to initialize a new git repository. 23 | 24 | $ mkdir test 25 | $ cd test 26 | $ git init 27 | 28 | Now you can instantiate a GitStore instance and store some data. The 29 | data will be serialized depending on the file extension. So for YAML 30 | storage you can use the 'yml' extension: 31 | 32 | store = GitStore.new('/path/to/repo') 33 | 34 | store['users/matthias.yml'] = User.new('Matthias') 35 | store['pages/home.yml'] = Page.new('matthias', 'Home') 36 | 37 | store.commit 'Added user and page' 38 | 39 | ### Transactions 40 | 41 | GitStore manages concurrent access by a file locking scheme. So only 42 | one process can start a transaction at one time. This is implemented 43 | by locking the `refs/head/.lock` file, which is also 44 | respected by the git binary. 45 | 46 | If you access the repository from different processes or threads, you 47 | should write to the store using transactions. If something goes wrong 48 | inside a transaction, all changes will be rolled back to the original 49 | state. 50 | 51 | store = GitStore.new('/path/to/repo') 52 | 53 | store.transaction do 54 | # If an exception happens here, the transaction will be aborted. 55 | store['pages/home.yml'] = Page.new('matthias', 'Home') 56 | end 57 | 58 | 59 | A transaction without a block looks like this: 60 | 61 | store.start_transaction 62 | 63 | store['pages/home.yml'] = Page.new('matthias', 'Home') 64 | 65 | store.rollback # This will restore the original state 66 | 67 | 68 | ### Data Storage 69 | 70 | When you call the `commit` method, your data is written back straight 71 | into the git repository. No intermediate file representation. So if 72 | you want to have a look at your data, you can use a git browser like 73 | [git-gui][6] or checkout the files: 74 | 75 | $ git checkout 76 | 77 | 78 | ### Iteration 79 | 80 | Iterating over the data objects is quite easy. Furthermore you can 81 | iterate over trees and subtrees, so you can partition your data in a 82 | meaningful way. For example you may separate the config files and the 83 | pages of a wiki: 84 | 85 | store['pages/home.yml'] = Page.new('matthias', 'Home') 86 | store['pages/about.yml'] = Page.new('matthias', 'About') 87 | store['config/wiki.yml'] = { 'name' => 'My Personal Wiki' } 88 | 89 | # Enumerate all objects 90 | store.each { |obj| ... } 91 | 92 | # Enumerate only pages 93 | store['pages'].each { |page| ... } 94 | 95 | 96 | ### Serialization 97 | 98 | Serialization is dependent on the filename extension. You can add more 99 | handlers if you like, the interface is like this: 100 | 101 | class YAMLHandler 102 | def read(data) 103 | YAML.load(data) 104 | end 105 | 106 | def write(data) 107 | data.to_yaml 108 | end 109 | end 110 | 111 | Shinmun uses its own handler for files with `md` extension: 112 | 113 | class PostHandler 114 | def read(data) 115 | Post.new(:src => data) 116 | end 117 | 118 | def write(post) 119 | post.dump 120 | end 121 | end 122 | 123 | store = GitStore.new('.') 124 | store.handler['md'] = PostHandler.new 125 | 126 | 127 | ### GitStore on GitHub 128 | 129 | Download or fork the project on its [Github page][5] 130 | 131 | 132 | 133 | [1]: http://git.or.cz/ 134 | [2]: http://github.com/mojombo/grit 135 | [5]: http://github.com/georgi/git_store 136 | [6]: http://www.kernel.org/pub/software/scm/git/docs/git-gui.html 137 | [7]: http://www.matthias-georgi.de/shinmun 138 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require 'rake' 2 | require 'rdoc/task' 3 | require 'rspec/core/rake_task' 4 | 5 | desc "Run all specs" 6 | RSpec::Core::RakeTask.new(:spec) do |spec| 7 | spec.pattern = 'test/**/*_spec.rb' 8 | spec.rspec_opts = ['--backtrace'] 9 | end 10 | 11 | desc "Generate the RDoc" 12 | RDoc::Task.new do |rdoc| 13 | files = ["README.md", "LICENSE", "lib/**/*.rb"] 14 | rdoc.rdoc_files.include(files) 15 | rdoc.main = "README.md" 16 | rdoc.title = "Git Store - using Git as versioned data store in Ruby" 17 | end 18 | 19 | desc "Run the rspec" 20 | task :default => :spec 21 | -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | * File history 2 | * File diffs 3 | * Retrieving all paths of a repository. -------------------------------------------------------------------------------- /git_store.gemspec: -------------------------------------------------------------------------------- 1 | Gem::Specification.new do |s| 2 | s.name = 'git_store' 3 | s.version = '0.3.3' 4 | s.summary = 'a simple data store based on git' 5 | s.author = 'Matthias Georgi' 6 | s.email = 'matti.georgi@gmail.com' 7 | s.homepage = 'http://georgi.github.com/git_store' 8 | s.description = <.lock` file, which is also respected 27 | # by the git binary. 28 | # 29 | # A regular commit should be atomic by the nature of git, as the only 30 | # critical part is writing the 40 bytes SHA1 hash of the commit object 31 | # to the file `refs/head/`, which is done atomically by the 32 | # operating system. 33 | # 34 | # So reading a repository should be always consistent in a git 35 | # repository. The head of a branch points to a commit object, which in 36 | # turn points to a tree object, which itself is a snapshot of the 37 | # GitStore at commit time. All involved objects are keyed by their 38 | # SHA1 value, so there is no chance for another process to write to 39 | # the same files. 40 | # 41 | class GitStore 42 | include Enumerable 43 | 44 | TYPE_CLASS = { 45 | 'tree' => Tree, 46 | 'blob' => Blob, 47 | 'commit' => Commit, 48 | 'tag' => Tag 49 | } 50 | 51 | CLASS_TYPE = { 52 | Tree => 'tree', 53 | Blob => 'blob', 54 | Commit => 'commit', 55 | Tag => 'tag' 56 | } 57 | 58 | attr_reader :path, :index, :root, :branch, :lock_file, :head, :packs, :handler, :bare, :objects 59 | 60 | # Initialize a store. 61 | def initialize(path, branch = 'master', bare = false) 62 | if bare && !File.exists?("#{path}") or 63 | !bare && !File.exists?("#{path}/.git") 64 | raise ArgumentError, "first argument must be a valid Git repository: `#{path}'" 65 | end 66 | 67 | @bare = bare 68 | @path = path.chomp('/') 69 | @branch = branch 70 | @root = Tree.new(self) 71 | @packs = {} 72 | @objects = {} 73 | 74 | @handler = { 75 | 'yml' => YAMLHandler.new 76 | } 77 | 78 | @handler.default = DefaultHandler.new 79 | 80 | load_packs("#{git_path}/objects/pack") 81 | 82 | load 83 | end 84 | 85 | # Returns the path to the current head file. 86 | def head_path 87 | "#{git_path}/refs/heads/#{branch}" 88 | end 89 | 90 | # Returns the path to the object file for given id. 91 | def object_path(id) 92 | "#{git_path}/objects/#{ id[0...2] }/#{ id[2..39] }" 93 | end 94 | 95 | # Returns the path to the git data directory. 96 | def git_path 97 | if bare 98 | "#{path}" 99 | else 100 | "#{path}/.git" 101 | end 102 | end 103 | 104 | # Read the id of the head commit. 105 | # 106 | # Returns the object id of the last commit. 107 | def read_head_id 108 | File.read(head_path).strip if File.exists?(head_path) 109 | end 110 | 111 | # Return a handler for a given path. 112 | def handler_for(path) 113 | handler[ path.split('.').last ] 114 | end 115 | 116 | # Read an object for the specified path. 117 | def [](path) 118 | root[path] 119 | end 120 | 121 | # Write an object to the specified path. 122 | def []=(path, data) 123 | root[path] = data 124 | end 125 | 126 | # Iterate over all key-values pairs found in this store. 127 | def each(&block) 128 | root.each(&block) 129 | end 130 | 131 | # Returns all paths found in this store. 132 | def paths 133 | root.paths 134 | end 135 | 136 | # Returns all values found in this store. 137 | def values 138 | root.values 139 | end 140 | 141 | # Remove given path from store. 142 | def delete(path) 143 | root.delete(path) 144 | end 145 | 146 | # Find or create a tree object with given path. 147 | def tree(path) 148 | root.tree(path) 149 | end 150 | 151 | # Returns the store as a hash tree. 152 | def to_hash 153 | root.to_hash 154 | end 155 | 156 | # Inspect the store. 157 | def inspect 158 | "#" 159 | end 160 | 161 | # Has our store been changed on disk? 162 | def changed? 163 | head.nil? or head.id != read_head_id 164 | end 165 | 166 | # Load the current head version from repository. 167 | def load(from_disk = false) 168 | if id = read_head_id 169 | @head = get(id) 170 | @root = @head.tree 171 | end 172 | 173 | load_from_disk if from_disk 174 | end 175 | 176 | def load_from_disk 177 | root.each_blob do |path, blob| 178 | file = "#{self.path}/#{path}" 179 | if File.file?(file) 180 | blob.data = File.read(file) 181 | end 182 | end 183 | end 184 | 185 | # Reload the store, if it has been changed on disk. 186 | def refresh! 187 | load if changed? 188 | end 189 | 190 | # Is there any transaction going on? 191 | def in_transaction? 192 | Thread.current['git_store_lock'] 193 | end 194 | 195 | # All changes made inside a transaction are atomic. If some 196 | # exception occurs the transaction will be rolled back. 197 | # 198 | # Example: 199 | # store.transaction { store['a'] = 'b' } 200 | # 201 | def transaction(message = "", author = User.from_config, committer = author) 202 | start_transaction 203 | result = yield 204 | commit message, author, committer 205 | 206 | result 207 | rescue 208 | rollback 209 | raise 210 | ensure 211 | finish_transaction 212 | end 213 | 214 | # Start a transaction. 215 | # 216 | # Tries to get lock on lock file, reload the this store if 217 | # has changed in the repository. 218 | def start_transaction 219 | file = open("#{head_path}.lock", "w") 220 | file.flock(File::LOCK_EX) 221 | 222 | Thread.current['git_store_lock'] = file 223 | 224 | load if changed? 225 | end 226 | 227 | # Restore the state of the store. 228 | # 229 | # Any changes made to the store are discarded. 230 | def rollback 231 | objects.clear 232 | load 233 | finish_transaction 234 | end 235 | 236 | # Finish the transaction. 237 | # 238 | # Release the lock file. 239 | def finish_transaction 240 | Thread.current['git_store_lock'].close rescue nil 241 | Thread.current['git_store_lock'] = nil 242 | 243 | File.unlink("#{head_path}.lock") rescue nil 244 | end 245 | 246 | # Write a commit object to disk and set the head of the current branch. 247 | # 248 | # Returns the commit object 249 | def commit(message = '', author = User.from_config, committer = author) 250 | root.write 251 | 252 | commit = Commit.new(self) 253 | commit.tree = root 254 | commit.parent << head.id if head 255 | commit.author = author 256 | commit.committer = committer 257 | commit.message = message 258 | commit.write 259 | 260 | open(head_path, "wb") do |file| 261 | file.write(commit.id) 262 | end 263 | 264 | @head = commit 265 | end 266 | 267 | # Returns a list of commits starting from head commit. 268 | def commits(limit = 10, start = head) 269 | entries = [] 270 | current = start 271 | 272 | while current and entries.size < limit 273 | entries << current 274 | current = get(current.parent.first) 275 | end 276 | 277 | entries 278 | end 279 | 280 | # Get an object by its id. 281 | # 282 | # Returns a tree, blob, commit or tag object. 283 | def get(id) 284 | return nil if id.nil? 285 | 286 | return objects[id] if objects.has_key?(id) 287 | 288 | type, content = get_object(id) 289 | 290 | klass = TYPE_CLASS[type] or raise NotImplementedError, "type not supported: #{type}" 291 | 292 | objects[id] = klass.new(self, id, content) 293 | end 294 | 295 | # Save a git object to the store. 296 | # 297 | # Returns the object id. 298 | def put(object) 299 | type = CLASS_TYPE[object.class] or raise NotImplementedError, "class not supported: #{object.class}" 300 | 301 | id = put_object(type, object.dump) 302 | 303 | objects[id] = object 304 | 305 | id 306 | end 307 | 308 | # Returns the hash value of an object string. 309 | def sha(str) 310 | Digest::SHA1.hexdigest(str)[0, 40] 311 | end 312 | 313 | # Calculate the id for a given type and raw data string. 314 | def id_for(type, content) 315 | sha "#{type} #{content.length}\0#{content}" 316 | end 317 | 318 | # Read the raw object with the given id from the repository. 319 | # 320 | # Returns a pair of content and type of the object 321 | def get_object(id) 322 | path = object_path(id) 323 | 324 | if File.exists?(path) 325 | buf = open(path, "rb") { |f| f.read } 326 | 327 | raise "not a loose object: #{id}" if not legacy_loose_object?(buf) 328 | 329 | header, content = Zlib::Inflate.inflate(buf).split(/\0/, 2) 330 | type, size = header.split(/ /, 2) 331 | 332 | raise "bad object: #{id}" if content.length != size.to_i 333 | else 334 | content, type = get_object_from_pack(id) 335 | end 336 | 337 | return type, content 338 | end 339 | 340 | # Write a raw object to the repository. 341 | # 342 | # Returns the object id. 343 | def put_object(type, content) 344 | data = "#{type} #{content.length}\0#{content}" 345 | id = sha(data) 346 | path = object_path(id) 347 | 348 | unless File.exists?(path) 349 | FileUtils.mkpath(File.dirname(path)) 350 | open(path, 'wb') do |f| 351 | f.write Zlib::Deflate.deflate(data) 352 | end 353 | end 354 | 355 | id 356 | end 357 | 358 | protected 359 | 360 | if 'String'[0].respond_to?(:ord) 361 | def legacy_loose_object?(buf) 362 | buf[0] == ?x && (((buf[0].ord << 8) + buf[1].ord) % 31 == 0) 363 | end 364 | else 365 | def legacy_loose_object?(buf) 366 | word = (buf[0] << 8) + buf[1] 367 | buf[0] == 0x78 && word % 31 == 0 368 | end 369 | end 370 | 371 | def get_object_from_pack(id) 372 | pack, offset = @packs[id] 373 | 374 | pack.parse_object(offset) if pack 375 | end 376 | 377 | def load_packs(path) 378 | if File.directory?(path) 379 | Dir.open(path) do |dir| 380 | entries = dir.select { |entry| entry =~ /\.pack$/i } 381 | entries.each do |entry| 382 | pack = PackStorage.new(File.join(path, entry)) 383 | pack.each_entry do |id, offset| 384 | id = id.unpack("H*").first 385 | @packs[id] = [pack, offset] 386 | end 387 | end 388 | end 389 | end 390 | end 391 | 392 | end 393 | -------------------------------------------------------------------------------- /lib/git_store/blob.rb: -------------------------------------------------------------------------------- 1 | class GitStore 2 | 3 | # This class stores the raw string data of a blob, but also the 4 | # deserialized data object. 5 | class Blob 6 | 7 | attr_accessor :store, :id, :data, :mode, :object 8 | 9 | # Initialize a Blob 10 | def initialize(store, id = nil, data = nil) 11 | @store = store 12 | @id = id || store.id_for('blob', data) 13 | @data = data 14 | @mode = "100644" 15 | end 16 | 17 | def ==(other) 18 | Blob === other and id == other.id 19 | end 20 | 21 | def dump 22 | @data 23 | end 24 | 25 | # Write the data to the git object store 26 | def write 27 | @id = store.put(self) 28 | end 29 | 30 | end 31 | 32 | end 33 | -------------------------------------------------------------------------------- /lib/git_store/commit.rb: -------------------------------------------------------------------------------- 1 | class GitStore 2 | 3 | class Commit 4 | attr_accessor :store, :id, :tree, :parent, :author, :committer, :message 5 | 6 | def initialize(store, id = nil, data = nil) 7 | @store = store 8 | @id = id 9 | @parent = [] 10 | 11 | parse(data) if data 12 | end 13 | 14 | def ==(other) 15 | Commit === other and id == other.id 16 | end 17 | 18 | def parse(data) 19 | headers, @message = data.split(/\n\n/, 2) 20 | 21 | headers.split(/\n/).each do |header| 22 | key, value = header.split(/ /, 2) 23 | case key 24 | when 'parent' 25 | @parent << value 26 | 27 | when 'author' 28 | @author = User.parse(value) 29 | 30 | when 'committer' 31 | @committer = User.parse(value) 32 | 33 | when 'tree' 34 | @tree = store.get(value) 35 | end 36 | end 37 | 38 | self 39 | end 40 | 41 | def diff(commit, path = nil) 42 | commit = commit.id if Commit === commit 43 | Diff.exec(store, "git diff --full-index #{commit} #{id} -- '#{path}'") 44 | end 45 | 46 | def diffs(path = nil) 47 | diff(parent.first, path) 48 | end 49 | 50 | def write 51 | @id = store.put(self) 52 | end 53 | 54 | def dump 55 | [ "tree #{ tree.id }", 56 | parent.map { |parent| "parent #{parent}" }, 57 | "author #{ author.dump }", 58 | "committer #{ committer.dump }", 59 | '', 60 | message ].flatten.join("\n") 61 | end 62 | 63 | end 64 | 65 | end 66 | -------------------------------------------------------------------------------- /lib/git_store/diff.rb: -------------------------------------------------------------------------------- 1 | class GitStore 2 | 3 | # adapted from Grit 4 | class Diff 5 | attr_reader :store 6 | attr_reader :a_path, :b_path 7 | attr_reader :a_blob, :b_blob 8 | attr_reader :a_mode, :b_mode 9 | attr_reader :new_file, :deleted_file 10 | attr_reader :diff 11 | 12 | def initialize(store, a_path, b_path, a_blob, b_blob, a_mode, b_mode, new_file, deleted_file, diff) 13 | @store = store 14 | @a_path = a_path 15 | @b_path = b_path 16 | @a_blob = a_blob =~ /^0{40}$/ ? nil : store.get(a_blob) 17 | @b_blob = b_blob =~ /^0{40}$/ ? nil : store.get(b_blob) 18 | @a_mode = a_mode 19 | @b_mode = b_mode 20 | @new_file = new_file 21 | @deleted_file = deleted_file 22 | @diff = diff 23 | end 24 | 25 | def self.exec(store, cmd) 26 | list(store, IO.popen(cmd) { |io| io.read }) 27 | end 28 | 29 | def self.list(store, text) 30 | lines = text.split("\n") 31 | 32 | diffs = [] 33 | 34 | while !lines.empty? 35 | m, a_path, b_path = *lines.shift.match(%r{^diff --git a/(.+?) b/(.+)$}) 36 | 37 | if lines.first =~ /^old mode/ 38 | m, a_mode = *lines.shift.match(/^old mode (\d+)/) 39 | m, b_mode = *lines.shift.match(/^new mode (\d+)/) 40 | end 41 | 42 | if lines.empty? || lines.first =~ /^diff --git/ 43 | diffs << Diff.new(store, a_path, b_path, nil, nil, a_mode, b_mode, false, false, nil) 44 | next 45 | end 46 | 47 | new_file = false 48 | deleted_file = false 49 | 50 | if lines.first =~ /^new file/ 51 | m, b_mode = lines.shift.match(/^new file mode (.+)$/) 52 | a_mode = nil 53 | new_file = true 54 | elsif lines.first =~ /^deleted file/ 55 | m, a_mode = lines.shift.match(/^deleted file mode (.+)$/) 56 | b_mode = nil 57 | deleted_file = true 58 | end 59 | 60 | m, a_blob, b_blob, b_mode = *lines.shift.match(%r{^index ([0-9A-Fa-f]+)\.\.([0-9A-Fa-f]+) ?(.+)?$}) 61 | b_mode.strip! if b_mode 62 | 63 | diff_lines = [] 64 | while lines.first && lines.first !~ /^diff/ 65 | diff_lines << lines.shift 66 | end 67 | diff = diff_lines.join("\n") 68 | 69 | diffs << Diff.new(store, a_path, b_path, a_blob, b_blob, a_mode, b_mode, new_file, deleted_file, diff) 70 | end 71 | 72 | diffs 73 | end 74 | end 75 | 76 | end 77 | -------------------------------------------------------------------------------- /lib/git_store/handlers.rb: -------------------------------------------------------------------------------- 1 | 2 | # This fix ensures sorted yaml maps. 3 | class Hash 4 | def to_yaml( opts = {} ) 5 | YAML::quick_emit( object_id, opts ) do |out| 6 | out.map( taguri, to_yaml_style ) do |map| 7 | sort_by { |k, v| k.to_s }.each do |k, v| 8 | map.add( k, v ) 9 | end 10 | end 11 | end 12 | end 13 | end 14 | 15 | class GitStore 16 | 17 | class DefaultHandler 18 | def read(data) 19 | data 20 | end 21 | 22 | def write(data) 23 | data.to_s 24 | end 25 | end 26 | 27 | class YAMLHandler 28 | def read(data) 29 | YAML.load(data) 30 | end 31 | 32 | def write(data) 33 | data.to_yaml 34 | end 35 | end 36 | end 37 | -------------------------------------------------------------------------------- /lib/git_store/pack.rb: -------------------------------------------------------------------------------- 1 | # 2 | # converted from the gitrb project 3 | # 4 | # authors: 5 | # Matthias Lederhofer 6 | # Simon 'corecode' Schubert 7 | # Scott Chacon 8 | # 9 | # provides native ruby access to git objects and pack files 10 | # 11 | 12 | require 'zlib' 13 | 14 | class GitStore 15 | PACK_SIGNATURE = "PACK" 16 | PACK_IDX_SIGNATURE = "\377tOc" 17 | 85 18 | OBJ_NONE = 0 19 | OBJ_COMMIT = 1 20 | OBJ_TREE = 2 21 | OBJ_BLOB = 3 22 | OBJ_TAG = 4 23 | 24 | OBJ_TYPES = [nil, 'commit', 'tree', 'blob', 'tag'].freeze 25 | 26 | class Mmap 27 | def initialize(file, version = 1) 28 | @file = file 29 | @offset = nil 30 | if version == 2 31 | @global_offset = 8 32 | else 33 | @global_offset = 0 34 | end 35 | end 36 | 37 | def unmap 38 | @file = nil 39 | end 40 | 41 | def [](*idx) 42 | idx = idx[0] if idx.length == 1 43 | case idx 44 | when Range 45 | offset = idx.first 46 | len = idx.last - idx.first + idx.exclude_end? ? 0 : 1 47 | when Fixnum 48 | offset = idx 49 | len = nil 50 | when Array 51 | offset, len = idx 52 | else 53 | raise RuntimeError, "invalid index param: #{idx.class}" 54 | end 55 | if @offset != offset 56 | @file.seek(offset + @global_offset) 57 | end 58 | @offset = offset + len ? len : 1 59 | if not len 60 | @file.read(1)[0] 61 | else 62 | @file.read(len) 63 | end 64 | end 65 | end 66 | 67 | class PackFormatError < StandardError 68 | end 69 | 70 | class PackStorage 71 | OBJ_OFS_DELTA = 6 72 | OBJ_REF_DELTA = 7 73 | 74 | FanOutCount = 256 75 | SHA1Size = 20 76 | IdxOffsetSize = 4 77 | OffsetSize = 4 78 | CrcSize = 4 79 | OffsetStart = FanOutCount * IdxOffsetSize 80 | SHA1Start = OffsetStart + OffsetSize 81 | EntrySize = OffsetSize + SHA1Size 82 | EntrySizeV2 = SHA1Size + CrcSize + OffsetSize 83 | 84 | def initialize(file) 85 | if file =~ /\.idx$/ 86 | file = file[0...-3] + 'pack' 87 | end 88 | @name = file 89 | @cache = {} 90 | init_pack 91 | end 92 | 93 | def with_idx(index_file = nil) 94 | if !index_file 95 | index_file = @name 96 | idxfile = File.open(@name[0...-4]+'idx') 97 | else 98 | idxfile = File.open(index_file) 99 | end 100 | 101 | # read header 102 | sig = idxfile.read(4) 103 | ver = idxfile.read(4).unpack("N")[0] 104 | 105 | if sig == PACK_IDX_SIGNATURE 106 | if(ver != 2) 107 | raise PackFormatError, "pack #@name has unknown pack file version #{ver}" 108 | end 109 | @version = 2 110 | else 111 | @version = 1 112 | end 113 | 114 | idx = Mmap.new(idxfile, @version) 115 | yield idx 116 | idx.unmap 117 | idxfile.close 118 | end 119 | 120 | def with_packfile 121 | packfile = File.open(@name) 122 | result = yield packfile 123 | packfile.close 124 | 125 | result 126 | end 127 | 128 | def cache_objects 129 | @cache = {} 130 | with_packfile do |packfile| 131 | each_entry do |sha, offset| 132 | data, type = unpack_object(packfile, offset, {:caching => true}) 133 | if data 134 | @cache[sha] = [type, data] 135 | end 136 | end 137 | end 138 | end 139 | 140 | def name 141 | @name 142 | end 143 | 144 | def close 145 | # shouldnt be anything open now 146 | end 147 | 148 | # given an index file, list out the shas that it's packfile contains 149 | def get_shas 150 | shas = [] 151 | each_sha1 { |sha| shas << sha.unpack("H*")[0] } 152 | shas 153 | end 154 | 155 | def [](sha1) 156 | if obj = @cache[sha1] 157 | return obj 158 | end 159 | 160 | offset = find_object(sha1) 161 | return nil if !offset 162 | @cache[sha1] = obj = parse_object(offset) 163 | return obj 164 | end 165 | 166 | def init_pack 167 | with_idx do |idx| 168 | @offsets = [0] 169 | FanOutCount.times do |i| 170 | pos = idx[i * IdxOffsetSize,IdxOffsetSize].unpack('N')[0] 171 | if pos < @offsets[i] 172 | raise PackFormatError, "pack #@name has discontinuous index #{i}" 173 | end 174 | @offsets << pos 175 | end 176 | @size = @offsets[-1] 177 | end 178 | end 179 | 180 | def each_entry 181 | with_idx do |idx| 182 | if @version == 2 183 | data = read_data_v2(idx) 184 | data.each do |sha1, crc, offset| 185 | yield sha1, offset 186 | end 187 | else 188 | pos = OffsetStart 189 | @size.times do 190 | offset = idx[pos,OffsetSize].unpack('N')[0] 191 | sha1 = idx[pos+OffsetSize,SHA1Size] 192 | pos += EntrySize 193 | yield sha1, offset 194 | end 195 | end 196 | end 197 | end 198 | 199 | def read_data_v2(idx) 200 | data = [] 201 | pos = OffsetStart 202 | @size.times do |i| 203 | data[i] = [idx[pos,SHA1Size], 0, 0] 204 | pos += SHA1Size 205 | end 206 | @size.times do |i| 207 | crc = idx[pos,CrcSize] 208 | data[i][1] = crc 209 | pos += CrcSize 210 | end 211 | @size.times do |i| 212 | offset = idx[pos,OffsetSize].unpack('N')[0] 213 | data[i][2] = offset 214 | pos += OffsetSize 215 | end 216 | data 217 | end 218 | private :read_data_v2 219 | 220 | def each_sha1 221 | with_idx do |idx| 222 | if @version == 2 223 | data = read_data_v2(idx) 224 | data.each do |sha1, crc, offset| 225 | yield sha1 226 | end 227 | else 228 | pos = SHA1Start 229 | @size.times do 230 | sha1 = idx[pos,SHA1Size] 231 | pos += EntrySize 232 | yield sha1 233 | end 234 | end 235 | end 236 | end 237 | 238 | def find_object_in_index(idx, sha1) 239 | slot = sha1[0] 240 | return nil if !slot 241 | first, last = @offsets[slot,2] 242 | while first < last 243 | mid = (first + last) / 2 244 | if @version == 2 245 | midsha1 = idx[OffsetStart + (mid * SHA1Size), SHA1Size] 246 | cmp = midsha1 <=> sha1 247 | 248 | if cmp < 0 249 | first = mid + 1 250 | elsif cmp > 0 251 | last = mid 252 | else 253 | pos = OffsetStart + (@size * (SHA1Size + CrcSize)) + (mid * OffsetSize) 254 | offset = idx[pos, OffsetSize].unpack('N')[0] 255 | return offset 256 | end 257 | else 258 | midsha1 = idx[SHA1Start + mid * EntrySize,SHA1Size] 259 | cmp = midsha1 <=> sha1 260 | 261 | if cmp < 0 262 | first = mid + 1 263 | elsif cmp > 0 264 | last = mid 265 | else 266 | pos = OffsetStart + mid * EntrySize 267 | offset = idx[pos,OffsetSize].unpack('N')[0] 268 | return offset 269 | end 270 | end 271 | end 272 | nil 273 | end 274 | 275 | def find_object(sha1) 276 | obj = nil 277 | with_idx do |idx| 278 | obj = find_object_in_index(idx, sha1) 279 | end 280 | obj 281 | end 282 | private :find_object 283 | 284 | def parse_object(offset) 285 | data, type = with_packfile do |packfile| 286 | unpack_object(packfile, offset) 287 | end 288 | 289 | return data, OBJ_TYPES[type] 290 | end 291 | 292 | def unpack_object(packfile, offset, options = {}) 293 | obj_offset = offset 294 | packfile.seek(offset) 295 | 296 | c = packfile.read(1)[0].ord 297 | size = c & 0xf 298 | type = (c >> 4) & 7 299 | shift = 4 300 | offset += 1 301 | while c & 0x80 != 0 302 | c = packfile.read(1)[0].ord 303 | size |= ((c & 0x7f) << shift) 304 | shift += 7 305 | offset += 1 306 | end 307 | 308 | return [false, false] if !(type == OBJ_COMMIT || type == OBJ_TREE) && options[:caching] 309 | 310 | case type 311 | when OBJ_OFS_DELTA, OBJ_REF_DELTA 312 | data, type = unpack_deltified(packfile, type, offset, obj_offset, size, options) 313 | #puts type 314 | when OBJ_COMMIT, OBJ_TREE, OBJ_BLOB, OBJ_TAG 315 | data = unpack_compressed(offset, size) 316 | else 317 | raise PackFormatError, "invalid type #{type}" 318 | end 319 | [data, type] 320 | end 321 | private :unpack_object 322 | 323 | def unpack_deltified(packfile, type, offset, obj_offset, size, options = {}) 324 | packfile.seek(offset) 325 | data = packfile.read(SHA1Size) 326 | 327 | if type == OBJ_OFS_DELTA 328 | i = 0 329 | c = data[i].ord 330 | base_offset = c & 0x7f 331 | while c & 0x80 != 0 332 | c = data[i += 1].ord 333 | base_offset += 1 334 | base_offset <<= 7 335 | base_offset |= c & 0x7f 336 | end 337 | base_offset = obj_offset - base_offset 338 | offset += i + 1 339 | else 340 | base_offset = find_object(data) 341 | offset += SHA1Size 342 | end 343 | 344 | base, type = unpack_object(packfile, base_offset) 345 | 346 | return [false, false] if !(type == OBJ_COMMIT || type == OBJ_TREE) && options[:caching] 347 | 348 | delta = unpack_compressed(offset, size) 349 | [patch_delta(base, delta), type] 350 | end 351 | private :unpack_deltified 352 | 353 | def unpack_compressed(offset, destsize) 354 | outdata = "" 355 | with_packfile do |packfile| 356 | packfile.seek(offset) 357 | zstr = Zlib::Inflate.new 358 | while outdata.size < destsize 359 | indata = packfile.read(4096) 360 | if indata.size == 0 361 | raise PackFormatError, 'error reading pack data' 362 | end 363 | outdata += zstr.inflate(indata) 364 | end 365 | if outdata.size > destsize 366 | raise PackFormatError, 'error reading pack data' 367 | end 368 | zstr.close 369 | end 370 | outdata 371 | end 372 | private :unpack_compressed 373 | 374 | def patch_delta(base, delta) 375 | src_size, pos = patch_delta_header_size(delta, 0) 376 | if src_size != base.size 377 | raise PackFormatError, 'invalid delta data' 378 | end 379 | 380 | dest_size, pos = patch_delta_header_size(delta, pos) 381 | dest = "" 382 | while pos < delta.size 383 | c = delta[pos].ord 384 | pos += 1 385 | if c & 0x80 != 0 386 | pos -= 1 387 | cp_off = cp_size = 0 388 | cp_off = delta[pos += 1].ord if c & 0x01 != 0 389 | cp_off |= delta[pos += 1].ord << 8 if c & 0x02 != 0 390 | cp_off |= delta[pos += 1].ord << 16 if c & 0x04 != 0 391 | cp_off |= delta[pos += 1].ord << 24 if c & 0x08 != 0 392 | cp_size = delta[pos += 1].ord if c & 0x10 != 0 393 | cp_size |= delta[pos += 1].ord << 8 if c & 0x20 != 0 394 | cp_size |= delta[pos += 1].ord << 16 if c & 0x40 != 0 395 | cp_size = 0x10000 if cp_size == 0 396 | pos += 1 397 | dest += base[cp_off, cp_size] 398 | elsif c != 0 399 | dest += delta[pos,c] 400 | pos += c 401 | else 402 | raise PackFormatError, 'invalid delta data' 403 | end 404 | end 405 | dest 406 | end 407 | private :patch_delta 408 | 409 | def patch_delta_header_size(delta, pos) 410 | size = 0 411 | shift = 0 412 | begin 413 | c = delta[pos] 414 | if c == nil 415 | raise PackFormatError, 'invalid delta header' 416 | end 417 | c = c.ord 418 | pos += 1 419 | size |= (c & 0x7f) << shift 420 | shift += 7 421 | end while c & 0x80 != 0 422 | [size, pos] 423 | end 424 | private :patch_delta_header_size 425 | end 426 | end 427 | -------------------------------------------------------------------------------- /lib/git_store/tag.rb: -------------------------------------------------------------------------------- 1 | class GitStore 2 | 3 | class Tag 4 | attr_accessor :store, :id, :object, :type, :tagger, :message 5 | 6 | def initialize(store, id = nil, data = nil) 7 | @store = store 8 | @id = id 9 | 10 | parse(data) if data 11 | end 12 | 13 | def ==(other) 14 | Tag === other and id == other.id 15 | end 16 | 17 | def parse(data) 18 | headers, @message = data.split(/\n\n/, 2) 19 | 20 | headers.split(/\n/).each do |header| 21 | key, value = header.split(/ /, 2) 22 | case key 23 | when 'type' 24 | @type = value 25 | 26 | when 'object' 27 | @object = store.get(value) 28 | 29 | when 'tagger' 30 | @tagger = User.parse(value) 31 | 32 | end 33 | end 34 | 35 | self 36 | end 37 | 38 | end 39 | 40 | end 41 | -------------------------------------------------------------------------------- /lib/git_store/tree.rb: -------------------------------------------------------------------------------- 1 | class GitStore 2 | 3 | class Tree 4 | include Enumerable 5 | 6 | attr_reader :store, :table 7 | attr_accessor :id, :data, :mode 8 | 9 | # Initialize a tree 10 | def initialize(store, id = nil, data = nil) 11 | @store = store 12 | @id = id 13 | @table = {} 14 | @mode = "040000" 15 | parse(data) if data 16 | end 17 | 18 | def ==(other) 19 | Tree === other and id == other.id 20 | end 21 | 22 | # Has this tree been modified? 23 | def modified? 24 | @modified or @table.values.any? { |entry| Tree === entry and entry.modified? } 25 | end 26 | 27 | # Find or create a subtree with specified name. 28 | def tree(name) 29 | get(name) or put(name, Tree.new(store)) 30 | end 31 | 32 | # Read the contents of a raw git object. 33 | def parse(data) 34 | @table.clear 35 | 36 | while data.size > 0 37 | mode, data = data.split(" ", 2) 38 | name, data = data.split("\0", 2) 39 | id = data.slice!(0, 20).unpack("H*").first 40 | 41 | @table[name] = store.get(id) 42 | end 43 | end 44 | 45 | def dump 46 | @table.map { |k, v| "#{ v.mode } #{ k }\0#{ [v.write].pack("H*") }" }.join 47 | end 48 | 49 | # Write this treetree back to the git repository. 50 | # 51 | # Returns the object id of the tree. 52 | def write 53 | return id if not modified? 54 | @modified = false 55 | @id = store.put(self) 56 | end 57 | 58 | # Read entry with specified name. 59 | def get(name) 60 | entry = @table[name] 61 | 62 | case entry 63 | when Blob 64 | entry.object ||= handler_for(name).read(entry.data) 65 | 66 | when Tree 67 | entry 68 | end 69 | end 70 | 71 | def handler_for(name) 72 | store.handler_for(name) 73 | end 74 | 75 | # Write entry with specified name. 76 | def put(name, value) 77 | if value.is_a?(Tree) 78 | @modified = true 79 | @table[name] = value 80 | else 81 | data = handler_for(name).write(value) 82 | if @table[name].nil? or data != @table[name].data 83 | @modified = true 84 | @table[name] = Blob.new(store, nil, data) 85 | end 86 | end 87 | 88 | value 89 | end 90 | 91 | # Remove entry with specified name. 92 | def remove(name) 93 | @modified = true 94 | @table.delete(name.to_s) 95 | end 96 | 97 | # Does this key exist in the table? 98 | def has_key?(name) 99 | @table.has_key?(name.to_s) 100 | end 101 | 102 | def normalize_path(path) 103 | (path[0, 1] == '/' ? path[1..-1] : path).split('/') 104 | end 105 | 106 | # Read a value on specified path. 107 | def [](path) 108 | normalize_path(path).inject(self) do |tree, key| 109 | tree.get(key) or return nil 110 | end 111 | end 112 | 113 | # Write a value on specified path. 114 | def []=(path, value) 115 | list = normalize_path(path) 116 | tree = list[0..-2].to_a.inject(self) { |tree, name| tree.tree(name) } 117 | tree.put(list.last, value) 118 | end 119 | 120 | # Delete a value on specified path. 121 | def delete(path) 122 | list = normalize_path(path) 123 | 124 | tree = list[0..-2].to_a.inject(self) do |tree, key| 125 | tree.get(key) or return 126 | end 127 | 128 | tree.remove(list.last) 129 | end 130 | 131 | # Iterate over all objects found in this subtree. 132 | def each(path = [], &block) 133 | @table.sort.each do |name, entry| 134 | child_path = path + [name] 135 | case entry 136 | when Blob 137 | entry.object ||= handler_for(name).read(entry.data) 138 | yield child_path.join("/"), entry.object 139 | 140 | when Tree 141 | entry.each(child_path, &block) 142 | end 143 | end 144 | end 145 | 146 | def each_blob(path = [], &block) 147 | @table.sort.each do |name, entry| 148 | child_path = path + [name] 149 | 150 | case entry 151 | when Blob 152 | yield child_path.join("/"), entry 153 | 154 | when Tree 155 | entry.each_blob(child_path, &block) 156 | end 157 | end 158 | end 159 | 160 | def paths 161 | map { |path, data| path } 162 | end 163 | 164 | def values 165 | map { |path, data| data } 166 | end 167 | 168 | # Convert this tree into a hash object. 169 | def to_hash 170 | @table.inject({}) do |hash, (name, entry)| 171 | if entry.is_a?(Tree) 172 | hash[name] = entry.to_hash 173 | else 174 | hash[name] = entry.object ||= handler_for(name).read(entry.data) 175 | end 176 | hash 177 | end 178 | end 179 | 180 | def inspect 181 | "#" 182 | end 183 | 184 | end 185 | 186 | end 187 | -------------------------------------------------------------------------------- /lib/git_store/user.rb: -------------------------------------------------------------------------------- 1 | class GitStore 2 | class User < Struct.new(:name, :email, :time) 3 | def dump 4 | "#{ name } <#{email}> #{ time.to_i } #{ time.strftime('%z') }" 5 | end 6 | alias to_s dump 7 | 8 | def inspect 9 | "#" % [name, email, time] 10 | end 11 | 12 | def self.from_config 13 | name, email = config_get('user.name'), config_get('user.email') 14 | new name, email, Time.now 15 | end 16 | 17 | def self.config_get(key) 18 | value = `git config #{key}`.chomp 19 | 20 | if $?.exitstatus == 0 21 | return value unless value.empty? 22 | raise RuntimError, "#{key} is empty" 23 | else 24 | raise RuntimError, "No #{key} found in git config" 25 | end 26 | end 27 | 28 | def self.config_user_email 29 | email = `git config user.email`.chomp 30 | end 31 | 32 | def self.parse(user) 33 | if match = user.match(/(.*)<(.*)> (\d+) ([+-]\d+)/) 34 | new match[1].strip, match[2].strip, Time.at(match[3].to_i + match[4].to_i * 3600) 35 | end 36 | end 37 | end 38 | end 39 | -------------------------------------------------------------------------------- /test/bare_store_spec.rb: -------------------------------------------------------------------------------- 1 | require "#{File.dirname(__FILE__)}/../lib/git_store" 2 | require "#{File.dirname(__FILE__)}/helper" 3 | require 'pp' 4 | 5 | describe GitStore do 6 | 7 | REPO = '/tmp/git_store_test.git' 8 | 9 | attr_reader :store 10 | 11 | before(:each) do 12 | FileUtils.rm_rf REPO 13 | Dir.mkdir REPO 14 | Dir.chdir REPO 15 | 16 | `git init --bare` 17 | @store = GitStore.new(REPO, 'master', true) 18 | end 19 | 20 | it 'should fail to initialize without a valid git repository' do 21 | lambda { 22 | GitStore.new('/foo', 'master', true) 23 | }.should raise_error(ArgumentError) 24 | end 25 | 26 | it 'should save and load entries' do 27 | store['a'] = 'Hello' 28 | store.commit 29 | store.load 30 | 31 | store['a'].should == 'Hello' 32 | end 33 | end 34 | -------------------------------------------------------------------------------- /test/benchmark.rb: -------------------------------------------------------------------------------- 1 | require 'git_store' 2 | require 'grit' 3 | require 'benchmark' 4 | require 'fileutils' 5 | 6 | REPO = '/tmp/git-store' 7 | 8 | FileUtils.rm_rf REPO 9 | FileUtils.mkpath REPO 10 | Dir.chdir REPO 11 | 12 | `git init` 13 | 14 | store = GitStore.new(REPO) 15 | 16 | Benchmark.bm 20 do |x| 17 | x.report 'store 1000 objects' do 18 | store.transaction { 'aaa'.upto('jjj') { |key| store[key] = rand.to_s } } 19 | end 20 | x.report 'commit one object' do 21 | store.transaction { store['aa'] = rand.to_s } 22 | end 23 | x.report 'load 1000 objects' do 24 | GitStore.new('.').values { |v| v } 25 | end 26 | x.report 'load 1000 with grit' do 27 | Grit::Repo.new('.').tree.contents.each { |e| e.data } 28 | end 29 | end 30 | 31 | -------------------------------------------------------------------------------- /test/commit_spec.rb: -------------------------------------------------------------------------------- 1 | require "#{File.dirname(__FILE__)}/../lib/git_store" 2 | require 'pp' 3 | 4 | describe GitStore::Commit do 5 | 6 | REPO = '/tmp/git_store_test' 7 | 8 | attr_reader :store 9 | 10 | before(:each) do 11 | FileUtils.rm_rf REPO 12 | Dir.mkdir REPO 13 | Dir.chdir REPO 14 | `git init` 15 | @store = GitStore.new(REPO) 16 | end 17 | 18 | it "should dump in right format" do 19 | user = GitStore::User.new("hanni", "hanni@email.de", Time.now) 20 | 21 | commit = GitStore::Commit.new(nil) 22 | commit.tree = @store.root 23 | commit.author = user 24 | commit.committer = user 25 | commit.message = "This is a message" 26 | 27 | commit.dump.should == "tree #{@store.root.id} 28 | author #{user.dump} 29 | committer #{user.dump} 30 | 31 | This is a message" 32 | end 33 | 34 | it "should be readable by git binary" do 35 | time = Time.utc(2009, 4, 20) 36 | author = GitStore::User.new("hans", "hans@email.de", time) 37 | 38 | store['a'] = "Yay" 39 | commit = store.commit("Commit Message", author, author) 40 | 41 | IO.popen("git log") do |io| 42 | io.gets.should == "commit #{commit.id}\n" 43 | io.gets.should == "Author: hans \n" 44 | io.gets.should == "Date: Mon Apr 20 00:00:00 2009 +0000\n" 45 | io.gets.should == "\n" 46 | io.gets.should == " Commit Message\n" 47 | end 48 | end 49 | 50 | it "should diff 2 commits" do 51 | store['x'] = 'a' 52 | store['y'] = " 53 | First Line. 54 | Second Line. 55 | Last Line. 56 | " 57 | a = store.commit 58 | 59 | store.delete('x') 60 | store['y'] = " 61 | First Line. 62 | Last Line. 63 | Another Line. 64 | " 65 | store['z'] = 'c' 66 | 67 | b = store.commit 68 | 69 | diff = b.diff(a) 70 | 71 | diff[0].a_path.should == 'x' 72 | diff[0].deleted_file.should be_true 73 | 74 | diff[1].a_path.should == 'y' 75 | diff[1].diff.should == "--- a/y\n+++ b/y\n@@ -1,4 +1,4 @@\n \n First Line.\n-Second Line.\n Last Line.\n+Another Line." 76 | 77 | diff[2].a_path.should == 'z' 78 | diff[2].new_file.should be_true 79 | end 80 | 81 | end 82 | -------------------------------------------------------------------------------- /test/git_store_spec.rb: -------------------------------------------------------------------------------- 1 | require "#{File.dirname(__FILE__)}/../lib/git_store" 2 | require "#{File.dirname(__FILE__)}/helper" 3 | require 'pp' 4 | 5 | describe GitStore do 6 | 7 | REPO = '/tmp/git_store_test' 8 | 9 | attr_reader :store 10 | 11 | before(:each) do 12 | FileUtils.rm_rf REPO 13 | Dir.mkdir REPO 14 | Dir.chdir REPO 15 | 16 | `git init` 17 | `git config user.name 'User Name'` 18 | `git config user.email 'user.name@email.com'` 19 | @store = GitStore.new(REPO) 20 | end 21 | 22 | def file(file, data) 23 | FileUtils.mkpath(File.dirname(file)) 24 | open(file, 'w') { |io| io << data } 25 | 26 | `git add #{file}` 27 | `git commit -m 'added #{file}'` 28 | File.unlink(file) 29 | end 30 | 31 | it 'should fail to initialize without a valid git repository' do 32 | lambda { 33 | GitStore.new('/') 34 | }.should raise_error(ArgumentError) 35 | end 36 | 37 | it 'should find modified entries' do 38 | store['a'] = 'Hello' 39 | 40 | store.root.should be_modified 41 | 42 | store.commit 43 | 44 | store.root.should_not be_modified 45 | 46 | store['a'] = 'Bello' 47 | 48 | store.root.should be_modified 49 | end 50 | 51 | it 'should load a repo' do 52 | file 'a', 'Hello' 53 | file 'b', 'World' 54 | 55 | store.load 56 | 57 | store['a'].should == 'Hello' 58 | store['b'].should == 'World' 59 | end 60 | 61 | it 'should load folders' do 62 | file 'x/a', 'Hello' 63 | file 'y/b', 'World' 64 | 65 | store.load 66 | store['x'].should be_kind_of(GitStore::Tree) 67 | store['y'].should be_kind_of(GitStore::Tree) 68 | 69 | store['x']['a'].should == 'Hello' 70 | store['y']['b'].should == 'World' 71 | end 72 | 73 | it 'should load yaml' do 74 | file 'x/a.yml', '[1, 2, 3, 4]' 75 | 76 | store.load 77 | 78 | store['x']['a.yml'].should == [1,2,3,4] 79 | store['x']['a.yml'] = [1,2,3,4,5] 80 | end 81 | 82 | it 'should save yaml' do 83 | store['x/a.yml'] = [1,2,3,4,5] 84 | store['x/a.yml'].should == [1,2,3,4,5] 85 | end 86 | 87 | it 'should detect modification' do 88 | store.transaction do 89 | store['x/a'] = 'a' 90 | end 91 | 92 | store.load 93 | 94 | store['x/a'].should == 'a' 95 | 96 | store.transaction do 97 | store['x/a'] = 'b' 98 | store['x'].should be_modified 99 | store.root.should be_modified 100 | end 101 | 102 | store.load 103 | 104 | store['x/a'].should == 'b' 105 | end 106 | 107 | it 'should resolve paths' do 108 | file 'x/a', 'Hello' 109 | file 'y/b', 'World' 110 | 111 | store.load 112 | 113 | store['x/a'].should == 'Hello' 114 | store['y/b'].should == 'World' 115 | 116 | store['y/b'] = 'Now this' 117 | 118 | store['y']['b'].should == 'Now this' 119 | end 120 | 121 | it 'should create new trees' do 122 | store['new/tree'] = 'This tree' 123 | store['new/tree'].should == 'This tree' 124 | end 125 | 126 | it 'should delete entries' do 127 | store['a'] = 'Hello' 128 | store.delete('a') 129 | 130 | store['a'].should be_nil 131 | end 132 | 133 | it 'should have a head commit' do 134 | file 'a', 'Hello' 135 | 136 | store.load 137 | store.head.should_not be_nil 138 | end 139 | 140 | it 'should detect changes' do 141 | file 'a', 'Hello' 142 | 143 | store.should be_changed 144 | end 145 | 146 | it 'should rollback a transaction' do 147 | file 'a/b', 'Hello' 148 | file 'c/d', 'World' 149 | 150 | begin 151 | store.transaction do 152 | store['a/b'] = 'Changed' 153 | store['x/a'] = 'Added' 154 | raise 155 | end 156 | rescue 157 | end 158 | 159 | store['a/b'].should == 'Hello' 160 | store['c/d'].should == 'World' 161 | store['x/a'].should be_nil 162 | end 163 | 164 | it 'should commit a transaction' do 165 | file 'a/b', 'Hello' 166 | file 'c/d', 'World' 167 | 168 | store.transaction do 169 | store['a/b'] = 'Changed' 170 | store['x/a'] = 'Added' 171 | end 172 | 173 | a = git_ls_tree(store['a'].id) 174 | x = git_ls_tree(store['x'].id) 175 | 176 | a.should == [["100644", "blob", "b653cf27cef08de46da49a11fa5016421e9e3b32", "b"]] 177 | x.should == [["100644", "blob", "87d2b203800386b1cc8735a7d540a33e246357fa", "a"]] 178 | 179 | git_show(a[0][2]).should == 'Changed' 180 | git_show(x[0][2]).should == 'Added' 181 | end 182 | 183 | it "should save blobs" do 184 | store['a'] = 'a' 185 | store['b'] = 'b' 186 | store['c'] = 'c' 187 | 188 | store.commit 189 | 190 | a = store.id_for('blob', 'a') 191 | b = store.id_for('blob', 'b') 192 | c = store.id_for('blob', 'c') 193 | 194 | git_show(a).should == 'a' 195 | git_show(b).should == 'b' 196 | git_show(c).should == 'c' 197 | end 198 | 199 | it 'should allow only one transaction' do 200 | file 'a/b', 'Hello' 201 | 202 | ready = false 203 | 204 | store.transaction do 205 | Thread.start do 206 | store.transaction do 207 | store['a/b'] = 'Changed by second thread' 208 | end 209 | ready = true 210 | end 211 | store['a/b'] = 'Changed' 212 | end 213 | 214 | sleep 0.01 until ready 215 | 216 | store.load 217 | 218 | store['a/b'].should == 'Changed by second thread' 219 | end 220 | 221 | it 'should find all objects' do 222 | store.load 223 | store['c'] = 'Hello' 224 | store['d'] = 'World' 225 | store.commit 226 | 227 | store.to_a.should == [['c', 'Hello'], ['d', 'World']] 228 | end 229 | 230 | it "should load commits" do 231 | store['a'] = 'a' 232 | store.commit 'added a' 233 | 234 | store['b'] = 'b' 235 | store.commit 'added b' 236 | 237 | store.commits[0].message.should == 'added b' 238 | store.commits[1].message.should == 'added a' 239 | end 240 | 241 | it "should load tags" do 242 | file 'a', 'init' 243 | 244 | `git tag -m 'message' 0.1` 245 | 246 | store.load 247 | 248 | id = File.read('.git/refs/tags/0.1') 249 | tag = store.get(id) 250 | 251 | tag.type.should == 'commit' 252 | tag.object.should == store.head 253 | tag.tagger.name.should == 'User Name' 254 | tag.tagger.email.should == 'user.name@email.com' 255 | tag.message.should =~ /message/ 256 | end 257 | 258 | end 259 | -------------------------------------------------------------------------------- /test/helper.rb: -------------------------------------------------------------------------------- 1 | 2 | def git_show(id) 3 | IO.popen("git show #{id}") do |io| 4 | io.gets 5 | end 6 | end 7 | 8 | def git_ls_tree(id) 9 | lines = [] 10 | 11 | IO.popen("git ls-tree #{id}") do |io| 12 | while line = io.gets 13 | lines << line.split(" ") 14 | end 15 | end 16 | 17 | lines 18 | end 19 | -------------------------------------------------------------------------------- /test/tree_spec.rb: -------------------------------------------------------------------------------- 1 | require "#{File.dirname(__FILE__)}/../lib/git_store" 2 | require "#{File.dirname(__FILE__)}/helper" 3 | require 'pp' 4 | 5 | describe GitStore::Tree do 6 | REPO = '/tmp/git_store_test' 7 | 8 | attr_reader :store, :tree 9 | 10 | before(:each) do 11 | FileUtils.rm_rf REPO 12 | Dir.mkdir REPO 13 | Dir.chdir REPO 14 | 15 | `git init` 16 | 17 | @store = GitStore.new(REPO) 18 | end 19 | 20 | it "should parse a table" do 21 | tree = GitStore::Tree.new(store) 22 | 23 | a = store.put_object("blob", "a") 24 | b = store.put_object("blob", "b") 25 | c = store.put_object("blob", "c") 26 | 27 | data = 28 | "100644 a\0#{ [a].pack("H*") }" + 29 | "100644 b\0#{ [b].pack("H*") }" + 30 | "100644 c\0#{ [c].pack("H*") }" 31 | 32 | tree.parse(data) 33 | 34 | tree.get('a').should == 'a' 35 | tree.get('b').should == 'b' 36 | tree.get('c').should == 'c' 37 | end 38 | 39 | it "should write a table" do 40 | tree = GitStore::Tree.new(store) 41 | 42 | tree['a'] = 'a' 43 | tree['b'] = 'b' 44 | tree['c'] = 'c' 45 | 46 | id = tree.write 47 | 48 | a = ["2e65efe2a145dda7ee51d1741299f848e5bf752e"].pack('H*') 49 | b = ["63d8dbd40c23542e740659a7168a0ce3138ea748"].pack('H*') 50 | c = ["3410062ba67c5ed59b854387a8bc0ec012479368"].pack('H*') 51 | 52 | data = 53 | "100644 a\0#{a}" + 54 | "100644 b\0#{b}" + 55 | "100644 c\0#{c}" 56 | 57 | store.get_object(id).should == ['tree', data] 58 | end 59 | 60 | it "should save trees" do 61 | tree = GitStore::Tree.new(store) 62 | 63 | tree['a'] = 'a' 64 | tree['b'] = 'b' 65 | tree['c'] = 'c' 66 | 67 | tree.write 68 | 69 | git_ls_tree(tree.id).should == 70 | [["100644", "blob", "2e65efe2a145dda7ee51d1741299f848e5bf752e", "a"], 71 | ["100644", "blob", "63d8dbd40c23542e740659a7168a0ce3138ea748", "b"], 72 | ["100644", "blob", "3410062ba67c5ed59b854387a8bc0ec012479368", "c"]] 73 | end 74 | 75 | it "should save nested trees" do 76 | tree = GitStore::Tree.new(store) 77 | 78 | tree['x/a'] = 'a' 79 | tree['x/b'] = 'b' 80 | tree['x/c'] = 'c' 81 | 82 | tree.write 83 | 84 | git_ls_tree(tree.id).should == 85 | [["040000", "tree", "24e88cb96c396400000ef706d1ca1ed9a88251aa", "x"]] 86 | 87 | git_ls_tree("24e88cb96c396400000ef706d1ca1ed9a88251aa").should == 88 | [["100644", "blob", "2e65efe2a145dda7ee51d1741299f848e5bf752e", "a"], 89 | ["100644", "blob", "63d8dbd40c23542e740659a7168a0ce3138ea748", "b"], 90 | ["100644", "blob", "3410062ba67c5ed59b854387a8bc0ec012479368", "c"]] 91 | end 92 | end 93 | -------------------------------------------------------------------------------- /test/user_spec.rb: -------------------------------------------------------------------------------- 1 | require "#{File.dirname(__FILE__)}/../lib/git_store" 2 | require 'pp' 3 | 4 | describe GitStore::User do 5 | it 'should parse a user string' do 6 | user = GitStore::User.parse('Mr. T 1234567890 -0600') 7 | user.name.should == 'Mr. T' 8 | user.email.should == 'mr.t@a-team.us' 9 | user.time.should == Time.at(1234567890 - 2160000) 10 | end 11 | end 12 | --------------------------------------------------------------------------------