├── .gitignore ├── .travis.yml ├── Gemfile ├── LICENSE.txt ├── README.md ├── Rakefile ├── bin ├── dump_model_schema └── repl ├── lib ├── model_schema.rb └── model_schema │ ├── constants.rb │ ├── dumper.rb │ ├── plugin.rb │ ├── schema_error.rb │ └── version.rb ├── model_schema.gemspec └── test ├── model_schema ├── dumper_test.rb ├── plugin_test.rb ├── schema_error_test.rb └── version_test.rb └── test_helper.rb /.gitignore: -------------------------------------------------------------------------------- 1 | /.bundle/ 2 | /.yardoc 3 | /Gemfile.lock 4 | /_yardoc/ 5 | /coverage/ 6 | /doc/ 7 | /pkg/ 8 | /spec/reports/ 9 | /tmp/ 10 | .DS_Store 11 | *.sw[pon] 12 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: ruby 2 | rvm: 3 | - 2.1.5 4 | before_install: gem install bundler -v 1.10.6 5 | 6 | services: 7 | - postgresql 8 | before_script: 9 | - psql -c 'create database model_schema;' -U postgres 10 | env: 11 | - DB_URL=postgres://localhost:5432/model_schema 12 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | gemspec 3 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Karthik Viswanathan 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ModelSchema [![Build Status](https://travis-ci.org/karthikv/model_schema.svg?branch=master)](https://travis-ci.org/karthikv/model_schema) 2 | ModelSchema lets you annotate a [Sequel](https://github.com/jeremyevans/sequel/) 3 | Model with its expected schema and immediately identify inconsistencies. 4 | Instead of seeing a Sequel Model file that looks like this: 5 | 6 | ```rb 7 | class User < Sequel::Model(:users) 8 | end 9 | ``` 10 | 11 | You'll see one that looks like this: 12 | 13 | ```rb 14 | class User < Sequel::Model(:users) 15 | model_schema do 16 | primary_key :id 17 | 18 | String :email, :null => false 19 | String :password, :null => false 20 | 21 | TrueClass :is_admin, :default => false 22 | 23 | DateTime :created_at, :null => false 24 | DateTime :updated_at 25 | 26 | index :email 27 | end 28 | end 29 | ``` 30 | 31 | Unlike other similar gems, ModelSchema provides *enforcement*; if the schema you 32 | specify doesn't match the table schema, ModelSchema will raise an error and 33 | tell you exactly what's wrong, like so: 34 | 35 | ``` 36 | ModelSchema::SchemaError: Table users does not match the expected schema. 37 | 38 | Table users has extra columns: 39 | 40 | Integer :age 41 | 42 | Table users is missing columns: 43 | 44 | TrueClass :is_admin, :default => false 45 | 46 | Table users has mismatched indexes: 47 | 48 | actual: index [:email], :unique => true 49 | expected: index [:email] 50 | 51 | You may disable schema checks by passing :disable => true to model_schema or by 52 | setting the ENV variable DISABLE_MODEL_SCHEMA=1. 53 | ``` 54 | 55 | When developing on a team, local databases of team members can easily get out 56 | of sync due to differing migrations. ModelSchema immediately lets you know if 57 | the schema you expect differs from the actual schema. This ensures you identify 58 | database inconsistencies before they cause problems. As a nice added benefit, 59 | ModelSchema lets you see a list of columns for a model directly within the 60 | class itself. 61 | 62 | ## Installation 63 | Add `model_schema` to your Gemfile: 64 | 65 | ```rb 66 | gem 'model_schema' 67 | ``` 68 | 69 | And then execute `bundle` in your terminal. You can also install `model_schema` 70 | with `gem` directly by running `gem install model_schema`. 71 | 72 | ## Usage 73 | Require `model_schema` and register the plugin with Sequel: 74 | 75 | ```rb 76 | require 'model_schema' 77 | Sequel::Model.plugin(ModelSchema::Plugin) 78 | ``` 79 | 80 | Then, in each model where you'd like to use ModelSchema, introduce a call to 81 | `model_schema`, passing in a block that defines the schema. The block operates 82 | exactly as a [Sequel `create_table` 83 | block](http://sequel.jeremyevans.net/rdoc/files/doc/schema_modification_rdoc.html). 84 | See the documentation on that page for further details. 85 | 86 | ```rb 87 | class Post < Sequel::Model(:posts) 88 | model_schema do 89 | primary_key :id 90 | 91 | String :title, :null => false 92 | String :description, :text => true, :null => false 93 | DateTime :date_posted, :null => false 94 | end 95 | end 96 | ``` 97 | 98 | When the class is loaded, ModelSchema will ensure the table schema matches the 99 | given schema. If there are any errors, it will raise 100 | a `ModelSchema::SchemaError`, notifying you of any inconsistencies. 101 | 102 | You may pass an optional hash to `model_schema` with the following options: 103 | 104 | `disable`: `true` to disable all schema checks, `false` otherwise 105 | `no_indexes`: `true` to disable schema checks for indexes (columns will still 106 | be checked), `false` otherwise 107 | 108 | For instance, to disable index checking: 109 | 110 | ```rb 111 | class Item < Sequel::Model(:items) 112 | model_schema(:no_indexes => true) do 113 | ... 114 | end 115 | end 116 | ``` 117 | 118 | Note that you can disable ModelSchema in two ways: either pass `:disable => 119 | true` to the `model_schema` method, or set the environment variable 120 | `DISABLE_MODEL_SCHEMA=1` . 121 | 122 | ## Bootstrap Existing Project 123 | To help bootstrap existing projects that don't yet use ModelSchema, you can use 124 | the `dump_model_schema` executable. It will automatically dump an up-to-date 125 | `model_schema` block in each Sequel Model class. Use it like so: 126 | 127 | ```sh 128 | $ dump_model_schema -c [connection_string] model_file [model_file ...] 129 | ``` 130 | 131 | where `model_file` is a path to a Ruby file that contains a single Sequel 132 | Model, and `connection_string` is the database connection string to pass to 133 | `Sequel.connect()`. 134 | 135 | `dump_model_schema` will insert a `model_schema` block right after the 136 | definition of the Sequel Model class in `model_file`. Specifically, it looks 137 | for a line of the form `class SomeClassName < Sequel::Model(:table_name)`, and 138 | inserts a valid schema for table `table_name` directly after that line. Note 139 | that `dump_model_schema` overwrites `model_file`. 140 | 141 | For instance, say you had a file `items.rb` that looks like this: 142 | 143 | ```rb 144 | module SomeModule 145 | class Item < Sequel::Model(:items) 146 | end 147 | end 148 | ``` 149 | 150 | If you run: 151 | 152 | ```sh 153 | $ dump_model_schema -c [connection_string] items.rb 154 | ``` 155 | 156 | `items.rb` might now look like: 157 | 158 | ```rb 159 | module SomeModule 160 | class Item < Sequel::Model(:items) 161 | model_schema do 162 | primary_key :id, 163 | 164 | String :name, :null => false 165 | Integer :quantity 166 | 167 | index [:name], :name => :items_name_key 168 | end 169 | end 170 | end 171 | ``` 172 | 173 | By default, `dump_model_schema` assumes a tab size of 2 spaces, but you can 174 | change this with the `-t` option. Pass an integer representing the number of 175 | spaces, or 0 if you want to use hard tabs. 176 | 177 | You may specify multiple `model_file`s as distinct arguments, and each will 178 | have its `model_schema` dumped. This can be done easily with shell expansion: 179 | 180 | ```sh 181 | $ dump_model_schema -c [connection_string] models/*.rb 182 | ``` 183 | 184 | You may see help text with `dump_model_schema -h` and view the version of 185 | ModelSchema with `dump_model_schema -v`. 186 | 187 | ## Limitations 188 | ModelSchema has a few notable limitations: 189 | 190 | - It checks columns independently from indexes. Say you create a table like so: 191 | 192 | ```rb 193 | DB.create_table(:items) do 194 | String :name, :unique => true 195 | Integer :value, :index => true 196 | end 197 | ``` 198 | 199 | The corresponding `model_schema` block would be: 200 | 201 | ```rb 202 | class Item < Sequel::Model(:items) 203 | model_schema do 204 | String :name 205 | Integer :value 206 | 207 | index :name, :unique => true 208 | index :value 209 | end 210 | end 211 | ``` 212 | 213 | You have to separate the columns from the indexes, since the schema dumper 214 | reads them independently of one another. 215 | 216 | - It relies on Sequel's [schema dumper extension](http://sequel.jeremyevans.net/rdoc/files/doc/migration_rdoc.html#label-Dumping+the+current+schema+as+a+migration) 217 | to read your table's schema. The schema dumper doesn't read constraints, 218 | triggers, special index types (e.g. gin, gist) or partial indexes; you'll 219 | have to omit these from your `model_schema` block. 220 | 221 | - It doesn't handle all type aliasing. For instance, the Postgres types 222 | `character varying(255)[]` and `varchar(255)[]` are equivalent, but 223 | ModelSchema is unaware of this. In turn, you might see this error message: 224 | 225 | ``` 226 | Table complex has mismatched columns: 227 | 228 | actual: column :advisors, "character varying(255)[]", :null=>false 229 | expected: column :advisors, "varchar(255)[]", :null=>false 230 | ``` 231 | 232 | In the above case, you'll need to change `varchar(255)[]` to `character 233 | varying(255)[]` in your `model_schema` block to fix the issue. 234 | 235 | A similar problem occurs with `numeric(x, 0)` and `numeric(x)`, where x is an 236 | integer; they are equivalent in Postgres, but ModelSchema doesn't know this. 237 | 238 | ## Support for ActiveRecord 239 | Currently, ModelSchema only works with [Sequel](https://github.com/jeremyevans/sequel/). 240 | If you'd like something similar for other Ruby ORMs, like ActiveRecord, please 241 | express your interest in [this issue](https://github.com/karthikv/model_schema/issues/1). 242 | 243 | ## Development and Contributing 244 | After cloning this repository, execute `bundle` to install dependencies. You 245 | may run tests with `rake test`, and open up a REPL using `bin/repl`. 246 | 247 | Note that tests require access to a Postgres database. Set the environment 248 | variable `DB_URL` to a Postgres connection string (e.g. 249 | `postgres://localhost:5432/model_schema`) prior to running tests. See 250 | [connecting to a database](http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html) 251 | for information about connection strings. 252 | 253 | To install this gem onto your local machine, run `bundle exec rake install`. 254 | 255 | Any bug reports and pull requests are welcome. 256 | 257 | ## License 258 | See the [LICENSE.txt](LICENSE.txt) file. 259 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require 'bundler/gem_tasks' 2 | require 'rake/testtask' 3 | 4 | Rake::TestTask.new(:test) do |t| 5 | t.libs << 'test' 6 | t.libs << 'lib' 7 | t.test_files = FileList['test/**/*_test.rb'] 8 | end 9 | 10 | task :default => :test 11 | -------------------------------------------------------------------------------- /bin/dump_model_schema: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | require 'bundler/setup' 3 | require 'model_schema' 4 | 5 | ModelSchema::Dumper.run(ARGV) 6 | -------------------------------------------------------------------------------- /bin/repl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | require 'bundler/setup' 3 | require 'model_schema' 4 | 5 | require 'pry' 6 | Pry.start 7 | -------------------------------------------------------------------------------- /lib/model_schema.rb: -------------------------------------------------------------------------------- 1 | module ModelSchema; end 2 | 3 | require 'model_schema/version' 4 | require 'model_schema/constants' 5 | require 'model_schema/schema_error' 6 | require 'model_schema/plugin' 7 | require 'model_schema/dumper' 8 | -------------------------------------------------------------------------------- /lib/model_schema/constants.rb: -------------------------------------------------------------------------------- 1 | module ModelSchema 2 | # ENV variable name to disable schema checks. 3 | DISABLE_MODEL_SCHEMA_KEY = 'DISABLE_MODEL_SCHEMA' 4 | 5 | # field types representing table columns and table indexes 6 | FIELD_COLUMNS = :columns 7 | FIELD_INDEXES = :indexes 8 | FIELDS = [FIELD_COLUMNS, FIELD_INDEXES] 9 | 10 | # default column parameters 11 | DEFAULT_COL = { 12 | :name => nil, 13 | :type => nil, 14 | :collate => nil, 15 | :default => nil, 16 | :deferrable => nil, 17 | :index => nil, 18 | :key => [:id], 19 | :null => nil, 20 | :on_delete => :no_action, 21 | :on_update => :no_action, 22 | :primary_key => nil, 23 | :primary_key_constraint_name => nil, 24 | :unique => nil, 25 | :unique_constraint_name => nil, 26 | :serial => nil, 27 | :table => nil, 28 | :text => nil, 29 | :fixed => nil, 30 | :size => nil, 31 | :only_time => nil, 32 | } 33 | 34 | # default index parameters 35 | DEFAULT_INDEX = { 36 | :columns => nil, 37 | :name => nil, 38 | :type => nil, 39 | :unique => nil, 40 | :where => nil, 41 | } 42 | end 43 | -------------------------------------------------------------------------------- /lib/model_schema/dumper.rb: -------------------------------------------------------------------------------- 1 | require 'optparse' 2 | 3 | SEQUEL_MODEL_REGEX = /class\s+.*?<\s+Sequel::Model\((.*?)\)/ 4 | LEADING_WHITESPACE_REGEX = /^\s*/ 5 | 6 | module ModelSchema 7 | module Dumper 8 | # Parses options and then dumps the model schema. 9 | def self.run(args) 10 | opts = {} 11 | opts[:tabbing] = 2 12 | 13 | parser = OptionParser.new do |p| 14 | p.banner = 'Usage: dump_model_schema [options] model_file [model_file ...]' 15 | p.separator "\nDumps a valid model_schema block in each given model_file.\n\n" 16 | 17 | p.on('-c', '--connection CONNECTION', 18 | 'Connection string for database') do |connection| 19 | opts[:connection] = connection 20 | end 21 | 22 | p.on('-t', '--tabbing TABBING', Integer, 23 | 'Number of spaces for tabbing, or 0 for hard tabs') do |tabbing| 24 | opts[:tabbing] = tabbing 25 | end 26 | 27 | p.on('-v', '--version', 'Print version') do 28 | puts ModelSchema::VERSION 29 | exit 30 | end 31 | 32 | p.on('-h', '--help', 'Print help') do 33 | puts parser 34 | exit 35 | end 36 | end 37 | 38 | model_files = parser.parse(args) 39 | 40 | # model and connection are required 41 | abort 'Must provide at least one model file.' if model_files.empty? 42 | abort 'Must provide a connection string with -c or --connection.' if !opts[:connection] 43 | 44 | db = Sequel.connect(opts[:connection]) 45 | db.extension(:schema_dumper) 46 | 47 | if db.is_a?(Sequel::Postgres::Database) 48 | # include all Postgres type extensions so schema dumps are accurate 49 | db.extension(:pg_array, :pg_enum, :pg_hstore, :pg_inet, :pg_json, 50 | :pg_range, :pg_row) 51 | end 52 | 53 | had_error = false 54 | model_files.each do |path| 55 | begin 56 | dump_model_schema(db, path, opts) 57 | rescue StandardError, SystemExit => error 58 | # SystemExit error messages are already printed by abort() 59 | $stderr.puts error.message if error.is_a?(StandardError) 60 | had_error = true 61 | end 62 | end 63 | 64 | exit 1 if had_error 65 | end 66 | 67 | # Dumps a valid model_schema into the given file path. Accepts options as 68 | # per the OptionParser above. 69 | def self.dump_model_schema(db, path, opts) 70 | model = parse_model_file(path) 71 | abort "In #{path}, couldn't find class that extends Sequel::Model" if !model 72 | 73 | klass = Class.new(Sequel::Model(model[:table_name])) 74 | klass.db = db 75 | klass.plugin(ModelSchema::Plugin) 76 | 77 | # dump table generator given by model_schema 78 | generator = klass.send(:table_generator) 79 | commands = [generator.dump_columns, generator.dump_constraints, 80 | generator.dump_indexes].reject{|s| s == ''}.join("\n\n") 81 | 82 | # account for indentation 83 | tab = opts[:tabbing] == 0 ? "\t" : ' ' * opts[:tabbing] 84 | schema_indentation = model[:indentation] + tab 85 | command_indentation = schema_indentation + tab 86 | 87 | commands = commands.lines.map {|l| l == "\n" ? l : command_indentation + l}.join 88 | commands = commands.gsub('=>', ' => ') 89 | 90 | dump_lines = ["#{schema_indentation}model_schema do\n", 91 | "#{commands}\n", 92 | "#{schema_indentation}end\n"] 93 | 94 | lines = model[:lines_before] + dump_lines + model[:lines_after] 95 | File.write(path, lines.join) 96 | end 97 | 98 | # Parses the model file at the given path, returning a hash of the form: 99 | # 100 | # :table_name => the model table name 101 | # :lines_before => an array of lines before the expected model schema dump 102 | # :lines_after => an array of lines after the expected model schema dump 103 | # :indentation => the indentation (leading whitespace) of the model class 104 | # 105 | # Returns nil if the file couldn't be parsed. 106 | def self.parse_model_file(path) 107 | lines = File.read(path).lines 108 | 109 | lines.each_with_index do |line, index| 110 | match = SEQUEL_MODEL_REGEX.match(line) 111 | 112 | if match 113 | # extract table name as symbol 114 | table_name = match[1] 115 | if table_name[0] == ':' 116 | table_name = table_name[1..-1].to_sym 117 | else 118 | abort "In #{path}, can't find a symbol table name in line: #{line}" 119 | end 120 | 121 | # indentation for model_schema block 122 | indentation = LEADING_WHITESPACE_REGEX.match(line)[0] 123 | 124 | return { 125 | :table_name => table_name.to_sym, 126 | :lines_before => lines[0..index], 127 | :lines_after => lines[(index + 1)..-1], 128 | :indentation => indentation, 129 | } 130 | end 131 | end 132 | 133 | nil 134 | end 135 | end 136 | end 137 | -------------------------------------------------------------------------------- /lib/model_schema/plugin.rb: -------------------------------------------------------------------------------- 1 | require 'sequel' 2 | 3 | module ModelSchema 4 | # Allows you to define an expected schema for a Sequel::Model class and fail 5 | # if that schema is not met. 6 | module Plugin 7 | module ClassMethods 8 | # Checks if the model's table schema matches the schema specified by the 9 | # given block. Raises a SchemaError if this isn't the case. 10 | # 11 | # options: 12 | # :disable => true to disable schema checks; 13 | # you may also set the ENV variable DISABLE_MODEL_SCHEMA=1 14 | # :no_indexes => true to disable index checks 15 | def model_schema(options={}, &block) 16 | return if ENV[DISABLE_MODEL_SCHEMA_KEY] == '1' || options[:disable] 17 | db.extension(:schema_dumper) 18 | 19 | # table generators are Sequel's way of representing schemas 20 | db_generator = table_generator 21 | exp_generator = db.create_table_generator(&block) 22 | 23 | schema_errors = check_all(FIELD_COLUMNS, db_generator, exp_generator) 24 | if !options[:no_indexes] 25 | schema_errors += check_all(FIELD_INDEXES, db_generator, exp_generator) 26 | end 27 | 28 | raise SchemaError.new(table_name, schema_errors) if schema_errors.length > 0 29 | end 30 | 31 | private 32 | 33 | # Returns the table generator representing this table. 34 | def table_generator 35 | begin 36 | db_generator_explicit = db.send(:dump_table_generator, table_name, 37 | :same_db => true) 38 | db_generator_generic = db.send(:dump_table_generator, table_name) 39 | rescue Sequel::DatabaseError => error 40 | if error.message.include?('PG::UndefinedTable:') 41 | fail NameError, "Table #{table_name} doesn't exist." 42 | end 43 | end 44 | 45 | # db_generator_explicit contains explicit string types for each field, 46 | # specific to the current database; db_generator_generic contains ruby 47 | # types for each field. When there's no corresponding ruby type, 48 | # db_generator_generic defaults to the String type. We'd like to 49 | # combine db_generator_explicit and db_generator_generic into one 50 | # generator, where ruby types are used if they are accurate. If there 51 | # is no accurate ruby type, we use the explicit database type. This 52 | # gives us cleaner column dumps, as ruby types have a better, more 53 | # generic interface (e.g. `String :col_name` as opposed to 54 | # `column :col_name, 'varchar(255)`). 55 | 56 | # start with db_generator_generic, and correct as need be 57 | db_generator = db_generator_generic.dup 58 | 59 | # avoid using Sequel::Model.db_schema because it has odd caching 60 | # behavior across classes that breaks tests 61 | db.schema(table_name).each do |name, col_schema| 62 | type_hash = db.column_schema_to_ruby_type(col_schema) 63 | 64 | if type_hash == {:type => String} 65 | # There's no corresponding ruby type, as per: 66 | # 68 | # Copy over the column from db_generator_explicit. 69 | index = db_generator.columns.find_index {|c| c[:name] == name} 70 | col = db_generator_explicit.columns.find {|c| c[:name] == name} 71 | db_generator.columns[index] = col 72 | end 73 | end 74 | 75 | db_generator 76 | end 77 | 78 | # Check if db_generator and exp_generator match for the given field 79 | # (FIELD_COLUMNS for columns or FIELD_INDEXES for indexes). 80 | def check_all(field, db_generator, exp_generator) 81 | # To find an accurate diff, we perform two passes on exp_array. In the 82 | # first pass, we find perfect matches between exp_array and db_array, 83 | # deleting the corresponding elements. In the second pass, for each 84 | # exp_elem in exp_array, we find the closest db_elem in db_array that 85 | # matches it. We then add a mismatch diff between db_elem and exp_elem 86 | # and remove db_elem from db_array. If no db_elem is deemed close 87 | # enough, we add a missing diff for exp_elem. Finally, we add an extra 88 | # diff for each remaining db_elem in db_array. 89 | 90 | # don't modify original arrays 91 | db_array = db_generator.send(field).dup 92 | exp_array = exp_generator.send(field).dup 93 | 94 | # first pass: remove perfect matches 95 | exp_array.select! do |exp_elem| 96 | diffs = db_array.map do |db_elem| 97 | check_single(field, :db_generator => db_generator, 98 | :exp_generator => exp_generator, 99 | :db_elem => db_elem, 100 | :exp_elem => exp_elem) 101 | end 102 | 103 | index = diffs.find_index(nil) 104 | if index 105 | # found perfect match; delete elem so it won't be matched again 106 | db_array.delete_at(index) 107 | false # we've accounted for this element 108 | else 109 | true # we still need to account for this element 110 | end 111 | end 112 | 113 | schema_diffs = [] 114 | 115 | # second pass: find diffs 116 | exp_array.each do |exp_elem| 117 | index = find_close_match(field, exp_elem, db_array) 118 | 119 | if index 120 | # add mismatch diff between exp_elem and db_array[index] 121 | schema_diffs << check_single(field, :db_generator => db_generator, 122 | :exp_generator => exp_generator, 123 | :db_elem => db_array[index], 124 | :exp_elem => exp_elem) 125 | db_array.delete_at(index) 126 | else 127 | # add missing diff, since no db_elem is deemed close enough 128 | schema_diffs << {:field => field, 129 | :type => SchemaError::TYPE_MISSING, 130 | :generator => exp_generator, 131 | :elem => exp_elem} 132 | end 133 | end 134 | 135 | # because we deleted as we went on, db_array holds extra elements 136 | db_array.each do |db_elem| 137 | schema_diffs << {:field => field, 138 | :type => SchemaError::TYPE_EXTRA, 139 | :generator => db_generator, 140 | :elem => db_elem} 141 | end 142 | 143 | schema_diffs 144 | end 145 | 146 | # Returns the index of an element in db_array that closely matches exp_elem, 147 | # or nil if no such element exists. 148 | def find_close_match(field, exp_elem, db_array) 149 | case field 150 | when FIELD_COLUMNS 151 | db_array.find_index {|e| e[:name] == exp_elem[:name]} 152 | when FIELD_INDEXES 153 | db_array.find_index do |e| 154 | e[:name] == exp_elem[:name] || e[:columns] == exp_elem[:columns] 155 | end 156 | end 157 | end 158 | 159 | # Check if the given database element matches the expected element. 160 | # 161 | # field: FIELD_COLUMNS for columns or FIELD_INDEXES for indexes 162 | # opts: 163 | # :db_generator => db table generator 164 | # :exp_generator => expected table generator 165 | # :db_elem => column, constraint, or index from db_generator 166 | # :exp_elem => column, constraint, or index from exp_generator 167 | def check_single(field, opts) 168 | db_generator, exp_generator = opts.values_at(:db_generator, :exp_generator) 169 | db_elem, exp_elem = opts.values_at(:db_elem, :exp_elem) 170 | 171 | error = {:field => field, 172 | :type => SchemaError::TYPE_MISMATCH, 173 | :db_generator => db_generator, 174 | :exp_generator => exp_generator, 175 | :db_elem => db_elem, 176 | :exp_elem => exp_elem} 177 | 178 | # db_elem and exp_elem now have the same keys; compare then 179 | case field 180 | when FIELD_COLUMNS 181 | db_elem_defaults = DEFAULT_COL.merge(db_elem) 182 | exp_elem_defaults = DEFAULT_COL.merge(exp_elem) 183 | return error if db_elem_defaults.length != exp_elem_defaults.length 184 | 185 | type_literal = db.method(:type_literal) 186 | # already accounted for in type check 187 | keys_accounted_for = [:text, :fixed, :size, :serial] 188 | 189 | match = db_elem_defaults.all? do |key, value| 190 | if key == :type 191 | # types could either be strings or ruby types; normalize them 192 | db_type = type_literal.call(db_elem_defaults).to_s 193 | exp_type = type_literal.call(exp_elem_defaults).to_s 194 | db_type == exp_type 195 | elsif keys_accounted_for.include?(key) 196 | true 197 | else 198 | value == exp_elem_defaults[key] 199 | end 200 | end 201 | 202 | when FIELD_INDEXES 203 | db_elem_defaults = DEFAULT_INDEX.merge(db_elem) 204 | exp_elem_defaults = DEFAULT_INDEX.merge(exp_elem) 205 | return error if db_elem_defaults.length != exp_elem_defaults.length 206 | 207 | # if no index name is specified, accept any name 208 | db_elem_defaults.delete(:name) if !exp_elem_defaults[:name] 209 | match = db_elem_defaults.all? {|key, value| value == exp_elem_defaults[key]} 210 | end 211 | 212 | match ? nil : error 213 | end 214 | end 215 | end 216 | end 217 | -------------------------------------------------------------------------------- /lib/model_schema/schema_error.rb: -------------------------------------------------------------------------------- 1 | module ModelSchema 2 | # Tracks differences between the expected schema and database table schema. 3 | class SchemaError < StandardError 4 | TYPE_EXTRA = :extra 5 | TYPE_MISSING = :missing 6 | TYPE_MISMATCH = :mismatch 7 | 8 | attr_reader :schema_diffs 9 | 10 | # Creates a SchemaError for the given table with an array of schema 11 | # differences. Each element of schema_diffs should be a hash of the 12 | # following form: 13 | # 14 | # :field => if a column is different, use FIELD_COLUMNS; 15 | # if an index is different, use FIELD_INDEXES 16 | # :type => if there's an extra column/index, use TYPE_EXTRA; 17 | # if there's a missing column/index, use TYPE_MISSING; 18 | # if there's a mismatched column/index, use TYPE_MISMATCH 19 | # 20 | # For TYPE_EXTRA and TYPE_MISSING: 21 | # :generator => the table generator that contains the extra/missing index/column 22 | # :elem => the missing index/column, as a hash from the table generator 23 | # 24 | # For TYPE_MISMATCH: 25 | # :db_generator => the db table generator 26 | # :exp_generator => the expected table generator 27 | # :db_elem => the index/column in the db table generator as a hash 28 | # :exp_elem => the index/column in the exp table generator as a hash 29 | def initialize(table_name, schema_diffs) 30 | @table_name = table_name 31 | @schema_diffs = schema_diffs 32 | end 33 | 34 | # Dumps a single column/index from the generator to its string representation. 35 | # 36 | # field: FIELD_COLUMNS for a column or FIELD_INDEXES for an index 37 | # generator: the table generator 38 | # elem: the index/column in the generator as a hash 39 | def dump_single(field, generator, elem) 40 | array = generator.send(field) 41 | index = array.find_index(elem) 42 | fail ArgumentError, "#{elem.inspect} not part of #{array.inspect}" if !index 43 | 44 | lines = generator.send(:"dump_#{field}").lines.map(&:strip) 45 | lines[index] 46 | end 47 | 48 | # Returns the diffs in schema_diffs that have the given field and type. 49 | def diffs_by_field_type(field, type) 50 | @schema_diffs.select {|diff| diff[:field] == field && diff[:type] == type} 51 | end 52 | 53 | # Dumps all diffs that have the given field and are of TYPE_EXTRA. 54 | def dump_extra_diffs(field) 55 | extra_diffs = diffs_by_field_type(field, TYPE_EXTRA) 56 | 57 | if extra_diffs.length > 0 58 | header = "Table #{@table_name} has extra #{field}:\n" 59 | diff_str = extra_diffs.map do |diff| 60 | dump_single(field, diff[:generator], diff[:elem]) 61 | end.join("\n\t") 62 | 63 | "#{header}\n\t#{diff_str}\n" 64 | end 65 | end 66 | 67 | # Dumps all diffs that have the given field and are of TYPE_MISSING. 68 | def dump_missing_diffs(field) 69 | missing_diffs = diffs_by_field_type(field, TYPE_MISSING) 70 | 71 | if missing_diffs.length > 0 72 | header = "Table #{@table_name} is missing #{field}:\n" 73 | diff_str = missing_diffs.map do |diff| 74 | dump_single(field, diff[:generator], diff[:elem]) 75 | end.join("\n\t") 76 | 77 | "#{header}\n\t#{diff_str}\n" 78 | end 79 | end 80 | 81 | # Dumps all diffs that have the given field and are of TYPE_MISMATCH. 82 | def dump_mismatch_diffs(field) 83 | mismatch_diffs = diffs_by_field_type(field, TYPE_MISMATCH) 84 | 85 | if mismatch_diffs.length > 0 86 | header = "Table #{@table_name} has mismatched #{field}:\n" 87 | diff_str = mismatch_diffs.map do |diff| 88 | "actual: #{dump_single(field, diff[:db_generator], diff[:db_elem])}\n\t" + 89 | "expected: #{dump_single(field, diff[:exp_generator], diff[:exp_elem])}" 90 | end.join("\n\n\t") 91 | 92 | "#{header}\n\t#{diff_str}\n" 93 | end 94 | end 95 | 96 | # Combines all dumps into one cohesive error message. 97 | def to_s 98 | parts = FIELDS.flat_map do |field| 99 | [dump_extra_diffs(field), 100 | dump_missing_diffs(field), 101 | dump_mismatch_diffs(field)] 102 | end 103 | 104 | [ 105 | "Table #{@table_name} does not match the expected schema.\n\n", 106 | parts.compact.join("\n"), 107 | "\nYou may disable schema checks by passing :disable => true to model_", 108 | "schema or by setting the ENV variable #{DISABLE_MODEL_SCHEMA_KEY}=1.\n" 109 | ].join 110 | end 111 | end 112 | end 113 | -------------------------------------------------------------------------------- /lib/model_schema/version.rb: -------------------------------------------------------------------------------- 1 | module ModelSchema 2 | VERSION = '0.1.3' 3 | end 4 | -------------------------------------------------------------------------------- /model_schema.gemspec: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | lib = File.expand_path('../lib', __FILE__) 3 | $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) 4 | require 'model_schema/version' 5 | 6 | Gem::Specification.new do |spec| 7 | spec.name = 'model_schema' 8 | spec.version = ModelSchema::VERSION 9 | spec.authors = ['Karthik Viswanathan'] 10 | spec.email = ['karthik.ksv@gmail.com'] 11 | 12 | spec.summary = %(Enforced, Annotated Schema for Ruby Sequel Models) 13 | spec.description = %(Annotate a Sequel Model with its expected schema 14 | and immediately identify inconsistencies.).gsub(/\n\s+/, '') 15 | spec.homepage = 'https://github.com/karthikv/model_schema' 16 | spec.license = 'MIT' 17 | 18 | spec.files = `git ls-files -z`.split("\x0").reject {|f| f.match(%r{^(test|spec|features)/})} 19 | spec.bindir = 'bin' 20 | spec.executables << 'dump_model_schema' 21 | spec.require_paths = ['lib'] 22 | 23 | spec.add_development_dependency('bundler', '~> 1.10') 24 | spec.add_development_dependency('rake', '~> 10.0') 25 | spec.add_development_dependency('minitest') 26 | spec.add_development_dependency('minitest-hooks') 27 | spec.add_development_dependency('mocha') 28 | spec.add_development_dependency('pg') 29 | spec.add_development_dependency('pry') 30 | spec.add_development_dependency('awesome_print') 31 | 32 | spec.add_runtime_dependency('sequel') 33 | end 34 | -------------------------------------------------------------------------------- /test/model_schema/dumper_test.rb: -------------------------------------------------------------------------------- 1 | require 'test_helper' 2 | 3 | class DumperTest < BaseTest 4 | def before_all 5 | @db_url = ENV['DB_URL'] 6 | @db = Sequel::Database.connect(@db_url) 7 | end 8 | 9 | def around 10 | @db.transaction(:rollback => :always, :auto_savepoint => true) {super} 11 | end 12 | 13 | def dumper 14 | ModelSchema::Dumper 15 | end 16 | 17 | def with_captured_stderr 18 | begin 19 | old_stderr = $stderr 20 | $stderr = StringIO.new('', 'w') 21 | yield 22 | $stderr.string 23 | ensure 24 | $stderr = old_stderr 25 | end 26 | end 27 | 28 | def test_no_model 29 | error = nil 30 | stderr = with_captured_stderr do 31 | begin 32 | dumper.run(['-c', @db_url]) 33 | rescue SystemExit => e 34 | error = e 35 | end 36 | end 37 | 38 | refute_nil error 39 | assert_includes stderr, 'provide at least one model file' 40 | end 41 | 42 | def test_no_connection 43 | error = nil 44 | stderr = with_captured_stderr do 45 | begin 46 | dumper.run(['some-model-file']) 47 | rescue SystemExit => e 48 | error = e 49 | end 50 | end 51 | 52 | refute_nil error 53 | assert_includes stderr, 'provide a connection' 54 | end 55 | 56 | def write_model(table_name) 57 | path = "model_schema/#{table_name}" 58 | contents = ['# A simple model for a simple table', 59 | 'module SomeApp', 60 | ' module Models', 61 | " class Item < Sequel::Model(:#{table_name})", 62 | ' belongs_to :other, :class => :Other', 63 | ' end', 64 | ' end', 65 | 'end'].join("\n") 66 | File.stubs(:read).with(path).returns(contents) 67 | path 68 | end 69 | 70 | def expected_simple_model(table_name, tab) 71 | ['# A simple model for a simple table', 72 | 'module SomeApp', 73 | ' module Models', 74 | " class Item < Sequel::Model(:#{table_name})", 75 | " #{tab}model_schema do", 76 | " #{tab * 2}String :name, :size => 50", 77 | " #{tab * 2}Integer :value, :null => false", 78 | '', 79 | " #{tab * 2}index [:name], :unique => true", 80 | " #{tab}end", 81 | ' belongs_to :other, :class => :Other', 82 | ' end', 83 | ' end', 84 | 'end'].join("\n") 85 | end 86 | 87 | def expected_complex_model(table_name) 88 | ['# A simple model for a simple table', 89 | 'module SomeApp', 90 | ' module Models', 91 | " class Item < Sequel::Model(:#{table_name})", 92 | " model_schema do", 93 | " primary_key :id", 94 | " foreign_key :other_id, :others, :null => false, :key => [:id], :on_delete => :cascade", 95 | " String :name, :text => true", 96 | " String :location, :size => 50, :fixed => true", 97 | " String :legal_name, :size => 200", 98 | " String :advisor, :text => true", 99 | " BigDecimal :amount, :size => [10, 0]", 100 | " Integer :value, :null => false", 101 | " column :advisors, \"character varying(255)[]\", :null => false", 102 | " column :interests, \"text[]\", :null => false", 103 | " TrueClass :is_right", 104 | " DateTime :created_at, :null => false", 105 | " Time :updated_at, :only_time => true", 106 | '', 107 | " index [:other_id, :name]", 108 | " index [:value], :name => :complex_value_key, :unique => true", 109 | " index [:interests], :name => :int_index", 110 | " end", 111 | ' belongs_to :other, :class => :Other', 112 | ' end', 113 | ' end', 114 | 'end'].join("\n") 115 | end 116 | 117 | def set_up_dump_simple(tab) 118 | simple_table = create_simple_table(@db) 119 | path = write_model(simple_table) 120 | contents = expected_simple_model(simple_table, tab) 121 | 122 | Sequel.stubs(:connect).with(@db_url).returns(@db) 123 | File.expects(:write).with do |p, c| 124 | assert_equal path, p 125 | assert_equal contents, c 126 | true 127 | end 128 | 129 | path 130 | end 131 | 132 | def test_dump_simple 133 | path = set_up_dump_simple(' ') 134 | dumper.run(['-c', @db_url, path]) 135 | end 136 | 137 | def test_dump_simple_four_spaces 138 | path = set_up_dump_simple(' ') 139 | dumper.run(['-c', @db_url, '-t', '4', path]) 140 | end 141 | 142 | def test_dump_simple_hard_tab 143 | path = set_up_dump_simple("\t") 144 | dumper.run(['-c', @db_url, '-t', '0', path]) 145 | end 146 | 147 | def test_dump_complex 148 | complex_table = create_complex_table(@db) 149 | path = write_model(complex_table) 150 | contents = expected_complex_model(complex_table) 151 | 152 | Sequel.stubs(:connect).with(@db_url).returns(@db) 153 | File.expects(:write).with do |p, c| 154 | assert_equal path, p 155 | assert_equal contents, c 156 | true 157 | end 158 | 159 | dumper.run(['-c', @db_url, path]) 160 | end 161 | 162 | def test_dump_multiple 163 | simple_table = create_simple_table(@db) 164 | complex_table = create_complex_table(@db) 165 | 166 | simple_path = write_model(simple_table) 167 | complex_path = write_model(complex_table) 168 | 169 | simple_contents = expected_simple_model(simple_table, ' ') 170 | complex_contents = expected_complex_model(complex_table) 171 | 172 | Sequel.stubs(:connect).with(@db_url).returns(@db) 173 | File.expects(:write).with(simple_path, simple_contents) 174 | File.expects(:write).with(complex_path, complex_contents) 175 | 176 | dumper.run(['-c', @db_url, simple_path, complex_path]) 177 | end 178 | 179 | def test_dump_multiple_abort_error 180 | path = set_up_dump_simple(' ') 181 | other_path = 'model_schema/other_path' 182 | File.stubs(:read).with(other_path).returns("Some bogus\n\tmodel\n\tfile") 183 | 184 | error = nil 185 | stderr = with_captured_stderr do 186 | begin 187 | dumper.run(['-c', @db_url, other_path, path]) 188 | rescue SystemExit => e 189 | error = e 190 | end 191 | end 192 | 193 | refute_nil error 194 | assert_includes stderr, "couldn't find class that extends Sequel::Model" 195 | end 196 | 197 | def test_dump_multiple_standard_error 198 | path = set_up_dump_simple(' ') 199 | other_path = 'model_schema/non_existent_path' 200 | File.stubs(:read).with(other_path).raises(Errno::ENOENT) 201 | 202 | error = nil 203 | stderr = with_captured_stderr do 204 | begin 205 | dumper.run(['-c', @db_url, other_path, path]) 206 | rescue SystemExit => e 207 | error = e 208 | end 209 | end 210 | 211 | refute_nil error 212 | assert_includes stderr, "No such file" 213 | end 214 | end 215 | -------------------------------------------------------------------------------- /test/model_schema/plugin_test.rb: -------------------------------------------------------------------------------- 1 | require 'test_helper' 2 | 3 | module Sequel 4 | module Schema 5 | class CreateTableGenerator 6 | def ==(other) 7 | other_db = other.instance_variable_get(:@db) 8 | 9 | (columns == other.columns && 10 | constraints == other.constraints && 11 | indexes == other.indexes && 12 | @db.serial_primary_key_options == other_db.serial_primary_key_options) 13 | end 14 | end 15 | end 16 | end 17 | 18 | class PluginTest < BaseTest 19 | def before_all 20 | @db = Sequel::Database.connect(ENV['DB_URL']) 21 | Sequel::Model.plugin(ModelSchema::Plugin) 22 | end 23 | 24 | def around 25 | @db.transaction(:rollback => :always, :auto_savepoint => true) {super} 26 | end 27 | 28 | def test_simple_schema 29 | db = @db # alias for use in class context below 30 | simple_table = create_simple_table(db) 31 | 32 | Class.new(Sequel::Model(simple_table)) do 33 | self.db = db 34 | model_schema do 35 | String :name, :size => 50 36 | Integer :value, :null => false 37 | index :name, :unique => true 38 | end 39 | end 40 | end 41 | 42 | def test_simple_schema_type_aliases 43 | db = @db # alias for use in class context below 44 | simple_table = create_simple_table(db) 45 | 46 | Class.new(Sequel::Model(simple_table)) do 47 | self.db = db 48 | model_schema do 49 | varchar :name, :size => 50 50 | column :value, 'integer', :null => false 51 | index :name, :unique => true 52 | end 53 | end 54 | end 55 | 56 | def test_simple_schema_no_indexes 57 | db = @db # alias for use in class context below 58 | simple_table = create_simple_table(db) 59 | 60 | Class.new(Sequel::Model(simple_table)) do 61 | self.db = db 62 | model_schema(:no_indexes => true) do 63 | varchar :name, :size => 50 64 | column :value, 'integer', :null => false 65 | end 66 | end 67 | end 68 | 69 | def test_disable_simple_schema 70 | db = @db # alias for use in class context below 71 | simple_table = create_simple_table(db) 72 | 73 | Class.new(Sequel::Model(simple_table)) do 74 | self.db = db 75 | model_schema(:disable => true) {} 76 | end 77 | end 78 | 79 | def test_disable_simple_schema_env 80 | ENV[ModelSchema::DISABLE_MODEL_SCHEMA_KEY] = '1' 81 | db = @db # alias for use in class context below 82 | simple_table = create_simple_table(db) 83 | 84 | Class.new(Sequel::Model(simple_table)) do 85 | self.db = db 86 | model_schema {} 87 | end 88 | ENV.delete(ModelSchema::DISABLE_MODEL_SCHEMA_KEY) 89 | end 90 | 91 | def test_simple_schema_extra_col 92 | db = @db # alias for use in class context below 93 | simple_table = create_simple_table(db) 94 | 95 | begin 96 | Class.new(Sequel::Model(simple_table)) do 97 | self.db = db 98 | model_schema {} 99 | end 100 | rescue error_class => error 101 | klass = Class.new(Sequel::Model(simple_table)) 102 | klass.db = db 103 | generator = klass.send(:table_generator) 104 | 105 | diffs = [create_extra_diff(field_columns, generator, :name => :name), 106 | create_extra_diff(field_columns, generator, :name => :value), 107 | create_extra_diff(field_indexes, generator, :columns => [:name])] 108 | 109 | diffs.each {|d| assert_includes error.schema_diffs, d} 110 | assert_equal diffs.length, error.schema_diffs.length 111 | else 112 | flunk 'Extra column not detected' 113 | end 114 | end 115 | 116 | def test_simple_schema_missing_col 117 | db = @db # alias for use in class context below 118 | simple_table = create_simple_table(db) 119 | 120 | table_proc = proc do 121 | String :name, :size => 50 122 | Integer :value, :null => false 123 | index :name, :unique => true 124 | 125 | TrueClass :is_valid, :default => true 126 | DateTime :completed_at 127 | index :value, :where => {:is_valid => true} 128 | end 129 | generator = db.create_table_generator(&table_proc) 130 | 131 | begin 132 | Class.new(Sequel::Model(simple_table)) do 133 | self.db = db 134 | model_schema(&table_proc) 135 | end 136 | rescue error_class => error 137 | diffs = [create_missing_diff(field_columns, generator, :name => :is_valid), 138 | create_missing_diff(field_columns, generator, :name => :completed_at), 139 | create_missing_diff(field_indexes, generator, :columns => [:value])] 140 | 141 | diffs.each {|d| assert_includes error.schema_diffs, d} 142 | assert_equal diffs.length, error.schema_diffs.length 143 | else 144 | flunk 'Missing column not detected' 145 | end 146 | end 147 | 148 | def test_simple_schema_mismatch_col 149 | db = @db # alias for use in class context below 150 | simple_table = create_simple_table(db) 151 | 152 | table_proc = proc do 153 | String :name, :size => 50, :default => true 154 | primary_key :value 155 | index :name 156 | end 157 | exp_generator = db.create_table_generator(&table_proc) 158 | 159 | begin 160 | Class.new(Sequel::Model(simple_table)) do 161 | self.db = db 162 | model_schema(&table_proc) 163 | end 164 | rescue error_class => error 165 | klass = Class.new(Sequel::Model(simple_table)) 166 | klass.db = db 167 | db_generator = klass.send(:table_generator) 168 | 169 | diffs = [create_mismatch_diff(field_columns, db_generator, exp_generator, 170 | :name => :name), 171 | create_mismatch_diff(field_columns, db_generator, exp_generator, 172 | :name => :value), 173 | create_mismatch_diff(field_indexes, db_generator, exp_generator, 174 | :columns => [:name])] 175 | 176 | diffs.each {|d| assert_includes error.schema_diffs, d} 177 | assert_equal diffs.length, error.schema_diffs.length 178 | else 179 | flunk 'Extra column not detected' 180 | end 181 | end 182 | 183 | def test_complex_schema 184 | db = @db # alias for use in class context below 185 | complex_table = create_complex_table(db) 186 | 187 | Class.new(Sequel::Model(complex_table)) do 188 | self.db = db 189 | model_schema do 190 | primary_key :id 191 | foreign_key :other_id, :others, :null => false, :on_delete => :cascade 192 | 193 | String :name 194 | String :location, :fixed => true, :size => 50 195 | String :legal_name, :size => 200 196 | String :advisor, :text => true 197 | 198 | BigDecimal :amount, :size => [10, 0] 199 | Integer :value, :null => false 200 | 201 | column :advisors, 'character varying(255)[]', :null => false 202 | column :interests, 'text[]', :null => false 203 | 204 | Time :created_at, :null => false 205 | Time :updated_at, :only_time => true 206 | 207 | TrueClass :is_right 208 | 209 | index [:other_id, :name] 210 | index :value, :unique => true 211 | index :interests 212 | end 213 | end 214 | end 215 | 216 | def test_complex_schema_many_errors_integration 217 | db = @db # alias for use in class context below 218 | complex_table = create_complex_table(db) 219 | 220 | table_proc = proc do 221 | primary_key :id, :serial => true # correct; serial is implied 222 | # wrong table name 223 | foreign_key :other_id, :some_others, :null => false, :on_delete => :cascade 224 | 225 | String :name # correct 226 | String :location # missing all attributes 227 | # db has extra legal_name field 228 | String :advisor # missing :text => true, but it's equivalent for postgres 229 | Float :pi # db is missing field 230 | 231 | # wrong name; missing field + extra field 232 | BigDecimal :amt, :size => [10, 0] 233 | Integer :value, :null => false # correct 234 | 235 | column :advisors, 'text[]', :null => false # wrong type 236 | # db has extra interests field 237 | TrueClass :is_missing, :null => false # db is missing field 238 | 239 | Time :created_at, :null => true # :null => true instead of false 240 | Time :updated_at # not only_time 241 | 242 | # correct w/ equivalent type and normal default 243 | FalseClass :is_right, :default => nil 244 | 245 | index [:other_id, :name], :unique => true # incorrectly unique 246 | index [:other_id, :name] # correct index, same columns as above 247 | index :amount, :name => :int_index # same name as other index 248 | index :advisor # db is missing index 249 | 250 | # db has extra value index 251 | end 252 | exp_generator = db.create_table_generator(&table_proc) 253 | 254 | begin 255 | Class.new(Sequel::Model(complex_table)) do 256 | self.db = db 257 | model_schema(&table_proc) 258 | end 259 | rescue error_class => error 260 | klass = Class.new(Sequel::Model(complex_table)) 261 | klass.db = db 262 | 263 | db_generator = klass.send(:table_generator) 264 | complex_table_str = complex_table.to_s 265 | 266 | parts = [complex_table_str, 'extra columns', 267 | # within sub-arrays, any order is OK 268 | [dump_column(db_generator, :legal_name), 269 | dump_column(db_generator, :amount), 270 | dump_column(db_generator, :interests)], 271 | 272 | complex_table_str, 'missing columns', 273 | [dump_column(exp_generator, :pi), 274 | dump_column(exp_generator, :amt), 275 | dump_column(exp_generator, :is_missing)], 276 | 277 | complex_table_str, 'mismatched columns', 278 | [dump_column(db_generator, :other_id), 279 | dump_column(exp_generator, :other_id), 280 | dump_column(db_generator, :location), 281 | dump_column(exp_generator, :location), 282 | dump_column(db_generator, :advisors), 283 | dump_column(exp_generator, :advisors), 284 | dump_column(db_generator, :created_at), 285 | dump_column(exp_generator, :created_at), 286 | dump_column(db_generator, :updated_at), 287 | dump_column(exp_generator, :updated_at)], 288 | 289 | complex_table_str, 'extra indexes', 290 | dump_index(db_generator, :value), 291 | 292 | complex_table_str, 'missing indexes', 293 | # refers to first, incorrectly unique index 294 | [dump_index(exp_generator, [:other_id, :name]), 295 | dump_index(exp_generator, :advisor)], 296 | 297 | complex_table_str, 'mismatched indexes', 298 | [dump_index(db_generator, :interests), 299 | dump_index(exp_generator, :amount)], 300 | 301 | 'disable', ModelSchema::DISABLE_MODEL_SCHEMA_KEY, '=1'] 302 | 303 | assert_includes_with_order error.message, parts 304 | else 305 | flunk 'Numerous errors not detected' 306 | end 307 | end 308 | end 309 | -------------------------------------------------------------------------------- /test/model_schema/schema_error_test.rb: -------------------------------------------------------------------------------- 1 | require 'test_helper' 2 | 3 | class SchemaErrorTest < BaseTest 4 | def setup 5 | @db = Sequel::Database.new 6 | @db.extension(:schema_dumper) 7 | @table_name = 'schema_error_table' 8 | end 9 | 10 | def create_table_generator 11 | Sequel::Schema::CreateTableGenerator.new(@db) do 12 | primary_key :id 13 | foreign_key :organization_id, :organizations, :null => false, 14 | :key => [:id], 15 | :on_delete => :cascade 16 | 17 | Integer :type 18 | String :first_name, :text => true, :null => false 19 | column :emails, 'text[]', :null => false 20 | DateTime :created_at, :null => false 21 | DateTime :updated_at 22 | 23 | index :type 24 | index :created_at, :name => :created_at_index 25 | index [:first_name, :emails], :unique => true, :type => :gin, 26 | :where => {:type => 1} 27 | end 28 | end 29 | 30 | def test_dump_single_column 31 | generator = create_table_generator 32 | assert_equal 'Integer :type', dump_column(generator, :type) 33 | end 34 | 35 | def test_dump_single_column_with_options 36 | generator = create_table_generator 37 | assert_equal 'String :first_name, :text=>true, :null=>false', 38 | dump_column(generator, :first_name) 39 | end 40 | 41 | def test_dump_single_foreign_key 42 | generator = create_table_generator 43 | fk_str = %(foreign_key :organization_id, :organizations, :null=>false, 44 | :key=>[:id], :on_delete=>:cascade).gsub(/\n\s+/, '') 45 | assert_equal fk_str, dump_column(generator, :organization_id) 46 | end 47 | 48 | def test_dump_single_primary_key 49 | generator = create_table_generator 50 | assert_equal 'primary_key :id', dump_column(generator, :id) 51 | end 52 | 53 | def test_dump_single_custom_type 54 | generator = create_table_generator 55 | assert_equal 'column :emails, "text[]", :null=>false', 56 | dump_column(generator, :emails) 57 | end 58 | 59 | def test_dump_single_index 60 | generator = create_table_generator 61 | assert_equal 'index [:type]', dump_index(generator, :type) 62 | end 63 | 64 | def test_dump_single_index_with_options 65 | generator = create_table_generator 66 | assert_equal 'index [:created_at], :name=>:created_at_index', 67 | dump_index(generator, :created_at) 68 | end 69 | 70 | def test_dump_single_index_complex 71 | generator = create_table_generator 72 | index_str = %(index [:first_name, :emails], :unique=>true, :type=>:gin, 73 | :where=>{:type=>1}).gsub(/\n\s+/, '') 74 | assert_equal index_str, dump_index(generator, [:first_name, :emails]) 75 | end 76 | 77 | def test_col_extra 78 | generator = create_table_generator 79 | schema_diffs = [create_extra_diff(field_columns, generator, :name => :created_at), 80 | create_extra_diff(field_columns, generator, :name => :first_name), 81 | create_extra_diff(field_indexes, generator, 82 | :columns => [:type]), 83 | create_extra_diff(field_indexes, generator, 84 | :columns => [:created_at])] 85 | 86 | message = schema_error(schema_diffs).to_s 87 | parts = [@table_name, 'extra columns', 88 | [dump_column(generator, :created_at), 89 | dump_column(generator, :first_name)], 90 | 91 | @table_name, 'extra indexes', 92 | [dump_index(generator, :type), 93 | dump_index(generator, :created_at)], 94 | 95 | 'disable', ModelSchema::DISABLE_MODEL_SCHEMA_KEY, '=1'] 96 | 97 | assert_includes_with_order message, parts 98 | end 99 | 100 | def test_col_missing 101 | generator = create_table_generator 102 | schema_diffs = [create_missing_diff(field_columns, generator, :name => :type), 103 | create_missing_diff(field_columns, generator, :name => :updated_at), 104 | create_missing_diff(field_indexes, generator, 105 | :columns => [:created_at]), 106 | create_missing_diff(field_indexes, generator, 107 | :columns => [:first_name, :emails])] 108 | 109 | message = schema_error(schema_diffs).to_s 110 | parts = [@table_name, 'missing columns', 111 | [dump_column(generator, :type), 112 | dump_column(generator, :updated_at)], 113 | 114 | @table_name, 'missing indexes', 115 | [dump_index(generator, :created_at), 116 | dump_index(generator, [:first_name, :emails])], 117 | 118 | 'disable', ModelSchema::DISABLE_MODEL_SCHEMA_KEY, '=1'] 119 | 120 | assert_includes_with_order message, parts 121 | end 122 | 123 | def create_exp_table_generator 124 | Sequel::Schema::CreateTableGenerator.new(@db) do 125 | foreign_key :organization_id, :org 126 | String :emails, :text => true, :null => false, :unique => true 127 | DateTime :created_at 128 | 129 | index :updated_at, :name => :created_at_index 130 | index [:first_name, :emails], :unique => true, :where => {:type => 2} 131 | end 132 | end 133 | 134 | def test_col_mismatch 135 | generator = create_table_generator 136 | exp_generator = create_exp_table_generator 137 | 138 | schema_diffs = [create_mismatch_diff(field_columns, generator, exp_generator, 139 | :name => :organization_id), 140 | create_mismatch_diff(field_columns, generator, exp_generator, 141 | :name => :emails), 142 | create_mismatch_diff(field_indexes, generator, exp_generator, 143 | :name => :created_at_index), 144 | create_mismatch_diff(field_indexes, generator, exp_generator, 145 | :columns => [:first_name, :emails])] 146 | 147 | message = schema_error(schema_diffs).to_s 148 | parts = [@table_name, 'mismatched columns', 149 | [dump_column(generator, :organization_id), 150 | dump_column(exp_generator, :organization_id), 151 | dump_column(generator, :emails), 152 | dump_column(exp_generator, :emails)], 153 | 154 | @table_name, 'mismatched indexes', 155 | [dump_index(generator, :created_at), 156 | dump_index(exp_generator, :updated_at), 157 | dump_index(generator, [:first_name, :emails]), 158 | dump_index(exp_generator, [:first_name, :emails])], 159 | 160 | 'disable', ModelSchema::DISABLE_MODEL_SCHEMA_KEY, '=1'] 161 | 162 | assert_includes_with_order message, parts 163 | end 164 | 165 | def test_col_all 166 | generator = create_table_generator 167 | exp_generator = create_exp_table_generator 168 | 169 | schema_diffs = [create_extra_diff(field_columns, generator, :name => :type), 170 | create_extra_diff(field_indexes, generator, 171 | :columns => [:created_at]), 172 | create_missing_diff(field_columns, exp_generator, 173 | :name => :organization_id), 174 | create_missing_diff(field_columns, generator, 175 | :name => :first_name), 176 | create_mismatch_diff(field_columns, generator, exp_generator, 177 | :name => :created_at), 178 | create_mismatch_diff(field_indexes, generator, exp_generator, 179 | :columns => [:first_name, :emails])] 180 | 181 | message = schema_error(schema_diffs).to_s 182 | parts = [@table_name, 'extra columns', 183 | dump_column(generator, :type), 184 | 185 | @table_name, 'missing columns', 186 | [dump_column(exp_generator, :organization_id), 187 | dump_column(generator, :first_name)], 188 | 189 | @table_name, 'mismatched columns', 190 | [dump_column(generator, :created_at), 191 | dump_column(exp_generator, :created_at)], 192 | 193 | @table_name, 'extra indexes', 194 | dump_index(generator, :created_at), 195 | 196 | @table_name, 'mismatched indexes', 197 | [dump_index(generator, [:first_name, :emails]), 198 | dump_index(exp_generator, [:first_name, :emails])], 199 | 200 | 'disable', ModelSchema::DISABLE_MODEL_SCHEMA_KEY, '=1'] 201 | 202 | assert_includes_with_order message, parts 203 | end 204 | end 205 | -------------------------------------------------------------------------------- /test/model_schema/version_test.rb: -------------------------------------------------------------------------------- 1 | require 'test_helper' 2 | 3 | class VersionTest < BaseTest 4 | def test_version 5 | refute_nil ModelSchema::VERSION 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /test/test_helper.rb: -------------------------------------------------------------------------------- 1 | require 'bundler/setup' 2 | 3 | $LOAD_PATH.unshift File.expand_path('../../lib', __FILE__) 4 | require 'model_schema' 5 | 6 | require 'minitest/autorun' 7 | require 'minitest/hooks/test' 8 | require 'mocha/mini_test' 9 | 10 | class BaseTest < Minitest::Test 11 | include Minitest::Hooks 12 | 13 | def error_class 14 | ModelSchema::SchemaError 15 | end 16 | 17 | def type_extra 18 | ModelSchema::SchemaError::TYPE_EXTRA 19 | end 20 | 21 | def type_missing 22 | ModelSchema::SchemaError::TYPE_MISSING 23 | end 24 | 25 | def type_mismatch 26 | ModelSchema::SchemaError::TYPE_MISMATCH 27 | end 28 | 29 | def field_columns 30 | ModelSchema::FIELD_COLUMNS 31 | end 32 | 33 | def field_indexes 34 | ModelSchema::FIELD_INDEXES 35 | end 36 | 37 | def find_match(array, condition) 38 | array.find do |elem| 39 | condition.keys.all? do |key| 40 | condition[key] == elem[key] 41 | end 42 | end 43 | end 44 | 45 | def create_diff(field, type, opts) 46 | opts.merge(:field => field, :type => type) 47 | end 48 | 49 | def create_extra_diff(field, generator, condition) 50 | elem = find_match(generator.send(field), condition) 51 | create_diff(field, type_extra, :generator => generator, :elem => elem) 52 | end 53 | 54 | def create_missing_diff(field, generator, condition) 55 | elem = find_match(generator.send(field), condition) 56 | create_diff(field, type_missing, :generator => generator, :elem => elem) 57 | end 58 | 59 | def create_mismatch_diff(field, db_generator, exp_generator, condition) 60 | db_elem = find_match(db_generator.send(field), condition) 61 | exp_elem = find_match(exp_generator.send(field), condition) 62 | 63 | create_diff(field, type_mismatch, :db_generator => db_generator, 64 | :exp_generator => exp_generator, 65 | :db_elem => db_elem, 66 | :exp_elem => exp_elem) 67 | end 68 | 69 | def schema_error(schema_diffs=[]) 70 | ModelSchema::SchemaError.new(@table_name, schema_diffs) 71 | end 72 | 73 | def dump_column(generator, name) 74 | schema_error.dump_single(field_columns, generator, 75 | find_match(generator.columns, :name => name)) 76 | end 77 | 78 | def dump_index(generator, columns) 79 | columns = columns.is_a?(Array) ? columns : [columns] 80 | schema_error.dump_single(field_indexes, generator, 81 | find_match(generator.indexes, :columns => columns)) 82 | end 83 | 84 | def assert_includes_with_order(message, parts) 85 | parts.each do |part| 86 | if part.is_a?(String) 87 | assert_includes message, part 88 | index = message.index(part) 89 | message = message[(index + part.length)..-1] 90 | else 91 | # part is an array of strings, each of which must be included in 92 | # message, but in any order with respect to one another 93 | pieces = part 94 | end_index = 0 95 | 96 | pieces.each do |p| 97 | assert_includes message, p 98 | cur_end_index = message.index(p) + p.length 99 | end_index = [end_index, cur_end_index].max 100 | end 101 | 102 | message = message[end_index..-1] 103 | end 104 | end 105 | end 106 | 107 | def create_simple_table(db) 108 | table_name = :simple 109 | db.create_table(table_name) do 110 | String :name, :size => 50 111 | Integer :value, :null => false 112 | index :name, :unique => true 113 | end 114 | 115 | table_name 116 | end 117 | 118 | def create_complex_table(db) 119 | # other table for referencing 120 | db.create_table(:others) do 121 | primary_key :id 122 | Integer :value 123 | end 124 | 125 | table_name = :complex 126 | db.create_table(table_name) do 127 | primary_key :id 128 | foreign_key :other_id, :others, :null => false, :on_delete => :cascade 129 | 130 | String :name 131 | String :location, :fixed => true, :size => 50 132 | String :legal_name, :size => 200 133 | String :advisor, :text => true 134 | 135 | BigDecimal :amount, :size => 10 136 | Integer :value, :null => false, :unique => true 137 | 138 | column :advisors, 'varchar(255)[]', :null => false 139 | column :interests, 'text[]', :null => false, :index => {:name => :int_index} 140 | 141 | TrueClass :is_right 142 | 143 | Time :created_at, :null => false 144 | Time :updated_at, :only_time => true 145 | 146 | index [:other_id, :name] 147 | end 148 | 149 | table_name 150 | end 151 | end 152 | --------------------------------------------------------------------------------