├── .gitignore ├── .rspec ├── .travis.yml ├── Gemfile ├── LICENSE.txt ├── README.md ├── Rakefile ├── lib └── rspec │ ├── page-regression.rb │ └── page-regression │ ├── file_paths.rb │ ├── image_comparison.rb │ ├── matcher.rb │ ├── renderer.rb │ └── version.rb ├── rspec-page-regression.gemspec └── spec ├── fixtures ├── A.png ├── ABdiff.png ├── B.png └── Small.png ├── match_expectation_spec.rb ├── spec_helper.rb └── support ├── helpers.rb └── matchers.rb /.gitignore: -------------------------------------------------------------------------------- 1 | *.gem 2 | *.rbc 3 | .bundle 4 | .config 5 | .yardoc 6 | Gemfile.lock 7 | InstalledFiles 8 | _yardoc 9 | coverage 10 | doc/ 11 | lib/bundler/man 12 | pkg 13 | rdoc 14 | spec/reports 15 | spec/expectation 16 | test/tmp 17 | test/version_tmp 18 | tmp 19 | -------------------------------------------------------------------------------- /.rspec: -------------------------------------------------------------------------------- 1 | --format documentation 2 | --color 3 | --tty 4 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | sudo: false 2 | rvm: 3 | - 1.9.3 4 | - 2.1.0 5 | - jruby 6 | script: bundle exec rake spec_with_coveralls 7 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | # Specify your gem's dependencies in rspec-page-regression.gemspec 4 | gemspec 5 | 6 | gem 'awesome_print' 7 | gem 'byebug' if RUBY_VERSION >= "2.0.0" && RUBY_PLATFORM != 'java' 8 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2013 ronen barzel 2 | 3 | MIT License 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining 6 | a copy of this software and associated documentation files (the 7 | "Software"), to deal in the Software without restriction, including 8 | without limitation the rights to use, copy, modify, merge, publish, 9 | distribute, sublicense, and/or sell copies of the Software, and to 10 | permit persons to whom the Software is furnished to do so, subject to 11 | the following conditions: 12 | 13 | The above copyright notice and this permission notice shall be 14 | included in all copies or substantial portions of the Software. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 19 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 20 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 21 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 22 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | [![Gem Version](https://badge.fury.io/rb/rspec-page-regression.png)](http://badge.fury.io/rb/rspec-page-regression) 3 | [![Build Status](https://secure.travis-ci.org/rprt/rspec-page-regression.png)](http://travis-ci.org/ronen/rspec-page-regression) 4 | [![Coverage Status](https://coveralls.io/repos/rprt/rspec-page-regression/badge.svg?branch=master&service=github)](https://coveralls.io/github/rprt/rspec-page-regression?branch=master) 5 | [![Dependency Status](https://gemnasium.com/rprt/rspec-page-regression.png)](https://gemnasium.com/ronen/rspec-page-regression) 6 | 7 | # rspec-page-regression 8 | 9 | 10 | Rspec-page-regression is an [RSpec](https://github.com/rspec/rspec) plugin 11 | that makes it easy to headlessly regression test your web application pages to make sure the pages continue to look the way you expect them to look, taking into account HTML, CSS, and JavaScript. 12 | 13 | It provides an RSpec matcher that compares the test snapshot to an expected image, and facilitates management of the images. 14 | 15 | Rspec-page-regression uses [PhantomJS](http://www.phantomjs.org/) to headlessly render web page snapshots, by virtue of the [Poltergeist](https://github.com/jonleighton/poltergeist) driver for [Capybara](https://github.com/jnicklas/capybara). You can also use the Selenium driver to test against real browsers. 16 | 17 | Rspec-page-regression is tested on ruby 1.9.3, 2.1.0, and jruby 18 | 19 | ## Installation 20 | 21 | 22 | Install PhantomJS as per [PhantomJS: Download and Install](http://phantomjs.org/download.html) and/or [Poltergeist: Installing PhantomJS](https://github.com/jonleighton/poltergeist#installing-phantomjs). There are no other external dependencies (no need for Qt, nor an X server, nor ImageMagick, etc.) 23 | 24 | In your Gemfile: 25 | 26 | gem 'rspec-page-regression' 27 | 28 | And in your spec_helper: 29 | 30 | require 'rspec' # or 'rspec/rails' if you're using Rails 31 | require 'rspec/page-regression' 32 | 33 | require 'capybara/rspec' 34 | require 'capybara/poltergeist' 35 | Capybara.javascript_driver = :poltergeist 36 | 37 | Rspec-page-regression presupposes the convention that your spec files are somwhere under a directory named `spec` (checked in to your repo), which has a sibling directory `tmp` (.gitignore'd) 38 | 39 | To install for use with Selenium, [see instructions below](#selenium). 40 | 41 | #### Note on versions: 42 | Rspec-page-regression has multiple versions that work in concert with the [significant changes in RSpec version 3](http://myronmars.to/n/dev-blog/2013/07/the-plan-for-rspec-3). If you're using bundler, the gem dependencies should automatically find the proper version of rspec-page-regression for your chosen version of RSpec. 43 | 44 | | Rspec Version | Rspec-page-regression | 45 | | ------------- | --------------------- | 46 | | >= 3.0.* | >= 0.3.0 | 47 | | 2.99 | 0.2.99 | 48 | | <= 2.14.* | <= 0.2.1 | 49 | 50 | 51 | ## Usage 52 | 53 | Rspec-page-regression provides a matcher that renders the page and compares 54 | the resulting image against an expected image. To use it, you need to enable Capybara and Poltergeist by specifying `:type => :feature` and `:js => true`: 55 | 56 | require 'spec_helper' 57 | 58 | describe "my page", :type => :feature, :js => true do 59 | 60 | before(:each) do 61 | visit my_page_path 62 | end 63 | 64 | it { expect(page).to match_expectation } 65 | 66 | context "popup help" do 67 | before(:each) do 68 | click_button "Help" 69 | end 70 | 71 | it { expect(page).to match_expectation } 72 | end 73 | end 74 | 75 | The spec will pass if the test rendered image contains the exact same pixel values as the expectated image. Otherwise it will fail with an error message along the lines of: 76 | 77 | Test image does not match expected image 78 | $ open tmp/spec/expectation/my_page/popup_help/test.png spec/expectation/my_page/popup_help/expected.png tmp/spec/expectation/my_page/popup_help/difference.png 79 | 80 | Notice that the second line gives a command you can copy and paste in order to visually compare the test and expected images. 81 | 82 | It also shows a "difference image" in which each pixel contains the per-channel absolute difference between the test and expected images. That is, the difference images is black everywhere except has some funky colored pixels where the test and expected images differ. To help you locate those, it also has a red bounding box drawn around the region with differences. 83 | 84 | ### How do I create expectation images? 85 | 86 | The easiest way to create an expectation image is to run the test for the first time and let it fail. You'll then get a failure message like: 87 | 88 | Missing expectation image spec/expectation/my_page/popup_help/expected.png 89 | $ open tmp/spec/expectation/my_page/popup_help/test.png 90 | To create it: 91 | $ mkdir -p spec/expectation/my_page/popup_help && cp tmp/spec/expectation/my_page/popup_help/test.png spec/expectation/my_page/popup_help/expected.png 92 | 93 | First view the test image to make sure it really is what you expect. Then copy and paste the last line to install it as the expected image. (And then of course commit the expected image into your repository.) 94 | 95 | ### How do I update expectation images? 96 | 97 | If you've deliberatly changed something that affects the look of your web page, your regression test will fail. The "test" image will contain the new look, and the "expected" image will contain the old. 98 | 99 | Once you've visually checked the test image to make sure it's really what you want, then simply copy the test image over the old expectation image. (And then of course commit it it into your repository.) 100 | 101 | The failure message doesn't include a ready-to-copy-and-paste `cp` command, but you can copy and paste the individual file paths from the message. (The reason not to have a ready-to-copy-and-paste command is if the failure is real, it shouldn't be too easy to mindlessly copy and paste to make it go away.) 102 | 103 | ### Where are the expectation images? 104 | 105 | As per the above examples, the expectation images default to being stored under `spec/expectation`, with the remainder of the path constructed from the example group descriptions. (If the `it` also has a description it will be used as well.) 106 | 107 | If that default scheme doesn't suit you, you can pass a path to where the expectation image should be found: 108 | 109 | expect(page).to match_expectation "/path/to/my/file.png" 110 | 111 | Everything will work normally, and the failure messages will refer to your path. 112 | 113 | ## Configuration 114 | 115 | ### Window size 116 | 117 | The default window size for the renders is 1024 x 768 pixels. You can specify another size as `[height, width]` in pixels: 118 | 119 | # in spec_helper.rb: 120 | RSpec::PageRegression.configure do |config| 121 | config.page_size = [1280, 1024] 122 | end 123 | 124 | Note that this specifies the size of the browser window viewport; but rspec-page-regression requests a render of the full page, which might extend beyond the window. So the rendered file dimensions may be larger than this configuration value. 125 | 126 | ### Image difference threshold 127 | 128 | By default, a test fails if only a single pixel in the screenshot differs from the expectation image. To account for minor rendering differences, you can set a threshold value that allows a certain amount of differences. The threshold value is the fraction of pixels that are allowed to differ before the test fails. 129 | 130 | RSpec::PageRegression.configure do |config| 131 | config.threshold = 0.01 132 | end 133 | 134 | This setting means that 1% of pixels are allowed to differ between the rendering result and the expectation image. For example, for an image size of 1024 x 768 and a threshold of 0.01, the maximum number of pixel differences between the images is 7864. 135 | 136 | ## [Using the selenium driver](id:selenium) 137 | 138 | You can also use the selenium driver with capybara. This offers the possiblity to visually test your pages against a range of real browsers. 139 | 140 | Add the [selenium-webdriver](https://rubygems.org/gems/selenium-webdriver) to your Gemfile: 141 | 142 | gem 'selenium-webdriver' 143 | 144 | And in your spec_helper replace: 145 | 146 | require 'capybara/poltergeist' 147 | Capybara.javascript_driver = :poltergeist 148 | 149 | With: 150 | 151 | require 'selenium/webdriver' 152 | Capybara.javascript_driver = :selenium 153 | 154 | 155 | See also the [capybara readme](https://github.com/jnicklas/capybara#selenium) and [selenium wiki](https://code.google.com/p/selenium/wiki/RubyBindings) for more information. 156 | 157 | ## Contributing 158 | 159 | Contributions are welcome! As usual, here's the drill: 160 | 161 | 1. Fork it 162 | 2. Create your feature branch (`git checkout -b my-new-feature`) 163 | 3. Commit your changes (`git commit -am 'Add some feature'`) 164 | 4. Push to the branch (`git push origin my-new-feature`) 165 | 5. Create new Pull Request 166 | 167 | Don't forget to include specs (`rake spec`) to verify your functionality. Code coverage should be 100% 168 | 169 | ## History 170 | 171 | Release Notes: 172 | 173 | * 0.4.2 - Now works with jruby. Thanks to [@paresharma](https://github.com/paresharma) 174 | 175 | * 0.4.1 - Bug fix: wasn't including example name in file path. Thanks to [@kurtisnelson](https://github.com/kurtisnelson) 176 | 177 | * 0.4.0 - Add difference threshold. Thanks to [@abersager](https://github.com/abersager) 178 | 179 | * 0.3.0 - Compatibility with rspec 3.0 180 | 181 | * 0.2.99 - Compatibility with rspec 2.99 182 | 183 | * 0.2.1 - Explicit dependency on rspec ~> 2.14.0 184 | 185 | * 0.2.0 - Support selenium. Thanks to [@edwinvdgraaf](https://github.com/edwinvdgraaf) 186 | 187 | * 0.1.2 - Remove existing difference images so they won't be shown in cases where files couldn't be differenced. 188 | 189 | 190 | [![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/ronen/rspec-page-regression/trend.png)](https://bitdeli.com/free "Bitdeli Badge") 191 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require "bundler/gem_tasks" 2 | 3 | task :default => :spec 4 | 5 | require 'rspec/core/rake_task' 6 | RSpec::Core::RakeTask.new(:spec) do |spec| 7 | spec.rspec_opts = '-Ispec' 8 | end 9 | 10 | require 'coveralls/rake/task' 11 | Coveralls::RakeTask.new 12 | task :spec_with_coveralls => [:spec, 'coveralls:push'] 13 | -------------------------------------------------------------------------------- /lib/rspec/page-regression.rb: -------------------------------------------------------------------------------- 1 | require "rspec/page-regression/file_paths" 2 | require "rspec/page-regression/image_comparison" 3 | require "rspec/page-regression/matcher" 4 | require "rspec/page-regression/renderer" 5 | require "rspec/page-regression/version" 6 | 7 | module RSpec::PageRegression 8 | def self.configure 9 | yield self 10 | end 11 | 12 | def self.page_size= (page_size) 13 | @@page_size = page_size 14 | end 15 | 16 | def self.page_size 17 | @@page_size ||= [1024, 768] 18 | end 19 | 20 | def self.threshold= (threshold) 21 | @@threshold = threshold 22 | end 23 | 24 | def self.threshold 25 | @@threshold ||= 0.0 26 | end 27 | end 28 | -------------------------------------------------------------------------------- /lib/rspec/page-regression/file_paths.rb: -------------------------------------------------------------------------------- 1 | require 'active_support/core_ext/string/inflections' 2 | 3 | module RSpec::PageRegression 4 | class FilePaths 5 | attr_reader :expected_image 6 | attr_reader :test_image 7 | attr_reader :difference_image 8 | 9 | def initialize(example, expected_path = nil) 10 | expected_path = Pathname.new(expected_path) if expected_path 11 | 12 | descriptions = description_ancestry(example.metadata[:example_group]) 13 | descriptions.push example.description unless example.description.parameterize('_') =~ %r{ 14 | ^ 15 | (then_+)? 16 | ( (expect_+) (page_+) (to_+) (not_+)? | (page_+) (should_+)? ) 17 | match_expectation 18 | (_#{Regexp.escape(expected_path.to_s)})? 19 | $ 20 | }xi 21 | canonical_path = descriptions.map{|s| s.parameterize('_')}.inject(Pathname.new(""), &:+) 22 | 23 | app_root = Pathname.new(example.metadata[:file_path]).realpath.each_filename.take_while{|c| c != "spec"}.inject(Pathname.new("/"), &:+) 24 | expected_root = app_root + "spec" + "expectation" 25 | test_root = app_root + "tmp" + "spec" + "expectation" 26 | cwd = Pathname.getwd 27 | 28 | @expected_image = expected_path || (expected_root + canonical_path + "expected.png").relative_path_from(cwd) 29 | @test_image = (test_root + canonical_path + "test.png").relative_path_from cwd 30 | @difference_image = (test_root + canonical_path + "difference.png").relative_path_from cwd 31 | end 32 | 33 | def all 34 | [test_image, expected_image, difference_image] 35 | end 36 | 37 | 38 | private 39 | 40 | def description_ancestry(metadata) 41 | return [] if metadata.nil? 42 | description_ancestry(metadata[:parent_example_group]) << metadata[:description].parameterize("_") 43 | end 44 | end 45 | end 46 | -------------------------------------------------------------------------------- /lib/rspec/page-regression/image_comparison.rb: -------------------------------------------------------------------------------- 1 | if RUBY_PLATFORM == 'java' 2 | require "chunky_png" 3 | else 4 | require 'oily_png' 5 | end 6 | 7 | module RSpec::PageRegression 8 | 9 | class ImageComparison 10 | include ChunkyPNG::Color 11 | 12 | attr_reader :result 13 | 14 | def initialize(filepaths) 15 | @filepaths = filepaths 16 | @result = compare 17 | end 18 | 19 | def expected_size 20 | [@iexpected.width , @iexpected.height] 21 | end 22 | 23 | def test_size 24 | [@itest.width , @itest.height] 25 | end 26 | 27 | private 28 | 29 | def compare 30 | @filepaths.difference_image.unlink if @filepaths.difference_image.exist? 31 | 32 | return :missing_expected unless @filepaths.expected_image.exist? 33 | return :missing_test unless @filepaths.test_image.exist? 34 | 35 | @iexpected = ChunkyPNG::Image.from_file(@filepaths.expected_image) 36 | @itest = ChunkyPNG::Image.from_file(@filepaths.test_image) 37 | 38 | return :size_mismatch if test_size != expected_size 39 | 40 | return :match if pixels_match? 41 | 42 | create_difference_image 43 | return :difference 44 | end 45 | 46 | def pixels_match? 47 | max_count = RSpec::PageRegression.threshold * @itest.width * @itest.height 48 | count = 0 49 | @itest.height.times do |y| 50 | next if @itest.row(y) == @iexpected.row(y) 51 | diff = @itest.row(y).zip(@iexpected.row(y)).select { |x, y| x != y } 52 | count += diff.count 53 | return false if count > max_count 54 | end 55 | return true 56 | end 57 | 58 | def create_difference_image 59 | idiff = ChunkyPNG::Image.from_file(@filepaths.expected_image) 60 | xmin = @itest.width + 1 61 | xmax = -1 62 | ymin = @itest.height + 1 63 | ymax = -1 64 | @itest.height.times do |y| 65 | @itest.row(y).each_with_index do |test_pixel, x| 66 | idiff[x,y] = if test_pixel != (expected_pixel = idiff[x,y]) 67 | xmin = x if x < xmin 68 | xmax = x if x > xmax 69 | ymin = y if y < ymin 70 | ymax = y if y > ymax 71 | rgb( 72 | (r(test_pixel) - r(expected_pixel)).abs, 73 | (g(test_pixel) - g(expected_pixel)).abs, 74 | (b(test_pixel) - b(expected_pixel)).abs 75 | ) 76 | else 77 | rgb(0,0,0) 78 | end 79 | end 80 | end 81 | 82 | idiff.rect(xmin-1,ymin-1,xmax+1,ymax+1,rgb(255,0,0)) 83 | 84 | idiff.save @filepaths.difference_image 85 | end 86 | end 87 | end 88 | -------------------------------------------------------------------------------- /lib/rspec/page-regression/matcher.rb: -------------------------------------------------------------------------------- 1 | require 'which_works' 2 | 3 | module RSpec::PageRegression 4 | 5 | RSpec::Matchers.define :match_expectation do |expectation_path| 6 | 7 | match do |page| 8 | @filepaths = FilePaths.new(RSpec.current_example, expectation_path) 9 | Renderer.render(page, @filepaths.test_image) 10 | @comparison = ImageComparison.new(@filepaths) 11 | @comparison.result == :match 12 | end 13 | 14 | failure_message do |page| 15 | msg = case @comparison.result 16 | when :missing_expected then "Missing expectation image #{@filepaths.expected_image}" 17 | when :missing_test then "Missing test image #{@filepaths.test_image}" 18 | when :size_mismatch then "Test image size #{@comparison.test_size.join('x')} does not match expectation #{@comparison.expected_size.join('x')}" 19 | when :difference then "Test image does not match expected image" 20 | end 21 | 22 | msg += "\n $ #{viewer} #{@filepaths.all.select(&:exist?).join(' ')}" 23 | 24 | case @comparison.result 25 | when :missing_expected 26 | msg += "\nCreate it via:\n $ mkdir -p #{@filepaths.expected_image.dirname} && cp #{@filepaths.test_image} #{@filepaths.expected_image}" 27 | end 28 | 29 | msg 30 | end 31 | 32 | failure_message_when_negated do |page| 33 | "Test image expected to not match expectation image" 34 | end 35 | 36 | def viewer 37 | File.basename(Which.which("open", "feh", "display", :array => true).first || "viewer") 38 | end 39 | end 40 | end 41 | -------------------------------------------------------------------------------- /lib/rspec/page-regression/renderer.rb: -------------------------------------------------------------------------------- 1 | module RSpec::PageRegression 2 | module Renderer 3 | 4 | def self.render(page, test_image_path) 5 | 6 | test_image_path.dirname.mkpath unless test_image_path.dirname.exist? 7 | # Capybara doesn't implement resize in API 8 | unless page.driver.respond_to? :resize 9 | page.driver.browser.manage.window.resize_to *RSpec::PageRegression.page_size 10 | else 11 | page.driver.resize *RSpec::PageRegression.page_size 12 | end 13 | page.driver.save_screenshot test_image_path, :full => true 14 | end 15 | end 16 | end 17 | -------------------------------------------------------------------------------- /lib/rspec/page-regression/version.rb: -------------------------------------------------------------------------------- 1 | module RSpec 2 | module PageRegression 3 | VERSION = "0.4.2" 4 | end 5 | end 6 | -------------------------------------------------------------------------------- /rspec-page-regression.gemspec: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | lib = File.expand_path('../lib', __FILE__) 3 | $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) 4 | require 'rspec/page-regression/version' 5 | 6 | Gem::Specification.new do |spec| 7 | spec.name = "rspec-page-regression" 8 | spec.version = RSpec::PageRegression::VERSION 9 | spec.authors = ["ronen barzel"] 10 | spec.email = ["ronen@barzel.org"] 11 | spec.summary = %q{Web page rendering (HTML, CSS, and JavasSript) regression for RSpec} 12 | spec.description = %q{Rspec-page-regression provides a mechanism for headless regression testing of web page renders in RSpec. It takes into account HTML, CSS, and JavaScript, by virtue of using PhantomJS (via the Poltergeist gem) to render snapshots. It provides an RSpec matcher that compares the test snapshot to an expected image, and facilitates management of the images.} 13 | spec.homepage = "https://github.com/ronen/rspec-page-regression" 14 | spec.license = "MIT" 15 | 16 | spec.files = `git ls-files`.split($/) 17 | spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) } 18 | spec.test_files = spec.files.grep(%r{^(test|spec|features)/}) 19 | spec.require_paths = ["lib"] 20 | 21 | spec.add_dependency "activesupport" 22 | 23 | if RUBY_PLATFORM == 'java' 24 | spec.add_dependency "chunky_png" 25 | else 26 | spec.add_dependency "oily_png" 27 | end 28 | 29 | spec.add_dependency "poltergeist" 30 | spec.add_dependency "rspec", "~> 3.0" 31 | spec.add_dependency "which_works" 32 | 33 | spec.add_development_dependency "bourne" 34 | spec.add_development_dependency "bundler", "~> 1.3" 35 | spec.add_development_dependency "mocha" 36 | spec.add_development_dependency "rake" 37 | spec.add_development_dependency "rspec-given" 38 | spec.add_development_dependency "simplecov" 39 | spec.add_development_dependency "simplecov-gem-adapter" 40 | spec.add_development_dependency "coveralls" 41 | end 42 | -------------------------------------------------------------------------------- /spec/fixtures/A.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rprt/rspec-page-regression/4b0bfa62e68d1034502a9afae74f1288e32e7ac7/spec/fixtures/A.png -------------------------------------------------------------------------------- /spec/fixtures/ABdiff.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rprt/rspec-page-regression/4b0bfa62e68d1034502a9afae74f1288e32e7ac7/spec/fixtures/ABdiff.png -------------------------------------------------------------------------------- /spec/fixtures/B.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rprt/rspec-page-regression/4b0bfa62e68d1034502a9afae74f1288e32e7ac7/spec/fixtures/B.png -------------------------------------------------------------------------------- /spec/fixtures/Small.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rprt/rspec-page-regression/4b0bfa62e68d1034502a9afae74f1288e32e7ac7/spec/fixtures/Small.png -------------------------------------------------------------------------------- /spec/match_expectation_spec.rb: -------------------------------------------------------------------------------- 1 | require 'spec_helper.rb' 2 | require 'fileutils' 3 | 4 | describe "match_expectation" do 5 | 6 | Given { 7 | @opts = { :full => true } 8 | @driver = mock("Driver") 9 | @driver.stubs :resize 10 | @driver.stubs :save_screenshot 11 | @page = mock("Page") 12 | @page.stubs(:driver).returns @driver 13 | @match_argument = nil 14 | } 15 | 16 | context "helpers" do 17 | it "use proper paths" do 18 | expect(expected_path).to eq Pathname.new("spec/expectation/match_expectation/helpers/use_proper_paths/expected.png") 19 | expect(test_path).to eq Pathname.new("tmp/spec/expectation/match_expectation/helpers/use_proper_paths/test.png") 20 | expect(difference_path).to eq Pathname.new("tmp/spec/expectation/match_expectation/helpers/use_proper_paths/difference.png") 21 | end 22 | end 23 | 24 | 25 | context "using expect().to" do 26 | 27 | When { 28 | begin 29 | expect(@page).to match_expectation @match_argument 30 | rescue RSpec::Expectations::ExpectationNotMetError => e 31 | @error = e 32 | end 33 | } 34 | 35 | context "framework" do 36 | Then { expect(@driver).to have_received(:resize).with(1024, 768) } 37 | Then { expect(@driver).to have_received(:save_screenshot).with(test_path, @opts) } 38 | 39 | context "selenium" do 40 | Given { 41 | @browser = mock("Browser") 42 | @window = mock("Window") 43 | @window.stubs(:resize_to) 44 | manage = mock("Manage") 45 | manage.stubs(:window).returns @window 46 | @browser.stubs(:manage => manage) 47 | @driver.stubs(:browser => @browser) 48 | @driver.unstub(:resize) 49 | } 50 | 51 | Then { expect(@window).to have_received(:resize_to).with(1024, 768) } 52 | end 53 | end 54 | 55 | context "when files match" do 56 | Given { use_test_image "A" } 57 | Given { use_expected_image "A" } 58 | 59 | Then { expect(@error).to be_nil } 60 | end 61 | 62 | 63 | context "when files do not match" do 64 | Given { use_test_image "A" } 65 | Given { use_expected_image "B" } 66 | 67 | Then { expect(@error).to_not be_nil } 68 | Then { expect(@error.message).to include "Test image does not match expected image" } 69 | Then { expect(@error.message).to match viewer_pattern(test_path, expected_path, difference_path) } 70 | 71 | Then { expect(difference_path.read).to eq fixture_image("ABdiff").read } 72 | end 73 | 74 | context "when difference threshold is set" do 75 | context "when difference threshold is configured below image difference" do 76 | Given do 77 | RSpec::PageRegression.configure do |config| 78 | config.threshold = 0.01 79 | end 80 | end 81 | Given { use_test_image "A" } 82 | Given { use_expected_image "B" } 83 | 84 | Then { expect(@error).to_not be_nil } 85 | Then { expect(@error.message).to include "Test image does not match expected image" } 86 | Then { expect(@error.message).to match viewer_pattern(test_path, expected_path, difference_path) } 87 | 88 | Then { expect(difference_path.read).to eq fixture_image("ABdiff").read } 89 | end 90 | 91 | context "when difference threshold is configured above image difference" do 92 | Given do 93 | RSpec::PageRegression.configure do |config| 94 | config.threshold = 0.02 95 | end 96 | end 97 | Given { use_test_image "A" } 98 | Given { use_expected_image "B" } 99 | 100 | Then { expect(@error).to be_nil } 101 | end 102 | 103 | after :each do 104 | RSpec::PageRegression.class_variable_set :@@threshold, nil 105 | end 106 | end 107 | 108 | context "when test image is missing" do 109 | Given { use_expected_image "A" } 110 | 111 | Then { expect(@error).to_not be_nil } 112 | Then { expect(@error.message).to include "Missing test image #{test_path}" } 113 | Then { expect(@error.message).to match viewer_pattern(expected_path) } 114 | context "with previously-created difference image" do 115 | Given { preexisting_difference_image } 116 | Then { expect(difference_path).to_not be_exist } 117 | end 118 | end 119 | 120 | context "when expected image is missing" do 121 | Given { use_test_image "A" } 122 | 123 | Then { expect(@error).to_not be_nil } 124 | Then { expect(@error.message).to include "Missing expectation image #{expected_path}" } 125 | Then { expect(@error.message).to match viewer_pattern(test_path) } 126 | Then { expect(@error.message).to include "mkdir -p #{expected_path.dirname} && cp #{test_path} #{expected_path}" } 127 | context "with previously-created difference image" do 128 | Given { preexisting_difference_image } 129 | Then { expect(difference_path).to_not be_exist } 130 | end 131 | end 132 | 133 | context "when sizes mismatch" do 134 | Given { use_test_image "Small" } 135 | Given { use_expected_image "A" } 136 | 137 | Then { expect(@error).to_not be_nil } 138 | Then { expect(@error.message).to include "Test image size 256x167 does not match expectation 512x334" } 139 | Then { expect(@error.message).to match viewer_pattern(test_path, expected_path) } 140 | context "with previously-created difference image" do 141 | Given { preexisting_difference_image } 142 | Then { expect(difference_path).to_not be_exist } 143 | end 144 | end 145 | 146 | context "with match argument" do 147 | Given { @match_argument = "/this/is/a/test.png" } 148 | Then { expect(@error.message).to include "Missing expectation image /this/is/a/test.png" } 149 | context "with previously-created difference image" do 150 | Given { preexisting_difference_image } 151 | Then { expect(difference_path).to_not be_exist } 152 | end 153 | end 154 | 155 | context "with trivial example description" do 156 | Given do 157 | RSpec::Core::Example.any_instance.stubs :metadata => { 158 | file_path: __FILE__, 159 | description: "Then expect(page).to match_expectation", 160 | example_group: { description: "parent" } 161 | } 162 | end 163 | Then { expect(@driver).to have_received(:save_screenshot).with(Pathname.new("tmp/spec/expectation/parent/test.png"), @opts) } 164 | Then { expect(@error.message).to include "Missing expectation image spec/expectation/parent/expected.png" } 165 | end 166 | 167 | context "with page size configuration" do 168 | Given do 169 | RSpec::PageRegression.configure do |config| 170 | config.page_size = [123, 456] 171 | end 172 | end 173 | Then { expect(@driver).to have_received(:resize).with(123, 456) } 174 | end 175 | 176 | end 177 | 178 | context "using expect().to_not" do 179 | When { 180 | begin 181 | expect(@page).to_not match_expectation 182 | rescue RSpec::Expectations::ExpectationNotMetError => e 183 | @error = e 184 | end 185 | } 186 | 187 | context "when files don't match" do 188 | Given { use_test_image "A" } 189 | Given { use_expected_image "B" } 190 | 191 | Then { expect(@error).to be_nil } 192 | end 193 | 194 | context "when files match" do 195 | Given { use_test_image "A" } 196 | Given { use_expected_image "A" } 197 | 198 | Then { expect(@error).to_not be_nil } 199 | Then { expect(@error.message).to eq "Test image expected to not match expectation image" } 200 | end 201 | end 202 | 203 | def fixture_image(name) 204 | FixturesDir + "#{name}.png" 205 | end 206 | 207 | def use_fixture_image(name, path) 208 | path.dirname.mkpath unless path.dirname.exist? 209 | FileUtils.cp fixture_image(name), path 210 | end 211 | 212 | def create_existing_difference_image 213 | end 214 | 215 | def use_test_image(name) 216 | use_fixture_image(name, test_path) 217 | end 218 | 219 | def use_expected_image(name) 220 | use_fixture_image(name, expected_path) 221 | end 222 | 223 | def preexisting_difference_image 224 | difference_path.dirname.mkpath unless difference_path.dirname.exist? 225 | FileUtils.touch difference_path 226 | end 227 | 228 | def viewer_pattern(*paths) 229 | %r{ 230 | \b 231 | (open|feh|display|viewer) 232 | \s 233 | #{paths.map{|path| Regexp.escape(path.to_s)}.join('\s')} 234 | \s*$ 235 | }x 236 | end 237 | 238 | 239 | end 240 | -------------------------------------------------------------------------------- /spec/spec_helper.rb: -------------------------------------------------------------------------------- 1 | require 'pathname' 2 | require 'simplecov' 3 | require 'simplecov-gem-adapter' 4 | SimpleCov.start "gem" 5 | 6 | $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) 7 | 8 | require 'rspec' 9 | require 'rspec/given' 10 | require 'rspec/page-regression' 11 | require 'bourne' 12 | 13 | SpecDir = Pathname.new(__FILE__).dirname 14 | RootDir = SpecDir.dirname 15 | TestDir = RootDir + "tmp/spec" 16 | FixturesDir = SpecDir + "fixtures" 17 | 18 | Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f} 19 | 20 | RSpec.configure do |config| 21 | config.mock_with :mocha 22 | config.include Helpers 23 | config.raise_errors_for_deprecations! 24 | end 25 | -------------------------------------------------------------------------------- /spec/support/helpers.rb: -------------------------------------------------------------------------------- 1 | module Helpers 2 | 3 | def test_path 4 | getpath(TestDir, "test") 5 | end 6 | 7 | def expected_path 8 | getpath(SpecDir, "expected") 9 | end 10 | 11 | def difference_path 12 | getpath(TestDir, "difference") 13 | end 14 | 15 | def getpath(root, base) 16 | (root + "expectation" + example_path(RSpec.current_example) + "#{base}.png").relative_path_from Pathname.getwd 17 | end 18 | 19 | def example_path(example) 20 | group_path(example.metadata[:example_group]) + example.description.parameterize("_") 21 | end 22 | 23 | def group_path(metadata) 24 | return Pathname.new("") if metadata.nil? 25 | group_path(metadata[:parent_example_group]) + metadata[:description].parameterize("_") 26 | end 27 | 28 | end 29 | -------------------------------------------------------------------------------- /spec/support/matchers.rb: -------------------------------------------------------------------------------- 1 | module RSpec 2 | module Matchers 3 | def fail 4 | raise_error(RSpec::Expectations::ExpectationNotMetError) 5 | end 6 | 7 | def fail_with(message) 8 | raise_error(RSpec::Expectations::ExpectationNotMetError, message) 9 | end 10 | 11 | def fail_matching(message) 12 | raise_error(RSpec::Expectations::ExpectationNotMetError, /#{Regexp.escape(message)}/) 13 | end 14 | end 15 | end 16 | --------------------------------------------------------------------------------