├── log └── .keep ├── app ├── mailers │ ├── .keep │ └── .gitkeep ├── models │ ├── .keep │ ├── .gitkeep │ └── concerns │ │ └── .keep ├── assets │ ├── images │ │ └── .keep │ ├── javascripts │ │ ├── bootstrap.js.coffee │ │ └── application.js │ └── stylesheets │ │ ├── bootstrap_and_overrides.css │ │ └── application.css ├── controllers │ ├── concerns │ │ └── .keep │ ├── admin_controller.rb │ ├── application_controller.rb │ ├── map_api_controller.rb │ └── changeset_api_controller.rb ├── helpers │ ├── application_helper.rb │ └── api_helper.rb ├── workers │ └── tiler_worker.rb └── views │ ├── api │ └── changesets.atom.builder │ ├── admin │ └── index.html.erb │ └── layouts │ └── admin.html.erb ├── lib ├── assets │ └── .keep ├── tasks │ ├── .keep │ ├── resque.rake │ └── owl.rake └── tiler │ ├── logging.rb │ ├── geos_utils.rb │ ├── cmdline_options.rb │ ├── test │ ├── perf_test.rb │ ├── utils_test.rb │ ├── tiler_unit_test.rb │ ├── common.rb │ └── tiler_realdata_test.rb │ ├── summary_tiler.rb │ ├── changeset.rb │ ├── utils.rb │ ├── change.rb │ └── changeset_tiler.rb ├── public ├── favicon.ico ├── owl_logo.png ├── owl_santa.png ├── robots.txt ├── 500.html ├── 422.html └── 404.html ├── test ├── helpers │ └── .keep ├── mailers │ └── .keep ├── models │ └── .keep ├── controllers │ ├── .keep │ ├── changeset_api_controller_realdata_test.rb │ └── changeset_api_controller_unit_test.rb ├── fixtures │ ├── .keep │ ├── tiler_unit_delete_node.sql │ ├── tiler_unit_move_node.sql │ ├── tiler_unit_tag_node.sql │ ├── tiler_unit_move_node_same_changeset.sql │ ├── tiler_unit_create_node.sql │ ├── tiler_unit_create_way.sql │ ├── tiler_unit_delete_way.sql │ ├── tiler_unit_move_way.sql │ └── tiler_unit_affect_way.sql ├── integration │ └── .keep └── test_helper.rb ├── INSTALL ├── db ├── sql │ ├── owl_load_changesets.sql │ ├── get_newest_changesets.sql │ ├── owl_load_data.sql │ ├── get_untiled_changesets.sql │ ├── owl_constraints.sql │ ├── owl_indexes.sql │ ├── owl_schema.sql │ └── owl_functions.sql ├── seeds.rb ├── schema.rb └── structure.sql ├── scripts ├── list_test_changeset_ids.sh ├── dump_way_revisions.rb ├── update_changesets.rb ├── way_tiler.rb ├── aggregate_changesets.rb ├── dump_testdata.sh └── tiler.rb ├── .gitmodules ├── bin ├── rake ├── bundle └── rails ├── config.ru ├── config ├── initializers │ ├── session_store.rb │ ├── filter_parameter_logging.rb │ ├── mime_types.rb │ ├── backtrace_silencers.rb │ ├── wrap_parameters.rb │ ├── secret_token.rb │ └── inflections.rb ├── environment.rb ├── boot.rb ├── locales │ ├── en.bootstrap.yml │ └── en.yml ├── database.yml ├── application.rb ├── environments │ ├── development.rb │ ├── test.rb │ └── production.rb └── routes.rb ├── replication ├── run_tiler.sh ├── replicate_changesets.sh ├── replicate.sh └── replicate_changesets.rb ├── Rakefile ├── .gitignore ├── README.md ├── Gemfile ├── COPYING └── Gemfile.lock /log/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /app/mailers/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /app/models/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /lib/assets/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /lib/tasks/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /public/favicon.ico: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/helpers/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/mailers/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/models/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /app/assets/images/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /app/mailers/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /app/models/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/controllers/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/fixtures/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/integration/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /app/models/concerns/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /app/controllers/concerns/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /lib/tasks/resque.rake: -------------------------------------------------------------------------------- 1 | require 'resque/tasks' 2 | -------------------------------------------------------------------------------- /INSTALL: -------------------------------------------------------------------------------- 1 | http://wiki.openstreetmap.org/wiki/OWL/Installation 2 | -------------------------------------------------------------------------------- /app/helpers/application_helper.rb: -------------------------------------------------------------------------------- 1 | module ApplicationHelper 2 | end 3 | -------------------------------------------------------------------------------- /db/sql/owl_load_changesets.sql: -------------------------------------------------------------------------------- 1 | \copy changesets FROM 'changesets.txt' 2 | -------------------------------------------------------------------------------- /scripts/list_test_changeset_ids.sh: -------------------------------------------------------------------------------- 1 | ls -1 ../testdata | grep -o '[0-9]\+' | uniq 2 | -------------------------------------------------------------------------------- /public/owl_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ppawel/openstreetmap-watch-list/HEAD/public/owl_logo.png -------------------------------------------------------------------------------- /public/owl_santa.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ppawel/openstreetmap-watch-list/HEAD/public/owl_santa.png -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "osmosis-plugin"] 2 | path = osmosis-plugin 3 | url = git://github.com/ppawel/osmosis.git 4 | -------------------------------------------------------------------------------- /bin/rake: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | require_relative '../config/boot' 3 | require 'rake' 4 | Rake.application.run 5 | -------------------------------------------------------------------------------- /bin/bundle: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__) 3 | load Gem.bin_path('bundler', 'bundle') 4 | -------------------------------------------------------------------------------- /app/assets/javascripts/bootstrap.js.coffee: -------------------------------------------------------------------------------- 1 | jQuery -> 2 | $("a[rel~=popover], .has-popover").popover() 3 | $("a[rel~=tooltip], .has-tooltip").tooltip() 4 | -------------------------------------------------------------------------------- /bin/rails: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | APP_PATH = File.expand_path('../../config/application', __FILE__) 3 | require_relative '../config/boot' 4 | require 'rails/commands' 5 | -------------------------------------------------------------------------------- /config.ru: -------------------------------------------------------------------------------- 1 | # This file is used by Rack-based servers to start the application. 2 | 3 | require ::File.expand_path('../config/environment', __FILE__) 4 | run Rails.application 5 | -------------------------------------------------------------------------------- /config/initializers/session_store.rb: -------------------------------------------------------------------------------- 1 | # Be sure to restart your server when you modify this file. 2 | 3 | Owl::Application.config.session_store :cookie_store, key: '_owl_session' 4 | -------------------------------------------------------------------------------- /config/environment.rb: -------------------------------------------------------------------------------- 1 | # Load the Rails application. 2 | require File.expand_path('../application', __FILE__) 3 | 4 | # Initialize the Rails application. 5 | Owl::Application.initialize! 6 | -------------------------------------------------------------------------------- /config/boot.rb: -------------------------------------------------------------------------------- 1 | # Set up gems listed in the Gemfile. 2 | ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__) 3 | 4 | require 'bundler/setup' if File.exists?(ENV['BUNDLE_GEMFILE']) 5 | -------------------------------------------------------------------------------- /db/sql/get_newest_changesets.sql: -------------------------------------------------------------------------------- 1 | SELECT DISTINCT changeset_id 2 | FROM nodes WHERE tstamp >= (SELECT MAX(tstamp) FROM tiles) 3 | UNION 4 | SELECT DISTINCT changeset_id 5 | FROM ways WHERE tstamp >= (SELECT MAX(tstamp) FROM tiles) 6 | -------------------------------------------------------------------------------- /test/fixtures/tiler_unit_delete_node.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO nodes VALUES (1, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(18.6023903 49.8778083)'), 4326)); 2 | INSERT INTO nodes VALUES (1, 2, 'f', 1, NOW(), 1, '', NULL); 3 | -------------------------------------------------------------------------------- /public/robots.txt: -------------------------------------------------------------------------------- 1 | # See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file 2 | # 3 | # To ban all spiders from the entire site uncomment the next two lines: 4 | # User-agent: * 5 | # Disallow: / 6 | -------------------------------------------------------------------------------- /config/initializers/filter_parameter_logging.rb: -------------------------------------------------------------------------------- 1 | # Be sure to restart your server when you modify this file. 2 | 3 | # Configure sensitive parameters which will be filtered from the log file. 4 | Rails.application.config.filter_parameters += [:password] 5 | -------------------------------------------------------------------------------- /config/initializers/mime_types.rb: -------------------------------------------------------------------------------- 1 | # Be sure to restart your server when you modify this file. 2 | 3 | # Add new mime types for use in respond_to blocks: 4 | # Mime::Type.register "text/richtext", :rtf 5 | # Mime::Type.register_alias "text/html", :iphone 6 | -------------------------------------------------------------------------------- /test/fixtures/tiler_unit_move_node.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO nodes VALUES (1, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(1 2)'), 4326)); 2 | INSERT INTO nodes VALUES (1, 2, 't', 1, NOW(), 2, '', ST_SetSRID(ST_GeomFromText('POINT(3 4)'), 4326)); 3 | -------------------------------------------------------------------------------- /lib/tiler/logging.rb: -------------------------------------------------------------------------------- 1 | require 'logger' 2 | 3 | module Tiler 4 | module Logger 5 | class <1', ST_SetSRID(ST_GeomFromText('POINT(3 4)'), 4326)); 2 | INSERT INTO nodes VALUES (1, 2, 't', 1, NOW(), 1, 'a=>1,b=>2', ST_SetSRID(ST_GeomFromText('POINT(1 2)'), 4326)); 3 | -------------------------------------------------------------------------------- /test/fixtures/tiler_unit_move_node_same_changeset.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO nodes VALUES (1, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(1 2)'), 4326)); 2 | INSERT INTO nodes VALUES (1, 2, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(3 4)'), 4326)); 3 | -------------------------------------------------------------------------------- /db/sql/owl_load_data.sql: -------------------------------------------------------------------------------- 1 | -- Import the table data from the data files using the fast COPY method. 2 | \copy users FROM 'users.txt' 3 | \copy nodes FROM 'nodes.txt' 4 | \copy ways FROM 'ways.txt' 5 | \copy relations FROM 'relations.txt' 6 | \copy relation_members FROM 'relation_members.txt' 7 | -------------------------------------------------------------------------------- /test/fixtures/tiler_unit_create_node.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO nodes VALUES (1, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(18.6023903 49.8778083)'), 4326)); 2 | INSERT INTO nodes VALUES (2, 1, 't', 1, NOW(), 1, 'a=>1', ST_SetSRID(ST_GeomFromText('POINT(18.6052464 49.8777568)'), 4326)); 3 | -------------------------------------------------------------------------------- /app/assets/stylesheets/bootstrap_and_overrides.css: -------------------------------------------------------------------------------- 1 | /* 2 | =require twitter-bootstrap-static/bootstrap 3 | 4 | Use Font Awesome icons (default) 5 | To use Glyphicons sprites instead of Font Awesome, replace with "require twitter-bootstrap-static/sprites" 6 | =require twitter-bootstrap-static/fontawesome 7 | */ -------------------------------------------------------------------------------- /replication/run_tiler.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | cd /home/ppawel/src/openstreetmap-watch-list/scripts 4 | 5 | set -e 6 | 7 | ( 8 | # Try to lock on the lock file (fd 200) 9 | flock -x -n 200 10 | psql -d owl -f ../sql/get_newest_changesets.sql | ./tiler.rb --changes --retile 11 | ) 200>/home/ppawel/.tiler.lock 12 | -------------------------------------------------------------------------------- /db/seeds.rb: -------------------------------------------------------------------------------- 1 | # This file should contain all the record creation needed to seed the database with its default values. 2 | # The data can then be loaded with the rake db:seed (or created alongside the db with db:setup). 3 | # 4 | # Examples: 5 | # 6 | # cities = City.create([{ name: 'Chicago' }, { name: 'Copenhagen' }]) 7 | # Mayor.create(name: 'Emanuel', city: cities.first) 8 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | # Add your own tasks in files placed in lib/tasks ending in .rake, 2 | # for example lib/tasks/capistrano.rake, and they will automatically be available to Rake. 3 | 4 | require File.expand_path('../config/application', __FILE__) 5 | 6 | Rake::TaskManager.class_eval do 7 | def remove_task(task_name) 8 | @tasks.delete(task_name.to_s) 9 | end 10 | end 11 | 12 | Owl::Application.load_tasks 13 | -------------------------------------------------------------------------------- /replication/replicate_changesets.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Add osmosis executable to the PATH. 4 | PATH=/home/ppawel/jdk/bin:/home/ppawel/bin:/home/ppawel/.gem/ruby/1.9.1/bin/:$PATH 5 | export PATH 6 | 7 | DIR=$(dirname $0) 8 | cd $DIR 9 | 10 | set -e 11 | 12 | ( 13 | # Try to lock on the lock file (fd 200) 14 | flock -x -n 200 15 | ./replicate_changesets.rb 16 | ) 200>/home/ppawel/.changesets.lock 17 | -------------------------------------------------------------------------------- /test/fixtures/tiler_unit_create_way.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO nodes VALUES (1, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.001 0.001)'), 4326)); 2 | INSERT INTO nodes VALUES (2, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.002 0.002)'), 4326)); 3 | INSERT INTO nodes VALUES (3, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.003 0.003)'), 4326)); 4 | INSERT INTO ways VALUES (3, 1, 't', 1, NOW(), 1, '', ARRAY[1, 2, 3]); 5 | -------------------------------------------------------------------------------- /replication/replicate.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Add osmosis executable to the PATH. 4 | PATH=/home/ppawel/jdk/bin:/home/ppawel/bin:/home/ppawel/.gem/ruby/1.9.1/bin/:$PATH 5 | export PATH 6 | 7 | cd /home/ppawel/replication 8 | 9 | set -e 10 | 11 | ( 12 | # Try to lock on the lock file (fd 200) 13 | flock -x -n 200 14 | osmosis --rri --lpc --write-owldb-change authFile=~/authFile invalidActionsMode=LOG 15 | ) 200>/home/ppawel/replication/lock 16 | -------------------------------------------------------------------------------- /config/initializers/backtrace_silencers.rb: -------------------------------------------------------------------------------- 1 | # Be sure to restart your server when you modify this file. 2 | 3 | # You can add backtrace silencers for libraries that you're using but don't wish to see in your backtraces. 4 | # Rails.backtrace_cleaner.add_silencer { |line| line =~ /my_noisy_library/ } 5 | 6 | # You can also remove all the silencers if you're trying to debug a problem that might stem from framework code. 7 | # Rails.backtrace_cleaner.remove_silencers! 8 | -------------------------------------------------------------------------------- /test/fixtures/tiler_unit_delete_way.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO nodes VALUES (1, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.001 0.001)'), 4326)); 2 | INSERT INTO nodes VALUES (2, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.002 0.002)'), 4326)); 3 | INSERT INTO nodes VALUES (3, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.003 0.003)'), 4326)); 4 | INSERT INTO ways VALUES (3, 1, 't', 1, NOW(), 1, '', ARRAY[1, 2, 3]); 5 | INSERT INTO ways VALUES (3, 2, 'f', 1, NOW(), 2, '', ARRAY[]::bigint[]); 6 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # See http://help.github.com/ignore-files/ for more about ignoring files. 2 | # 3 | # If you find yourself ignoring temporary files generated by your text editor 4 | # or operating system, you probably want to add a global ignore instead: 5 | # git config --global core.excludesfile '~/.gitignore_global' 6 | 7 | # Ignore bundler config. 8 | /.bundle 9 | 10 | # Ignore the default SQLite database. 11 | /db/*.sqlite3 12 | /db/*.sqlite3-journal 13 | 14 | # Ignore all logfiles and tempfiles. 15 | /log/*.log 16 | /tmp 17 | 18 | testdata 19 | vendor 20 | -------------------------------------------------------------------------------- /scripts/dump_way_revisions.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | require 'pg' 4 | require 'yaml' 5 | 6 | $config = YAML.load_file('../rails/config/database.yml')['development'] 7 | 8 | @conn = PGconn.open(:host => $config['host'], :port => $config['port'], :dbname => $config['database'], 9 | :user => $config['username'], :password => $config['password']) 10 | 11 | ARGF.each_line do |way_id| 12 | @conn.exec("COPY (SELECT * FROM OWL_GenerateWayRevisions(#{way_id}, true)) TO STDOUT") 13 | while not (data = @conn.get_copy_data).nil? 14 | puts data 15 | end 16 | end 17 | -------------------------------------------------------------------------------- /test/test_helper.rb: -------------------------------------------------------------------------------- 1 | ENV["RAILS_ENV"] ||= "test" 2 | require File.expand_path('../../config/environment', __FILE__) 3 | require 'rails/test_help' 4 | 5 | class ActiveSupport::TestCase 6 | #ActiveRecord::Migration.check_pending! 7 | 8 | # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order. 9 | # 10 | # Note: You'll currently still have to declare fixtures explicitly in integration tests 11 | # -- they do not yet inherit this setting 12 | #fixtures :all 13 | 14 | # Add more helper methods to be used by all tests here... 15 | end 16 | -------------------------------------------------------------------------------- /app/helpers/api_helper.rb: -------------------------------------------------------------------------------- 1 | module ApiHelper 2 | def get_xyz(params) 3 | return params[:x].to_i, params[:y].to_i, params[:zoom].to_i 4 | end 5 | 6 | def get_limit_sql(params) 7 | limit = (params[:limit] || 5).to_i 8 | limit = 5 if limit <= 0 9 | sql = " LIMIT #{limit}" 10 | sql += " OFFSET #{params[:offset].to_i}" if params[:offset] 11 | return sql 12 | end 13 | 14 | def get_timelimit_sql(params) 15 | return (params[:timelimit] and params[:timelimit].to_i > 0) ? " AND tstamp >= (NOW() - interval '#{params[:timelimit].to_i} hour')" : '' 16 | end 17 | end 18 | -------------------------------------------------------------------------------- /config/locales/en.bootstrap.yml: -------------------------------------------------------------------------------- 1 | # Sample localization file for English. Add more files in this directory for other locales. 2 | # See https://github.com/svenfuchs/rails-i18n/tree/master/rails%2Flocale for starting points. 3 | 4 | en: 5 | helpers: 6 | actions: "Actions" 7 | links: 8 | back: "Back" 9 | cancel: "Cancel" 10 | confirm: "Are you sure?" 11 | destroy: "Delete" 12 | new: "New" 13 | edit: "Edit" 14 | titles: 15 | edit: "Edit %{model}" 16 | save: "Save %{model}" 17 | new: "New %{model}" 18 | delete: "Delete %{model}" 19 | -------------------------------------------------------------------------------- /db/sql/get_untiled_changesets.sql: -------------------------------------------------------------------------------- 1 | SELECT changeset_id 2 | FROM 3 | 4 | (SELECT changeset_id, MAX(tstamp) AS tstamp 5 | FROM nodes n 6 | WHERE n.tstamp >= NOW() - INTERVAL '12 hour' 7 | GROUP BY changeset_id 8 | 9 | UNION 10 | 11 | SELECT changeset_id, MAX(tstamp) AS tstamp 12 | FROM ways w 13 | WHERE w.tstamp >= NOW() - INTERVAL '12 hour' 14 | GROUP BY changeset_id) x 15 | 16 | GROUP BY changeset_id 17 | HAVING (NOT EXISTS (SELECT 1 FROM tiles WHERE changeset_id = x.changeset_id) OR 18 | MAX(x.tstamp) < (SELECT MAX(tstamp) FROM tiles WHERE changeset_id = x.changeset_id)) 19 | -------------------------------------------------------------------------------- /app/workers/tiler_worker.rb: -------------------------------------------------------------------------------- 1 | class TilerWorker 2 | @queue = :tiler 3 | 4 | def self.perform(changeset_id) 5 | @conn = ActiveRecord::Base.connection.raw_connection() 6 | @conn.set_error_verbosity(0) 7 | @tiler = Tiler::ChangesetTiler.new(@conn) 8 | zoom = 18 9 | before = Time.now 10 | puts "Generating tiles for changeset #{changeset_id}..." 11 | tile_count = @tiler.generate(zoom, changeset_id, {:retile => true}) 12 | puts "Done, tile count: #{tile_count}" 13 | puts "Changeset #{changeset_id} took #{Time.now - before}s" 14 | { tiles: tile_count } 15 | end 16 | end 17 | -------------------------------------------------------------------------------- /config/initializers/wrap_parameters.rb: -------------------------------------------------------------------------------- 1 | # Be sure to restart your server when you modify this file. 2 | 3 | # This file contains settings for ActionController::ParamsWrapper which 4 | # is enabled by default. 5 | 6 | # Enable parameter wrapping for JSON. You can disable this by setting :format to an empty array. 7 | ActiveSupport.on_load(:action_controller) do 8 | wrap_parameters format: [:json] if respond_to?(:wrap_parameters) 9 | end 10 | 11 | # To enable root element in JSON for ActiveRecord objects. 12 | # ActiveSupport.on_load(:active_record) do 13 | # self.include_root_in_json = true 14 | # end 15 | -------------------------------------------------------------------------------- /app/views/api/changesets.atom.builder: -------------------------------------------------------------------------------- 1 | atom_feed do |feed| 2 | feed.title("OpenStreetMap Changesets") 3 | #feed.updated(@changesets[0].created_at) if @changesets.length > 0 4 | 5 | @changesets.each do |changeset| 6 | feed.entry(changeset, :url => "http://www.openstreetmap.org/browse/changeset/#{changeset.id}") do |entry| 7 | entry.title("Changeset #{changeset.id} by #{changeset.user_name}") 8 | entry.content("Changes: #{changeset.entity_changes}", :type => 'html') 9 | 10 | entry.author do |author| 11 | author.name(changeset.user_name) 12 | end 13 | end 14 | end 15 | end 16 | -------------------------------------------------------------------------------- /lib/tiler/geos_utils.rb: -------------------------------------------------------------------------------- 1 | require 'ffi-geos' 2 | 3 | def get_tile_geom(x, y, zoom) 4 | cs = Geos::CoordinateSequence.new(5, 2) 5 | y1, x1 = tile2latlon(x, y, zoom) 6 | y2, x2 = tile2latlon(x + 1, y + 1, zoom) 7 | cs.y[0], cs.x[0] = y1, x1 8 | cs.y[1], cs.x[1] = y1, x2 9 | cs.y[2], cs.x[2] = y2, x2 10 | cs.y[3], cs.x[3] = y2, x1 11 | cs.y[4], cs.x[4] = y1, x1 12 | Geos::create_polygon(cs, :srid => 4326) 13 | end 14 | 15 | 16 | def envelope_to_bbox(envelope) 17 | return nil if not envelope.is_a?(Geos::Polygon) 18 | ring = envelope.exterior_ring 19 | [ring[0].x, ring[0].y, ring[2].x, ring[2].y] 20 | end 21 | -------------------------------------------------------------------------------- /lib/tiler/cmdline_options.rb: -------------------------------------------------------------------------------- 1 | require 'optparse' 2 | 3 | module Tiler 4 | 5 | def self.parse_cmdline_options 6 | options = {} 7 | 8 | opt = OptionParser.new do |opts| 9 | opts.banner = "Usage: tiler.rb [options]" 10 | 11 | opts.on("--changes", "Generate changes for each changeset before doing the tiling") do |o| 12 | options[:changes] = o 13 | end 14 | 15 | opts.separator('') 16 | 17 | opts.on("--retile", "Remove existing tiles and regenerate tiles from scratch (optional, default is false)") do |o| 18 | options[:retile] = o 19 | end 20 | end 21 | 22 | opt.parse! 23 | options 24 | end 25 | 26 | end 27 | -------------------------------------------------------------------------------- /scripts/update_changesets.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | STDOUT.sync = true 4 | 5 | require 'pg' 6 | require 'yaml' 7 | 8 | $config = YAML.load_file('../rails/config/database.yml')['development'] 9 | 10 | @conn = PGconn.open(:host => $config['host'], :port => $config['port'], :dbname => $config['database'], 11 | :user => $config['username'], :password => $config['password']) 12 | 13 | changesets = @conn.query("SELECT DISTINCT changeset_id FROM nodes").to_a 14 | 15 | changesets.each_with_index do |row, index| 16 | puts "#{Time.now} - Changeset #{row['id']} (#{index + 1} / #{changesets.size})" 17 | @conn.query("SELECT OWL_GenerateChanges(#{row['changeset_id']})") 18 | end 19 | -------------------------------------------------------------------------------- /config/initializers/secret_token.rb: -------------------------------------------------------------------------------- 1 | # Be sure to restart your server when you modify this file. 2 | 3 | # Your secret key is used for verifying the integrity of signed cookies. 4 | # If you change this key, all old signed cookies will become invalid! 5 | 6 | # Make sure the secret is at least 30 characters and all random, 7 | # no regular words or you'll be exposed to dictionary attacks. 8 | # You can use `rake secret` to generate a secure secret key. 9 | 10 | # Make sure your secret_key_base is kept private 11 | # if you're sharing your code publicly. 12 | Owl::Application.config.secret_key_base = '3d5f55846dff206020d13b824016084bbd884e711f7ae061abe602d6f7da4e2ffe505b11112f2554f7a0b475fcc7865ed3ef69c08179f71bae4c177f03bbcc48' 13 | -------------------------------------------------------------------------------- /db/sql/owl_constraints.sql: -------------------------------------------------------------------------------- 1 | -- Primary keys 2 | ALTER TABLE ONLY changes ADD CONSTRAINT pk_changes PRIMARY KEY (id); 3 | ALTER TABLE ONLY changesets ADD CONSTRAINT pk_changesets PRIMARY KEY (id); 4 | ALTER TABLE ONLY changeset_tiles ADD CONSTRAINT pk_changeset_tiles PRIMARY KEY (change_id, x, y, zoom); 5 | 6 | ALTER TABLE ONLY nodes ADD CONSTRAINT pk_nodes PRIMARY KEY (id, version); 7 | ALTER TABLE ONLY ways ADD CONSTRAINT pk_ways PRIMARY KEY (id, version); 8 | ALTER TABLE ONLY relations ADD CONSTRAINT pk_relations PRIMARY KEY (id, version); 9 | ALTER TABLE ONLY relation_members ADD CONSTRAINT pk_relation_members PRIMARY KEY (relation_id, version, sequence_id); 10 | ALTER TABLE ONLY users ADD CONSTRAINT pk_users PRIMARY KEY (id); 11 | -------------------------------------------------------------------------------- /app/assets/javascripts/application.js: -------------------------------------------------------------------------------- 1 | // This is a manifest file that'll be compiled into application.js, which will include all the files 2 | // listed below. 3 | // 4 | // Any JavaScript/Coffee file within this directory, lib/assets/javascripts, vendor/assets/javascripts, 5 | // or vendor/assets/javascripts of plugins, if any, can be referenced here using a relative path. 6 | // 7 | // It's not advisable to add code directly here, but if you do, it'll appear at the bottom of the 8 | // the compiled file. 9 | // 10 | // WARNING: THE FIRST BLANK LINE MARKS THE END OF WHAT'S TO BE PROCESSED, ANY BLANK LINE SHOULD 11 | // GO AFTER THE REQUIRES BELOW. 12 | // 13 | //= require jquery 14 | //= require jquery_ujs 15 | //= require_tree . 16 | -------------------------------------------------------------------------------- /scripts/way_tiler.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | $:.unshift File.absolute_path(File.dirname(__FILE__) + '/../tiler/lib/') 4 | 5 | require 'pg' 6 | require 'yaml' 7 | require 'way_tiler' 8 | 9 | $config = YAML.load_file('../rails/config/database.yml')['development'] 10 | 11 | @conn = PGconn.open(:host => $config['host'], :port => $config['port'], :dbname => $config['database'], 12 | :user => $config['username'], :password => $config['password']) 13 | @conn.set_error_verbosity(0) 14 | 15 | @way_tiler = ::Tiler::WayTiler.new(@conn) 16 | 17 | i = 0 18 | ARGF.each_line do |way_id| 19 | @conn.transaction do |c| 20 | @way_tiler.create_way_tiles(way_id.to_i) 21 | end 22 | i += 1 23 | puts "---- #{i}" if i % 1000 24 | end 25 | -------------------------------------------------------------------------------- /config/initializers/inflections.rb: -------------------------------------------------------------------------------- 1 | # Be sure to restart your server when you modify this file. 2 | 3 | # Add new inflection rules using the following format. Inflections 4 | # are locale specific, and you may define rules for as many different 5 | # locales as you wish. All of these examples are active by default: 6 | # ActiveSupport::Inflector.inflections(:en) do |inflect| 7 | # inflect.plural /^(ox)$/i, '\1en' 8 | # inflect.singular /^(ox)en/i, '\1' 9 | # inflect.irregular 'person', 'people' 10 | # inflect.uncountable %w( fish sheep ) 11 | # end 12 | 13 | # These inflection rules are supported but not enabled by default: 14 | # ActiveSupport::Inflector.inflections(:en) do |inflect| 15 | # inflect.acronym 'RESTful' 16 | # end 17 | -------------------------------------------------------------------------------- /config/locales/en.yml: -------------------------------------------------------------------------------- 1 | # Files in the config/locales directory are used for internationalization 2 | # and are automatically loaded by Rails. If you want to use locales other 3 | # than English, add the necessary files in this directory. 4 | # 5 | # To use the locales, use `I18n.t`: 6 | # 7 | # I18n.t 'hello' 8 | # 9 | # In views, this is aliased to just `t`: 10 | # 11 | # <%= t('hello') %> 12 | # 13 | # To use a different locale, set it with `I18n.locale`: 14 | # 15 | # I18n.locale = :es 16 | # 17 | # This would use the information in config/locales/es.yml. 18 | # 19 | # To learn more, please read the Rails Internationalization guide 20 | # available at http://guides.rubyonrails.org/i18n.html. 21 | 22 | en: 23 | hello: "Hello world" 24 | -------------------------------------------------------------------------------- /config/database.yml: -------------------------------------------------------------------------------- 1 | development: 2 | adapter: postgresql 3 | host: localhost 4 | # port: 54320 5 | #database: europe_osm 6 | database: owl 7 | username: ppawel 8 | # password: RineX212 9 | pool: 5 10 | 11 | # Warning: The database defined as "test" will be erased and 12 | # re-generated from your development database when you run "rake". 13 | # Do not set this db to the same as development or production. 14 | test: 15 | adapter: postgresql 16 | host: localhost 17 | #port: 3333 18 | #database: europe_osm 19 | database: owl_test 20 | username: ppawel 21 | password: aa 22 | pool: 5 23 | 24 | production: 25 | adapter: postgresql 26 | host: localhost 27 | port: 54320 28 | database: owl 29 | username: ppawel 30 | password: RineX212 31 | pool: 5 32 | -------------------------------------------------------------------------------- /scripts/aggregate_changesets.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | STDOUT.sync = true 4 | 5 | require 'pg' 6 | require 'yaml' 7 | 8 | $config = YAML.load_file('../rails/config/database.yml')['development'] 9 | 10 | @conn = PGconn.open(:host => $config['host'], :port => $config['port'], :dbname => $config['database'], 11 | :user => $config['username'], :password => $config['password']) 12 | 13 | count = 0 14 | 15 | ARGF.each_line do |line| 16 | changeset_id = line.to_i 17 | next if changeset_id == 0 18 | count += 1 19 | puts "#{Time.now} Aggregating changeset #{changeset_id}... (#{count})" 20 | @conn.transaction do |c| 21 | (3..16).reverse_each do |i| 22 | @conn.query("SELECT OWL_AggregateChangeset(#{changeset_id}, #{i}, #{i - 1})") 23 | end 24 | end 25 | end 26 | -------------------------------------------------------------------------------- /lib/tiler/test/perf_test.rb: -------------------------------------------------------------------------------- 1 | $:.unshift File.absolute_path(File.dirname(__FILE__)) + '/../lib' 2 | 3 | require 'pg' 4 | require 'test/unit' 5 | require 'yaml' 6 | require 'changeset_tiler' 7 | 8 | require './common' 9 | 10 | class TilerTest < Test::Unit::TestCase 11 | include TestCommon 12 | 13 | def test_lots_of_tiles2 14 | setup_changeset_test(10822980) 15 | assert_equal(48043, @tiles.size) 16 | end 17 | 18 | def test_lots_of_tiles 19 | setup_changeset_test(8146963) 20 | assert_equal(2396, @tiles.size) 21 | end 22 | 23 | def test_lots_of_changes 24 | setup_changeset_test(8193200) 25 | assert_equal(111, @tiles.size) 26 | end 27 | 28 | def test_dateline 29 | setup_changeset_test(14483444) 30 | end 31 | 32 | def test_memory 33 | setup_changeset_test(3519977) 34 | end 35 | 36 | def test_oom 37 | setup_changeset_test(14806915) 38 | end 39 | end 40 | -------------------------------------------------------------------------------- /lib/tiler/test/utils_test.rb: -------------------------------------------------------------------------------- 1 | $:.unshift File.absolute_path(File.dirname(__FILE__) + '/../lib/') 2 | 3 | require 'test/unit' 4 | require 'utils' 5 | 6 | class UtilsTest < Test::Unit::TestCase 7 | def test_bbox_to_tiles 8 | tiles = bbox_to_tiles(18, [-95.2174366, 18.4330814, -95.2174366, 18.4330814]) 9 | assert_equal(61736, tiles.to_a[0][0]) 10 | assert_equal(117411, tiles.to_a[0][1]) 11 | 12 | tiles = bbox_to_tiles(12, [-95.2143046, 18.450548, -95.2143046, 18.450548]) 13 | assert_equal(964, tiles.to_a[0][0]) 14 | assert_equal(1834, tiles.to_a[0][1]) 15 | end 16 | 17 | def test_box2d_to_bbox 18 | bbox = box2d_to_bbox('BOX(5.8243191 45.1378079,5.8243191 45.1378079)') 19 | assert_equal(5.8243191, bbox[0]) 20 | assert_equal(45.1378079, bbox[1]) 21 | assert_equal(5.8243191, bbox[2]) 22 | assert_equal(45.1378079, bbox[3]) 23 | end 24 | 25 | def test_south_pole_tile 26 | puts latlon2tile(-90.0, 0.0, 18) 27 | end 28 | end 29 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | NOTE: Code rewrite is currently in progress, things probably don't work as described! 2 | 3 | OWL: OpenStreetMap Watch List 4 | ============================= 5 | 6 | OWL is a service that allows monitoring and analyzing changes in OpenStreetMap data. 7 | 8 | Wiki: http://wiki.openstreetmap.org/wiki/OWL 9 | Installation: http://wiki.openstreetmap.org/wiki/OWL/Installation 10 | API: http://wiki.openstreetmap.org/wiki/OWL/API 11 | 12 | Contents: 13 | --------- 14 | 15 | osmosis-plugin/ 16 | Git submodule containing a plugin for Osmosis that takes care of 17 | populating the database. 18 | 19 | See the `INSTALL.md` file for installation and usage instructions. 20 | 21 | rails/ 22 | A Rails project which hosts the API which allows applications 23 | to interface with OWL. 24 | 25 | scripts/ 26 | Miscellaneous scripts to keep bits of the database up-to-date, 27 | like the OWL changeset details scraper. 28 | 29 | sql/ 30 | SQL scripts for setting up a database with OWL schema. 31 | 32 | tiler/ 33 | A tool for creating geometry tiles that are served by the API. 34 | -------------------------------------------------------------------------------- /test/fixtures/tiler_unit_move_way.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO nodes VALUES (1, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.001 0.001)'), 4326)); 2 | INSERT INTO nodes VALUES (2, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.002 0.002)'), 4326)); 3 | INSERT INTO nodes VALUES (3, 1, 't', 1, NOW(), 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.003 0.003)'), 4326)); 4 | INSERT INTO ways VALUES (3, 1, 't', 1, NOW(), 1, '', ARRAY[1, 2, 3]); 5 | INSERT INTO nodes VALUES (3, 2, 't', 1, NOW(), 2, '', ST_SetSRID(ST_GeomFromText('POINT(0.0035 0.0035)'), 4326)); 6 | 7 | INSERT INTO nodes VALUES (4, 1, 't', 1, NOW(), 3, '', ST_SetSRID(ST_GeomFromText('POINT(0.001 0.001)'), 4326)); 8 | INSERT INTO nodes VALUES (5, 1, 't', 1, NOW(), 3, '', ST_SetSRID(ST_GeomFromText('POINT(0.002 0.002)'), 4326)); 9 | INSERT INTO nodes VALUES (6, 1, 't', 1, NOW(), 3, '', ST_SetSRID(ST_GeomFromText('POINT(0.003 0.003)'), 4326)); 10 | INSERT INTO ways VALUES (4, 1, 't', 1, NOW(), 3, '', ARRAY[4, 5, 6]); 11 | INSERT INTO nodes VALUES (5, 2, 't', 1, NOW(), 3, '', ST_SetSRID(ST_GeomFromText('POINT(0.0035 0.0035)'), 4326)); 12 | -------------------------------------------------------------------------------- /app/controllers/admin_controller.rb: -------------------------------------------------------------------------------- 1 | class AdminController < ActionController::Base 2 | protect_from_forgery 3 | 4 | def index 5 | end 6 | 7 | def go 8 | params[:ids].each_line do |line| 9 | Resque.enqueue(TilerWorker, line.to_i) 10 | end 11 | render nothing: true 12 | end 13 | 14 | def go_latest 15 | sql = "SELECT DISTINCT changeset_id FROM ways w WHERE NOT EXISTS 16 | (SELECT 1 FROM changeset_tiles WHERE changeset_id = w.changeset_id) 17 | ORDER BY changeset_id DESC 18 | LIMIT #{params[:limit]}" 19 | go_from_sql(sql) 20 | render nothing: true 21 | end 22 | 23 | def go_nearby 24 | sql = "SELECT DISTINCT changeset_id FROM ways w WHERE NOT EXISTS 25 | (SELECT 1 FROM changeset_tiles WHERE changeset_id = w.changeset_id) 26 | ORDER BY changeset_id DESC 27 | LIMIT #{params[:limit]}" 28 | go_from_sql(sql) 29 | render nothing: true 30 | end 31 | 32 | def go_from_sql(sql) 33 | for row in ActiveRecord::Base.connection.execute(sql) do 34 | Resque.enqueue(TilerWorker, row['changeset_id'].to_i) 35 | end 36 | end 37 | end 38 | -------------------------------------------------------------------------------- /app/controllers/application_controller.rb: -------------------------------------------------------------------------------- 1 | class ApplicationController < ActionController::Base 2 | protect_from_forgery 3 | 4 | before_filter :cors_preflight_check 5 | after_filter :cors_set_access_control_headers 6 | 7 | # For all responses in this controller, return the CORS access control headers. 8 | 9 | def cors_set_access_control_headers 10 | headers['Access-Control-Allow-Origin'] = '*' 11 | headers['Access-Control-Allow-Methods'] = 'POST, GET, OPTIONS' 12 | headers['Access-Control-Max-Age'] = "1728000" 13 | end 14 | 15 | # If this is a preflight OPTIONS request, then short-circuit the 16 | # request, return only the necessary headers and return an empty 17 | # text/plain. 18 | 19 | def cors_preflight_check 20 | if request.method == :options 21 | headers['Access-Control-Allow-Origin'] = '*' 22 | headers['Access-Control-Allow-Methods'] = 'POST, GET, OPTIONS' 23 | headers['Access-Control-Allow-Headers'] = 'X-Requested-With, X-Prototype-Version' 24 | headers['Access-Control-Max-Age'] = '1728000' 25 | render :text => '', :content_type => 'text/plain' 26 | end 27 | end 28 | end 29 | -------------------------------------------------------------------------------- /test/controllers/changeset_api_controller_realdata_test.rb: -------------------------------------------------------------------------------- 1 | require 'test_helper' 2 | require 'test/common' 3 | 4 | class ChangesetApiControllerTest < ActionController::TestCase 5 | include TestCommon 6 | 7 | test "13473782" do 8 | setup_changeset_test(13473782) 9 | verify_api_response 10 | end 11 | 12 | def verify_api_response 13 | # For every tile we request it through the API and verify JSON from the response 14 | for tile in @conn.query("SELECT * FROM changeset_tiles WHERE zoom = #{TEST_ZOOM}").to_a do 15 | get(:changesets_tile_json, {:x => tile['x'].to_i, :y => tile['y'].to_i, :zoom => TEST_ZOOM}) 16 | json = JSON[@response.body] 17 | assert(!json.empty?, 'Response is empty for tile: ' + tile.to_s) 18 | for changeset in json do 19 | change_ids = [] 20 | for change in changeset['changes'] do 21 | change_ids << change['id'] 22 | assert((!change['geom'].nil? or !change['prev_geom'].nil?), 'No geometry for change: ' + change.to_s) 23 | end 24 | assert_equal(change_ids.uniq, change_ids, 'Changeset has duplicate changes: ' + changeset.to_s) 25 | end 26 | end 27 | end 28 | end 29 | -------------------------------------------------------------------------------- /config/application.rb: -------------------------------------------------------------------------------- 1 | require File.expand_path('../boot', __FILE__) 2 | 3 | require 'rails/all' 4 | 5 | # Require the gems listed in Gemfile, including any gems 6 | # you've limited to :test, :development, or :production. 7 | Bundler.require(:default, Rails.env) 8 | 9 | module Owl 10 | class Application < Rails::Application 11 | # Settings in config/environments/* take precedence over those specified here. 12 | # Application configuration should go into files in config/initializers 13 | # -- all .rb files in that directory are automatically loaded. 14 | 15 | # Set Time.zone default to the specified zone and make Active Record auto-convert to this zone. 16 | # Run "rake -D time" for a list of tasks for finding time zone names. Default is UTC. 17 | # config.time_zone = 'Central Time (US & Canada)' 18 | 19 | # The default locale is :en and all translations from config/locales/*.rb,yml are auto loaded. 20 | # config.i18n.load_path += Dir[Rails.root.join('my', 'locales', '*.{rb,yml}').to_s] 21 | # config.i18n.default_locale = :de 22 | 23 | #config.active_record.schema_format = :sql 24 | 25 | config.autoload_paths << "#{Rails.root}/lib/tiler" 26 | end 27 | end 28 | -------------------------------------------------------------------------------- /config/environments/development.rb: -------------------------------------------------------------------------------- 1 | Owl::Application.configure do 2 | # Settings specified here will take precedence over those in config/application.rb. 3 | 4 | # In the development environment your application's code is reloaded on 5 | # every request. This slows down response time but is perfect for development 6 | # since you don't have to restart the web server when you make code changes. 7 | config.cache_classes = false 8 | 9 | # Do not eager load code on boot. 10 | config.eager_load = false 11 | 12 | # Show full error reports and disable caching. 13 | config.consider_all_requests_local = true 14 | config.action_controller.perform_caching = false 15 | 16 | # Don't care if the mailer can't send. 17 | config.action_mailer.raise_delivery_errors = false 18 | 19 | # Print deprecation notices to the Rails logger. 20 | config.active_support.deprecation = :log 21 | 22 | # Raise an error on page load if there are pending migrations 23 | config.active_record.migration_error = :page_load 24 | 25 | # Debug mode disables concatenation and preprocessing of assets. 26 | # This option may cause significant delays in view rendering with a large 27 | # number of complex assets. 28 | config.assets.debug = true 29 | end 30 | -------------------------------------------------------------------------------- /lib/tasks/owl.rake: -------------------------------------------------------------------------------- 1 | Rake::TestTask.new do |t| 2 | t.name = 'owl:test:tiler:realdata' 3 | t.loader = :testrb 4 | t.libs << 'lib/tiler' 5 | t.test_files = FileList['lib/tiler/test/tiler_realdata_test.rb'] 6 | t.verbose = true 7 | end 8 | 9 | Rake::TestTask.new do |t| 10 | t.name = 'owl:test:tiler:unit' 11 | t.loader = :testrb 12 | t.libs << 'lib/tiler' 13 | t.test_files = FileList['lib/tiler/test/tiler_unit_test.rb'] 14 | t.verbose = true 15 | end 16 | 17 | Rake::TestTask.new do |t| 18 | t.name = 'owl:test:api:realdata' 19 | t.libs << 'test' 20 | t.test_files = FileList['test/controllers/*realdata_test.rb'] 21 | t.verbose = true 22 | end 23 | 24 | Rake::TestTask.new do |t| 25 | t.name = 'owl:test:api:unit' 26 | t.libs << 'test' 27 | t.test_files = FileList['test/controllers/*unit_test.rb'] 28 | t.verbose = true 29 | end 30 | 31 | # Don't need those, database is managed manually. 32 | 33 | Rake.application.remove_task 'db:test:load' 34 | Rake.application.remove_task 'db:test:purge' 35 | 36 | namespace :db do 37 | namespace :test do 38 | task :load do |t| 39 | # rewrite the task to not do anything you don't want 40 | end 41 | task :purge do |t| 42 | # rewrite the task to not do anything you don't want 43 | end 44 | end 45 | end -------------------------------------------------------------------------------- /lib/tiler/summary_tiler.rb: -------------------------------------------------------------------------------- 1 | module Tiler 2 | 3 | # Implements tiling logic for summary tiles. 4 | class SummaryTiler 5 | include ::Tiler::Logger 6 | 7 | attr_accessor :conn 8 | 9 | def initialize(conn) 10 | @conn = conn 11 | end 12 | 13 | def generate_summary_tiles(summary_zoom) 14 | clear_summary_tiles(summary_zoom) 15 | subtiles_per_tile = 2**16 / 2**summary_zoom 16 | 17 | for x in (0..2**summary_zoom - 1) 18 | for y in (0..2**summary_zoom - 1) 19 | num_changesets = @conn.query(" 20 | SELECT COUNT(DISTINCT changeset_id) AS num_changesets 21 | FROM changeset_tiles 22 | WHERE zoom = 16 23 | AND x >= #{x * subtiles_per_tile} AND x < #{(x + 1) * subtiles_per_tile} 24 | AND y >= #{y * subtiles_per_tile} AND y < #{(y + 1) * subtiles_per_tile} 25 | ").to_a[0]['num_changesets'].to_i 26 | 27 | @@log.debug "Tile (#{x}, #{y}), num_changesets = #{num_changesets}" 28 | 29 | @conn.query("INSERT INTO summary_tiles (num_changesets, zoom, x, y) 30 | VALUES (#{num_changesets}, #{summary_zoom}, #{x}, #{y})") 31 | end 32 | end 33 | end 34 | 35 | def clear_summary_tiles(zoom) 36 | @conn.query("DELETE FROM summary_tiles WHERE zoom = #{zoom}").cmd_tuples 37 | end 38 | end 39 | -------------------------------------------------------------------------------- /db/sql/owl_indexes.sql: -------------------------------------------------------------------------------- 1 | -- OWL is meant to be very conservative with indexes because of the amount of data it processes (indexes are expensive). 2 | -- So we only create indexes that we really use and need - for every index there needs to be a short explanation. 3 | 4 | -- Used for looking up ways that a node belongs to (@> operator for the "nodes" column). 5 | CREATE INDEX idx_ways_nodes_id ON ways USING gin (nodes); 6 | 7 | -- Used for selecting changeset elements. 8 | CREATE INDEX idx_nodes_changeset_id ON nodes USING btree (changeset_id); 9 | CREATE INDEX idx_relations_changeset_id ON relations USING btree (changeset_id); 10 | CREATE INDEX idx_ways_changeset_id ON ways USING btree (changeset_id); 11 | 12 | -- Used for selecting nodes (by id) for a specific way when constructing way geometry. 13 | CREATE INDEX idx_nodes_node_id ON nodes USING btree (id); 14 | 15 | -- Used by the changeset API to locate tiles by specific tile or tile range. 16 | CREATE INDEX idx_changeset_tiles_xyz ON changeset_tiles USING btree (zoom, x, y); 17 | CREATE INDEX idx_changeset_tiles_xcyz ON changeset_tiles USING btree (zoom, changeset_id, x, y); 18 | 19 | -- Used during replication to select latest objects. 20 | --CREATE INDEX idx_nodes_tstamp ON nodes USING btree (tstamp); 21 | --CREATE INDEX idx_ways_tstamp ON ways USING btree (tstamp); 22 | 23 | -- Used by the vector tiles API. 24 | --CREATE INDEX idx_nodes_geom ON nodes USING gist (geom) WHERE visible AND current; 25 | -------------------------------------------------------------------------------- /scripts/dump_testdata.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | PSQL_OPTIONS=$1 4 | CHANGESET_ID=$2 5 | 6 | if [ "$CHANGESET_ID" = "" ]; then 7 | echo "Usage: dump_testdata.sh " 8 | exit 1 9 | fi 10 | 11 | psql $PSQL_OPTIONS -a -c "\copy (SELECT * FROM changesets WHERE id = $CHANGESET_ID) TO '../testdata/$CHANGESET_ID-changeset.csv'" 12 | 13 | psql $PSQL_OPTIONS -a -c "\copy ( \ 14 | WITH way_ids AS \ 15 | (SELECT id FROM ways WHERE nodes && (SELECT array_agg(id) FROM nodes WHERE changeset_id = $CHANGESET_ID) UNION \ 16 | SELECT id FROM ways WHERE changeset_id = $CHANGESET_ID), \ 17 | node_data AS (SELECT * FROM nodes WHERE changeset_id = $CHANGESET_ID \ 18 | UNION SELECT * FROM nodes WHERE id IN (SELECT unnest(nodes) FROM ways WHERE id IN (SELECT id FROM way_ids) )) \ 19 | (SELECT id, version, visible, user_id, tstamp, changeset_id, tags, geom FROM nodes n WHERE n.id IN (SELECT id FROM node_data)) \ 20 | ORDER BY id, version \ 21 | ) TO '../testdata/$CHANGESET_ID-nodes.csv'" 22 | 23 | psql $PSQL_OPTIONS -a -c "\copy ( \ 24 | WITH way_ids AS (SELECT id FROM ways WHERE changeset_id = $CHANGESET_ID UNION \ 25 | SELECT id FROM ways WHERE nodes && (SELECT array_agg(id) FROM nodes WHERE changeset_id = $CHANGESET_ID)) \ 26 | (SELECT id, version, visible, user_id, tstamp, changeset_id, tags, nodes FROM ways w WHERE w.id IN (SELECT id FROM way_ids)) \ 27 | ORDER BY id, version \ 28 | ) TO '../testdata/$CHANGESET_ID-ways.csv'" 29 | -------------------------------------------------------------------------------- /scripts/tiler.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | $:.unshift File.absolute_path(File.dirname(__FILE__) + '/../lib/') 4 | 5 | # Useful when redirecting log output to a file. 6 | STDOUT.sync = true 7 | 8 | require 'pg' 9 | require 'yaml' 10 | 11 | require 'tiler/cmdline_options' 12 | require 'tiler/logging' 13 | require 'tiler/changeset_tiler' 14 | 15 | GC.enable 16 | 17 | $config = YAML.load_file(File.absolute_path(File.dirname(__FILE__) + '/../config/database.yml'))['development'] 18 | options = Tiler::parse_cmdline_options 19 | puts options.inspect 20 | zoom = 18 21 | changeset_ids = ARGF.each_line.collect {|line| line.to_i} 22 | 23 | conn = PGconn.open(:host => $config['host'], :port => $config['port'], :dbname => $config['database'], 24 | :user => $config['username'], :password => $config['password']) 25 | conn.set_error_verbosity(0) 26 | tiler = Tiler::ChangesetTiler.new(conn) 27 | 28 | changeset_ids.each_with_index do |changeset_id, count| 29 | next if changeset_id == 0 30 | 31 | # Print out some diagnostic information. 32 | if count % 1000 == 0 33 | GC.start 34 | p GC::stat 35 | p GC::Profiler.result 36 | p GC::Profiler.total_time 37 | GC::Profiler.report 38 | end 39 | 40 | before = Time.now 41 | puts "Generating tiles for changeset #{changeset_id}... (#{count})" 42 | tile_count = tiler.generate(zoom, changeset_id, options) 43 | puts "Done, tile count: #{tile_count}" 44 | puts "Changeset #{changeset_id} took #{Time.now - before}s (#{count})" 45 | end 46 | -------------------------------------------------------------------------------- /app/assets/stylesheets/application.css: -------------------------------------------------------------------------------- 1 | /* 2 | * This is a manifest file that'll be compiled into application.css, which will include all the files 3 | * listed below. 4 | * 5 | * Any CSS and SCSS file within this directory, lib/assets/stylesheets, vendor/assets/stylesheets, 6 | * or vendor/assets/stylesheets of plugins, if any, can be referenced here using a relative path. 7 | * 8 | * You're free to add application-wide styles to this file and they'll appear at the top of the 9 | * compiled file, but it's generally better to create a new file per style scope. 10 | * 11 | *= require_self 12 | *= require_tree . 13 | */ 14 | 15 | body { 16 | font-family: "HelveticaNeue", "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif; 17 | font-weight: normal; 18 | font-size: 14px; 19 | } 20 | 21 | #wrap { 22 | width: 100%; 23 | } 24 | 25 | #left_col_container { 26 | width: 20%; 27 | height: 100%; 28 | float: left; 29 | } 30 | 31 | #left_col { 32 | padding: 5px; 33 | } 34 | 35 | #right_col { 36 | float: right; 37 | width: 80%; 38 | } 39 | 40 | #map { 41 | width: 100%; 42 | height: 100%; 43 | } 44 | 45 | #changeset_list { 46 | max-height: 500px; 47 | overflow: auto; 48 | clear: both; 49 | } 50 | 51 | .changeset_item { 52 | background: #eeeeee; 53 | margin: 3px; 54 | line-height: 20px; 55 | padding: 5px; 56 | } 57 | 58 | .changeset_item_selected { 59 | background-color: lightblue; 60 | margin: 3px; 61 | line-height: 20px; 62 | padding: 5px; 63 | } 64 | -------------------------------------------------------------------------------- /public/500.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | We're sorry, but something went wrong (500) 5 | 48 | 49 | 50 | 51 | 52 |
53 |

We're sorry, but something went wrong.

54 |
55 |

If you are the application owner check the logs for more information.

56 | 57 | 58 | -------------------------------------------------------------------------------- /config/routes.rb: -------------------------------------------------------------------------------- 1 | require 'resque/server' 2 | #require 'sidekiq/web' 3 | #require 'sidekiq_status/web' 4 | 5 | Owl::Application.routes.draw do 6 | @xyz_constraints = {:zoom => /\d+/, :x => /\d+/, :y => /\d+/} 7 | @range_constraints = {:zoom => /\d+/, :x1 => /\d+/, :y1 => /\d+/, :x2 => /\d+/, :y2 => /\d+/} 8 | 9 | # The priority is based upon order of creation: 10 | # first created -> highest priority. 11 | 12 | # Tile operations 13 | get 'api/0.1/changesets/:zoom/:x/:y.atom' => 'changeset_api#changesets_tile_atom', :constraints => @xyz_constrains, :format => 'atom' 14 | get 'api/0.1/changesets/:zoom/:x/:y.json' => 'changeset_api#changesets_tile_json', :constraints => @xyz_constrains, :format => 'json' 15 | 16 | get 'api/0.1/summary/:zoom/:x/:y' => 'changeset_api#summary_tile', :constraints => @xyz_constrains 17 | get 'api/0.1/summary/:zoom/:x1/:y1/:x2/:y2' => 'changeset_api#summary_tilerange', :constraints => @range_constrains, :format => 'json' 18 | 19 | # Map API 20 | get 'api/0.1/kothic/:zoom/:x/:y.js' => 'map_api#kothic', :constraints => @xyz_constrains 21 | 22 | #get 'admin/:changeset_id' => 'application#test' 23 | get 'admin' => 'admin#index' 24 | get 'admin/process' => 'admin#process' 25 | post 'admin/go' => 'admin#go' 26 | get 'admin/go_latest' => 'admin#go_latest' 27 | post 'admin/spawn_workers' => 'admin#spawn_workers' 28 | 29 | mount Resque::Server.new, :at => "/resque" 30 | 31 | #mount Sidekiq::Web => '/sidekiq' 32 | #mount Sidekiq::Monitor::Engine => '/sidekiqm' 33 | end 34 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | # Bundle edge Rails instead: gem 'rails', github: 'rails/rails' 4 | gem 'rails', '4.1.9' 5 | 6 | # Use SCSS for stylesheets 7 | gem 'sass-rails', '~> 4.0.0' 8 | 9 | # Use Uglifier as compressor for JavaScript assets 10 | gem 'uglifier', '>= 1.3.0' 11 | 12 | # Use CoffeeScript for .js.coffee assets and views 13 | gem 'coffee-rails', '~> 4.0.0' 14 | 15 | # See https://github.com/sstephenson/execjs#readme for more supported runtimes 16 | # gem 'therubyracer', platforms: :ruby 17 | 18 | # Use jquery as the JavaScript library 19 | gem 'jquery-rails' 20 | 21 | # Turbolinks makes following links in your web application faster. Read more: https://github.com/rails/turbolinks 22 | gem 'turbolinks' 23 | 24 | # Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder 25 | gem 'jbuilder', '~> 1.2' 26 | 27 | group :doc do 28 | # bundle exec rake doc:rails generates the API under doc/api. 29 | gem 'sdoc', require: false 30 | end 31 | 32 | gem 'pg' 33 | gem 'ffi-geos' 34 | 35 | gem 'sinatra', '>= 1.3.0', :require => nil 36 | #gem 'sidekiq' 37 | #gem 'sidekiq_status' 38 | #gem 'sidekiq_monitor' 39 | 40 | gem 'minitest' 41 | 42 | gem 'resque' 43 | 44 | gem 'twitter-bootstrap-rails' 45 | 46 | # Use ActiveModel has_secure_password 47 | # gem 'bcrypt-ruby', '~> 3.0.0' 48 | 49 | # Use unicorn as the app server 50 | # gem 'unicorn' 51 | 52 | # Use Capistrano for deployment 53 | # gem 'capistrano', group: :development 54 | 55 | # Use debugger 56 | # gem 'debugger', group: [:development, :test] 57 | -------------------------------------------------------------------------------- /app/views/admin/index.html.erb: -------------------------------------------------------------------------------- 1 |

OWL

2 | <%= form_tag("/admin/go", method: "post", :remote => true, :class => 'form-horizontal') do %> 3 |
4 | <%= label_tag(:ids, "changeset ids", :class => 'control-label') %> 5 |
6 | <%= text_area_tag :ids %> 7 |
8 |
9 |
10 | <%= submit_tag("Go", :class => 'btn btn-primary') %> 11 |
12 | <% end %> 13 | 14 | <%= form_tag("/admin/go_latest", method: "get", :remote => true, :class => 'form-horizontal') do %> 15 |
16 | <%= label_tag(:limit, "limit", :class => 'control-label') %> 17 |
18 | <%= text_field_tag :limit, 100 %> 19 |
20 |
21 |
22 | <%= submit_tag("Go", :class => 'btn btn-primary') %> 23 |
24 | <% end %> 25 | 26 | <%= form_tag("/admin/go_nearby", method: "get", :remote => true, :class => 'form-horizontal') do %> 27 |
28 | <%= label_tag(:coords, "coords", :class => 'control-label') %> 29 |
30 | <%= text_field_tag :coords, '-34.0512/151.0203' %> 31 |
32 |
33 |
34 | <%= label_tag(:limit, "limit", :class => 'control-label') %> 35 |
36 | <%= text_field_tag :limit, 100 %> 37 |
38 |
39 |
40 | <%= submit_tag("Go", :class => 'btn btn-primary') %> 41 |
42 | <% end %> 43 | -------------------------------------------------------------------------------- /COPYING: -------------------------------------------------------------------------------- 1 | Copyright (c) 2010, Matt Amos 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are 6 | met: 7 | 8 | * Redistributions of source code must retain the above copyright 9 | notice, this list of conditions and the following disclaimer. 10 | 11 | * Redistributions in binary form must reproduce the above copyright 12 | notice, this list of conditions and the following disclaimer in the 13 | documentation and/or other materials provided with the distribution. 14 | 15 | Neither the name "OWL" nor the names of any contributors may be used 16 | to endorse or promote products derived from this software without 17 | specific prior written permission. 18 | 19 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 22 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 23 | HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 24 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 25 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /public/422.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | The change you wanted was rejected (422) 5 | 48 | 49 | 50 | 51 | 52 |
53 |

The change you wanted was rejected.

54 |

Maybe you tried to change something you didn't have access to.

55 |
56 |

If you are the application owner check the logs for more information.

57 | 58 | 59 | -------------------------------------------------------------------------------- /public/404.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | The page you were looking for doesn't exist (404) 5 | 48 | 49 | 50 | 51 | 52 |
53 |

The page you were looking for doesn't exist.

54 |

You may have mistyped the address or the page may have moved.

55 |
56 |

If you are the application owner check the logs for more information.

57 | 58 | 59 | -------------------------------------------------------------------------------- /config/environments/test.rb: -------------------------------------------------------------------------------- 1 | Owl::Application.configure do 2 | # Settings specified here will take precedence over those in config/application.rb. 3 | 4 | # The test environment is used exclusively to run your application's 5 | # test suite. You never need to work with it otherwise. Remember that 6 | # your test database is "scratch space" for the test suite and is wiped 7 | # and recreated between test runs. Don't rely on the data there! 8 | config.cache_classes = true 9 | 10 | # Do not eager load code on boot. This avoids loading your whole application 11 | # just for the purpose of running a single test. If you are using a tool that 12 | # preloads Rails for running tests, you may have to set it to true. 13 | config.eager_load = false 14 | 15 | # Configure static asset server for tests with Cache-Control for performance. 16 | config.serve_static_assets = true 17 | config.static_cache_control = "public, max-age=3600" 18 | 19 | # Show full error reports and disable caching. 20 | config.consider_all_requests_local = true 21 | config.action_controller.perform_caching = false 22 | 23 | # Raise exceptions instead of rendering exception templates. 24 | config.action_dispatch.show_exceptions = false 25 | 26 | # Disable request forgery protection in test environment. 27 | config.action_controller.allow_forgery_protection = false 28 | 29 | # Tell Action Mailer not to deliver emails to the real world. 30 | # The :test delivery method accumulates sent emails in the 31 | # ActionMailer::Base.deliveries array. 32 | config.action_mailer.delivery_method = :test 33 | 34 | # Print deprecation notices to the stderr. 35 | config.active_support.deprecation = :stderr 36 | 37 | config.active_record.schema_format = :sql 38 | end 39 | -------------------------------------------------------------------------------- /test/fixtures/tiler_unit_affect_way.sql: -------------------------------------------------------------------------------- 1 | INSERT INTO nodes VALUES (1, 1, 't', 1, '2013-03-03 18:00:00', 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.001 0.001)'), 4326)); 2 | INSERT INTO nodes VALUES (2, 1, 't', 1, '2013-03-03 18:00:00', 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.002 0.002)'), 4326)); 3 | INSERT INTO nodes VALUES (3, 1, 't', 1, '2013-03-03 18:00:00', 1, '', ST_SetSRID(ST_GeomFromText('POINT(0.003 0.003)'), 4326)); 4 | INSERT INTO ways VALUES (3, 1, 't', 1, '2013-03-03 18:00:00', 1, '', ARRAY[1, 2, 3]); 5 | 6 | INSERT INTO nodes VALUES (3, 2, 't', 1, '2013-03-03 19:00:00', 2, 'some=>tag', ST_SetSRID(ST_GeomFromText('POINT(0.003 0.003)'), 4326)); 7 | 8 | INSERT INTO nodes VALUES (10, 1, 't', 1, '2013-03-03 20:00:00', 3, '', ST_SetSRID(ST_GeomFromText('POINT(18.6023903 49.8778083)'), 4326)); 9 | INSERT INTO nodes VALUES (11, 1, 't', 1, '2013-03-03 20:00:01', 3, '', ST_SetSRID(ST_GeomFromText('POINT(18.6024903 49.8778083)'), 4326)); 10 | INSERT INTO nodes VALUES (12, 1, 't', 1, '2013-03-03 20:00:05', 3, '', ST_SetSRID(ST_GeomFromText('POINT(18.6025903 49.8778083)'), 4326)); 11 | 12 | INSERT INTO ways VALUES (5, 1, 't', 1, '2013-03-03 20:00:10', 3, '', ARRAY[10, 11, 12]); 13 | 14 | INSERT INTO nodes VALUES (11, 2, 't', 1, '2013-03-03 20:00:21', 3, '', ST_SetSRID(ST_GeomFromText('POINT(18.6024903 49.8779083)'), 4326)); 15 | 16 | INSERT INTO nodes VALUES (20, 1, 't', 1, '2013-03-03 21:00:00', 4, '', ST_SetSRID(ST_GeomFromText('POINT(18.6023903 49.8778083)'), 4326)); 17 | INSERT INTO nodes VALUES (21, 1, 't', 1, '2013-03-03 21:00:01', 4, '', ST_SetSRID(ST_GeomFromText('POINT(18.6024903 49.8778083)'), 4326)); 18 | INSERT INTO nodes VALUES (22, 1, 't', 1, '2013-03-03 21:00:05', 4, '', ST_SetSRID(ST_GeomFromText('POINT(18.6025903 49.8778083)'), 4326)); 19 | 20 | INSERT INTO ways VALUES (6, 1, 't', 1, '2013-03-03 21:00:10', 4, '', ARRAY[20, 21, 22]); 21 | INSERT INTO nodes VALUES (21, 2, 't', 1, '2013-03-03 21:00:21', 4, '', ST_SetSRID(ST_GeomFromText('POINT(18.6024903 49.8779083)'), 4326)); 22 | INSERT INTO ways VALUES (6, 2, 't', 1, '2013-03-03 21:10:10', 4, 'new=>tag', ARRAY[20, 21, 22]); 23 | INSERT INTO ways VALUES (6, 3, 't', 1, '2013-03-03 21:10:10', 4, 'new=>tag', ARRAY[20, 21, 22, 21]); 24 | -------------------------------------------------------------------------------- /app/controllers/map_api_controller.rb: -------------------------------------------------------------------------------- 1 | require 'utils' 2 | 3 | ## 4 | # Implements OWL API operations. 5 | # See: http://wiki.openstreetmap.org/wiki/OWL_(OpenStreetMap_Watch_List)/API 6 | # 7 | class MapApiController < ApplicationController 8 | include ApiHelper 9 | 10 | def kothic 11 | @x, @y, @zoom = get_xyz(params) 12 | tile_bbox = tile2bbox(@x, @y, @zoom) 13 | p tile_bbox 14 | tiles = ActiveRecord::Base.connection.select_all(" 15 | WITH tile_bbox AS ( 16 | SELECT ST_Transform(ST_SetSRID('LINESTRING(#{tile_bbox[0]} #{tile_bbox[1]}, 17 | #{tile_bbox[2]} #{tile_bbox[3]})'::geometry, 4326), 900913)::box2d g 18 | ) 19 | (SELECT DISTINCT ON (w.id, t.x, t.y) ST_AsGeoJSON(ST_TransScale(ST_Transform(geom, 900913), 20 | (SELECT -ST_XMin(g) FROM tile_bbox), 21 | (SELECT -ST_YMin(g) FROM tile_bbox), 22 | (SELECT 10000 / (ST_XMax(g) - ST_XMin(g)) FROM tile_bbox), 23 | (SELECT 10000 / (ST_YMax(g) - ST_YMin(g)) FROM tile_bbox)), 0) AS geojson, 24 | w.tags 25 | FROM way_tiles t 26 | INNER JOIN ways w ON (w.id = t.way_id AND w.version = t.version) 27 | WHERE x >= #{@x * (2 ** (16 - @zoom))} AND y >= #{@y * (2 ** (16 - @zoom))} AND 28 | x <= #{(@x + 1) * (2 ** (16 - @zoom))} AND y <= #{(@y + 1) * (2 ** (16 - @zoom))} 29 | ORDER BY w.id, t.x, t.y, t.rev DESC) 30 | UNION 31 | (SELECT DISTINCT ON (n.id) ST_AsGeoJSON(ST_TransScale(ST_Transform(n.geom, 900913), 32 | (SELECT -ST_XMin(g) FROM tile_bbox), 33 | (SELECT -ST_YMin(g) FROM tile_bbox), 34 | (SELECT 10000 / (ST_XMax(g) - ST_XMin(g)) FROM tile_bbox), 35 | (SELECT 10000 / (ST_YMax(g) - ST_YMin(g)) FROM tile_bbox)), 0) AS geojson, 36 | n.tags 37 | FROM nodes n 38 | WHERE n.visible AND n.current AND 39 | geom && (SELECT ST_Transform(ST_Setsrid(g, 900913), 4326) FROM tile_bbox) AND tags != ''::hstore 40 | ORDER BY n.id, n.version DESC) 41 | --LIMIT 10000 42 | ").to_a 43 | 44 | geojson = [] 45 | 46 | for tile in tiles 47 | kothic_tile = JSON[tile['geojson']] 48 | kothic_tile['properties'] = eval("{#{tile['tags']}}") 49 | geojson << kothic_tile 50 | end 51 | text = "onKothicDataResponse({\"features\": #{JSON[geojson].as_json}, \"bbox\":#{tile_bbox}, \"granularity\": 10000 }, 52 | #{@zoom}, #{@x}, #{@y})" 53 | render :js => text 54 | end 55 | 56 | end 57 | -------------------------------------------------------------------------------- /lib/tiler/changeset.rb: -------------------------------------------------------------------------------- 1 | class Changeset 2 | attr_accessor :id 3 | attr_accessor :user_id 4 | attr_accessor :user_name 5 | attr_accessor :created_at 6 | attr_accessor :closed_at 7 | attr_accessor :open 8 | attr_accessor :tags 9 | attr_accessor :entity_changes 10 | attr_accessor :num_changes 11 | attr_accessor :change_ids 12 | attr_accessor :bboxes 13 | attr_accessor :changes 14 | attr_accessor :geom_geojson 15 | attr_accessor :prev_geom_geojson 16 | 17 | def initialize(hash) 18 | @id = hash['id'].to_i 19 | @user_id = hash['user_id'].to_i 20 | @user_name = hash['user_name'] 21 | @created_at = hash['created_at'] ? Time.parse(hash['created_at']) : nil 22 | @closed_at = hash['closed_at'] ? Time.parse(hash['closed_at']) : nil 23 | @open = hash['open'] == 't' 24 | @tags = eval("{#{hash['tags']}}") 25 | @bboxes = box2d_to_bbox(hash['bboxes']) if hash['bboxes'] 26 | @changes = Change.from_pg_array(hash['changes']) 27 | 28 | geojsons = pg_string_to_array(hash['geojson']) 29 | prev_geojsons = pg_string_to_array(hash['prev_geojson']) 30 | change_tags = pg_string_to_array(hash['change_tags']) 31 | change_prev_tags = pg_string_to_array(hash['change_prev_tags']) 32 | 33 | @changes.each_with_index do |change, index| 34 | change.geom = JSON[geojsons[index]] if geojsons[index] 35 | change.prev_geom = JSON[prev_geojsons[index]] if prev_geojsons[index] 36 | change.tags = eval("{#{change_tags[index]}}") if change_tags[index] 37 | change.prev_tags = eval("{#{change_prev_tags[index]}}") if change_prev_tags[index] 38 | end 39 | end 40 | 41 | def generate_json(options = {:include_changes => true}) 42 | { 43 | "id" => id, 44 | "created_at" => created_at, 45 | "closed_at" => closed_at, 46 | "user_id" => user_id, 47 | "user_name" => user_name, 48 | "tags" => tags, 49 | #"bbox" => bbox ? box2d_to_bbox(total_bbox)[0] : nil, 50 | "changes" => @changes.as_json, 51 | "bboxes" => bboxes 52 | } 53 | end 54 | 55 | ## 56 | # Converts PostGIS' BOX2D string representation to a list. 57 | # bbox is [xmin, ymin, xmax, ymax] 58 | # 59 | def box2d_to_bbox(box2d) 60 | return [] if !box2d 61 | result = [] 62 | box2d.scan(/BOX\(([\d\.]+) ([\d\.]+),([\d\.]+) ([\d\.]+)\)/).each do |m| 63 | result << m.map(&:to_f) 64 | end 65 | result.uniq 66 | end 67 | 68 | def pg_string_to_array(str) 69 | return [] unless str 70 | dup = str.dup 71 | dup[0] = '[' 72 | dup[-1] = ']' 73 | dup.gsub!('NULL', 'nil') 74 | eval(dup) 75 | end 76 | end 77 | -------------------------------------------------------------------------------- /lib/tiler/test/tiler_unit_test.rb: -------------------------------------------------------------------------------- 1 | $:.unshift File.absolute_path(File.dirname(__FILE__)) + '/../lib' 2 | 3 | require 'pg' 4 | require 'minitest/autorun' 5 | require 'yaml' 6 | require 'tiler/changeset_tiler' 7 | require 'tiler/test/common' 8 | 9 | class TilerUnitTest < Minitest::Test 10 | include TestCommon 11 | 12 | def initialize(name = nil) 13 | @test_name = name 14 | super(name) unless name.nil? 15 | end 16 | 17 | def test_create_node 18 | setup_unit_test(@test_name) 19 | assert_equal(2, find_changes('el_type' => 'N').size) 20 | assert_equal(0, find_changes('el_id' => '1')[0]['tags'].size) 21 | end 22 | 23 | def test_delete_node 24 | setup_unit_test(@test_name) 25 | assert_equal(2, find_changes('el_type' => 'N').size) 26 | end 27 | 28 | def test_move_node 29 | setup_unit_test(@test_name) 30 | assert_equal(2, find_changes('el_type' => 'N').size) 31 | end 32 | 33 | def test_move_node_same_changeset 34 | setup_unit_test(@test_name) 35 | assert_equal(1, find_changes('el_type' => 'N').size) 36 | end 37 | 38 | def test_tag_node 39 | setup_unit_test(@test_name) 40 | assert_equal(1, find_changes('el_type' => 'N').size) 41 | assert_equal(1, find_changes('el_type' => 'N', 'version' => '2').size) 42 | end 43 | 44 | def test_create_way 45 | setup_unit_test(@test_name) 46 | assert_equal(0, find_changes('el_type' => 'N').size) 47 | assert_equal(1, find_changes('el_type' => 'W').size) 48 | end 49 | 50 | def test_move_way 51 | setup_unit_test(@test_name) 52 | assert_equal(0, find_changes('el_type' => 'N', 'changeset_id' => 1).size) 53 | assert_equal(0, find_changes('el_type' => 'N', 'changeset_id' => 2).size) 54 | assert_equal(1, find_changes('el_type' => 'W', 'changeset_id' => 2).size) 55 | assert_equal(0, find_changes('el_type' => 'N', 'changeset_id' => 3).size) 56 | assert_equal(1, find_changes('el_type' => 'W', 'action' => 'CREATE', 'changeset_id' => 3).size) 57 | end 58 | 59 | def test_delete_way 60 | setup_unit_test(@test_name) 61 | assert_equal(1, find_changes('el_type' => 'W', 'changeset_id' => 2).size) 62 | end 63 | 64 | def test_affect_way 65 | setup_unit_test(@test_name) 66 | assert_equal(1, find_changes('el_type' => 'N', 'changeset_id' => 2).size) 67 | assert_equal(0, find_changes('el_type' => 'W', 'changeset_id' => 2).size) 68 | 69 | assert_equal(0, find_changes('el_type' => 'N', 'changeset_id' => 3).size) 70 | assert_equal(1, find_changes('el_type' => 'W', 'changeset_id' => 3).size) 71 | assert_equal(1, find_changes('el_type' => 'W', 'action' => 'CREATE', 'changeset_id' => 3).size) 72 | assert_equal(1, find_changes('el_type' => 'W', 'action' => 'CREATE', 'changeset_id' => 4).size) 73 | end 74 | end 75 | -------------------------------------------------------------------------------- /test/controllers/changeset_api_controller_unit_test.rb: -------------------------------------------------------------------------------- 1 | require 'test_helper' 2 | 3 | class ChangesetApiControllerTest < ActionController::TestCase 4 | def initialize(name = nil) 5 | @test_name = name 6 | super(name) unless name.nil? 7 | end 8 | 9 | test "create_node" do 10 | setup_test 11 | get(:changesets_tile_json, {:x => 36154, :y => 22260, :zoom => 16}) 12 | changesets = assigns['changesets'] 13 | json = JSON[@response.body] 14 | verify_response(json) 15 | assert_equal(1, changesets.size) 16 | assert_equal(1, json.size) 17 | end 18 | 19 | test "delete_node" do 20 | setup_test 21 | get(:changesets_tile_json, {:x => 36154, :y => 22260, :zoom => 16}) 22 | changesets = assigns['changesets'] 23 | json = JSON[@response.body] 24 | verify_response(json) 25 | assert_equal(1, changesets.size) 26 | assert_equal(1, json.size) 27 | end 28 | 29 | test "move_way" do 30 | setup_test 31 | get(:changesets_tile_json, {:x => 36154, :y => 22260, :zoom => 16}) 32 | changesets = assigns['changesets'] 33 | json = JSON[@response.body] 34 | verify_response(json) 35 | assert_equal(1, changesets.size) 36 | assert_equal(1, json.size) 37 | end 38 | 39 | # Does some generic checks. 40 | def verify_response(changesets) 41 | for changeset in changesets 42 | for change in changeset['changes'] 43 | if change['el_action'] == 'CREATE' 44 | assert(change['prev_tags'].nil?) 45 | assert(!change['tags'].nil?) 46 | assert(change['prev_geom'].nil?) 47 | assert(!change['geom'].nil?) 48 | elsif change['el_action'] == 'DELETE' 49 | #assert(change['tags'].nil?) 50 | assert(!change['prev_tags'].nil?) 51 | assert(change['geom'].nil?) 52 | assert(!change['prev_geom'].nil?) 53 | end 54 | end 55 | end 56 | end 57 | 58 | def setup_test 59 | reset_db 60 | load_data 61 | end 62 | 63 | def load_data 64 | system("psql -a -d owl_test -f test/fixtures/tiler_unit_#{@test_name.gsub('test_', '')}.sql") 65 | #system('psql -a -d owl_test -c "\copy nodes from ' + Rails.root.to_s + '/testdata/' + changeset_id.to_s + '-nodes.csv"') 66 | #system('psql -a -d owl_test -c "\copy ways from ' + Rails.root.to_s + '/testdata/' + changeset_id.to_s + '-ways.csv"') 67 | 68 | conn = PGconn.open(:dbname => 'owl_test', :user => 'ppawel') 69 | for i in 1..10 do 70 | conn.exec("INSERT INTO changesets VALUES (#{i}, 1, 'testuser', NOW(), NOW(), 'f', '', NULL, 0, NULL);") 71 | end 72 | tiler = Tiler::ChangesetTiler.new(conn) 73 | for id in conn.exec("SELECT changeset_id FROM nodes UNION SELECT changeset_id FROM ways").to_a.uniq do 74 | tiler.generate(16, id['changeset_id'].to_i, {:retile => true, :changes => true}) 75 | end 76 | end 77 | 78 | def reset_db 79 | system('psql -a -d owl_test -c "TRUNCATE changesets; TRUNCATE changeset_tiles; TRUNCATE nodes; TRUNCATE ways;"') 80 | end 81 | end 82 | -------------------------------------------------------------------------------- /replication/replicate_changesets.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | STDOUT.sync = true 4 | 5 | require 'nokogiri' 6 | require 'open-uri' 7 | require 'pg' 8 | require 'yaml' 9 | 10 | def parse_tags(changeset_el) 11 | tags = {} 12 | changeset_el.children.each do |tag_el| 13 | next unless tag_el.element? 14 | tags[tag_el['k']] = tag_el['v'] 15 | end 16 | tags 17 | end 18 | 19 | def insert_changeset(changeset_el) 20 | id = changeset_el['id'].to_i 21 | count = @conn.exec("INSERT INTO changesets (id, user_id, user_name, created_at, closed_at, open, tags, num_changes) VALUES ( 22 | #{changeset_el['id'].to_i}, 23 | #{changeset_el['uid'].to_i}, 24 | '#{PGconn.escape(changeset_el['user'])}', 25 | '#{changeset_el['created_at']}', 26 | #{changeset_el['closed_at'] ? '\'' + changeset_el['closed_at'] + '\'' : 'NULL'}, 27 | '#{changeset_el['open']}', 28 | '#{PGconn.escape(parse_tags(changeset_el).to_s.gsub('{', '').gsub('}', ''))}'::hstore, 29 | #{changeset_el['num_changes'].to_i})") 30 | end 31 | 32 | def changeset_exists(changeset_id) 33 | @conn.exec("SELECT COUNT(*) FROM changesets WHERE id = #{changeset_id}").getvalue(0, 0).to_i > 0 34 | end 35 | 36 | def update_changeset(changeset_el) 37 | id = changeset_el['id'].to_i 38 | count = @conn.exec(" 39 | UPDATE changesets 40 | SET 41 | closed_at = #{!changeset_el['closed_at'] ? 'NULL' : '\'' + changeset_el['closed_at'] + '\''}, 42 | tags = '#{PGconn.escape(parse_tags(changeset_el).to_s.gsub('{', '').gsub('}', ''))}'::hstore 43 | WHERE id = #{id}").cmd_tuples 44 | end 45 | 46 | def save_state(state) 47 | File.open('state.yaml', 'w') {|f| f.puts(state.to_yaml)} 48 | end 49 | 50 | $config = YAML.load_file('../config/database.yml')['development'] 51 | current_state = YAML.load_file('state.yaml') 52 | remote_state = YAML.load(open('http://planet.openstreetmap.org/replication/changesets/state.yaml').read) 53 | 54 | @conn = PGconn.open(:host => $config['host'], :port => $config['port'], :dbname => $config['database'], 55 | :user => $config['username'], :password => $config['password']) 56 | 57 | for id in (current_state['sequence'].to_i + 1..remote_state['sequence'].to_i) 58 | file_id = id.to_s.rjust(9, '0') 59 | puts "#{Time.now} -- file_id = #{file_id}" 60 | 61 | begin 62 | @conn.transaction do |c| 63 | open("http://planet.openstreetmap.org/replication/changesets/#{file_id[0..2]}/#{file_id[3..5]}/#{file_id[6..8]}.osm.gz") do |f| 64 | next if f.size == 0 65 | gz = Zlib::GzipReader.new(f) 66 | text = gz.read 67 | xml = Nokogiri::XML(text) 68 | xml.root.children.each do |changeset_el| 69 | next unless changeset_el.element? 70 | changeset_id = changeset_el['id'].to_i 71 | if !changeset_exists(changeset_id) 72 | puts "#{Time.now} -- Inserting changeset #{changeset_id}... " 73 | insert_changeset(changeset_el) 74 | else 75 | puts "#{Time.now} -- Updating changeset #{changeset_id}... " 76 | update_changeset(changeset_el) 77 | end 78 | end 79 | end 80 | end 81 | 82 | current_state['sequence'] = id 83 | save_state(current_state) 84 | rescue 85 | puts $!.inspect, $@ 86 | exit 87 | end 88 | end 89 | -------------------------------------------------------------------------------- /db/sql/owl_schema.sql: -------------------------------------------------------------------------------- 1 | -- Database creation script for the OWL schema. 2 | 3 | DROP TABLE IF EXISTS changes; 4 | DROP TABLE IF EXISTS nodes; 5 | DROP TABLE IF EXISTS ways; 6 | DROP TABLE IF EXISTS changeset_tiles; 7 | DROP TABLE IF EXISTS changesets; 8 | DROP TABLE IF EXISTS relation_members; 9 | DROP TABLE IF EXISTS relations; 10 | DROP TABLE IF EXISTS users; 11 | 12 | DROP TYPE IF EXISTS element_type CASCADE; 13 | CREATE TYPE element_type AS ENUM ('N', 'W', 'R'); 14 | 15 | DROP TYPE IF EXISTS action CASCADE; 16 | CREATE TYPE action AS ENUM ('CREATE', 'MODIFY', 'DELETE', 'AFFECT'); 17 | 18 | DROP AGGREGATE IF EXISTS array_accum(anyarray); 19 | CREATE AGGREGATE array_accum (anyarray) ( 20 | sfunc = array_cat, 21 | stype = anyarray, 22 | initcond = '{}' 23 | ); 24 | 25 | CREATE TABLE changes ( 26 | id bigserial NOT NULL, 27 | tstamp timestamp without time zone, 28 | el_type element_type, 29 | action action, 30 | el_id bigint, 31 | version int, 32 | changeset_id bigint, 33 | tags hstore, 34 | prev_tags hstore, 35 | geom geometry(GEOMETRY, 4326), 36 | prev_geom geometry(GEOMETRY, 4326) 37 | ); 38 | 39 | -- Create a table for changesets. 40 | CREATE TABLE changesets ( 41 | id bigint NOT NULL, 42 | user_id bigint, 43 | user_name varchar(255), 44 | created_at timestamp without time zone NOT NULL, 45 | closed_at timestamp without time zone, -- If NULL, changeset is still open for business. 46 | open boolean NOT NULL, 47 | tags hstore, 48 | num_changes int, -- Comes from the official changeset metadata. 49 | bbox geometry -- Bounding box of all changes for this changeset. 50 | ); 51 | 52 | -- Create a table for nodes. 53 | CREATE TABLE nodes ( 54 | id bigint NOT NULL, 55 | version int NOT NULL, 56 | visible boolean NOT NULL, 57 | user_id int NOT NULL, 58 | tstamp timestamp without time zone NOT NULL, 59 | changeset_id bigint NOT NULL, 60 | tags hstore NOT NULL, 61 | geom geometry(POINT, 4326) 62 | ); 63 | 64 | -- Create a table for ways. 65 | CREATE TABLE ways ( 66 | id bigint NOT NULL, 67 | version int NOT NULL, 68 | visible boolean NOT NULL, 69 | user_id int NOT NULL, 70 | tstamp timestamp without time zone NOT NULL, 71 | changeset_id bigint NOT NULL, 72 | tags hstore NOT NULL, 73 | nodes bigint[] NOT NULL 74 | ); 75 | 76 | -- Create a table for relations. 77 | CREATE TABLE relations ( 78 | id bigint NOT NULL, 79 | version int NOT NULL, 80 | visible boolean NOT NULL, 81 | user_id int NOT NULL, 82 | tstamp timestamp without time zone NOT NULL, 83 | changeset_id bigint NOT NULL, 84 | tags hstore NOT NULL 85 | ); 86 | 87 | -- Create a table for representing relation member relationships. 88 | CREATE TABLE relation_members ( 89 | relation_id bigint NOT NULL, 90 | version bigint NOT NULL, 91 | member_id bigint NOT NULL, 92 | member_type character(1) NOT NULL, 93 | member_role text NOT NULL, 94 | sequence_id int NOT NULL 95 | ); 96 | 97 | -- Create a table for users. 98 | CREATE TABLE users ( 99 | id int NOT NULL, 100 | name text NOT NULL 101 | ); 102 | 103 | -- Create a table for changeset tiles. 104 | CREATE TABLE changeset_tiles ( 105 | changeset_id bigint NOT NULL, 106 | tstamp timestamp without time zone NOT NULL, 107 | x int NOT NULL, 108 | y int NOT NULL, 109 | zoom int NOT NULL, 110 | change_id bigint NOT NULL, 111 | geom geometry(GEOMETRY, 4326), 112 | prev_geom geometry(GEOMETRY, 4326) 113 | ); 114 | -------------------------------------------------------------------------------- /config/environments/production.rb: -------------------------------------------------------------------------------- 1 | Owl::Application.configure do 2 | # Settings specified here will take precedence over those in config/application.rb. 3 | 4 | # Code is not reloaded between requests. 5 | config.cache_classes = true 6 | 7 | # Eager load code on boot. This eager loads most of Rails and 8 | # your application in memory, allowing both thread web servers 9 | # and those relying on copy on write to perform better. 10 | # Rake tasks automatically ignore this option for performance. 11 | config.eager_load = true 12 | 13 | # Full error reports are disabled and caching is turned on. 14 | config.consider_all_requests_local = false 15 | config.action_controller.perform_caching = true 16 | 17 | # Enable Rack::Cache to put a simple HTTP cache in front of your application 18 | # Add `rack-cache` to your Gemfile before enabling this. 19 | # For large-scale production use, consider using a caching reverse proxy like nginx, varnish or squid. 20 | # config.action_dispatch.rack_cache = true 21 | 22 | # Disable Rails's static asset server (Apache or nginx will already do this). 23 | config.serve_static_assets = false 24 | 25 | # Compress JavaScripts and CSS. 26 | config.assets.js_compressor = :uglifier 27 | # config.assets.css_compressor = :sass 28 | 29 | # Do not fallback to assets pipeline if a precompiled asset is missed. 30 | config.assets.compile = false 31 | 32 | # Generate digests for assets URLs. 33 | config.assets.digest = true 34 | 35 | # Version of your assets, change this if you want to expire all your assets. 36 | config.assets.version = '1.0' 37 | 38 | # Specifies the header that your server uses for sending files. 39 | # config.action_dispatch.x_sendfile_header = "X-Sendfile" # for apache 40 | # config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for nginx 41 | 42 | # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies. 43 | # config.force_ssl = true 44 | 45 | # Set to :debug to see everything in the log. 46 | config.log_level = :info 47 | 48 | # Prepend all log lines with the following tags. 49 | # config.log_tags = [ :subdomain, :uuid ] 50 | 51 | # Use a different logger for distributed setups. 52 | # config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new) 53 | 54 | # Use a different cache store in production. 55 | # config.cache_store = :mem_cache_store 56 | 57 | # Enable serving of images, stylesheets, and JavaScripts from an asset server. 58 | # config.action_controller.asset_host = "http://assets.example.com" 59 | 60 | # Precompile additional assets. 61 | # application.js, application.css, and all non-JS/CSS in app/assets folder are already added. 62 | # config.assets.precompile += %w( search.js ) 63 | 64 | # Ignore bad email addresses and do not raise email delivery errors. 65 | # Set this to true and configure the email server for immediate delivery to raise delivery errors. 66 | # config.action_mailer.raise_delivery_errors = false 67 | 68 | # Enable locale fallbacks for I18n (makes lookups for any locale fall back to 69 | # the I18n.default_locale when a translation can not be found). 70 | config.i18n.fallbacks = true 71 | 72 | # Send deprecation notices to registered listeners. 73 | config.active_support.deprecation = :notify 74 | 75 | # Disable automatic flushing of the log to improve performance. 76 | # config.autoflush_log = false 77 | 78 | # Use default logging formatter so that PID and timestamp are not suppressed. 79 | config.log_formatter = ::Logger::Formatter.new 80 | end 81 | -------------------------------------------------------------------------------- /app/views/layouts/admin.html.erb: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | <%= content_for?(:title) ? yield(:title) : "Owl" %> 8 | <%= csrf_meta_tags %> 9 | 10 | 11 | 14 | 15 | <%= stylesheet_link_tag "application", :media => "all" %> 16 | 17 | 18 | 19 | <%= favicon_link_tag 'apple-touch-icon-144x144-precomposed.png', :rel => 'apple-touch-icon-precomposed', :type => 'image/png', :sizes => '144x144' %> 20 | 21 | 22 | 23 | <%= favicon_link_tag 'apple-touch-icon-114x114-precomposed.png', :rel => 'apple-touch-icon-precomposed', :type => 'image/png', :sizes => '114x114' %> 24 | 25 | 26 | 27 | <%= favicon_link_tag 'apple-touch-icon-72x72-precomposed.png', :rel => 'apple-touch-icon-precomposed', :type => 'image/png', :sizes => '72x72' %> 28 | 29 | 30 | 31 | <%= favicon_link_tag 'apple-touch-icon-precomposed.png', :rel => 'apple-touch-icon-precomposed', :type => 'image/png' %> 32 | 33 | 34 | 35 | <%= favicon_link_tag 'favicon.ico', :rel => 'shortcut icon' %> 36 | 37 | <%= javascript_include_tag "application" %> 38 | 39 | 40 | 41 |
42 |
43 |
44 | 45 | 46 | 47 | 48 | 49 | OWL 50 |
51 |
    52 | 55 |
56 |
57 |
58 |
59 |
60 | 61 |
62 |
63 |
64 |
65 |
    66 |
  • Process changesets
    • 67 | 69 |
70 |
71 |
72 |
73 | <%= bootstrap_flash %> 74 | <%= yield %> 75 |
76 |
77 | 78 |
79 |

© Company 2013

80 |
81 | 82 |
83 | 84 | 85 | 86 | -------------------------------------------------------------------------------- /lib/tiler/utils.rb: -------------------------------------------------------------------------------- 1 | require 'set' 2 | 3 | def degrees(rad) 4 | rad * 180 / Math::PI 5 | end 6 | 7 | def radians(angle) 8 | angle / 180 * Math::PI 9 | end 10 | 11 | # Translated to Ruby rom http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames 12 | def latlon2tile(lat_deg, lon_deg, zoom) 13 | lat_deg = -89.999999 if lat_deg == -90.0 # Hack 14 | lat_rad = radians(lat_deg) 15 | n = 2.0 ** zoom 16 | xtile = ((lon_deg + 180.0) / 360.0 * n).floor 17 | ytile = ((1.0 - Math.log(Math.tan(lat_rad) + 1.0 / Math.cos(lat_rad)) / Math::PI) / 2.0 * n).floor 18 | return xtile, ytile 19 | end 20 | 21 | # Translated to Ruby rom http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames 22 | def tile2latlon(x, y, zoom) 23 | n = 2.0 ** zoom 24 | lon_deg = x / n * 360.0 - 180.0 25 | lat_rad = Math.atan(Math.sinh(Math::PI * (1 - 2 * y / n.to_f))) 26 | lat_deg = degrees(lat_rad) 27 | return lat_deg, lon_deg 28 | end 29 | 30 | def tile2bbox(x, y, zoom) 31 | lat1, lon1 = tile2latlon(x, y, zoom) 32 | lat2, lon2 = tile2latlon(x + 1, y + 1, zoom) 33 | [lon1, lat1, lon2, lat2] 34 | end 35 | 36 | ## 37 | # Converts PostGIS' BOX2D string representation to a list. 38 | # 39 | def box2d_to_bbox(box2d) 40 | box2d.gsub(',', ' ').gsub('BOX(', '').gsub(')', '').split(' ').map(&:to_f) 41 | end 42 | 43 | ## 44 | # bbox is [xmin, ymin, xmax, ymax] 45 | # 46 | def bbox_to_tiles(zoom, bbox) 47 | tiles = Set.new 48 | bounds = bbox_bound_tiles(zoom, bbox) 49 | (bounds[0][0]..bounds[1][0]).each do |x| 50 | (bounds[0][1]..bounds[1][1]).each do |y| 51 | tiles << [x, y] 52 | end 53 | end 54 | tiles 55 | end 56 | 57 | def bbox_bound_tiles(zoom, bbox) 58 | top_left = latlon2tile(bbox[1], bbox[0], zoom) 59 | bottom_right = latlon2tile(bbox[3], bbox[2], zoom) 60 | min_y = [top_left[1], bottom_right[1]].min 61 | max_y = [top_left[1], bottom_right[1]].max 62 | [[top_left[0], min_y], [bottom_right[0], max_y]] 63 | end 64 | 65 | def bbox_tile_count(zoom, bbox) 66 | tiles = Set.new 67 | top_left = latlon2tile(bbox[1], bbox[0], zoom) 68 | bottom_right = latlon2tile(bbox[3], bbox[2], zoom) 69 | min_y = [top_left[1], bottom_right[1]].min 70 | max_y = [top_left[1], bottom_right[1]].max 71 | (bottom_right[0] - top_left[0] + 1) * (max_y - min_y + 1) 72 | end 73 | 74 | def subtiles(tile, source_zoom, target_zoom) 75 | tiles = Set.new 76 | subtiles_per_tile = 2**target_zoom / 2**source_zoom 77 | (tile[0] * subtiles_per_tile..(tile[0] + 1) * subtiles_per_tile - 1).each do |x| 78 | (tile[1] * subtiles_per_tile..(tile[1] + 1) * subtiles_per_tile - 1).each do |y| 79 | tiles << [x, y] 80 | end 81 | end 82 | tiles 83 | end 84 | 85 | def pg_parse_array(str) 86 | eval(str.gsub('{', '[').gsub('}', ']')) 87 | end 88 | 89 | def pg_parse_geom_array(str) 90 | a = eval(str.gsub('{', '[\'').gsub('}', '\']').gsub(':', '\',\'')) 91 | a.collect {|v| v == 'NULL' ? nil : v} 92 | end 93 | 94 | def to_postgres_geom_array(geom_arr) 95 | result = '' 96 | geom_arr.each_with_index do |geom, index| 97 | result << ':' if index > 0 98 | if geom.nil? 99 | result << 'NULL' 100 | next 101 | end 102 | result << geom 103 | end 104 | "{#{result}}" 105 | end 106 | 107 | def memory_usage 108 | `ps -o rss= -p #{$$}`.to_i 109 | end 110 | 111 | def prepare_tiles(tiles_to_check, geom, source_zoom, zoom) 112 | tiles = Set.new 113 | for tile in tiles_to_check 114 | tile_geom = get_tile_geom(tile[0], tile[1], source_zoom) 115 | intersects = geom.intersects?(tile_geom) 116 | tiles.merge(subtiles(tile, source_zoom, zoom)) if intersects 117 | end 118 | tiles 119 | end 120 | 121 | def prepare_tiles_to_check(geom, bbox, source_zoom) 122 | tiles = Set.new 123 | test_zoom = 11 124 | bbox_to_tiles(test_zoom, bbox).select {|tile| geom.intersects?(get_tile_geom(tile[0], tile[1], test_zoom))}.each do |tile| 125 | tiles.merge(subtiles(tile, test_zoom, source_zoom)) 126 | end 127 | tiles 128 | end 129 | -------------------------------------------------------------------------------- /app/controllers/changeset_api_controller.rb: -------------------------------------------------------------------------------- 1 | require 'utils' 2 | require 'changeset_tiler' 3 | 4 | ## 5 | # Implements OWL API operations. 6 | # See: http://wiki.openstreetmap.org/wiki/OWL/API 7 | # 8 | class ChangesetApiController < ApplicationController 9 | include ApiHelper 10 | 11 | def changesets_tile_json 12 | @changesets = find_changesets_by_tile 13 | render :json => JSON[@changesets.map(&:generate_json)], :callback => params[:callback] 14 | end 15 | 16 | def changesets_tile_atom 17 | @changesets = find_changesets_by_tile 18 | render :template => 'api/changesets' 19 | end 20 | 21 | def summary_tile 22 | @summary = generate_summary_tile || {'num_changesets' => 0, 'latest_changeset' => nil} 23 | render :json => @summary.as_json, :callback => params[:callback] 24 | end 25 | 26 | def summary_tilerange 27 | @summary_list = generate_summary_tilerange || [{'num_changesets' => 0, 'latest_changeset' => nil}] 28 | render :json => @summary_list.as_json, :callback => params[:callback] 29 | end 30 | 31 | private 32 | def find_changesets_by_tile 33 | @x, @y, @zoom = get_xyz(params) 34 | changesets = ActiveRecord::Base.connection.raw_connection().exec(" 35 | SELECT 36 | t.changeset_id, 37 | MAX(c.tstamp) AS max_tstamp, 38 | cs.*, 39 | cs.bbox AS total_bbox, 40 | array_agg(row(c.*)) AS changes, 41 | array_agg(c.tags) AS change_tags, 42 | array_agg(c.prev_tags) AS change_prev_tags, 43 | array_agg(ST_AsGeoJSON(t.geom)) AS geojson, 44 | array_agg(ST_AsGeoJSON(t.prev_geom)) AS prev_geojson 45 | FROM changeset_tiles t 46 | INNER JOIN changesets cs ON (cs.id = t.changeset_id) 47 | INNER JOIN changes c ON (c.id = t.change_id) 48 | WHERE x = #{@x} AND y = #{@y} AND zoom = #{@zoom} 49 | AND c.changeset_id IN ( 50 | SELECT DISTINCT changeset_id 51 | FROM changeset_tiles 52 | WHERE x = #{@x} AND y = #{@y} AND zoom = #{@zoom} 53 | #{get_timelimit_sql(params)} 54 | ORDER BY changeset_id DESC 55 | #{get_limit_sql(params)} 56 | ) 57 | GROUP BY t.changeset_id, cs.id 58 | ORDER BY cs.created_at DESC").collect {|row| Changeset.new(row)} 59 | changesets 60 | end 61 | 62 | def generate_summary_tile 63 | @x, @y, @zoom = get_xyz(params) 64 | rows = ActiveRecord::Base.connection.select_all("WITH agg AS ( 65 | SELECT changeset_id, MAX(tstamp) AS max_tstamp 66 | FROM changeset_tiles 67 | WHERE x = #{@x} AND y = #{@y} AND zoom = #{@zoom} 68 | #{get_timelimit_sql(params)} 69 | GROUP BY changeset_id 70 | ) SELECT * FROM 71 | (SELECT COUNT(*) AS num_changesets FROM agg) a, 72 | (SELECT changeset_id FROM agg ORDER BY max_tstamp DESC NULLS LAST LIMIT 1) b") 73 | return if rows.empty? 74 | row = rows[0] 75 | summary_tile = {'num_changesets' => row['num_changesets']} 76 | summary_tile['latest_changeset'] = 77 | ActiveRecord::Base.connection.select_all("SELECT *, NULL AS tile_bbox, 78 | bbox::box2d::text AS total_bbox 79 | FROM changesets WHERE id = #{row['changeset_id']}")[0] 80 | summary_tile 81 | end 82 | 83 | def generate_summary_tilerange 84 | @zoom, @x1, @y1, @x2, @y2 = get_range(params) 85 | rows = ActiveRecord::Base.connection.execute(" 86 | SELECT x, y, COUNT(*) AS num_changesets, MAX(tstamp) AS max_tstamp, MAX(changeset_id) AS changeset_id 87 | FROM changeset_tiles 88 | INNER JOIN changesets cs ON (cs.id = changeset_id) 89 | WHERE x >= #{@x1} AND x <= #{@x2} AND y >= #{@y1} AND y <= #{@y2} AND zoom = #{@zoom} 90 | #{get_timelimit_sql(params)} 91 | GROUP BY x, y").to_a 92 | rows.to_a 93 | end 94 | 95 | def pg_string_to_array(str) 96 | dup = str.dup 97 | dup[0] = '[' 98 | dup[-1] = ']' 99 | dup.gsub!('NULL', 'nil') 100 | eval(dup) 101 | end 102 | end 103 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | actionmailer (4.1.9) 5 | actionpack (= 4.1.9) 6 | actionview (= 4.1.9) 7 | mail (~> 2.5, >= 2.5.4) 8 | actionpack (4.1.9) 9 | actionview (= 4.1.9) 10 | activesupport (= 4.1.9) 11 | rack (~> 1.5.2) 12 | rack-test (~> 0.6.2) 13 | actionview (4.1.9) 14 | activesupport (= 4.1.9) 15 | builder (~> 3.1) 16 | erubis (~> 2.7.0) 17 | activemodel (4.1.9) 18 | activesupport (= 4.1.9) 19 | builder (~> 3.1) 20 | activerecord (4.1.9) 21 | activemodel (= 4.1.9) 22 | activesupport (= 4.1.9) 23 | arel (~> 5.0.0) 24 | activesupport (4.1.9) 25 | i18n (~> 0.6, >= 0.6.9) 26 | json (~> 1.7, >= 1.7.7) 27 | minitest (~> 5.1) 28 | thread_safe (~> 0.1) 29 | tzinfo (~> 1.1) 30 | arel (5.0.1.20140414130214) 31 | builder (3.2.2) 32 | coffee-rails (4.0.1) 33 | coffee-script (>= 2.2.0) 34 | railties (>= 4.0.0, < 5.0) 35 | coffee-script (2.3.0) 36 | coffee-script-source 37 | execjs 38 | coffee-script-source (1.8.0) 39 | erubis (2.7.0) 40 | execjs (2.2.2) 41 | ffi (1.9.6) 42 | ffi-geos (1.0.0) 43 | ffi (>= 1.0.0) 44 | hike (1.2.3) 45 | i18n (0.7.0) 46 | jbuilder (1.5.3) 47 | activesupport (>= 3.0.0) 48 | multi_json (>= 1.2.0) 49 | jquery-rails (3.1.2) 50 | railties (>= 3.0, < 5.0) 51 | thor (>= 0.14, < 2.0) 52 | json (1.8.2) 53 | mail (2.6.3) 54 | mime-types (>= 1.16, < 3) 55 | mime-types (2.4.3) 56 | minitest (5.5.1) 57 | mono_logger (1.1.0) 58 | multi_json (1.10.1) 59 | pg (0.18.1) 60 | rack (1.5.2) 61 | rack-protection (1.5.3) 62 | rack 63 | rack-test (0.6.3) 64 | rack (>= 1.0) 65 | rails (4.1.9) 66 | actionmailer (= 4.1.9) 67 | actionpack (= 4.1.9) 68 | actionview (= 4.1.9) 69 | activemodel (= 4.1.9) 70 | activerecord (= 4.1.9) 71 | activesupport (= 4.1.9) 72 | bundler (>= 1.3.0, < 2.0) 73 | railties (= 4.1.9) 74 | sprockets-rails (~> 2.0) 75 | railties (4.1.9) 76 | actionpack (= 4.1.9) 77 | activesupport (= 4.1.9) 78 | rake (>= 0.8.7) 79 | thor (>= 0.18.1, < 2.0) 80 | rake (10.4.2) 81 | rdoc (4.2.0) 82 | json (~> 1.4) 83 | redis (3.2.0) 84 | redis-namespace (1.5.1) 85 | redis (~> 3.0, >= 3.0.4) 86 | resque (1.25.2) 87 | mono_logger (~> 1.0) 88 | multi_json (~> 1.0) 89 | redis-namespace (~> 1.3) 90 | sinatra (>= 0.9.2) 91 | vegas (~> 0.1.2) 92 | sass (3.2.19) 93 | sass-rails (4.0.5) 94 | railties (>= 4.0.0, < 5.0) 95 | sass (~> 3.2.2) 96 | sprockets (~> 2.8, < 3.0) 97 | sprockets-rails (~> 2.0) 98 | sdoc (0.4.1) 99 | json (~> 1.7, >= 1.7.7) 100 | rdoc (~> 4.0) 101 | sinatra (1.4.5) 102 | rack (~> 1.4) 103 | rack-protection (~> 1.4) 104 | tilt (~> 1.3, >= 1.3.4) 105 | sprockets (2.12.3) 106 | hike (~> 1.2) 107 | multi_json (~> 1.0) 108 | rack (~> 1.0) 109 | tilt (~> 1.1, != 1.3.0) 110 | sprockets-rails (2.2.2) 111 | actionpack (>= 3.0) 112 | activesupport (>= 3.0) 113 | sprockets (>= 2.8, < 4.0) 114 | thor (0.19.1) 115 | thread_safe (0.3.4) 116 | tilt (1.4.1) 117 | turbolinks (2.5.3) 118 | coffee-rails 119 | twitter-bootstrap-rails (3.2.0) 120 | actionpack (~> 4.1) 121 | execjs (~> 2.2) 122 | rails (~> 4.1) 123 | railties (~> 4.1) 124 | tzinfo (1.2.2) 125 | thread_safe (~> 0.1) 126 | uglifier (2.7.0) 127 | execjs (>= 0.3.0) 128 | json (>= 1.8.0) 129 | vegas (0.1.11) 130 | rack (>= 1.0.0) 131 | 132 | PLATFORMS 133 | ruby 134 | 135 | DEPENDENCIES 136 | coffee-rails (~> 4.0.0) 137 | ffi-geos 138 | jbuilder (~> 1.2) 139 | jquery-rails 140 | minitest 141 | pg 142 | rails (= 4.1.9) 143 | resque 144 | sass-rails (~> 4.0.0) 145 | sdoc 146 | sinatra (>= 1.3.0) 147 | turbolinks 148 | twitter-bootstrap-rails 149 | uglifier (>= 1.3.0) 150 | -------------------------------------------------------------------------------- /lib/tiler/change.rb: -------------------------------------------------------------------------------- 1 | require 'ffi-geos' 2 | require 'time' 3 | 4 | class Change 5 | attr_accessor :id 6 | attr_accessor :changeset_id 7 | attr_accessor :tstamp 8 | attr_accessor :el_type 9 | attr_accessor :el_id 10 | attr_accessor :el_version 11 | attr_accessor :action 12 | attr_accessor :geom 13 | attr_accessor :prev_geom 14 | attr_accessor :tags 15 | attr_accessor :prev_tags 16 | attr_accessor :nodes 17 | attr_accessor :prev_nodes 18 | 19 | def self.from_pg_array(changes) 20 | result = [] 21 | for change_str in convert_array(changes) do 22 | result << Change.from_string(change_str) 23 | end 24 | result 25 | end 26 | 27 | def self.from_string(str) 28 | change = Change.new 29 | a = str.delete('(').delete(')').split(',') 30 | change.id = a[0].to_i 31 | change.tstamp = Time.parse(a[1]) 32 | change.el_type = a[2] 33 | change.action = a[3] 34 | change.el_id = a[4].to_i 35 | change.el_version = a[5].to_i 36 | #p eval(str.gsub(/""/, '"').gsub(/^(.*?)"""/, '{').gsub(/"([^"]*)$/, '}')) 37 | #hash['geom_geojson'] = geojson(a[-2]) unless a[-2].empty? 38 | #hash['prev_geom_geojson'] = geojson(a[-1]) unless a[-1].empty? 39 | change 40 | end 41 | 42 | def as_json(options = {}) 43 | Hash[instance_variables.collect {|key| [key.to_s.gsub('@', ''), instance_variable_get(key)]}] 44 | end 45 | 46 | # 47 | # Parse a PostgreSQL-Array output and convert into ruby array. This 48 | # does the real parsing work. 49 | # 50 | def self.convert_array(str) 51 | array_nesting = 0 # nesting level of the array 52 | in_string = false # currently inside a quoted string ? 53 | escaped = false # if the character is escaped 54 | sbuffer = '' # buffer for the current element 55 | result_array = ::Array.new # the resulting Array 56 | 57 | str.each_byte { |char| # parse character by character 58 | char = char.chr # we need the Character, not it's Integer 59 | 60 | if escaped then # if this character is escaped, just add it to the buffer 61 | sbuffer += char 62 | escaped = false 63 | next 64 | end 65 | 66 | case char # let's see what kind of character we have 67 | #------------- {: beginning of an array ----# 68 | when '{' 69 | if in_string then # ignore inside a string 70 | sbuffer += char 71 | next 72 | end 73 | 74 | if array_nesting >= 1 then # if it's an nested array, defer for recursion 75 | sbuffer += char 76 | end 77 | array_nesting += 1 # inside another array 78 | 79 | #------------- ": string deliminator --------# 80 | when '"' 81 | in_string = !in_string 82 | 83 | #------------- \: escape character, next is regular character # 84 | when "\\" # single \, must be extra escaped in Ruby 85 | if array_nesting > 1 86 | sbuffer += char 87 | else 88 | escaped = true 89 | end 90 | 91 | #------------- ,: element separator ---------# 92 | when ',' 93 | if in_string or array_nesting > 1 then # don't care if inside string or 94 | sbuffer += char # nested array 95 | else 96 | if !sbuffer.is_a? ::Array then 97 | #sbuffer = @base_type.parse(sbuffer) 98 | #sbuffer = 'bla' 99 | end 100 | result_array << sbuffer # otherwise, here ends an element 101 | sbuffer = '' 102 | end 103 | 104 | #------------- }: End of Array --------------# 105 | when '}' 106 | if in_string then # ignore if inside quoted string 107 | sbuffer += char 108 | next 109 | end 110 | 111 | array_nesting -=1 # decrease nesting level 112 | 113 | if array_nesting == 1 # must be the end of a nested array 114 | sbuffer += char 115 | sbuffer = convert_array( sbuffer ) # recurse, using the whole nested array 116 | elsif array_nesting > 1 # inside nested array, keep it for later 117 | sbuffer += char 118 | else # array_nesting = 0, must be the last } 119 | if !sbuffer.is_a? ::Array then 120 | #sbuffer = @base_type.parse( sbuffer ) 121 | #sbuffer = 'ble' 122 | end 123 | 124 | result_array << sbuffer unless sbuffer.nil? # upto here was the last element 125 | end 126 | 127 | #------------- all other characters ---------# 128 | else 129 | sbuffer += char # simply append 130 | end 131 | } 132 | return result_array 133 | end # convert_array() 134 | end 135 | -------------------------------------------------------------------------------- /db/schema.rb: -------------------------------------------------------------------------------- 1 | # encoding: UTF-8 2 | # This file is auto-generated from the current state of the database. Instead 3 | # of editing this file, please use the migrations feature of Active Record to 4 | # incrementally modify your database, and then regenerate this schema definition. 5 | # 6 | # Note that this schema.rb definition is the authoritative source for your 7 | # database schema. If you need to create the application database on another 8 | # system, you should be using db:schema:load, not running all the migrations 9 | # from scratch. The latter is a flawed and unsustainable approach (the more migrations 10 | # you'll amass, the slower it'll run and the greater likelihood for issues). 11 | # 12 | # It's strongly recommended that you check this file into your version control system. 13 | 14 | ActiveRecord::Schema.define(version: 20130928181457) do 15 | 16 | # These are extensions that must be enabled in order to support this database 17 | enable_extension "plpgsql" 18 | enable_extension "hstore" 19 | enable_extension "postgis" 20 | 21 | # Could not dump table "changeset_tiles" because of following StandardError 22 | # Unknown type 'change' for column 'changes' 23 | 24 | # Could not dump table "changesets" because of following StandardError 25 | # Unknown type 'geometry' for column 'bbox' 26 | 27 | create_table "nodes", id: false, force: true do |t| 28 | t.integer "id", limit: 8, null: false 29 | t.integer "version", null: false 30 | t.integer "rev", null: false 31 | t.boolean "visible", null: false 32 | t.boolean "current", null: false 33 | t.integer "user_id", null: false 34 | t.datetime "tstamp", null: false 35 | t.integer "changeset_id", limit: 8, null: false 36 | t.hstore "tags", null: false 37 | t.integer "geom", limit: 0 38 | end 39 | 40 | add_index "nodes", ["changeset_id"], name: "idx_nodes_changeset_id", using: :btree 41 | add_index "nodes", ["geom"], name: "idx_nodes_geom", where: "(visible AND current)", using: :gist 42 | add_index "nodes", ["id"], name: "idx_nodes_node_id", using: :btree 43 | 44 | create_table "relation_members", id: false, force: true do |t| 45 | t.integer "relation_id", limit: 8, null: false 46 | t.integer "version", limit: 8, null: false 47 | t.integer "member_id", limit: 8, null: false 48 | t.string "member_type", limit: 1, null: false 49 | t.text "member_role", null: false 50 | t.integer "sequence_id", null: false 51 | end 52 | 53 | create_table "relations", id: false, force: true do |t| 54 | t.integer "id", limit: 8, null: false 55 | t.integer "version", null: false 56 | t.integer "rev", null: false 57 | t.boolean "visible", null: false 58 | t.boolean "current", null: false 59 | t.integer "user_id", null: false 60 | t.datetime "tstamp", null: false 61 | t.integer "changeset_id", limit: 8, null: false 62 | t.hstore "tags", null: false 63 | end 64 | 65 | add_index "relations", ["changeset_id"], name: "idx_relations_changeset_id", using: :btree 66 | 67 | create_table "sidekiq_jobs", force: true do |t| 68 | t.string "jid" 69 | t.string "queue" 70 | t.string "class_name" 71 | t.text "args" 72 | t.boolean "retry" 73 | t.datetime "enqueued_at" 74 | t.datetime "started_at" 75 | t.datetime "finished_at" 76 | t.string "status" 77 | t.string "name" 78 | t.text "result" 79 | end 80 | 81 | add_index "sidekiq_jobs", ["class_name"], name: "index_sidekiq_jobs_on_class_name", using: :btree 82 | add_index "sidekiq_jobs", ["enqueued_at"], name: "index_sidekiq_jobs_on_enqueued_at", using: :btree 83 | add_index "sidekiq_jobs", ["finished_at"], name: "index_sidekiq_jobs_on_finished_at", using: :btree 84 | add_index "sidekiq_jobs", ["jid"], name: "index_sidekiq_jobs_on_jid", using: :btree 85 | add_index "sidekiq_jobs", ["queue"], name: "index_sidekiq_jobs_on_queue", using: :btree 86 | add_index "sidekiq_jobs", ["retry"], name: "index_sidekiq_jobs_on_retry", using: :btree 87 | add_index "sidekiq_jobs", ["started_at"], name: "index_sidekiq_jobs_on_started_at", using: :btree 88 | add_index "sidekiq_jobs", ["status"], name: "index_sidekiq_jobs_on_status", using: :btree 89 | 90 | create_table "spatial_ref_sys", id: false, force: true do |t| 91 | t.integer "srid", null: false 92 | t.string "auth_name", limit: 256 93 | t.integer "auth_srid" 94 | t.string "srtext", limit: 2048 95 | t.string "proj4text", limit: 2048 96 | end 97 | 98 | create_table "users", id: false, force: true do |t| 99 | t.integer "id", null: false 100 | t.text "name", null: false 101 | end 102 | 103 | create_table "ways", id: false, force: true do |t| 104 | t.integer "id", limit: 8, null: false 105 | t.integer "version", null: false 106 | t.boolean "visible", null: false 107 | t.boolean "current", null: false 108 | t.integer "user_id", null: false 109 | t.datetime "tstamp", null: false 110 | t.integer "changeset_id", limit: 8, null: false 111 | t.hstore "tags", null: false 112 | t.integer "nodes", limit: 8, null: false, array: true 113 | end 114 | 115 | add_index "ways", ["changeset_id"], name: "idx_ways_changeset_id", using: :btree 116 | add_index "ways", ["nodes"], name: "idx_ways_nodes_id", using: :gin 117 | 118 | end 119 | -------------------------------------------------------------------------------- /lib/tiler/test/common.rb: -------------------------------------------------------------------------------- 1 | require 'tiler/change' 2 | 3 | ## 4 | # Utility methods for tiler tests. 5 | # 6 | module TestCommon 7 | TEST_ZOOM = 18 8 | 9 | def setup_unit_test(test_name) 10 | setup_db 11 | exec_sql_file("test/fixtures/tiler_unit_#{test_name.gsub('test_', '')}.sql") 12 | @tiler = Tiler::ChangesetTiler.new(@conn) 13 | for id in @conn.exec("SELECT changeset_id FROM nodes UNION SELECT changeset_id FROM ways").to_a.uniq do 14 | @tiler.generate(TEST_ZOOM, id['changeset_id'].to_i, {:retile => true, :changes => true}) 15 | end 16 | @changes = get_changes 17 | verify_changes(1) 18 | @tiles = get_tiles 19 | verify_tiles 20 | end 21 | 22 | def setup_changeset_test(id) 23 | puts "setup_changeset_test(#{id})" 24 | setup_db 25 | load_changeset(id) 26 | @tiler = Tiler::ChangesetTiler.new(@conn) 27 | @tiler.generate(TEST_ZOOM, id, {:retile => true, :changes => true}) 28 | @changes = get_changes 29 | @changes_h = Hash[@changes.collect {|row| [row['id'].to_i, row]}] 30 | verify_changes(id) 31 | @tiles = get_tiles 32 | verify_tiles 33 | end 34 | 35 | def setup_db 36 | $config = YAML.load_file('config/database.yml')['test'] 37 | @conn = PGconn.open(:host => $config['host'], :port => $config['port'], :dbname => $config['database'], 38 | :user => $config['username'], :password => $config['password']) 39 | @conn.set_error_verbosity(1) 40 | exec_sql_file('db/sql/owl_schema.sql') 41 | exec_sql_file('db/sql/owl_constraints.sql') 42 | exec_sql_file('db/sql/owl_indexes.sql') 43 | exec_sql_file('db/sql/owl_functions.sql') 44 | end 45 | 46 | def exec_sql_file(file) 47 | @conn.exec(File.open(file).read) 48 | end 49 | 50 | def load_changeset(id) 51 | if not File.exists?("testdata/#{id}-nodes.csv") 52 | raise "No test data for changeset #{id}" 53 | end 54 | 55 | puts 'Loading data...' 56 | 57 | @conn.exec("COPY changesets FROM STDIN;") 58 | File.open("testdata/#{id}-changeset.csv").read.each_line do |line| @conn.put_copy_data(line) end 59 | @conn.put_copy_end 60 | 61 | @conn.exec("COPY nodes FROM STDIN;") 62 | File.open("testdata/#{id}-nodes.csv").read.each_line do |line| @conn.put_copy_data(line) end 63 | @conn.put_copy_end 64 | 65 | @conn.exec("COPY ways FROM STDIN;") 66 | File.open("testdata/#{id}-ways.csv").read.each_line do |line| @conn.put_copy_data(line) end 67 | @conn.put_copy_end 68 | 69 | @conn.exec("VACUUM ANALYZE") 70 | end 71 | 72 | def verify_changes(changeset_id) 73 | assert(@changes.size > 0, 'NO CHANGES?!') 74 | 75 | for change in @changes 76 | geom_changed = geom_changed(change) 77 | tags_changed = change['tags'] != change['prev_tags'] 78 | #nodes_changed = change['nodes'] != change['prev_nodes'] 79 | #puts "changed #{change['el_id']} -- #{geom_changed} #{tags_changed}" 80 | #assert((nodes_changed or geom_changed or tags_changed or change['action'] == 'CREATE' or change['action'] == 'DELETE'), 81 | # "Change doesn't change anything: #{change}") 82 | 83 | if change['action'] == 'CREATE' 84 | assert(!change['geom'].nil?, 'geom should not be null for change: ' + change.to_s) 85 | assert(change['prev_geom'].nil?, 'prev_geom should be null for change: ' + change.to_s) 86 | #assert(!change['tags'].empty?, 'tags should not be null for change: ' + change.to_s) 87 | #assert(change['prev_tags'].empty?, 'prev_tags should be null for change: ' + change.to_s) 88 | end 89 | 90 | if change['action'] == 'DELETE' 91 | assert(change['geom'].nil?, 'geom should be null for change: ' + change.to_s) 92 | assert(!change['prev_geom'].empty?, 'prev_geom should not be null for change: ' + change.to_s) 93 | #assert(change['tags'].empty?, 'tags should be null for change: ' + change.to_s) 94 | #assert(!change['prev_tags'].empty?, 'prev_tags should not be null for change: ' + change.to_s) 95 | end 96 | 97 | if change['action'] == 'MODIFY' 98 | assert(!change['geom'].nil?) 99 | #assert(!change['prev_geom'].nil?, 'prev_geom should not be null for change: ' + change.to_s) 100 | end 101 | end 102 | 103 | for way in find_changes('el_type' => 'W') 104 | # There should be at most 2 versions of a way (unless there are more of them in the changeset). 105 | if way['el_changeset_id'].to_i != changeset_id and way['version'].to_i > 1 106 | # assert_equal(2, find_changes('el_type' => 'W', 'el_id' => way['el_id']).size, 107 | # "Too many versions for way: #{way}") 108 | end 109 | end 110 | end 111 | 112 | def geom_changed(change) 113 | if change['el_type'] == 'N' 114 | return (@conn.exec("SELECT NOT n.geom = n2.geom 115 | FROM nodes n 116 | LEFT JOIN nodes n2 ON (n2.id = n.id AND n2.version = n.version - 1) 117 | WHERE n.id = #{change['el_id']} AND n.version = #{change['version']}").getvalue(0, 0) == 't') 118 | end 119 | if change['el_type'] == 'W' 120 | return (@conn.exec("SELECT NOT ST_Equals(OWL_MakeLine(w.nodes, '#{change['tstamp']}'), 121 | OWL_MakeLine(w.nodes, TIMESTAMP '#{change['tstamp']}' - INTERVAL '5 seconds')) 122 | OR OWL_MakeLine(w.nodes, '#{change['tstamp']}') IS NULL 123 | FROM ways w 124 | WHERE w.id = #{change['el_id']} AND w.version = #{change['version']}").getvalue(0, 0) == 't') 125 | end 126 | end 127 | 128 | def get_changes 129 | for change in @conn.exec("SELECT 130 | changeset_id, 131 | id, 132 | tstamp, 133 | tags, 134 | prev_tags, 135 | el_type, 136 | action, 137 | el_id, 138 | version, 139 | (SELECT ST_Union(geom) FROM changeset_tiles WHERE change_id = c.id AND zoom = #{TEST_ZOOM}) AS geom, 140 | (SELECT ST_Union(prev_geom) FROM changeset_tiles WHERE change_id = c.id AND zoom = #{TEST_ZOOM}) AS prev_geom 141 | FROM changes c").to_a 142 | end 143 | end 144 | 145 | def get_tiles 146 | @conn.exec("SELECT * FROM changeset_tiles WHERE zoom = #{TEST_ZOOM}").to_a 147 | end 148 | 149 | # Performs sanity checks on tiles. 150 | def verify_tiles 151 | end 152 | 153 | def find_changes(filters) 154 | a = [] 155 | for change in @changes 156 | match = true 157 | for k, v in filters 158 | match = (match and (change[k].to_s == v.to_s)) 159 | end 160 | a << change if match 161 | end 162 | a 163 | end 164 | end 165 | -------------------------------------------------------------------------------- /lib/tiler/test/tiler_realdata_test.rb: -------------------------------------------------------------------------------- 1 | $:.unshift File.absolute_path(File.dirname(__FILE__)) + '/../lib' 2 | 3 | require 'pg' 4 | require 'test/unit' 5 | require 'yaml' 6 | require 'changeset_tiler' 7 | 8 | require 'test/common' 9 | 10 | class ChangesetTilerTest < Test::Unit::TestCase 11 | include TestCommon 12 | 13 | # Tag changes in Zagreb and Budapest place nodes. 14 | def test_12917265 15 | setup_changeset_test(12917265) 16 | assert_equal(6, find_changes('el_type' => 'N').size) 17 | #assert_equal(3, @tiles.size) 18 | end 19 | 20 | def test_13477045 21 | setup_changeset_test(13477045) 22 | assert_equal(0, find_changes('el_type' => 'N').size) 23 | assert_equal(26, find_changes('el_type' => 'W').size) 24 | end 25 | 26 | def test_14234906_multiple_way_versions 27 | setup_changeset_test(14234906) 28 | end 29 | 30 | def test_14459096_affected_way_with_version_1 31 | setup_changeset_test(14459096) 32 | assert_equal(0, find_changes('el_type' => 'N').size) 33 | assert_equal(1, find_changes('el_type' => 'W').size) 34 | end 35 | 36 | def test_14458340_affected_way 37 | setup_changeset_test(14458340) 38 | assert_equal(0, find_changes('el_type' => 'N').size) 39 | assert_equal(1, find_changes('el_type' => 'W').size) 40 | end 41 | 42 | def test_14698811 43 | setup_changeset_test(14698811) 44 | assert_equal(1, find_changes('el_type' => 'W', 'el_id' => '201787145', 'version' => '1').size) 45 | end 46 | 47 | def test_14698916 48 | setup_changeset_test(14698916) 49 | assert_equal(1, find_changes('el_type' => 'W', 'el_id' => '201787145', 'version' => '2').size) 50 | end 51 | 52 | def test_14820799_created_way 53 | setup_changeset_test(14820799) 54 | assert_equal(0, find_changes('el_type' => 'N').size) 55 | assert_equal(1, find_changes('el_type' => 'W', 'action' => 'CREATE', 'el_id' => '203232361').size) 56 | end 57 | 58 | def test_14797380_created_area 59 | setup_changeset_test(14797380) 60 | assert_equal(0, find_changes('el_type' => 'N').size) 61 | assert_equal(1, find_changes('el_type' => 'W', 'action' => 'CREATE', 'el_id' => '203011417').size) 62 | end 63 | 64 | def test_14796945 65 | setup_changeset_test(14796945) 66 | assert_equal(0, find_changes('el_type' => 'N').size) 67 | assert_equal(4, find_changes('el_type' => 'W').size) 68 | assert_equal(1, find_changes('el_type' => 'W', 'action' => 'CREATE', 'el_id' => '203008418').size) 69 | end 70 | 71 | def test_13387349 72 | setup_changeset_test(13387349) 73 | end 74 | 75 | def test_13394677 76 | setup_changeset_test(13394677) 77 | end 78 | 79 | def test_13477568 80 | setup_changeset_test(13477568) 81 | for tile in @tiles 82 | if (tile['x'].to_i == 36175) and (tile['y'].to_i == 22851) 83 | #geom_arr = pg_parse_geom_array(tile['geom']) 84 | #prev_geom_arr = pg_parse_geom_array(tile['prev_geom']) 85 | #p @changes_h[30]['tags'] != @changes_h[30]['prev_tags'] 86 | #p @changes_h[30] 87 | #p geom_arr[0] 88 | #p prev_geom_arr[0] 89 | end 90 | end 91 | end 92 | 93 | def test_13473237 94 | setup_changeset_test(13473237) 95 | end 96 | 97 | def test_14699204 98 | setup_changeset_test(14699204) 99 | end 100 | 101 | def test_13426127 102 | setup_changeset_test(13426127) 103 | end 104 | 105 | def test_13258073 106 | setup_changeset_test(13258073) 107 | assert_equal(1, find_changes('el_type' => 'N', 'el_id' => '420302052', 'action' => 'DELETE').size) 108 | end 109 | 110 | def test_14429223_deleted_ways 111 | setup_changeset_test(14429223) 112 | assert_equal(1, find_changes('el_type' => 'W', 'el_id' => '198336783', 'version' => '2').size) 113 | end 114 | 115 | def test_11193918 116 | setup_changeset_test(11193918) 117 | assert_equal(2, find_changes('el_type' => 'N').size) 118 | assert_equal(1, find_changes('el_id' => '1703304298').size) 119 | assert_equal(10, find_changes('el_type' => 'W').size) 120 | end 121 | 122 | def test_13018562 123 | setup_changeset_test(13018562) 124 | end 125 | 126 | def test_13223248_misaligned_way 127 | setup_changeset_test(13223248) 128 | 129 | way = find_changes('el_type' => 'W', 'el_id' => '166444532', 'version' => '4') 130 | p way 131 | assert_equal(1, way.size) 132 | #assert_equal(11, way[0]['nodes_len'].to_i) 133 | #assert(way[0]['geom_astext'].include?('18.650061')) 134 | 135 | way = find_changes('el_type' => 'W', 'el_id' => '169856888', 'version' => '1') 136 | assert_equal(1, way.size) 137 | #assert_equal('4', way[0]['nodes_len']) 138 | #assert(way[0]['geom_astext'].include?('18.650061')) 139 | end 140 | 141 | def test_13294164 142 | setup_changeset_test(13294164) 143 | assert_equal(9, find_changes('el_type' => 'W').size) 144 | 145 | # traffic_signals changed position - should be a change for that. 146 | changes = find_changes('el_type' => 'N', 'el_id' => '244942711') 147 | assert_equal(1, changes.size) 148 | end 149 | 150 | def test_13294164 151 | setup_changeset_test(13294164) 152 | assert_equal(9, find_changes('el_type' => 'W').size) 153 | 154 | # traffic_signals changed position - should be a change for that. 155 | changes = find_changes('el_type' => 'N', 'el_id' => '244942711') 156 | assert_equal(1, changes.size) 157 | end 158 | 159 | def test_16951072 160 | setup_changeset_test(16951072) 161 | assert_equal(2, find_changes('el_type' => 'W').size) 162 | end 163 | 164 | def test_16863679 165 | setup_changeset_test(16863679) 166 | assert_equal(3, find_changes('el_type' => 'W').size) 167 | end 168 | 169 | def test_14530383_forest_with_small_change_and_multiple_changes_of_one_object 170 | setup_changeset_test(14530383) 171 | forest_change = find_changes('el_type' => 'W', 'el_id' => '161116311', 'version' => '3') 172 | p forest_change 173 | assert_equal(1, forest_change.size) 174 | forest_tile = @tiles.find {|tile| tile['changes'].include?(forest_change[0]['id'])} 175 | # Tile geom should not include the whole forest (it was a bug once). 176 | #assert(!forest_tile['geom_astext'].include?('50.5478683')) 177 | 178 | # Now let's test way 174644591 which has 5 revisions in this changeset. 179 | assert_equal(4, find_changes('el_type' => 'W', 'action' => 'DELETE').size) 180 | end 181 | 182 | def test_18915_empty_geomcollection_error 183 | setup_changeset_test(18915) 184 | end 185 | 186 | def test_nan 187 | setup_changeset_test(14863424) 188 | end 189 | 190 | def test_collection 191 | setup_changeset_test(14836474) 192 | end 193 | 194 | # Way 16105282 has no-change versions. 195 | def test_7082 196 | setup_changeset_test(7082) 197 | p find_changes('el_type' => 'W', 'el_id' => '16105282')[0] 198 | p find_changes('el_type' => 'W', 'el_id' => '16105282')[1] 199 | assert_equal(1, find_changes('el_type' => 'W', 'el_id' => '16105282').size) 200 | end 201 | 202 | def test_13473782 203 | setup_changeset_test(13473782) 204 | p find_changes('el_type' => 'W', 'el_id' => '104749747') 205 | #assert_equal(3, @tiles.size) 206 | end 207 | 208 | # Way 172609358 changed nodes only. 209 | def test_13476841 210 | setup_changeset_test(13476841) 211 | assert_equal(1, find_changes('el_type' => 'W', 'el_id' => '172609358').size) 212 | assert(!find_changes('el_type' => 'W', 'el_id' => '172609358')[0]['prev_tags'].nil?) 213 | end 214 | 215 | def test_13472650 216 | setup_changeset_test(13472650) 217 | assert_equal(1, find_changes('el_type' => 'W', 'el_id' => '32284425').size) 218 | p find_changes('el_type' => 'W', 'el_id' => '32284425') 219 | end 220 | end 221 | -------------------------------------------------------------------------------- /lib/tiler/changeset_tiler.rb: -------------------------------------------------------------------------------- 1 | require 'tiler/logging' 2 | require 'tiler/utils' 3 | require 'ffi-geos' 4 | 5 | module Tiler 6 | 7 | # Implements tiling logic. 8 | class ChangesetTiler 9 | include ::Tiler::Logger 10 | 11 | attr_accessor :conn 12 | 13 | def initialize(conn) 14 | @conn = conn 15 | setup_prepared_statements 16 | init_geos 17 | end 18 | 19 | def init_geos 20 | @wkb_reader = Geos::WkbReader.new 21 | @wkt_reader = Geos::WktReader.new 22 | @wkb_writer = Geos::WkbWriter.new 23 | @wkb_writer.include_srid = true 24 | end 25 | 26 | ## 27 | # Generates tiles for given zoom and changeset. 28 | # 29 | def generate(zoom, changeset_id, options = {}) 30 | tile_count = nil 31 | @@log.debug "mem = #{memory_usage} (before)" 32 | @conn.transaction do |c| 33 | tile_count = generate_tiles(zoom, changeset_id, options) 34 | end 35 | cleanup 36 | @@log.debug "mem = #{memory_usage} (after)" 37 | tile_count 38 | end 39 | 40 | ## 41 | # Removes tiles for given zoom and changeset. This is useful when retiling (creating new tiles) to avoid 42 | # duplicate primary key error during insert. 43 | # 44 | def clear_tiles(changeset_id, zoom) 45 | count = @conn.exec("DELETE FROM changeset_tiles WHERE changeset_id = #{changeset_id} AND zoom = #{zoom}").cmd_tuples 46 | @@log.debug "Removed existing tiles: #{count}" 47 | count 48 | end 49 | 50 | def has_tiles(changeset_id) 51 | @conn.exec("SELECT COUNT(*) FROM changeset_tiles WHERE changeset_id = #{changeset_id}").getvalue(0, 0).to_i > 0 52 | end 53 | 54 | protected 55 | 56 | def cleanup 57 | @conn.exec('TRUNCATE _tiles') 58 | end 59 | 60 | def generate_tiles(zoom, changeset_id, options = {}) 61 | if options[:retile] 62 | clear_tiles(changeset_id, zoom) 63 | else 64 | return -1 if has_tiles(changeset_id) 65 | end 66 | 67 | @conn.exec_prepared('generate_changes', [changeset_id]) 68 | 69 | count = 0 70 | 71 | for change in @conn.exec_prepared('select_changes', [changeset_id]).to_a 72 | #p change 73 | if change['geom'] 74 | change['geom_obj'] = @wkb_reader.read_hex(change['geom']) 75 | change['geom_obj_prep'] = change['geom_obj'].to_prepared 76 | end 77 | 78 | if change['prev_geom'] 79 | change['prev_geom_obj'] = @wkb_reader.read_hex(change['prev_geom']) 80 | change['prev_geom_obj_prep'] = change['prev_geom_obj'].to_prepared 81 | end 82 | 83 | if change['diff_bbox'] 84 | change['diff_geom_obj'] = change['geom_obj'].difference(change['prev_geom_obj']) 85 | change['diff_geom_obj_prep'] = change['diff_geom_obj'].to_prepared 86 | end 87 | 88 | @@log.debug "#{change['action']} #{change['el_type']} #{change['el_id']} (#{change['version']})" 89 | 90 | count += create_change_tiles(changeset_id, change, change['id'].to_i, zoom) 91 | 92 | # GC has problems if we don't do this explicitly... 93 | change['geom_obj'] = nil 94 | change['prev_geom_obj'] = nil 95 | change['diff_geom_obj'] = nil 96 | end 97 | 98 | @conn.exec_prepared('generate_changeset_tiles', [changeset_id, zoom]) 99 | 100 | @@log.debug "Aggregating tiles..." 101 | 102 | # Now generate tiles at lower zoom levels. 103 | (12..zoom).reverse_each do |i| 104 | @conn.exec("SELECT OWL_AggregateChangeset(#{changeset_id}, #{i}, #{i - 1})") 105 | end 106 | 107 | count 108 | end 109 | 110 | def create_change_tiles(changeset_id, change, change_id, zoom) 111 | if change['el_type'] == 'N' 112 | count = 1 113 | if change['geom'] and change['prev_geom'] 114 | bbox_tile = bbox_to_tiles(zoom, box2d_to_bbox(change['geom_bbox'])).to_a[0] 115 | prev_bbox_tile = bbox_to_tiles(zoom, box2d_to_bbox(change['prev_geom_bbox'])).to_a[0] 116 | if bbox_tile == prev_bbox_tile 117 | add_change_tile(bbox_tile[0], bbox_tile[1], zoom, change, change['geom_obj'], change['prev_geom_obj']) 118 | else 119 | add_change_tile(bbox_tile[0], bbox_tile[1], zoom, change, change['geom_obj'], nil) 120 | add_change_tile(prev_bbox_tile[0], prev_bbox_tile[1], zoom, change, nil, change['prev_geom_obj']) 121 | count = 2 122 | end 123 | elsif change['geom'] 124 | bbox_tile = bbox_to_tiles(zoom, box2d_to_bbox(change['geom_bbox'])).to_a[0] 125 | add_change_tile(bbox_tile[0], bbox_tile[1], zoom, change, change['geom_obj'], change['prev_geom_obj']) 126 | elsif change['prev_geom'] 127 | prev_bbox_tile = bbox_to_tiles(zoom, box2d_to_bbox(change['prev_geom_bbox'])).to_a[0] 128 | add_change_tile(prev_bbox_tile[0], prev_bbox_tile[1], zoom, change, nil, change['prev_geom_obj']) 129 | end 130 | return count 131 | end 132 | 133 | if change['diff_bbox'] 134 | count = create_geom_tiles_diff(changeset_id, change, zoom) 135 | else 136 | count = create_geom_tiles(changeset_id, change, change['geom_obj'], change['geom_obj_prep'], change_id, zoom, false) 137 | if !change['equal'] 138 | count += create_geom_tiles(changeset_id, change, change['prev_geom_obj'], change['prev_geom_obj_prep'], change_id, zoom, true) 139 | end 140 | end 141 | 142 | @@log.debug " Created #{count} tile(s)" 143 | count 144 | end 145 | 146 | def create_geom_tiles_diff(changeset_id, change, zoom) 147 | bbox_to_use = 'diff_bbox' 148 | bbox = box2d_to_bbox(change[bbox_to_use]) 149 | tile_count = bbox_tile_count(zoom, bbox) 150 | 151 | @@log.debug " tile_count = #{tile_count} (using #{bbox_to_use})" 152 | 153 | tiles = prepare_tiles(zoom, change, change['diff_geom_obj_prep'], bbox, tile_count) 154 | 155 | if tiles.size == 1 156 | add_change_tile(tiles.to_a[0][0], tiles.to_a[0][1], zoom, change, change['geom_obj'], change['prev_geom_obj']) 157 | return 1 158 | end 159 | 160 | @@log.debug " Processing #{tiles.size} tile(s)..." 161 | 162 | count = 0 163 | 164 | for tile in tiles 165 | x, y = tile[0], tile[1] 166 | tile_geom = get_tile_geom(x, y, zoom) 167 | intersection = nil 168 | intersection_prev = nil 169 | 170 | if change['geom_obj'].intersects?(tile_geom) 171 | intersection = change['geom_obj'].intersection(tile_geom) 172 | intersection.srid = 4326 173 | end 174 | 175 | if change['prev_geom_obj'].intersects?(tile_geom) 176 | intersection_prev = change['prev_geom_obj'].intersection(tile_geom) 177 | intersection_prev.srid = 4326 178 | end 179 | 180 | add_change_tile(x, y, zoom, change, intersection, intersection_prev) 181 | count += 1 182 | end 183 | count 184 | end 185 | 186 | def create_geom_tiles(changeset_id, change, geom, geom_prep, change_id, zoom, is_prev) 187 | return 0 if geom.nil? 188 | 189 | bbox_to_use = (is_prev ? 'prev_geom' : 'geom') + '_bbox' 190 | bbox = box2d_to_bbox(change[bbox_to_use]) 191 | tile_count = bbox_tile_count(zoom, bbox) 192 | 193 | @@log.debug " tile_count = #{tile_count} (using #{bbox_to_use})" 194 | 195 | tiles = prepare_tiles(zoom, change, geom_prep, bbox, tile_count) 196 | 197 | if tiles.size == 1 198 | add_change_tile(tiles.to_a[0][0], tiles.to_a[0][1], zoom, change, is_prev ? nil : geom, is_prev ? geom : nil) 199 | return 1 200 | end 201 | 202 | @@log.debug " Processing #{tiles.size} tile(s)..." 203 | 204 | test_geom = change['diff_geom_obj_prep'] || geom_prep 205 | count = 0 206 | 207 | for tile in tiles 208 | x, y = tile[0], tile[1] 209 | tile_geom = get_tile_geom(x, y, zoom) 210 | 211 | if test_geom.intersects?(tile_geom) 212 | intersection = geom.intersection(tile_geom) 213 | intersection.srid = 4326 214 | add_change_tile(x, y, zoom, change, is_prev ? nil : intersection, is_prev ? intersection : nil) 215 | count += 1 216 | end 217 | end 218 | count 219 | end 220 | 221 | def prepare_tiles(zoom, change, geom_prep, bbox, tile_count) 222 | tiles = [] 223 | if tile_count < 64 224 | # Does not make sense to try to reduce small geoms. 225 | tiles = bbox_to_tiles(zoom, bbox) 226 | else 227 | tiles_to_check = (tile_count < 2048 ? bbox_to_tiles(14, bbox) : prepare_tiles_to_check(geom_prep, bbox, 14)) 228 | @@log.debug " tiles_to_check = #{tiles_to_check.size}" 229 | tiles = reduce_tiles(tiles_to_check, geom_prep, 14, zoom) 230 | end 231 | tiles 232 | end 233 | 234 | def add_change_tile(x, y, zoom, change, geom, prev_geom) 235 | @conn.exec_prepared('insert_tile', [ 236 | x, 237 | y, 238 | change['id'], 239 | change['tstamp'], 240 | (geom ? @wkb_writer.write_hex(geom) : nil), 241 | (prev_geom ? @wkb_writer.write_hex(prev_geom) : nil) 242 | ]) 243 | end 244 | 245 | def reduce_tiles(tiles_to_check, geom, source_zoom, zoom) 246 | tiles = Set.new 247 | for tile in tiles_to_check 248 | tile_geom = get_tile_geom(tile[0], tile[1], source_zoom) 249 | intersects = geom.intersects?(tile_geom) 250 | tiles.merge(subtiles(tile, source_zoom, zoom)) if intersects 251 | end 252 | tiles 253 | end 254 | 255 | def prepare_tiles_to_check(geom, bbox, source_zoom) 256 | tiles = Set.new 257 | test_zoom = 11 258 | bbox_to_tiles(test_zoom, bbox).select {|tile| geom.intersects?(get_tile_geom(tile[0], tile[1], test_zoom))}.each do |tile| 259 | tiles.merge(subtiles(tile, test_zoom, source_zoom)) 260 | end 261 | tiles 262 | end 263 | 264 | def get_tile_geom(x, y, zoom) 265 | cs = Geos::CoordinateSequence.new(5, 2) 266 | y1, x1 = tile2latlon(x, y, zoom) 267 | y2, x2 = tile2latlon(x + 1, y + 1, zoom) 268 | cs.y[0], cs.x[0] = y1, x1 269 | cs.y[1], cs.x[1] = y1, x2 270 | cs.y[2], cs.x[2] = y2, x2 271 | cs.y[3], cs.x[3] = y2, x1 272 | cs.y[4], cs.x[4] = y1, x1 273 | Geos::create_polygon(cs, :srid => 4326) 274 | end 275 | 276 | def setup_prepared_statements 277 | @conn.exec('DROP TABLE IF EXISTS _tiles') 278 | @conn.exec('CREATE TEMPORARY TABLE _tiles (x int, y int, change_id bigint, 279 | tstamp timestamp without time zone, 280 | geom geometry(GEOMETRY, 4326), prev_geom geometry(GEOMETRY, 4326))') 281 | 282 | @conn.prepare('generate_changes', "SELECT OWL_GenerateChanges($1)") 283 | @conn.prepare('select_changes', " 284 | SELECT *, 285 | CASE WHEN el_type = 'N' THEN ST_X(prev_geom) ELSE NULL END AS prev_lon, 286 | CASE WHEN el_type = 'N' THEN ST_Y(prev_geom) ELSE NULL END AS prev_lat, 287 | CASE WHEN el_type = 'N' THEN ST_X(geom) ELSE NULL END AS lon, 288 | CASE WHEN el_type = 'N' THEN ST_Y(geom) ELSE NULL END AS lat, 289 | Box2D(geom) AS geom_bbox, 290 | Box2D(prev_geom) AS prev_geom_bbox, 291 | Box2D(ST_Difference(geom, prev_geom)) AS diff_bbox, 292 | ST_Equals(geom, prev_geom) AS equal 293 | FROM changes 294 | WHERE changeset_id = $1") 295 | 296 | @conn.prepare('insert_tile', 297 | "INSERT INTO _tiles (x, y, change_id, tstamp, geom, prev_geom) VALUES ($1, $2, $3, $4, $5, $6)") 298 | 299 | @conn.prepare('generate_changeset_tiles', 300 | "INSERT INTO changeset_tiles (changeset_id, tstamp, zoom, x, y, change_id, geom, prev_geom) 301 | SELECT $1, tstamp, $2, x, y, change_id, geom, prev_geom 302 | FROM _tiles") 303 | end 304 | end 305 | 306 | end 307 | -------------------------------------------------------------------------------- /db/sql/owl_functions.sql: -------------------------------------------------------------------------------- 1 | -- 2 | -- Returns index of an element in given array. 3 | -- 4 | -- Source: http://wiki.postgresql.org/wiki/Array_Index 5 | -- 6 | CREATE OR REPLACE FUNCTION idx(anyarray, anyelement) 7 | RETURNS int AS 8 | $$ 9 | SELECT i FROM ( 10 | SELECT generate_series(array_lower($1,1),array_upper($1,1)) 11 | ) g(i) 12 | WHERE $1[i] = $2 13 | LIMIT 1; 14 | $$ LANGUAGE sql IMMUTABLE; 15 | 16 | -- 17 | -- Returns the intersection of two arrays. 18 | -- 19 | CREATE OR REPLACE FUNCTION array_intersect(anyarray, anyarray) 20 | RETURNS anyarray 21 | language sql 22 | as $FUNCTION$ 23 | SELECT ARRAY( 24 | SELECT UNNEST($1) 25 | INTERSECT 26 | SELECT UNNEST($2) 27 | ); 28 | $FUNCTION$; 29 | 30 | CREATE OR REPLACE FUNCTION OWL_IsPolygon(hstore) RETURNS boolean AS 31 | $$ 32 | SELECT $1 IS NOT NULL AND 33 | ($1 ? 'area' OR $1 ? 'landuse' OR $1 ? 'leisure' OR $1 ? 'amenity' OR $1 ? 'building') 34 | $$ LANGUAGE sql IMMUTABLE; 35 | 36 | -- 37 | -- OWL_MakeLine 38 | -- 39 | -- Creates a linestring from given list of node ids. If timestamp is given, 40 | -- the node versions used are filtered by this timestamp (are no newer than). 41 | -- This is useful for creating way geometry for historic versions. 42 | -- 43 | -- Note that it returns NULL when it cannot recreate the geometry, e.g. when 44 | -- there is not enough historical node versions in the database. 45 | -- 46 | CREATE OR REPLACE FUNCTION OWL_MakeLine(bigint[], timestamp without time zone) RETURNS geometry(GEOMETRY, 4326) AS $$ 47 | DECLARE 48 | way_geom geometry(GEOMETRY, 4326); 49 | 50 | BEGIN 51 | way_geom := (SELECT ST_MakeLine(q.geom ORDER BY x.seq) 52 | FROM (SELECT row_number() OVER () AS seq, n AS node_id FROM unnest($1) n) x 53 | INNER JOIN ( 54 | SELECT DISTINCT ON (id) id, geom 55 | FROM nodes n 56 | WHERE n.id IN (SELECT unnest($1)) 57 | AND tstamp <= $2 AND visible 58 | ORDER BY id, version DESC, tstamp DESC) q ON (q.id = x.node_id)); 59 | 60 | -- Now check if the linestring has exactly the right number of points. 61 | IF ST_NumPoints(way_geom) != array_length($1, 1) THEN 62 | RETURN NULL; 63 | END IF; 64 | 65 | RETURN ST_MakeValid(way_geom); 66 | END; 67 | $$ LANGUAGE plpgsql IMMUTABLE; 68 | 69 | CREATE OR REPLACE FUNCTION OWL_MakeLineFromTmpNodes(bigint[]) RETURNS geometry(GEOMETRY, 4326) AS $$ 70 | DECLARE 71 | way_geom geometry(GEOMETRY, 4326); 72 | 73 | BEGIN 74 | way_geom := (SELECT ST_MakeLine(geom ORDER BY seq) FROM _tmp_current_nodes); 75 | 76 | -- Now check if the linestring has exactly the right number of points. 77 | IF ST_NumPoints(way_geom) != array_length($1, 1) THEN 78 | --raise notice '--------------- %, %', $1, (select array_agg(id) from _tmp_nodes); 79 | RETURN NULL; 80 | END IF; 81 | 82 | RETURN ST_MakeValid(way_geom); 83 | END; 84 | $$ LANGUAGE plpgsql VOLATILE; 85 | 86 | -- 87 | -- OWL_MakeMinimalLine 88 | -- 89 | -- Creates a line from given nodes just as OWL_MakeLine does but also 90 | -- considers additional array of "minimal" nodes. If it's possible to 91 | -- build a shorter line using this subset of all nodes then do it. 92 | -- 93 | -- $1 - all nodes 94 | -- $2 - tstamp to use for constructing geometry 95 | -- $3 - "minimal" nodes (needs to be a subset of $1) 96 | -- 97 | CREATE OR REPLACE FUNCTION OWL_MakeMinimalLine(bigint[], timestamp without time zone, bigint[]) RETURNS geometry(GEOMETRY, 4326) AS $$ 98 | BEGIN 99 | RETURN OWL_MakeLine( 100 | (SELECT $1[MIN(idx($1, minimal_node)) - 2:MAX(idx($1, minimal_node)) + 2] 101 | FROM unnest($3) AS minimal_node), $2); 102 | END 103 | $$ LANGUAGE plpgsql IMMUTABLE; 104 | 105 | -- 106 | -- OWL_InterestingTags 107 | -- 108 | CREATE OR REPLACE FUNCTION OWL_InterestingTags(hstore) RETURNS boolean AS $$ 109 | SELECT $1 IS NOT NULL AND $1 - ARRAY['created_by', 'source'] != '' 110 | $$ LANGUAGE sql IMMUTABLE; 111 | 112 | -- 113 | -- OWL_Equals 114 | -- 115 | -- Determines if two geometries are the same or not in OWL sense. That is, 116 | -- very small changes in node geometry are ignored as they are not really 117 | -- useful to consider as data changes. 118 | -- 119 | CREATE OR REPLACE FUNCTION OWL_Equals(geometry(GEOMETRY, 4326), geometry(GEOMETRY, 4326)) RETURNS boolean AS $$ 120 | BEGIN 121 | IF $1 IS NULL OR $2 IS NULL THEN 122 | RETURN NULL; 123 | END IF; 124 | 125 | IF GeometryType($1) = 'POINT' AND GeometryType($2) = 'POINT' THEN 126 | RETURN $1 = $2; 127 | END IF; 128 | 129 | IF ST_OrderingEquals($1, $2) THEN 130 | RETURN true; 131 | END IF; 132 | 133 | RETURN OWL_Equals_Simple($1, $2); 134 | END 135 | $$ LANGUAGE plpgsql IMMUTABLE; 136 | 137 | CREATE OR REPLACE FUNCTION OWL_Equals_Buffer(geometry(GEOMETRY, 4326), geometry(GEOMETRY, 4326)) RETURNS boolean AS $$ 138 | DECLARE 139 | buf1 geometry(GEOMETRY, 4326); 140 | buf2 geometry(GEOMETRY, 4326); 141 | union_area float; 142 | intersection_area float; 143 | 144 | BEGIN 145 | buf1 := ST_Buffer($1, 0.0002); 146 | buf2 := ST_Buffer($2, 0.0002); 147 | union_area := ST_Area(ST_Union(buf1, buf2)); 148 | intersection_area := ST_Area(ST_Intersection(buf1, buf2)); 149 | IF intersection_area = 0.0 THEN RETURN false; END IF; 150 | RETURN ABS(1.0 - union_area / intersection_area) < 0.0002; 151 | END 152 | $$ LANGUAGE plpgsql IMMUTABLE; 153 | 154 | CREATE OR REPLACE FUNCTION OWL_Equals_Snap(geometry(GEOMETRY, 4326), geometry(GEOMETRY, 4326)) RETURNS boolean AS $$ 155 | SELECT ST_Equals(ST_SnapToGrid(st_Segmentize($1, 0.00002), 0.0002), ST_SnapToGrid(st_Segmentize($2, 0.00002), 0.0002)); 156 | $$ LANGUAGE sql IMMUTABLE; 157 | 158 | CREATE OR REPLACE FUNCTION OWL_Equals_Simple(geometry(GEOMETRY, 4326), geometry(GEOMETRY, 4326)) RETURNS boolean AS $$ 159 | SELECT ABS(ST_Area($1::box2d) - ST_Area($2::box2d)) < 0.0002 AND 160 | ABS(ST_Length($1) - ST_Length($2)) < 0.0002; 161 | $$ LANGUAGE sql IMMUTABLE; 162 | 163 | -- 164 | -- OWL_GenerateChanges 165 | -- 166 | CREATE OR REPLACE FUNCTION OWL_GenerateChanges(bigint) RETURNS VOID AS $$ 167 | DECLARE 168 | min_tstamp timestamp without time zone; 169 | max_tstamp timestamp without time zone; 170 | moved_nodes_ids bigint[]; 171 | row_count int; 172 | 173 | BEGIN 174 | RAISE NOTICE '% -- Generating changes for changeset %', clock_timestamp(), $1; 175 | 176 | CREATE TEMPORARY TABLE _tmp_result ( 177 | tstamp timestamp without time zone, 178 | el_type element_type, 179 | action action, 180 | el_id bigint, 181 | version int, 182 | tags hstore, 183 | prev_tags hstore, 184 | geom geometry(GEOMETRY, 4326), 185 | prev_geom geometry(GEOMETRY, 4326), 186 | nodes bigint[], 187 | prev_nodes bigint[] 188 | ) ON COMMIT DROP; 189 | 190 | INSERT INTO _tmp_result 191 | SELECT 192 | n.tstamp, 193 | 'N'::element_type, 194 | CASE 195 | WHEN n.version = 1 THEN 'CREATE'::action 196 | WHEN n.version > 1 AND n.visible THEN 'MODIFY'::action 197 | WHEN NOT n.visible THEN 'DELETE'::action 198 | END::action, 199 | n.id, 200 | n.version, 201 | n.tags, 202 | prev.tags, 203 | n.geom, 204 | prev.geom, 205 | NULL::bigint[], 206 | NULL::bigint[] 207 | FROM nodes n 208 | LEFT JOIN nodes prev ON (prev.id = n.id AND prev.version = n.version - 1) 209 | WHERE n.changeset_id = $1 AND (prev.version IS NOT NULL OR n.version = 1); 210 | 211 | GET DIAGNOSTICS row_count = ROW_COUNT; 212 | RAISE NOTICE '% -- Changeset nodes selected (%)', clock_timestamp(), row_count; 213 | 214 | INSERT INTO _tmp_result 215 | SELECT 216 | w.tstamp, 217 | 'W'::element_type, 218 | CASE 219 | WHEN w.version = 1 THEN 'CREATE'::action 220 | WHEN w.version > 1 AND w.visible THEN 'MODIFY'::action 221 | WHEN NOT w.visible THEN 'DELETE'::action 222 | END, 223 | w.id, 224 | w.version, 225 | w.tags, 226 | prev.tags, 227 | NULL, 228 | NULL, 229 | w.nodes, 230 | prev.nodes 231 | FROM ways w 232 | LEFT JOIN ways prev ON (prev.id = w.id AND prev.version = w.version - 1) 233 | WHERE w.changeset_id = $1 AND (prev.version IS NOT NULL OR w.version = 1); 234 | 235 | GET DIAGNOSTICS row_count = ROW_COUNT; 236 | RAISE NOTICE '% -- Changeset ways selected (%)', clock_timestamp(), row_count; 237 | 238 | SELECT MAX(tstamp), MIN(tstamp) 239 | INTO max_tstamp, min_tstamp 240 | FROM _tmp_result; 241 | 242 | moved_nodes_ids := (SELECT array_agg(el_id) FROM _tmp_result 243 | WHERE el_type = 'N' AND version > 1 AND NOT geom = prev_geom AND action != 'DELETE'); 244 | 245 | RAISE NOTICE '% -- Prepared data (min = %, max = %, moved nodes = %)', clock_timestamp(), 246 | min_tstamp, max_tstamp, array_length(moved_nodes_ids, 1); 247 | 248 | -- Affected ways 249 | 250 | INSERT INTO _tmp_result 251 | SELECT 252 | w.tstamp, 253 | 'W'::element_type, 254 | 'AFFECT'::action, 255 | w.id, 256 | version, 257 | w.tags, 258 | w.tags, 259 | NULL, 260 | NULL, 261 | w.nodes, 262 | w.nodes 263 | --OWL_MakeMinimalLine(w.nodes, max_tstamp, array_intersect(w.nodes, moved_nodes_ids)) AS geom, 264 | --OWL_MakeMinimalLine(w.nodes, min_tstamp, array_intersect(w.nodes, moved_nodes_ids)) AS prev_geom 265 | --OWL_MakeLine(w.nodes, max_tstamp) AS geom, 266 | --OWL_MakeLine(w.nodes, min_tstamp) AS prev_geom 267 | FROM ways w 268 | WHERE w.nodes && moved_nodes_ids AND 269 | w.version = (SELECT version FROM ways WHERE id = w.id AND tstamp <= max_tstamp ORDER BY version DESC LIMIT 1); 270 | 271 | GET DIAGNOSTICS row_count = ROW_COUNT; 272 | RAISE NOTICE '% -- Affected ways done (%)', clock_timestamp(), row_count; 273 | 274 | DELETE FROM _tmp_result 275 | WHERE el_type = 'N' AND el_id IN (SELECT unnest(nodes) FROM _tmp_result UNION SELECT unnest(prev_nodes) FROM _tmp_result) 276 | AND NOT OWL_InterestingTags(tags) AND NOT OWL_InterestingTags(prev_tags); 277 | 278 | GET DIAGNOSTICS row_count = ROW_COUNT; 279 | RAISE NOTICE '% -- Removed not interesting nodes (%)', clock_timestamp(), row_count; 280 | 281 | UPDATE _tmp_result w 282 | SET action = 'CREATE', prev_geom = NULL 283 | WHERE el_type = 'W' AND EXISTS 284 | (SELECT 1 FROM _tmp_result w2 WHERE w2.el_type = 'W' AND w2.el_id = w.el_id AND w2.action = 'CREATE'); 285 | 286 | DELETE FROM _tmp_result w 287 | WHERE el_type = 'W' AND version < (SELECT MAX(w2.version) FROM _tmp_result w2 WHERE w2.el_type = 'W' AND w2.el_id = w.el_id); 288 | 289 | GET DIAGNOSTICS row_count = ROW_COUNT; 290 | RAISE NOTICE '% -- Removed old changes (%)', clock_timestamp(), row_count; 291 | 292 | UPDATE _tmp_result 293 | SET 294 | geom = 295 | CASE 296 | WHEN action = 'DELETE' THEN NULL 297 | ELSE OWL_MakeLine(nodes, max_tstamp) 298 | END, 299 | prev_geom = 300 | CASE 301 | WHEN action = 'CREATE' THEN NULL 302 | ELSE OWL_MakeLine(prev_nodes, min_tstamp) 303 | END 304 | WHERE el_type = 'W'; 305 | 306 | GET DIAGNOSTICS row_count = ROW_COUNT; 307 | RAISE NOTICE '% -- Updated way geoms (%)', clock_timestamp(), row_count; 308 | 309 | DELETE FROM _tmp_result 310 | WHERE el_type = 'W' AND geom IS NOT NULL AND prev_geom IS NOT NULL 311 | AND OWL_Equals(geom, prev_geom) AND tags = prev_tags AND nodes = prev_nodes; 312 | 313 | GET DIAGNOSTICS row_count = ROW_COUNT; 314 | RAISE NOTICE '% -- Removed not interesting ways (%)', clock_timestamp(), row_count; 315 | 316 | RAISE NOTICE '% -- Returning % change(s)', clock_timestamp(), (SELECT COUNT(*) FROM _tmp_result); 317 | 318 | WITH _tmp_changes AS ( 319 | SELECT *, ROW_NUMBER() OVER(PARTITION BY el_id ORDER BY version DESC, tstamp DESC) AS rownum 320 | FROM _tmp_result 321 | ) 322 | INSERT INTO changes ( 323 | tstamp, 324 | el_type, 325 | action, 326 | el_id, 327 | version, 328 | changeset_id, 329 | tags, 330 | prev_tags, 331 | geom, 332 | prev_geom) 333 | SELECT 334 | tstamp, 335 | el_type, 336 | action, 337 | el_id, 338 | version, 339 | $1, 340 | tags, 341 | prev_tags, 342 | geom, 343 | prev_geom 344 | FROM 345 | _tmp_changes 346 | WHERE rownum = 1; 347 | END 348 | $$ LANGUAGE plpgsql; 349 | 350 | -- 351 | -- OWL_UpdateChangeset 352 | -- 353 | CREATE OR REPLACE FUNCTION OWL_UpdateChangeset(bigint) RETURNS void AS $$ 354 | DECLARE 355 | row record; 356 | idx int; 357 | result int[9]; 358 | result_bbox geometry; 359 | BEGIN 360 | result := ARRAY[0, 0, 0, 0, 0, 0, 0, 0, 0]; 361 | FOR row IN 362 | SELECT 363 | CASE el_type WHEN 'N' THEN 0 WHEN 'W' THEN 1 WHEN 'R' THEN 2 END AS el_type_idx, 364 | CASE action WHEN 'CREATE' THEN 0 WHEN 'MODIFY' THEN 1 WHEN 'DELETE' THEN 2 END AS action_idx, 365 | COUNT(*) as cnt 366 | FROM changes 367 | WHERE changeset_id = $1 368 | GROUP BY el_type, action 369 | LOOP 370 | result[row.el_type_idx * 3 + row.action_idx + 1] := row.cnt; 371 | END LOOP; 372 | 373 | result_bbox := (SELECT ST_Envelope(ST_Collect(ST_Collect(current_geom, new_geom))) FROM changes WHERE changeset_id = $1); 374 | 375 | UPDATE changesets cs SET entity_changes = result, bbox = result_bbox 376 | WHERE cs.id = $1; 377 | END; 378 | $$ LANGUAGE plpgsql; 379 | 380 | -- 381 | -- OWL_UpdateChangeset 382 | -- 383 | CREATE OR REPLACE FUNCTION OWL_UpdateChangeset(bigint) RETURNS void AS $$ 384 | DECLARE 385 | row record; 386 | idx int; 387 | result int[9]; 388 | result_bbox geometry; 389 | BEGIN 390 | result := ARRAY[0, 0, 0, 0, 0, 0, 0, 0, 0]; 391 | FOR row IN 392 | SELECT 393 | CASE el_type WHEN 'N' THEN 0 WHEN 'W' THEN 1 WHEN 'R' THEN 2 END AS el_type_idx, 394 | CASE action WHEN 'CREATE' THEN 0 WHEN 'MODIFY' THEN 1 WHEN 'DELETE' THEN 2 END AS action_idx, 395 | COUNT(*) as cnt 396 | FROM changes 397 | WHERE changeset_id = $1 398 | GROUP BY el_type, action 399 | LOOP 400 | result[row.el_type_idx * 3 + row.action_idx + 1] := row.cnt; 401 | END LOOP; 402 | 403 | result_bbox := (SELECT ST_Envelope(ST_Collect(ST_Collect(current_geom, new_geom))) FROM changes WHERE changeset_id = $1); 404 | 405 | UPDATE changesets cs SET entity_changes = result, bbox = result_bbox 406 | WHERE cs.id = $1; 407 | END; 408 | $$ LANGUAGE plpgsql; 409 | 410 | -- 411 | -- OWL_AggregateChangeset 412 | -- 413 | CREATE OR REPLACE FUNCTION OWL_AggregateChangeset(bigint, int, int) RETURNS void AS $$ 414 | DECLARE 415 | subtiles_per_tile bigint; 416 | 417 | BEGIN 418 | subtiles_per_tile := POW(2, $2) / POW(2, $3); 419 | 420 | DELETE FROM changeset_tiles WHERE changeset_id = $1 AND zoom = $3; 421 | 422 | INSERT INTO changeset_tiles (changeset_id, change_id, tstamp, x, y, zoom, geom, prev_geom) 423 | SELECT 424 | $1, 425 | change_id, 426 | MAX(tstamp), 427 | x/subtiles_per_tile, 428 | y/subtiles_per_tile, 429 | $3, 430 | ST_Union(geom), 431 | ST_Union(prev_geom) 432 | FROM changeset_tiles 433 | WHERE changeset_id = $1 AND zoom = $2 434 | GROUP BY x/subtiles_per_tile, y/subtiles_per_tile, change_id; 435 | END; 436 | $$ LANGUAGE plpgsql; 437 | -------------------------------------------------------------------------------- /db/structure.sql: -------------------------------------------------------------------------------- 1 | -- 2 | -- PostgreSQL database dump 3 | -- 4 | 5 | SET statement_timeout = 0; 6 | SET lock_timeout = 0; 7 | SET client_encoding = 'UTF8'; 8 | SET standard_conforming_strings = on; 9 | SET check_function_bodies = false; 10 | SET client_min_messages = warning; 11 | 12 | -- 13 | -- Name: plpgsql; Type: EXTENSION; Schema: -; Owner: - 14 | -- 15 | 16 | CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog; 17 | 18 | 19 | -- 20 | -- Name: EXTENSION plpgsql; Type: COMMENT; Schema: -; Owner: - 21 | -- 22 | 23 | COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language'; 24 | 25 | 26 | -- 27 | -- Name: hstore; Type: EXTENSION; Schema: -; Owner: - 28 | -- 29 | 30 | CREATE EXTENSION IF NOT EXISTS hstore WITH SCHEMA public; 31 | 32 | 33 | -- 34 | -- Name: EXTENSION hstore; Type: COMMENT; Schema: -; Owner: - 35 | -- 36 | 37 | COMMENT ON EXTENSION hstore IS 'data type for storing sets of (key, value) pairs'; 38 | 39 | 40 | -- 41 | -- Name: postgis; Type: EXTENSION; Schema: -; Owner: - 42 | -- 43 | 44 | CREATE EXTENSION IF NOT EXISTS postgis WITH SCHEMA public; 45 | 46 | 47 | -- 48 | -- Name: EXTENSION postgis; Type: COMMENT; Schema: -; Owner: - 49 | -- 50 | 51 | COMMENT ON EXTENSION postgis IS 'PostGIS geometry, geography, and raster spatial types and functions'; 52 | 53 | 54 | SET search_path = public, pg_catalog; 55 | 56 | -- 57 | -- Name: action; Type: TYPE; Schema: public; Owner: - 58 | -- 59 | 60 | CREATE TYPE action AS ENUM ( 61 | 'CREATE', 62 | 'MODIFY', 63 | 'DELETE', 64 | 'AFFECT' 65 | ); 66 | 67 | 68 | -- 69 | -- Name: element_type; Type: TYPE; Schema: public; Owner: - 70 | -- 71 | 72 | CREATE TYPE element_type AS ENUM ( 73 | 'N', 74 | 'W', 75 | 'R' 76 | ); 77 | 78 | 79 | -- 80 | -- Name: change; Type: TYPE; Schema: public; Owner: - 81 | -- 82 | 83 | CREATE TYPE change AS ( 84 | id integer, 85 | tstamp timestamp without time zone, 86 | el_type element_type, 87 | action action, 88 | el_id bigint, 89 | version integer, 90 | tags hstore, 91 | prev_tags hstore, 92 | geom geometry(Geometry,4326), 93 | prev_geom geometry(Geometry,4326) 94 | ); 95 | 96 | 97 | -- 98 | -- Name: array_intersect(anyarray, anyarray); Type: FUNCTION; Schema: public; Owner: - 99 | -- 100 | 101 | CREATE FUNCTION array_intersect(anyarray, anyarray) RETURNS anyarray 102 | LANGUAGE sql 103 | AS $_$ 104 | SELECT ARRAY( 105 | SELECT UNNEST($1) 106 | INTERSECT 107 | SELECT UNNEST($2) 108 | ); 109 | $_$; 110 | 111 | 112 | -- 113 | -- Name: idx(anyarray, anyelement); Type: FUNCTION; Schema: public; Owner: - 114 | -- 115 | 116 | CREATE FUNCTION idx(anyarray, anyelement) RETURNS integer 117 | LANGUAGE sql IMMUTABLE 118 | AS $_$ 119 | SELECT i FROM ( 120 | SELECT generate_series(array_lower($1,1),array_upper($1,1)) 121 | ) g(i) 122 | WHERE $1[i] = $2 123 | LIMIT 1; 124 | $_$; 125 | 126 | 127 | -- 128 | -- Name: owl_aggregatechangeset(bigint, integer, integer); Type: FUNCTION; Schema: public; Owner: - 129 | -- 130 | 131 | CREATE FUNCTION owl_aggregatechangeset(bigint, integer, integer) RETURNS void 132 | LANGUAGE plpgsql 133 | AS $_$ 134 | DECLARE 135 | subtiles_per_tile bigint; 136 | 137 | BEGIN 138 | subtiles_per_tile := POW(2, $2) / POW(2, $3); 139 | 140 | DELETE FROM changeset_tiles WHERE changeset_id = $1 AND zoom = $3; 141 | 142 | INSERT INTO changeset_tiles (changeset_id, tstamp, x, y, zoom, geom, prev_geom, changes) 143 | SELECT 144 | $1, 145 | MAX(tstamp), 146 | x/subtiles_per_tile * subtiles_per_tile, 147 | y/subtiles_per_tile * subtiles_per_tile, 148 | $3, 149 | CASE 150 | WHEN $3 >= 14 THEN array_accum(geom) 151 | ELSE array_accum((SELECT array_agg(ST_Envelope(unnest)) FROM unnest(geom))) 152 | END, 153 | CASE 154 | WHEN $3 >= 14 THEN array_accum(prev_geom) 155 | ELSE array_accum((SELECT array_agg(ST_Envelope(unnest)) FROM unnest(prev_geom))) 156 | END, 157 | array_accum(changes) 158 | FROM changeset_tiles 159 | WHERE changeset_id = $1 AND zoom = $2 160 | GROUP BY x/subtiles_per_tile, y/subtiles_per_tile; 161 | END; 162 | $_$; 163 | 164 | 165 | -- 166 | -- Name: owl_equals(geometry, geometry); Type: FUNCTION; Schema: public; Owner: - 167 | -- 168 | 169 | CREATE FUNCTION owl_equals(geometry, geometry) RETURNS boolean 170 | LANGUAGE plpgsql IMMUTABLE 171 | AS $_$ 172 | BEGIN 173 | IF $1 IS NULL OR $2 IS NULL THEN 174 | RETURN NULL; 175 | END IF; 176 | 177 | IF GeometryType($1) = 'POINT' AND GeometryType($2) = 'POINT' THEN 178 | RETURN $1 = $2; 179 | END IF; 180 | 181 | IF ST_OrderingEquals($1, $2) THEN 182 | RETURN true; 183 | END IF; 184 | 185 | RETURN OWL_Equals_Simple($1, $2); 186 | END 187 | $_$; 188 | 189 | 190 | -- 191 | -- Name: owl_equals_buffer(geometry, geometry); Type: FUNCTION; Schema: public; Owner: - 192 | -- 193 | 194 | CREATE FUNCTION owl_equals_buffer(geometry, geometry) RETURNS boolean 195 | LANGUAGE plpgsql IMMUTABLE 196 | AS $_$ 197 | DECLARE 198 | buf1 geometry(GEOMETRY, 4326); 199 | buf2 geometry(GEOMETRY, 4326); 200 | union_area float; 201 | intersection_area float; 202 | 203 | BEGIN 204 | buf1 := ST_Buffer($1, 0.0002); 205 | buf2 := ST_Buffer($2, 0.0002); 206 | union_area := ST_Area(ST_Union(buf1, buf2)); 207 | intersection_area := ST_Area(ST_Intersection(buf1, buf2)); 208 | IF intersection_area = 0.0 THEN RETURN false; END IF; 209 | RETURN ABS(1.0 - union_area / intersection_area) < 0.0002; 210 | END 211 | $_$; 212 | 213 | 214 | -- 215 | -- Name: owl_equals_simple(geometry, geometry); Type: FUNCTION; Schema: public; Owner: - 216 | -- 217 | 218 | CREATE FUNCTION owl_equals_simple(geometry, geometry) RETURNS boolean 219 | LANGUAGE sql IMMUTABLE 220 | AS $_$ 221 | SELECT ABS(ST_Area($1::box2d) - ST_Area($2::box2d)) < 0.0002 AND 222 | ABS(ST_Length($1) - ST_Length($2)) < 0.0002; 223 | $_$; 224 | 225 | 226 | -- 227 | -- Name: owl_equals_snap(geometry, geometry); Type: FUNCTION; Schema: public; Owner: - 228 | -- 229 | 230 | CREATE FUNCTION owl_equals_snap(geometry, geometry) RETURNS boolean 231 | LANGUAGE sql IMMUTABLE 232 | AS $_$ 233 | SELECT ST_Equals(ST_SnapToGrid(st_Segmentize($1, 0.00002), 0.0002), ST_SnapToGrid(st_Segmentize($2, 0.00002), 0.0002)); 234 | $_$; 235 | 236 | 237 | -- 238 | -- Name: owl_generatechanges(bigint); Type: FUNCTION; Schema: public; Owner: - 239 | -- 240 | 241 | CREATE FUNCTION owl_generatechanges(bigint) RETURNS change[] 242 | LANGUAGE plpgsql 243 | AS $_$ 244 | DECLARE 245 | min_tstamp timestamp without time zone; 246 | max_tstamp timestamp without time zone; 247 | moved_nodes_ids bigint[]; 248 | row_count int; 249 | 250 | BEGIN 251 | RAISE NOTICE '% -- Generating changes for changeset %', clock_timestamp(), $1; 252 | 253 | CREATE TEMPORARY TABLE _tmp_changeset_nodes ON COMMIT DROP AS 254 | SELECT 255 | $1, 256 | n.tstamp, 257 | n.changeset_id, 258 | 'N'::element_type AS type, 259 | n.id, 260 | n.version, 261 | CASE 262 | WHEN n.version = 1 THEN 'CREATE'::action 263 | WHEN n.version > 1 AND n.visible THEN 'MODIFY'::action 264 | WHEN NOT n.visible THEN 'DELETE'::action 265 | END AS el_action, 266 | NOT OWL_Equals(n.geom, prev.geom) AS geom_changed, 267 | n.tags != prev.tags AS tags_changed, 268 | NULL::boolean AS nodes_changed, 269 | NULL::boolean AS members_changed, 270 | CASE WHEN NOT n.visible THEN NULL ELSE n.geom END AS geom, 271 | CASE WHEN NOT n.visible OR NOT OWL_Equals(n.geom, prev.geom) THEN prev.geom ELSE NULL END AS prev_geom, 272 | n.tags, 273 | prev.tags AS prev_tags, 274 | NULL::bigint[] AS nodes, 275 | NULL::bigint[] AS prev_nodes 276 | FROM nodes n 277 | LEFT JOIN nodes prev ON (prev.id = n.id AND prev.version = n.version - 1) 278 | WHERE n.changeset_id = $1 AND (prev.version IS NOT NULL OR n.version = 1); 279 | 280 | CREATE TEMPORARY TABLE _tmp_moved_nodes ON COMMIT DROP AS 281 | SELECT * FROM _tmp_changeset_nodes n WHERE n.version > 1 AND n.geom_changed AND n.el_action != 'DELETE'; 282 | 283 | SELECT MAX(x.tstamp), MIN(x.tstamp) - INTERVAL '1 second' 284 | INTO max_tstamp, min_tstamp 285 | FROM ( 286 | SELECT n.tstamp 287 | FROM nodes n 288 | WHERE n.changeset_id = $1 289 | UNION 290 | SELECT w.tstamp 291 | FROM ways w 292 | WHERE w.changeset_id = $1 293 | ) x; 294 | 295 | moved_nodes_ids := (SELECT array_agg(id) FROM _tmp_moved_nodes); 296 | 297 | RAISE NOTICE '% -- Prepared data (min = %, max = %, moved nodes = %)', clock_timestamp(), 298 | min_tstamp, max_tstamp, (SELECT COUNT(*) FROM _tmp_moved_nodes); 299 | 300 | CREATE TEMPORARY TABLE _tmp_result ON COMMIT DROP AS 301 | SELECT * 302 | FROM _tmp_changeset_nodes n 303 | WHERE (n.el_action IN ('CREATE', 'DELETE') OR n.tags_changed OR n.geom_changed) AND 304 | (OWL_RemoveTags(n.tags) != ''::hstore OR OWL_RemoveTags(n.prev_tags) != ''::hstore); 305 | 306 | GET DIAGNOSTICS row_count = ROW_COUNT; 307 | RAISE NOTICE '% -- Nodes done (%)', clock_timestamp(), row_count; 308 | 309 | -- Created ways 310 | 311 | INSERT INTO _tmp_result 312 | SELECT 313 | $1, 314 | w.tstamp, 315 | w.changeset_id, 316 | 'W'::element_type AS type, 317 | w.id, 318 | w.version, 319 | 'CREATE', 320 | NULL, 321 | NULL, 322 | NULL, 323 | NULL, 324 | w.geom, 325 | NULL, 326 | w.tags, 327 | NULL, 328 | w.nodes, 329 | NULL 330 | FROM ( 331 | SELECT w.*, OWL_MakeLine(w.nodes, max_tstamp) AS geom 332 | FROM ways w 333 | WHERE w.changeset_id = $1 AND w.version = 1) w 334 | WHERE w.geom IS NOT NULL; 335 | 336 | GET DIAGNOSTICS row_count = ROW_COUNT; 337 | RAISE NOTICE '% -- Created ways done (%)', clock_timestamp(), row_count; 338 | 339 | -- Modified ways 340 | 341 | INSERT INTO _tmp_result 342 | SELECT 343 | $1, 344 | w.tstamp, 345 | w.changeset_id, 346 | 'W'::element_type AS type, 347 | w.id, 348 | w.version, 349 | 'MODIFY', 350 | w.geom_changed, 351 | w.tags_changed, 352 | w.nodes_changed, 353 | NULL, 354 | CASE WHEN w.geom_changed AND NOT w.tags_changed AND NOT w.nodes_changed THEN 355 | OWL_MakeMinimalLine(w.nodes, max_tstamp, (SELECT array_agg(id) FROM _tmp_changeset_nodes n WHERE w.nodes @> ARRAY[n.id])) 356 | ELSE 357 | OWL_MakeLine(w.nodes, max_tstamp) 358 | END, 359 | CASE WHEN w.geom_changed AND NOT w.tags_changed AND NOT w.nodes_changed THEN 360 | OWL_MakeMinimalLine(w.prev_nodes, min_tstamp, (SELECT array_agg(id) FROM _tmp_changeset_nodes n WHERE w.prev_nodes @> ARRAY[n.id])) 361 | ELSE 362 | CASE WHEN w.visible AND NOT w.geom_changed THEN NULL ELSE OWL_MakeLine(w.prev_nodes, min_tstamp) END 363 | END, 364 | w.tags, 365 | w.prev_tags, 366 | w.nodes, 367 | CASE WHEN w.nodes = w.prev_nodes THEN NULL ELSE w.prev_nodes END 368 | FROM ( 369 | SELECT w.*, 370 | prev.tags AS prev_tags, 371 | prev.nodes AS prev_nodes, 372 | NOT OWL_Equals(OWL_MakeLine(w.nodes, w.tstamp), OWL_MakeLine(prev.nodes, prev.tstamp)) AS geom_changed, 373 | CASE WHEN NOT w.visible OR w.version = 1 THEN NULL ELSE w.tags != prev.tags END AS tags_changed, 374 | w.nodes != prev.nodes AS nodes_changed 375 | FROM ways w 376 | INNER JOIN ways prev ON (prev.id = w.id AND prev.version = w.version - 1) 377 | WHERE w.changeset_id = $1 AND w.visible) w 378 | WHERE w.geom_changed IS NOT NULL AND (w.geom_changed OR w.tags_changed OR w.nodes_changed); 379 | 380 | GET DIAGNOSTICS row_count = ROW_COUNT; 381 | RAISE NOTICE '% -- Modified ways done (%)', clock_timestamp(), row_count; 382 | 383 | -- Deleted ways 384 | 385 | INSERT INTO _tmp_result 386 | SELECT 387 | $1, 388 | w.tstamp, 389 | w.changeset_id, 390 | 'W'::element_type AS type, 391 | w.id, 392 | w.version, 393 | 'DELETE', 394 | NULL, 395 | NULL, 396 | NULL, 397 | NULL, 398 | NULL, 399 | w.prev_geom, 400 | w.tags, 401 | w.prev_tags, 402 | w.nodes, 403 | w.prev_nodes 404 | FROM ( 405 | SELECT w.*, 406 | prev.tags AS prev_tags, 407 | prev.nodes AS prev_nodes, 408 | OWL_MakeLine(prev.nodes, max_tstamp) AS prev_geom 409 | FROM ways w 410 | INNER JOIN ways prev ON (prev.id = w.id AND prev.version = w.version - 1) 411 | WHERE w.changeset_id = $1 AND NOT w.visible) w 412 | WHERE w.prev_geom IS NOT NULL; 413 | 414 | GET DIAGNOSTICS row_count = ROW_COUNT; 415 | RAISE NOTICE '% -- Deleted ways done (%)', clock_timestamp(), row_count; 416 | 417 | -- Affected ways 418 | 419 | INSERT INTO _tmp_result 420 | SELECT 421 | $1, 422 | w.tstamp, 423 | w.changeset_id, 424 | 'W'::element_type AS type, 425 | w.id, 426 | version, 427 | 'AFFECT'::action, 428 | true, 429 | false, 430 | false, 431 | NULL, 432 | w.geom, 433 | w.prev_geom, 434 | w.tags, 435 | NULL, 436 | w.nodes, 437 | NULL 438 | FROM ( 439 | SELECT 440 | *, 441 | OWL_MakeMinimalLine(w.nodes, max_tstamp, array_intersect(w.nodes, moved_nodes_ids)) AS geom, 442 | OWL_MakeMinimalLine(w.nodes, min_tstamp, array_intersect(w.nodes, moved_nodes_ids)) AS prev_geom 443 | --OWL_MakeLine(w.nodes, max_tstamp) AS geom, 444 | --OWL_MakeLine(w.nodes, min_tstamp) AS prev_geom 445 | FROM ways w 446 | WHERE w.nodes && moved_nodes_ids AND 447 | w.version = (SELECT version FROM ways WHERE id = w.id AND tstamp <= max_tstamp ORDER BY version DESC LIMIT 1) AND 448 | w.changeset_id != $1) w; -- AND 449 | --OWL_MakeLine(w.nodes, max_tstamp) IS NOT NULL AND 450 | --(w.version = 1 OR OWL_MakeLine(w.nodes, min_tstamp) IS NOT NULL)) w; 451 | 452 | GET DIAGNOSTICS row_count = ROW_COUNT; 453 | RAISE NOTICE '% -- Affected ways done (%)', clock_timestamp(), row_count; 454 | 455 | RAISE NOTICE '% -- Returning % change(s)', clock_timestamp(), (SELECT COUNT(*) FROM _tmp_result); 456 | 457 | RETURN (SELECT array_agg(x.c) FROM (SELECT ROW( 458 | (row_number() OVER ())::int, 459 | tstamp, 460 | type, 461 | el_action, 462 | id, 463 | version, 464 | tags, 465 | prev_tags, 466 | geom, 467 | prev_geom 468 | )::change AS c FROM _tmp_result ORDER BY tstamp) x); 469 | END 470 | $_$; 471 | 472 | 473 | -- 474 | -- Name: owl_ispolygon(hstore); Type: FUNCTION; Schema: public; Owner: - 475 | -- 476 | 477 | CREATE FUNCTION owl_ispolygon(hstore) RETURNS boolean 478 | LANGUAGE sql IMMUTABLE 479 | AS $_$ 480 | SELECT $1 IS NOT NULL AND 481 | ($1 ? 'area' OR $1 ? 'landuse' OR $1 ? 'leisure' OR $1 ? 'amenity' OR $1 ? 'building') 482 | $_$; 483 | 484 | 485 | -- 486 | -- Name: owl_jointilegeometriesbychange(change[]); Type: FUNCTION; Schema: public; Owner: - 487 | -- 488 | 489 | CREATE FUNCTION owl_jointilegeometriesbychange(change[]) RETURNS text[] 490 | LANGUAGE sql IMMUTABLE 491 | AS $_$ 492 | WITH joined_arrays AS ( 493 | SELECT (c.unnest).id AS change_id, (c.unnest).geom, GeometryType((c.unnest).geom) AS geom_type 494 | FROM (SELECT unnest($1)) c 495 | ) 496 | SELECT 497 | array_agg(CASE WHEN c.g IS NOT NULL AND NOT ST_IsEmpty(c.g) AND ST_NumGeometries(c.g) > 0 THEN ST_AsGeoJSON(ST_CollectionHomogenize(c.g)) ELSE NULL END order by c.change_id) 498 | FROM ( 499 | SELECT ST_Collect(g) AS g, change_id 500 | FROM 501 | (SELECT NULL AS g, change_id 502 | FROM joined_arrays 503 | WHERE geom_type IS NULL 504 | GROUP BY change_id 505 | UNION 506 | SELECT ST_LineMerge(ST_Union(geom)) AS g, change_id 507 | FROM joined_arrays 508 | WHERE geom_type IS NOT NULL AND geom_type LIKE '%LINESTRING' 509 | GROUP BY change_id 510 | UNION 511 | SELECT ST_Union(geom) AS g, change_id 512 | FROM joined_arrays 513 | WHERE geom_type IS NOT NULL AND geom_type NOT LIKE '%LINESTRING' 514 | GROUP BY change_id) x 515 | GROUP BY change_id) c 516 | $_$; 517 | 518 | 519 | -- 520 | -- Name: owl_latlontotile(integer, geometry); Type: FUNCTION; Schema: public; Owner: - 521 | -- 522 | 523 | CREATE FUNCTION owl_latlontotile(integer, geometry) RETURNS TABLE(x integer, y integer) 524 | LANGUAGE sql 525 | AS $_$ 526 | SELECT 527 | floor((POW(2, $1) * ((ST_X($2) + 180) / 360)))::int AS tile_x, 528 | floor((1.0 - ln(tan(radians(ST_Y($2))) + 1.0 / cos(radians(ST_Y($2)))) / pi()) / 2.0 * POW(2, $1))::int AS tile_y; 529 | $_$; 530 | 531 | 532 | -- 533 | -- Name: owl_makeline(bigint[], timestamp without time zone); Type: FUNCTION; Schema: public; Owner: - 534 | -- 535 | 536 | CREATE FUNCTION owl_makeline(bigint[], timestamp without time zone) RETURNS geometry 537 | LANGUAGE plpgsql IMMUTABLE 538 | AS $_$ 539 | DECLARE 540 | way_geom geometry(GEOMETRY, 4326); 541 | 542 | BEGIN 543 | way_geom := (SELECT ST_MakeLine(q.geom ORDER BY x.seq) 544 | FROM (SELECT row_number() OVER () AS seq, n AS node_id FROM unnest($1) n) x 545 | INNER JOIN ( 546 | SELECT DISTINCT ON (id) id, geom 547 | FROM nodes n 548 | WHERE n.id IN (SELECT unnest($1)) 549 | AND tstamp <= $2 AND visible 550 | ORDER BY id, tstamp DESC) q ON (q.id = x.node_id)); 551 | 552 | -- Now check if the linestring has exactly the right number of points. 553 | IF ST_NumPoints(way_geom) != array_length($1, 1) THEN 554 | RETURN NULL; 555 | END IF; 556 | 557 | RETURN ST_MakeValid(way_geom); 558 | END; 559 | $_$; 560 | 561 | 562 | -- 563 | -- Name: owl_makelinefromtmpnodes(bigint[]); Type: FUNCTION; Schema: public; Owner: - 564 | -- 565 | 566 | CREATE FUNCTION owl_makelinefromtmpnodes(bigint[]) RETURNS geometry 567 | LANGUAGE plpgsql 568 | AS $_$ 569 | DECLARE 570 | way_geom geometry(GEOMETRY, 4326); 571 | 572 | BEGIN 573 | way_geom := (SELECT ST_MakeLine(geom ORDER BY seq) FROM _tmp_current_nodes); 574 | 575 | -- Now check if the linestring has exactly the right number of points. 576 | IF ST_NumPoints(way_geom) != array_length($1, 1) THEN 577 | --raise notice '--------------- %, %', $1, (select array_agg(id) from _tmp_nodes); 578 | RETURN NULL; 579 | END IF; 580 | 581 | RETURN ST_MakeValid(way_geom); 582 | END; 583 | $_$; 584 | 585 | 586 | -- 587 | -- Name: owl_makeminimalline(bigint[], timestamp without time zone, bigint[]); Type: FUNCTION; Schema: public; Owner: - 588 | -- 589 | 590 | CREATE FUNCTION owl_makeminimalline(bigint[], timestamp without time zone, bigint[]) RETURNS geometry 591 | LANGUAGE plpgsql IMMUTABLE 592 | AS $_$ 593 | BEGIN 594 | RETURN OWL_MakeLine( 595 | (SELECT $1[MIN(idx($1, minimal_node)) - 2:MAX(idx($1, minimal_node)) + 2] 596 | FROM unnest($3) AS minimal_node), $2); 597 | END 598 | $_$; 599 | 600 | 601 | -- 602 | -- Name: owl_mergechanges(change[]); Type: FUNCTION; Schema: public; Owner: - 603 | -- 604 | 605 | CREATE FUNCTION owl_mergechanges(change[]) RETURNS change[] 606 | LANGUAGE sql 607 | AS $_$ 608 | SELECT array_agg(x.ch ORDER BY (x.ch).id) FROM ( 609 | SELECT DISTINCT ROW( 610 | id, 611 | tstamp, 612 | el_type, 613 | action, 614 | el_id, 615 | version, 616 | tags, 617 | prev_tags, 618 | NULL, 619 | NULL)::change ch 620 | FROM unnest($1) c 621 | GROUP BY c.id, c.tstamp, c.el_type, c.action, c.el_id, c.version, c.tags, c.prev_tags) x 622 | $_$; 623 | 624 | 625 | -- 626 | -- Name: owl_removetags(hstore); Type: FUNCTION; Schema: public; Owner: - 627 | -- 628 | 629 | CREATE FUNCTION owl_removetags(hstore) RETURNS hstore 630 | LANGUAGE sql IMMUTABLE STRICT 631 | AS $_$ 632 | SELECT $1 - ARRAY['created_by', 'source'] 633 | $_$; 634 | 635 | 636 | -- 637 | -- Name: owl_updatechangeset(bigint); Type: FUNCTION; Schema: public; Owner: - 638 | -- 639 | 640 | CREATE FUNCTION owl_updatechangeset(bigint) RETURNS void 641 | LANGUAGE plpgsql 642 | AS $_$ 643 | DECLARE 644 | row record; 645 | idx int; 646 | result int[9]; 647 | result_bbox geometry; 648 | BEGIN 649 | result := ARRAY[0, 0, 0, 0, 0, 0, 0, 0, 0]; 650 | FOR row IN 651 | SELECT 652 | CASE el_type WHEN 'N' THEN 0 WHEN 'W' THEN 1 WHEN 'R' THEN 2 END AS el_type_idx, 653 | CASE action WHEN 'CREATE' THEN 0 WHEN 'MODIFY' THEN 1 WHEN 'DELETE' THEN 2 END AS action_idx, 654 | COUNT(*) as cnt 655 | FROM changes 656 | WHERE changeset_id = $1 657 | GROUP BY el_type, action 658 | LOOP 659 | result[row.el_type_idx * 3 + row.action_idx + 1] := row.cnt; 660 | END LOOP; 661 | 662 | result_bbox := (SELECT ST_Envelope(ST_Collect(ST_Collect(current_geom, new_geom))) FROM changes WHERE changeset_id = $1); 663 | 664 | UPDATE changesets cs SET entity_changes = result, bbox = result_bbox 665 | WHERE cs.id = $1; 666 | END; 667 | $_$; 668 | 669 | 670 | -- 671 | -- Name: array_accum(anyarray); Type: AGGREGATE; Schema: public; Owner: - 672 | -- 673 | 674 | CREATE AGGREGATE array_accum(anyarray) ( 675 | SFUNC = array_cat, 676 | STYPE = anyarray, 677 | INITCOND = '{}' 678 | ); 679 | 680 | 681 | SET default_tablespace = ''; 682 | 683 | SET default_with_oids = false; 684 | 685 | -- 686 | -- Name: changeset_tiles; Type: TABLE; Schema: public; Owner: -; Tablespace: 687 | -- 688 | 689 | CREATE TABLE changeset_tiles ( 690 | changeset_id bigint NOT NULL, 691 | tstamp timestamp without time zone NOT NULL, 692 | x integer NOT NULL, 693 | y integer NOT NULL, 694 | zoom integer NOT NULL, 695 | changes change[] NOT NULL 696 | ); 697 | 698 | 699 | -- 700 | -- Name: changesets; Type: TABLE; Schema: public; Owner: -; Tablespace: 701 | -- 702 | 703 | CREATE TABLE changesets ( 704 | id bigint NOT NULL, 705 | user_id bigint, 706 | user_name character varying(255), 707 | created_at timestamp without time zone NOT NULL, 708 | closed_at timestamp without time zone, 709 | open boolean NOT NULL, 710 | tags hstore NOT NULL, 711 | entity_changes integer[], 712 | num_changes integer, 713 | bbox geometry 714 | ); 715 | 716 | 717 | -- 718 | -- Name: nodes; Type: TABLE; Schema: public; Owner: -; Tablespace: 719 | -- 720 | 721 | CREATE TABLE nodes ( 722 | id bigint NOT NULL, 723 | version integer NOT NULL, 724 | rev integer NOT NULL, 725 | visible boolean NOT NULL, 726 | current boolean NOT NULL, 727 | user_id integer NOT NULL, 728 | tstamp timestamp without time zone NOT NULL, 729 | changeset_id bigint NOT NULL, 730 | tags hstore NOT NULL, 731 | geom geometry(Point,4326) 732 | ); 733 | 734 | 735 | -- 736 | -- Name: relation_members; Type: TABLE; Schema: public; Owner: -; Tablespace: 737 | -- 738 | 739 | CREATE TABLE relation_members ( 740 | relation_id bigint NOT NULL, 741 | version bigint NOT NULL, 742 | member_id bigint NOT NULL, 743 | member_type character(1) NOT NULL, 744 | member_role text NOT NULL, 745 | sequence_id integer NOT NULL 746 | ); 747 | 748 | 749 | -- 750 | -- Name: relations; Type: TABLE; Schema: public; Owner: -; Tablespace: 751 | -- 752 | 753 | CREATE TABLE relations ( 754 | id bigint NOT NULL, 755 | version integer NOT NULL, 756 | rev integer NOT NULL, 757 | visible boolean NOT NULL, 758 | current boolean NOT NULL, 759 | user_id integer NOT NULL, 760 | tstamp timestamp without time zone NOT NULL, 761 | changeset_id bigint NOT NULL, 762 | tags hstore NOT NULL 763 | ); 764 | 765 | 766 | -- 767 | -- Name: users; Type: TABLE; Schema: public; Owner: -; Tablespace: 768 | -- 769 | 770 | CREATE TABLE users ( 771 | id integer NOT NULL, 772 | name text NOT NULL 773 | ); 774 | 775 | 776 | -- 777 | -- Name: ways; Type: TABLE; Schema: public; Owner: -; Tablespace: 778 | -- 779 | 780 | CREATE TABLE ways ( 781 | id bigint NOT NULL, 782 | version integer NOT NULL, 783 | visible boolean NOT NULL, 784 | current boolean NOT NULL, 785 | user_id integer NOT NULL, 786 | tstamp timestamp without time zone NOT NULL, 787 | changeset_id bigint NOT NULL, 788 | tags hstore NOT NULL, 789 | nodes bigint[] NOT NULL 790 | ); 791 | 792 | 793 | -- 794 | -- Name: pk_changeset_tiles; Type: CONSTRAINT; Schema: public; Owner: -; Tablespace: 795 | -- 796 | 797 | ALTER TABLE ONLY changeset_tiles 798 | ADD CONSTRAINT pk_changeset_tiles PRIMARY KEY (changeset_id, x, y, zoom); 799 | 800 | 801 | -- 802 | -- Name: pk_changesets; Type: CONSTRAINT; Schema: public; Owner: -; Tablespace: 803 | -- 804 | 805 | ALTER TABLE ONLY changesets 806 | ADD CONSTRAINT pk_changesets PRIMARY KEY (id); 807 | 808 | 809 | -- 810 | -- Name: pk_nodes; Type: CONSTRAINT; Schema: public; Owner: -; Tablespace: 811 | -- 812 | 813 | ALTER TABLE ONLY nodes 814 | ADD CONSTRAINT pk_nodes PRIMARY KEY (id, version); 815 | 816 | 817 | -- 818 | -- Name: pk_relation_members; Type: CONSTRAINT; Schema: public; Owner: -; Tablespace: 819 | -- 820 | 821 | ALTER TABLE ONLY relation_members 822 | ADD CONSTRAINT pk_relation_members PRIMARY KEY (relation_id, version, sequence_id); 823 | 824 | 825 | -- 826 | -- Name: pk_relations; Type: CONSTRAINT; Schema: public; Owner: -; Tablespace: 827 | -- 828 | 829 | ALTER TABLE ONLY relations 830 | ADD CONSTRAINT pk_relations PRIMARY KEY (id, version); 831 | 832 | 833 | -- 834 | -- Name: pk_users; Type: CONSTRAINT; Schema: public; Owner: -; Tablespace: 835 | -- 836 | 837 | ALTER TABLE ONLY users 838 | ADD CONSTRAINT pk_users PRIMARY KEY (id); 839 | 840 | 841 | -- 842 | -- Name: pk_ways; Type: CONSTRAINT; Schema: public; Owner: -; Tablespace: 843 | -- 844 | 845 | ALTER TABLE ONLY ways 846 | ADD CONSTRAINT pk_ways PRIMARY KEY (id, version); 847 | 848 | 849 | -- 850 | -- PostgreSQL database dump complete 851 | -- 852 | 853 | SET search_path TO "$user",public; 854 | 855 | --------------------------------------------------------------------------------