├── LICENSE
├── .gitignore
├── README.md
└── mbtilesToPngs.py


/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2018 Stu
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
  1 | # Byte-compiled / optimized / DLL files
  2 | __pycache__/
  3 | *.py[cod]
  4 | *$py.class
  5 | 
  6 | # C extensions
  7 | *.so
  8 | 
  9 | # Distribution / packaging
 10 | .Python
 11 | build/
 12 | develop-eggs/
 13 | dist/
 14 | downloads/
 15 | eggs/
 16 | .eggs/
 17 | lib/
 18 | lib64/
 19 | parts/
 20 | sdist/
 21 | var/
 22 | wheels/
 23 | *.egg-info/
 24 | .installed.cfg
 25 | *.egg
 26 | MANIFEST
 27 | 
 28 | # PyInstaller
 29 | #  Usually these files are written by a python script from a template
 30 | #  before PyInstaller builds the exe, so as to inject date/other infos into it.
 31 | *.manifest
 32 | *.spec
 33 | 
 34 | # Installer logs
 35 | pip-log.txt
 36 | pip-delete-this-directory.txt
 37 | 
 38 | # Unit test / coverage reports
 39 | htmlcov/
 40 | .tox/
 41 | .coverage
 42 | .coverage.*
 43 | .cache
 44 | nosetests.xml
 45 | coverage.xml
 46 | *.cover
 47 | .hypothesis/
 48 | .pytest_cache/
 49 | 
 50 | # Translations
 51 | *.mo
 52 | *.pot
 53 | 
 54 | # Django stuff:
 55 | *.log
 56 | local_settings.py
 57 | db.sqlite3
 58 | 
 59 | # Flask stuff:
 60 | instance/
 61 | .webassets-cache
 62 | 
 63 | # Scrapy stuff:
 64 | .scrapy
 65 | 
 66 | # Sphinx documentation
 67 | docs/_build/
 68 | 
 69 | # PyBuilder
 70 | target/
 71 | 
 72 | # Jupyter Notebook
 73 | .ipynb_checkpoints
 74 | 
 75 | # pyenv
 76 | .python-version
 77 | 
 78 | # celery beat schedule file
 79 | celerybeat-schedule
 80 | 
 81 | # SageMath parsed files
 82 | *.sage.py
 83 | 
 84 | # Environments
 85 | .env
 86 | .venv
 87 | env/
 88 | venv/
 89 | ENV/
 90 | env.bak/
 91 | venv.bak/
 92 | 
 93 | # Spyder project settings
 94 | .spyderproject
 95 | .spyproject
 96 | 
 97 | # Rope project settings
 98 | .ropeproject
 99 | 
100 | # mkdocs documentation
101 | /site
102 | 
103 | # mypy
104 | .mypy_cache/
105 | 
106 | # mbtile files
107 | 16/
108 | *.mbtiles
109 | 
110 | # os files
111 | .DS_Store
112 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # mbtilesToPngs
 2 | 
 3 | This python program takes __rasterised__ .mbtiles files and extracts the images and places them into the correct directory structure `/${z}/${x}/${y}.png` that mimicks OpenStreetMap URI tile structure. This is to allow for offline use of OpenStreetMaps, and I will be using [Flutter_Map](https://github.com/apptreesoftware/flutter_map) along with my own images for offline maps. <br>
 4 | 
 5 | ## What is .mbtiles?
 6 | 
 7 | In short, .mbtiles is a SQLite database file, with tile data stored as binary objects [(BLOBS)](https://en.wikipedia.org/wiki/Binary_large_object) in the database. There are two main formats these BLOBS can be for .mbtiles:
 8 | 
 9 | 1. PBF for vectored tiles (think mathematical points, lines, polygons)
10 | 2. PNG for rasterised tiles (images).
11 | 
12 | This script only extracts the data from within the .mbtiles (it does not convert the data). This script only works for **Rasterised** .mbtiles that are already in .PNG format.
13 | 
14 | ## Where to get .mbtiles?
15 | 
16 | OpenStreetMap says:
17 | > Apart from very limited testing purposes, you should not use the tiles supplied by OpenStreetMap.org itself. OpenStreetMap is a volunteer-run non-profit body and cannot supply tiles for large-scale commercial use. Rather, you should use a third party provider that makes tiles from OSM data, or generate your own.
18 | 
19 | As far as I see, OpenStreetMap only offers MBTiles in [PBF format](https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/osmpbf/osmpbf.htm) which is a vectored based format. This script will not work with this format.
20 | 
21 | There are various ways to download .mbtiles in PNG format, below is a OSX method:
22 | 
23 | [A Guide to generate your own](https://tilemill-project.github.io/tilemill/docs/guides/osm-bright-mac-quickstart/)
24 | 
25 | This guide will go through the steps for compiling map data, downloading a specific area, customise the map, then export to mbtile. Warning! For Mac, when you install TileMill.app, it currently fails to start. Go into *TileMill* > *Updates Preference* > *check Install Developer Builds*.<br>
26 | 
27 | ## How to use
28 | 
29 | Make sure you have python3 install.<br>
30 | ```bash
31 | python3 mbtilesToPngs.py -i path/to/.mbtiles
32 | ```
33 | 
34 | For example:<br>
35 | > python3 sqliteReader.py -i ./OSMBright.mbtiles
36 | 
37 | ## My map says file not found! Why are the filenames incorrect?
38 | 
39 | [Read Here](http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/)<br>
40 | [And here](https://alastaira.wordpress.com/2011/07/06/converting-tms-tile-coordinates-to-googlebingosm-tile-coordinates/)<br>
41 | I've added option to convert TMS Tile Coordinates to Google/Bing/OSM Tile Coordinates. Just use `-tms` when running.
42 | 
43 | ## Links
44 | 
45 | [MBTiles](https://wiki.openstreetmap.org/wiki/MBTiles)<br>
46 | [Tiles](https://wiki.openstreetmap.org/wiki/Tiles)
47 | 
48 | ## License
49 | 
50 | This project is licensed under the [MIT License](LICENSE.md)
51 | 
52 | ## Acknowledgments
53 | 
54 | Thank you © OpenStreetMap (and) contributors, ODbL for the data! <br>
55 | Thank you AppTreeSoftware for [Flutter_Map](https://github.com/apptreesoftware/flutter_map)
56 | 


--------------------------------------------------------------------------------
/mbtilesToPngs.py:
--------------------------------------------------------------------------------
  1 | #!/usr/bin/env python3
  2 | import sqlite3
  3 | from sqlite3 import Error
  4 | import errno
  5 | import os
  6 | import sys
  7 | import getopt
  8 | import time
  9 | 
 10 | 
 11 | def create_connection(db_file):
 12 |     """ create a database connection to the SQLite database
 13 |         specified by the db_file
 14 |     :param db_file: database file
 15 |     :return: Connection object or None
 16 |     """
 17 |     try:
 18 |         conn = sqlite3.connect(db_file)
 19 |         return conn
 20 |     except Error as e:
 21 |         print(e)
 22 | 
 23 |     return None
 24 | 
 25 | 
 26 | def select_all_tiles(conn, tms, databaseName):
 27 |     cur = conn.cursor()
 28 |     cur.execute("SELECT * FROM map")
 29 | 
 30 |     rows = cur.fetchall()
 31 |     assets = {}
 32 |     print("Map count: " + str(len(rows)))
 33 |     print("TMS: " + str(tms))
 34 |     for row in rows:
 35 |         image = get_image(conn, row[3])
 36 | 
 37 |         # for printing
 38 |         dir = str(row[0]) + "/" + str(row[1])
 39 |         assets[dir] = dir
 40 | 
 41 |         # tsm check
 42 |         yValue = row[2]
 43 |         if(tms):
 44 |             ymax = 1 << row[0]
 45 |             yValue = ymax - row[2] - 1
 46 | 
 47 |         # generate png
 48 |         blob_to_file(
 49 |             databaseName,
 50 |             str(row[0]),  # {z}
 51 |             str(row[1]),  # {x}
 52 |             str(yValue),  # {y}
 53 |             image,  # data
 54 |         )
 55 |     for asset in assets:
 56 |         print ('- assets/map/' + databaseName + '/' + assets[asset] + "/")
 57 | 
 58 | 
 59 | def get_image(conn, id):
 60 |     cur = conn.cursor()
 61 |     cur.execute("SELECT * FROM images WHERE tile_id = ?", [id])
 62 | 
 63 |     row = cur.fetchone()
 64 |     return(row[0])
 65 | 
 66 | 
 67 | def mkdir_p(path):
 68 |     try:
 69 |         os.makedirs(path)
 70 |     except OSError as exc:  # Python >2.5
 71 |         if exc.errno == errno.EEXIST and os.path.isdir(path):
 72 |             pass
 73 |         else:
 74 |             raise
 75 | 
 76 | 
 77 | def blob_to_file(dir0, dir1, dir2, dir3, ablob):
 78 |     directory = os.path.join(os.path.expanduser('~'),
 79 |                              "Desktop", dir0, "map", dir1, dir2)
 80 |     if not os.path.exists(directory):
 81 |         os.makedirs(directory)
 82 |     # mkdir_p("~/Desktop/" + dir0)  # database
 83 |     # mkdir_p("~/Desktop/" + dir0 + "/map")  # map
 84 |     # mkdir_p("~/Desktop/" + dir0 + "/map/" + dir1 + "/" + dir2)  # zoom
 85 |     # mkdir_p("/" + dir0 + "/map/" + dir1 +
 86 |     #         "/" + dir2 + "/" + dir3)  # column
 87 |     filename = os.path.join(directory, (dir3 + ".png"))
 88 |     with open(filename, 'wb') as output_file:
 89 |         output_file.write(ablob)
 90 | 
 91 | 
 92 | def update_tms_row_to_zxy(conn):
 93 |     cur = conn.cursor()
 94 |     cur.execute("SELECT * FROM map")
 95 |     rows = cur.fetchall()
 96 |     for row in rows:
 97 |         ymax = 1 << row[0]
 98 |         yValue = ymax - row[2] - 1
 99 |         cur.execute("UPDATE map SET tile_row = ? WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?",
100 |                     (yValue, row[0], row[1], row[2]))
101 |         conn.commit()
102 | 
103 | 
104 | def begin_convertion(database, tms):
105 |     # create a database connection
106 |     conn = sqlite3.connect(database)
107 |     if database.__contains__("/"):
108 |         database = database.split("/")[-1]
109 |     with conn:
110 |         print("Processing mbtiles..\n***********\nIf you find your y coordinates (filename.png) are incorrect, use the -tsm option\n***********\n")
111 |         select_all_tiles(conn, tms, database.split(".")[0])
112 |         if tms:
113 |             print("Do you want to update the mbtiles database to zxy index?\n(y/n)")
114 |             strInput = input()
115 |             if "y" in strInput:
116 |                 update_tms_row_to_zxy(conn)
117 |     conn.close()
118 | 
119 | 
120 | def main(argv):
121 |     inputDir = ''
122 |     tms = False
123 |     try:
124 |         opts, args = getopt.getopt(argv, "hi:o:tms", ["ifile="])
125 |     except getopt.GetoptError:
126 |         print ('mbtilesToPngs.py -i <path_to_file> (.mbtiles only)')
127 |         sys.exit(2)
128 |     for opt, arg in opts:
129 |         if opt == '-h':
130 |             print ('mbtilesToPngs.py -i <path_to_file> (.mbtiles only)')
131 |             sys.exit()
132 |         elif opt in ("-i", "--ifile"):
133 |             inputDir = str(arg)
134 |             if not os.path.isfile(inputDir):
135 |                 print (inputDir, " file not found")
136 |                 sys.exit()
137 |         elif opt in ("-tms"):
138 |             print("-tsm checked")
139 |             tms = True
140 |     if inputDir == '':
141 |         print ('mbtilesToPngs.py -i <path_to_file> (.mbtiles only)')
142 |         sys.exit(2)
143 |     start = time.time()
144 |     begin_convertion(inputDir, tms)
145 |     end = time.time()
146 |     print("Time taken to complete: ", str(round((end - start), 2)), "s")
147 | 
148 | 
149 | if __name__ == "__main__":
150 |     main(sys.argv[1:])
151 | 


--------------------------------------------------------------------------------