├── .gitignore
├── CHANGELOG.md
├── LICENSE.txt
├── README.md
├── lgpl-v3.tmpl
├── prune_gpx.py
├── roadmaptools
├── __init__.py
├── adjectancy.py
├── calculate_curvature.py
├── clean_geojson.py
├── common.py
├── compute_edge_parameters.py
├── config
│ ├── __init__.py
│ ├── cities_envelopes_item.py
│ └── roadmaptools_config.py
├── const.py
├── download_map.py
├── estimate_speed_from_osm.py
├── export_nodes_and_id_maker.py
├── filter.py
├── generate_config.py
├── geojson_linestrings.py
├── geojson_shp.py
├── geometry.py
├── gmaps.py
├── gpx.py
├── gpx_ load_test.py
├── gpx_shp.py
├── graph.py
├── init.py
├── inout.py
├── osmfilter.py
├── plotting.py
├── prepare_geojson_to_agentpolisdemo.py
├── printer.py
├── resources
│ ├── __init__.py
│ └── config.cfg
├── road_graph_rtree.py
├── road_structures.py
├── sanitize.py
├── shp.py
├── simplify_graph.py
├── test
│ ├── __init__.py
│ ├── angle_test.py
│ ├── big_map_test.py
│ ├── cut_trace.py
│ ├── distance_test.py
│ ├── int.py
│ ├── networkx_pickle_test.py
│ ├── shapely_project_test.py
│ ├── shp_test.py
│ └── test_tmp.py
└── utm.py
├── setup.cfg
└── setup.py
/.gitignore:
--------------------------------------------------------------------------------
1 | #IDE
2 | .idea
3 | #PyPI
4 | .eggs/
5 | dist/
6 | roadmaptools.egg-info/
7 | #compiled files
8 | *.pyc
9 | .pytest_cache/
10 |
--------------------------------------------------------------------------------
/CHANGELOG.md:
--------------------------------------------------------------------------------
1 |
20 | # 5.0.0
21 | ## Changed
22 | - clean.geojson.py updated so that osmids are kept through the process
23 | - max speed unit is now saved in separate edge property, instead in the max speed string
24 |
25 | ## Addded
26 | - map download from overpass by area name
27 |
28 | ## Removed
29 | - the `osmtogeojson` script together with the broken `osmread` dependency
30 |
31 |
32 | # 4.1.0
33 |
34 | ## Added
35 | - delimiter option added to `inout.save_csv` method
36 |
37 | ## Fixed
38 | - country specific speed codes extended by US codes
39 | - string detection bug fixed in get_posted_speed method
40 | - config bug with negative coordinates treated as float fixed in download_map.py
41 |
42 |
43 | # 4.0.0
44 | ## Added
45 | - GeoJSON node iterator added to plotting
46 |
47 | ## Changed
48 | - inout.load_json now accepts encoding parameter
49 | - GeoJSON iterator renamed to GeoJSON edge iterator
50 |
51 | ## Fixed
52 | - GeoJSON edge iterator in plotting now handles feature collections that contains other feature types than LineStirng
53 |
54 |
55 | # 3.0.0
56 | ## Added
57 | - new filter module with a generic method for edge filtering
58 | - new method export_nodes_for_matplotlib in plotting
59 |
60 | ## Changed
61 | - method export_for_matplotlib in plotting renamed to export_edges_for_matplotlib
62 |
63 | # 2.0.1
64 | ## Added
65 | - Changelog
66 |
67 | ## Fixed
68 | - utm package added to setup requirements
69 | - readme updated
70 |
--------------------------------------------------------------------------------
/LICENSE.txt:
--------------------------------------------------------------------------------
1 | The MIT License (MIT)
2 |
3 | Copyright (c) 2017 Martin Korytak
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in
13 | all copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21 | THE SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
20 | # roadmap-processing
21 |
22 | Python module for working with road network graphs, mostly from osm. Usefull tools for working with `geojson` and `gpx`
23 | formats.
24 |
25 | ## Prerequisites
26 |
27 | We will work with Python [pip](https://pypi.python.org/pypi/pip). You should have all these installed before beginning
28 | the installation.
29 |
30 |
31 | ## Installing
32 |
33 | ```
34 | pip install roadmaptools
35 | ```
36 |
37 | ## Examples of usage
38 | Download map from osm:
39 |
40 | ```Python
41 | roadmaptools.download_map.download_cities(
42 | [(49.94, 14.22, 50.17, 14.71)], './raw_map.geojson')
43 | ```
44 |
45 |
46 |
47 | ## Versioning
48 |
49 | We use [GitHub](https://github.com) for versioning. For the versions available, see the
50 | [tags on this repository](https://github.com/aicenter/roadmap-processing/tags).
51 |
52 | ## Authors
53 |
54 | * **David Fiedler** - *Maintainer*
55 | * **Martin Korytak** - *Initial work*
56 |
57 | See also the list of [contributors](https://github.com/aicenter/roadmap-processing/graphs/contributors) who participated in this project.
58 |
59 | ## License
60 |
61 | This project is licensed under the MIT License - see the [LICENSE.txt](LICENSE.txt) file for details
62 |
63 |
--------------------------------------------------------------------------------
/lgpl-v3.tmpl:
--------------------------------------------------------------------------------
1 | Copyright (c) ${years} ${owner}.
2 |
3 | This file is part of ${projectname}
4 | (see ${projecturl}).
5 |
6 | This program is free software: you can redistribute it and/or modify
7 | it under the terms of the GNU Lesser General Public License as published by
8 | the Free Software Foundation, either version 3 of the License, or
9 | (at your option) any later version.
10 |
11 | This program is distributed in the hope that it will be useful,
12 | but WITHOUT ANY WARRANTY; without even the implied warranty of
13 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14 | GNU Lesser General Public License for more details.
15 |
16 | You should have received a copy of the GNU Lesser General Public License
17 | along with this program. If not, see .
18 |
--------------------------------------------------------------------------------
/prune_gpx.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from roadmaptools.inout import load_gpx, save_gpx
21 | from tqdm import tqdm
22 | import os
23 |
24 |
25 | # data = load_gpx('/home/martin/Stažené/traces-raw-old.gpx')
26 | # save_gpx(data,'neexistujici slozka')
27 |
28 | def file_len(fname):
29 | letters = 0
30 | with open(fname, 'r') as f:
31 | for line in f.readlines():
32 | letters += len(line)
33 | return letters
34 |
35 |
36 | def cut_gpx(in_file, out_file):
37 | number_of_chars = file_len(in_file)
38 | print(number_of_chars)
39 | number_of_chars /= 2
40 | number_of_chars = int(number_of_chars)
41 | # number_of_lines = 300000000
42 | print(number_of_chars)
43 | result = ''
44 | with open(in_file, 'r') as fh:
45 | for line in fh.readlines():
46 | if len(result) < number_of_chars:
47 | result += line
48 | elif not line.startswith(' \n\n'
55 |
56 | with open(out_file, 'w') as ofh:
57 | ofh.write(result)
58 |
59 |
60 | # data = load_gpx('/home/martin/Stažené/traces-raw.gpx')
61 |
62 |
63 |
64 | # cut_gpx('/home/martin/Stažené/traces-raw.gpx','/home/martin/Stažené/traces-raw-mensi.gpx')
65 |
66 | # def _read_all_csv_from_dir(directory: str):
67 | # traces_all = []
68 | #
69 | # for filename in tqdm(os.listdir(directory), desc="Loading and parsing traces"):
70 | # if os.path.splitext(filename)[-1] == '.csv':
71 | # abs_filename = os.path.join(directory, filename)
72 | # traces_all.append(_load_traces_from_csv(abs_filename))
73 | # return traces_all
74 |
75 | f = lambda x: True if x != "STREETPICKUP" else False
76 |
77 | import pandas as pd
78 | import numpy as np
79 | import roadmaptools
80 |
81 | ls = []
82 | len_ids = set()
83 | pd.set_option('display.max_columns', None)
84 | column_names = ['id_record', 'id_car', 'status', 'lat', 'lon', 'time']
85 | use_columns = ['id_car', 'status', 'lat', 'lon', 'time']
86 | # data_types = [str,int,str,float,float,str]
87 | arr = np.empty(shape=(len(os.listdir('/home/martin/MOBILITY/data/traces')),),dtype=object)
88 | for idx, filename in tqdm(enumerate(os.listdir('/home/martin/MOBILITY/data/traces'))):
89 | abs_filename = os.path.join('/home/martin/MOBILITY/data/traces', filename)
90 | filename_parts = abs_filename.split(sep='.')
91 | file_extension = filename_parts[-2]
92 | df = pd.read_csv(abs_filename, header=None, names=column_names, usecols=['id_car', 'status', 'lat', 'lon', 'time'], converters={'status': f})
93 | df['id_car'] = pd.to_numeric(file_extension + df['id_car'].astype(str))
94 | # print(df.head())
95 | # print(df.dtypes)
96 | # q = sort_one_by_one(df,'id_car','time')
97 | # print(q.head())
98 | filtered = df[df.loc[:, 'status']].loc[:, ['id_car', 'lat', 'lon', 'time']]
99 | filtered.sort_values(by='time', ascending=True, inplace=True)
100 | filtered.sort_values(by='id_car', kind='mergesort', ascending=True, inplace=True)
101 | # print(filtered)
102 | # ls.append(filtered)
103 | arr[idx] = filtered
104 |
105 | df = pd.concat(arr, ignore_index=True)
106 | # # exit(0)
107 | # print(df)
108 | # df.sort_values(by=['id_car','time'],ascending=True,inplace=True)
109 | # print(df)
110 | # for idx, row in df.iterrows():
111 | # print(idx)
112 |
113 | # iter_csv = pd.read_csv(abs_filename,iterator=True,chunksize=1000)
114 | # df = pd.concat([chunk[chunk['field'] > constant] for chunk in iter_csv])
115 | # for row in df:
116 | # len_id = len(row[1])
117 | # if len_id not in len_ids:
118 | # len_ids.add(len_id)
119 | #
120 | # print(len_ids)
121 |
122 | # from numpy import genfromtxt
123 | # from numpy.core.defchararray import add
124 | #
125 | # # my_data = genfromtxt('my_file.csv', delimiter=',')
126 | # arr = np.empty(shape=(len(os.listdir('/home/martin/MOBILITY/data/traces')),), dtype=object)
127 | # for idx, filename in tqdm(enumerate(os.listdir('/home/martin/MOBILITY/data/traces')[:5])):
128 | # abs_filename = os.path.join('/home/martin/MOBILITY/data/traces', filename)
129 | # filename_parts = abs_filename.split(sep='.')
130 | # file_extension = filename_parts[-2]
131 | # arr[idx] = genfromtxt(abs_filename, delimiter=',', usecols=(1, 2, 3, 4, 5), encoding=None,dtype=None)
132 | # # arr[idx][0] = add(file_extension, arr[idx][0])
133 | # arr[idx][0].astype(int)
134 | # print(arr[idx].dtype)
135 | # print(arr)
136 | # arr = np.empty(hash(5,))
137 | # l = []
138 | # for idx, filename in tqdm(enumerate(os.listdir('/home/martin/MOBILITY/data/traces'))):
139 | # abs_filename = os.path.join('/home/martin/MOBILITY/data/traces', filename)
140 | # filename_parts = abs_filename.split(sep='.')
141 | # file_extension = filename_parts[-2]
142 | # iterator = roadmaptools.inout.load_csv(abs_filename)
143 | #
144 | # ls = []
145 | # for row in iterator:
146 | # if row[2] == "STREETPICKUP":
147 | # continue
148 | #
149 | # ls.append(np.array([int(row[1]),float(row[3]),float(4),row[5]]))
150 | # l.append(ls)
151 |
--------------------------------------------------------------------------------
/roadmaptools/__init__.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
--------------------------------------------------------------------------------
/roadmaptools/adjectancy.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import numpy as np
21 | import roadmaptools.inout
22 |
23 | from typing import List, Callable
24 | from tqdm import tqdm
25 | from geojson import FeatureCollection
26 |
27 |
28 | def create_adj_matrix(nodes_filepath: str,
29 | edges_filepath: str,
30 | out_filepath: str,
31 | cost_function: Callable[[dict], int]):
32 | nodes = roadmaptools.inout.load_geojson(nodes_filepath)
33 | edges = roadmaptools.inout.load_geojson(edges_filepath)
34 | dm = get_adj_matrix(nodes, edges, cost_function)
35 | roadmaptools.inout.save_csv(dm, out_filepath)
36 |
37 |
38 | def get_adj_matrix(nodes: FeatureCollection, edges: FeatureCollection,
39 | cost_function: Callable[[dict], int]) -> np.ndarray:
40 | nodes = nodes['features']
41 | size = len(nodes)
42 | adj = np.full((size, size), np.nan)
43 | node_dict = {node['properties']['node_id']: node for node in nodes}
44 | for edge in tqdm(edges['features'], desc='filling the adjectancy matrix'):
45 | from_node = node_dict[edge['properties']['from_id']]
46 | to_node = node_dict[edge['properties']['to_id']]
47 | # cost = edge['properties']['length']
48 | cost = cost_function(edge)
49 | adj[from_node['properties']['index'], to_node['properties']['index']] = cost
50 |
51 | return adj
52 |
53 |
--------------------------------------------------------------------------------
/roadmaptools/calculate_curvature.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import math
21 | import geojson
22 | import codecs
23 | import sys
24 | import argparse
25 | import time
26 |
27 | from roadmaptools.printer import print_info
28 | from roadmaptools.init import config
29 |
30 |
31 | def compute_edge_curvatures(input_filename: str, output_filename: str):
32 |
33 | print_info('Computing average edge curvatures.')
34 | start_time = time.time()
35 |
36 | input_stream = open(input_filename, encoding='utf8')
37 | output_stream = open(output_filename, 'w')
38 |
39 | print_info("Loading geojson from: {}".format(input_filename))
40 | geojson_file = load_geojson(input_stream)
41 |
42 | print_info("Calculating curvature")
43 | geojson_out = get_geojson_with_curvature(geojson_file)
44 |
45 | print_info("Saving geojson to: {}".format(output_filename))
46 | save_geojson(geojson_out, output_stream)
47 | input_stream.close()
48 | output_stream.close()
49 |
50 | print_info('Curvature computation process finished. (%.2f secs)' % (time.time() - start_time))
51 |
52 |
53 | def calculate_curvature(input_stream, output_stream):
54 | json_dict = load_geojson(input_stream)
55 | analyse_roads(json_dict)
56 | save_geojson(json_dict, output_stream)
57 |
58 |
59 | def get_geojson_with_curvature(json_dict):
60 | analyse_roads(json_dict)
61 | return json_dict
62 |
63 |
64 | def get_node(node): # latlon
65 | return (node[1], node[0])
66 |
67 |
68 | def get_distance_between_coords(point1, point2):
69 | """
70 | Computes the distance between to GPS coordinates.
71 | :param point1:
72 | :param point2:
73 | :return: distance between two GPS coordinates in meters
74 | """
75 | R = 6371000
76 | lat1 = math.radians(point1[0])
77 | lat2 = math.radians(point2[0])
78 | lat = math.radians(point2[0] - point1[0])
79 | lon = math.radians(point2[1] - point1[1])
80 |
81 | a = math.sin(lat / 2) * math.sin(lat / 2) + math.cos(lat1) * math.cos(lat2) * math.sin(lon / 2) * math.sin(lon / 2)
82 | c = 2 * math.atan2(math.sqrt(a), math.sqrt(1 - a))
83 | distance = R * c
84 | return distance
85 |
86 |
87 | # C
88 | # /\
89 | # c / \ a
90 | # / \
91 | # A ----- B
92 | # b
93 | def calculate_angle_in_degree(a, b, c):
94 | if a + c > b: # check if it is a triangle
95 | angle = math.acos((a ** 2 + c ** 2 - b ** 2) / (2 * a * c)) # in radians
96 | else:
97 | angle = 0
98 | result = abs(180 - math.degrees(angle))
99 | return result
100 |
101 |
102 | def get_length(coords):
103 | length = 0
104 | for i in range(0, len(coords) - 1):
105 | point1 = get_node(coords[i])
106 | point2 = get_node(coords[i + 1])
107 | length += get_distance_between_coords(point1, point2)
108 | return length
109 |
110 |
111 | def get_curvature(coords):
112 | if len(coords) < 3:
113 | return [0, 0] # no curvature on edge
114 | else:
115 | total_curvature = 0
116 | max_curvature = -1
117 | length_of_edge = 0
118 | for i in range(0, len(coords) - 2):
119 | point_a = get_node(coords[i])
120 | point_b = get_node(coords[i + 1])
121 | point_c = get_node(coords[i + 2])
122 |
123 | length_c = get_distance_between_coords(point_a, point_b)
124 | length_a = get_distance_between_coords(point_b, point_c)
125 | length_b = get_distance_between_coords(point_c, point_a)
126 |
127 | # k = 0.5 * (length_b+length_a+length_c)
128 | # area = math.sqrt(k*(k-length_a)*(k-length_b)*(k-length_c))
129 | # radius = (length_b*length_a*length_c)/(4*area)
130 | angle = calculate_angle_in_degree(length_a, length_b, length_c)
131 | distance = length_c + length_a
132 | curvature = angle / distance
133 | if curvature > max_curvature:
134 | max_curvature = curvature
135 | total_curvature += angle
136 | length_of_edge += distance
137 |
138 | return [total_curvature / length_of_edge, max_curvature]
139 |
140 |
141 | def load_geojson(in_stream):
142 | json_dict = geojson.load(in_stream)
143 | return json_dict
144 |
145 |
146 | def analyse_roads(json_dict):
147 | for item in json_dict['features']:
148 | cur = get_curvature(item['geometry']['coordinates'])
149 | item['properties']['curvature'] = cur[0]
150 | item['properties']['max_curvature'] = cur[1]
151 |
152 |
153 | def save_geojson(json_dict, out_stream):
154 | geojson.dump(json_dict, out_stream)
155 |
156 |
157 | def get_args():
158 | parser = argparse.ArgumentParser()
159 | parser.add_argument('-i', dest="input", type=str, action='store', help='input file')
160 | parser.add_argument('-o', dest="output", type=str, action='store', help='output file')
161 | return parser.parse_args()
162 |
163 |
164 | # EXAMPLE OF USAGE
165 | if __name__ == '__main__':
166 | args = get_args()
167 | input_stream = sys.stdin
168 | output_stream = sys.stdout
169 |
170 | if args.input is not None:
171 | input_stream = codecs.open(args.input, encoding='utf8')
172 | if args.output is not None:
173 | output_stream = codecs.open(args.output, 'w')
174 |
175 | calculate_curvature(input_stream, output_stream)
176 | input_stream.close()
177 | output_stream.close()
178 |
--------------------------------------------------------------------------------
/roadmaptools/clean_geojson.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import geojson
21 | import codecs
22 | import copy
23 | import argparse
24 | import sys
25 | import time
26 | import pandas as pd
27 | import roadmaptools.inout
28 | import roadmaptools.road_structures
29 |
30 | from typing import Dict, Set, List, Tuple
31 | from tqdm import tqdm, trange
32 | from geojson import FeatureCollection, Feature
33 | from pandas import DataFrame
34 | from roadmaptools.printer import print_info
35 | from roadmaptools.init import config
36 |
37 | # properties that will not be deleted
38 | SET_OF_USEFUL_PROPERTIES = {'highway', 'id', 'lanes', 'maxspeed', 'oneway', 'bridge', 'width', 'tunnel',
39 | 'traffic_calming', 'lanes:forward', 'lanes:backward', 'junction'}
40 |
41 | # for correct type conversion
42 | dict_of_useful_properties = {'highway': str, 'id': int, 'lanes': int, 'maxspeed': int, 'oneway': str, 'bridge': str,
43 | 'width': float, 'tunnel': str, 'traffic_calming': str, 'lanes:forward': int,
44 | 'lanes:backward': int, 'junction': str}
45 |
46 | nonempty_columns = set()
47 |
48 |
49 | def _is_int(string: str) -> bool:
50 | try:
51 | int(string)
52 | return True
53 | except ValueError:
54 | return False
55 |
56 |
57 | def clean_geojson_files(input_file_path: str = config.geojson_file,
58 | output_file_path: str = config.cleaned_geojson_file,
59 | keep_attributes: Set[str] = SET_OF_USEFUL_PROPERTIES,
60 | remove_attributes: Set[str] = None):
61 | print_info('Cleaning geoJSON - input file: {}, cleaned file: {}'.format(input_file_path, output_file_path))
62 |
63 | start_time = time.time()
64 | feature_collection = roadmaptools.inout.load_geojson(input_file_path)
65 | prune_geojson_file(feature_collection, keep_attributes, remove_attributes)
66 |
67 | print_info('Cleaning complete. (%.2f secs)' % (time.time() - start_time))
68 |
69 | roadmaptools.inout.save_geojson(feature_collection, output_file_path)
70 |
71 |
72 | def clean_geojson(input_stream, output_stream):
73 | json_dict = _load_geojson(input_stream)
74 | json_deleted = get_geojson_with_deleted_features(json_dict)
75 | # save_geojson(output_stream, json_deleted)
76 | prune_geojson_file(json_dict)
77 | save_geojson(json_dict, output_stream)
78 |
79 |
80 | # def get_cleaned_geojson(json_dict):
81 | # print_info("Cleaning geojson")
82 | # prune_geojson_file(json_dict)
83 | # print_info("Removing empty features")
84 | # json_dict['features'] = [i for i in json_dict["features"] if i] # remove empty dicts
85 | # return json_dict
86 |
87 |
88 | def remove_properties(item, keep_attributes: Set[str], remove_attributes: Set[str]) -> Feature:
89 | if remove_attributes:
90 | item['properties'] = {k: v for k, v in item['properties'].items() if k not in remove_attributes}
91 | else:
92 | item['properties'] = {k: v for k, v in item['properties'].items() if k in keep_attributes}
93 | return item
94 |
95 |
96 | def _load_geojson(in_stream):
97 | json_dict = geojson.load(in_stream)
98 | return json_dict
99 |
100 |
101 | def get_geojson_with_deleted_features(json_dict):
102 | json_deleted = dict()
103 | json_deleted['type'] = json_dict['type']
104 | json_deleted['features'] = list()
105 |
106 | for item in json_dict['features']:
107 | if item['geometry']['type'] != 'LineString':
108 | json_deleted['features'].append(item)
109 |
110 | # with codecs.open("data/deleted_items.geojson", 'w') as output:
111 | # geojson.dump(json_deleted, output)
112 | # output.close()
113 | return json_deleted
114 |
115 |
116 | def create_desimplified_edge(coord_u: List[float],
117 | coord_v: List[float],
118 | item: Feature,
119 | is_forward: bool,
120 | from_id: int,
121 | to_id: int):
122 | item['properties']['id'] = str(roadmaptools.road_structures.get_edge_id_from_coords(coord_u, coord_v))
123 | del item['geometry']['coordinates']
124 | del item['properties']['nodes']
125 | item['geometry']['coordinates'] = [coord_u, coord_v]
126 |
127 | # setting from/to nodes
128 | item['properties']['from_osm_id'] = from_id
129 | item['properties']['to_osm_id'] = to_id
130 |
131 | # lane config for two way roads
132 | if 'oneway' not in item['properties'] or item['properties']['oneway'] != 'yes':
133 | if 'lanes:forward' in item['properties'] and is_forward:
134 | item['properties']['lanes'] = int(item['properties']['lanes:forward'])
135 | elif 'lanes:backward' in item['properties'] and not is_forward:
136 | item['properties']['lanes'] = int(item['properties']['lanes:backward'])
137 | # elif is_forward and 'lanes' in item['properties']:
138 | # item['properties']['lanes'] = int(item['properties']['lanes']) - 1
139 | # elif not is_forward and 'lanes' in item['properties']:
140 | # item['properties']['lanes'] = 1
141 | elif 'lanes' in item['properties'] and _is_int(item['properties']['lanes']) and int(
142 | item['properties']['lanes']) >= 2:
143 | item['properties']['lanes'] = int(item['properties']['lanes']) / 2
144 | else:
145 | item['properties']['lanes'] = 1
146 | # lane config for one way roads
147 | else:
148 | if 'lanes' not in item['properties'] or int(item['properties']['lanes']) < 1:
149 | item['properties']['lanes'] = 1
150 | else:
151 | item['properties']['lanes'] = int(item['properties']['lanes'])
152 |
153 | # item['properties']['oneway'] = 'yes'
154 |
155 | return item
156 |
157 |
158 | def check_types(item: Feature):
159 | for prop in dict_of_useful_properties:
160 | if prop in item['properties'] and not isinstance(item['properties'][prop], dict_of_useful_properties[prop]):
161 | if dict_of_useful_properties[prop] == int:
162 | try:
163 | if " mph" in item['properties'][prop]:
164 | temp = item['properties'][prop].split()
165 | item['properties'][prop] = float(temp[0]) * 1.609344
166 | elif " knots" in item['properties'][prop]:
167 | temp = item['properties'][prop].split()
168 | item['properties'][prop] = float(temp[0]) * 1.85200
169 | else:
170 | int(item['properties'][prop])
171 | except:
172 | del item['properties'][prop]
173 | elif dict_of_useful_properties[prop] == str:
174 | try:
175 | str(item['properties'][prop])
176 | except:
177 | del item['properties'][prop]
178 | elif dict_of_useful_properties[prop] == int:
179 | try:
180 | int(item['properties'][prop])
181 | except:
182 | del item['properties'][prop]
183 | elif dict_of_useful_properties[prop] == float:
184 | try:
185 | if " m" in item['properties'][prop]:
186 | temp = item['properties'][prop].split()
187 | item['properties'][prop] = float(temp[0])
188 | elif " km" in item['properties'][prop]:
189 | temp = item['properties'][prop].split()
190 | item['properties'][prop] = float(temp[0]) * 1000
191 | elif " mi" in item['properties'][prop]:
192 | temp = item['properties'][prop].split()
193 | item['properties'][prop] = float(temp[0]) * 1609.344
194 | else:
195 | float(item['properties'][prop])
196 | except:
197 | del item['properties'][prop]
198 |
199 |
200 | def prune_geojson_file(json_dict: FeatureCollection, keep_attributes: Set[str], remove_attributes: Set[str],
201 | desimplify=True):
202 | """
203 | Transforms the geojson file into the version to be used with roadmaptools.
204 | Output file contains only edges. Each edge receives an id, starting from 0 to edge count.
205 | Feature collection is changed inplace.
206 |
207 | If desimplify=True, than the edges are split on coordinates. Note that this is usually needed even when we
208 | want the graph to be simplified, because in raw data from openstreetmap, the roads are not split on crossroads.
209 |
210 | :param json_dict: Input data
211 | :param keep_attributes: edge attributes to keep
212 | :param remove_attributes: edge attributes to remove
213 | :param desimplify: True, means than the edges are split on coordinates.
214 | :return: None, the feature collection is changed in place
215 | """
216 |
217 | edges = [item for item in json_dict['features'] if item['geometry']['type'] == 'LineString']
218 |
219 | json_dict['features'] = []
220 |
221 | # we need to keep the list of nodes
222 | keep_attributes.add("nodes")
223 |
224 | for item in tqdm(edges, desc="Pruning geojson"):
225 | remove_properties(item, keep_attributes, remove_attributes)
226 | # check_types(item)
227 | if desimplify:
228 | for j in range(0, len(item['geometry']['coordinates']) - 1):
229 | new_edge = copy.deepcopy(item)
230 | u = item['geometry']['coordinates'][j]
231 | v = item['geometry']['coordinates'][j + 1]
232 | from_id = item['properties']['nodes'][j]
233 | to_id = item['properties']['nodes'][j + 1]
234 | new_item = create_desimplified_edge(u, v, new_edge, True, from_id, to_id)
235 | json_dict['features'].append(new_item)
236 |
237 | # create the opposite direction edge if it is not oneway
238 | if 'oneway' not in item['properties'] or item['properties']['oneway'] != 'yes':
239 | new_edge = copy.deepcopy(item)
240 | new_item = create_desimplified_edge(v, u, new_edge, False, to_id, from_id)
241 | json_dict['features'].append(new_item)
242 | else:
243 | json_dict["features"].append(item)
244 |
245 |
246 | def save_geojson(json_dict, out_stream):
247 | json_dict['features'] = [i for i in json_dict["features"] if i] # remove empty dicts
248 | geojson.dump(json_dict, out_stream)
249 |
250 |
251 | def get_args():
252 | parser = argparse.ArgumentParser()
253 | parser.add_argument('-i', dest="input", type=str, action='store', help='input file')
254 | parser.add_argument('-o', dest="output", type=str, action='store', help='output file')
255 | return parser.parse_args()
256 |
257 |
258 | def get_non_empty_columns(edges: List[Feature], empty_ratio: float = 0) -> Set[str]:
259 | # dataframe init
260 | rows = []
261 | for item in tqdm(edges, desc="Counting non-empty properties"):
262 | row = item['properties']
263 | rows.append(row)
264 | table = DataFrame.from_records(rows)
265 |
266 | nonempty_columns = set()
267 | columns = set()
268 |
269 | for column in table.columns:
270 | non_empty_count = count_nonempty_in_column(table, column)
271 | ratio = non_empty_count / len(table)
272 | if ratio > empty_ratio:
273 | nonempty_columns.add(column)
274 | columns.add(column)
275 |
276 | print("removed {} empty columns".format(len(columns) - len(nonempty_columns)))
277 |
278 | return nonempty_columns
279 |
280 |
281 | def count_nonempty_in_column(dataframe: DataFrame, column: str) -> int:
282 | return (dataframe[column].values != '').sum()
283 |
284 |
285 | # EXAMPLE OF USAGE
286 | if __name__ == '__main__':
287 | args = get_args()
288 | input_stream = sys.stdin
289 | output_stream = sys.stdout
290 |
291 | if args.input is not None:
292 | input_stream = codecs.open(args.input, encoding='utf8')
293 | if args.output is not None:
294 | output_stream = codecs.open(args.output, 'w')
295 |
296 | clean_geojson(input_stream, output_stream)
297 | input_stream.close()
298 | output_stream.close()
299 |
--------------------------------------------------------------------------------
/roadmaptools/common.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import os
21 |
22 | from typing import Callable
23 | from tqdm import tqdm
24 |
25 |
26 | def process_dir(dir_path: str, function: Callable[[str], None]):
27 | walk = list(os.walk(dir_path))[0]
28 | path = walk[0]
29 | files = walk[2]
30 |
31 | # # remove the link to parent dir
32 | # files = files[1:]
33 | for filename in tqdm(files, desc="Processing directory"):
34 | function(path + filename)
--------------------------------------------------------------------------------
/roadmaptools/compute_edge_parameters.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import roadmaptools.inout
21 | import geojson.feature
22 | import networkx as nx
23 | import roadmaptools.utm
24 | import roadmaptools.geometry
25 | import roadmaptools.estimate_speed_from_osm
26 |
27 | from typing import List, Dict
28 | from roadmaptools.init import config
29 |
30 |
31 | _computations = []
32 |
33 |
34 | def compute_edge_parameters(input_filename: str, output_filename: str):
35 | geojson_content = roadmaptools.inout.load_geojson(input_filename)
36 |
37 | # projection determination
38 | for item in geojson_content['features']:
39 | if item["geometry"]["type"] == "LineString":
40 | first_coord = item['geometry']['coordinates'][0]
41 | break
42 |
43 | projection = roadmaptools.utm.TransposedUTM.from_gps(first_coord[1], first_coord[0])
44 |
45 | for item in geojson_content['features']:
46 | # transformed coordianates
47 | coords = item['geometry']['coordinates']
48 | projected_coords = []
49 | for coord in coords:
50 | projected_coords.append(roadmaptools.utm.wgs84_to_utm_1E2(coord[1], coord[0]))
51 | item['properties']['utm_coords'] = projected_coords
52 |
53 | # edge length
54 | item['properties']["length"] = roadmaptools.geometry.get_length_from_coords(projected_coords)
55 |
56 | # max speed
57 | speed, unit = roadmaptools.estimate_speed_from_osm.get_posted_speed(item)
58 | item['properties']['maxspeed'] = speed
59 | item['properties']['speed_unit'] = unit
60 |
61 |
62 | # graph = roadmaptools.inout.load_graph(geojson_content)
63 |
64 | # edge_map = _create_edge_map(graph)
65 |
66 | # graph_multi_test(graph)
67 |
68 | # _computations.append(compute_centrality)
69 |
70 | # for computation in _computations:
71 | # computation(graph, geojson_content, edge_map)
72 |
73 | roadmaptools.inout.save_geojson(geojson_content, output_filename)
74 |
75 |
76 | def compute_centrality(graph: nx.DiGraph, data: geojson.feature.FeatureCollection, edge_map: Dict):
77 | for item in data['features']:
78 | edge = edge_map[item['properties']['id']]
79 | from_degree = graph.degree(edge[0])
80 | to_degree = graph.degree(edge[1])
81 | item['properties']["from_degree"] = from_degree
82 | item['properties']["to_degree"] = to_degree
83 |
84 |
85 | def _create_edge_map(graph: nx.DiGraph) -> Dict:
86 | edge_map = {}
87 | for edge in graph.edges():
88 | # edge_map[graph[edge[0]][edge[1]][0]["id"]] = edge
89 | edge_map[graph[edge[0]][edge[1]]["id"]] = edge
90 | return edge_map
91 |
92 |
93 | def graph_multi_test(graph: nx.DiGraph):
94 | for edge in graph.edges():
95 | if len(graph[edge[0]][edge[1]]) > 1:
96 | a=1
97 |
--------------------------------------------------------------------------------
/roadmaptools/config/__init__.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
--------------------------------------------------------------------------------
/roadmaptools/config/cities_envelopes_item.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
21 | class CitiesEnvelopesItem:
22 | def __init__(self, properties: dict=None):
23 | self.south = properties.get("south")
24 | self.east = properties.get("east")
25 | self.north = properties.get("north")
26 | self.west = properties.get("west")
27 |
28 |
29 | pass
30 |
31 |
--------------------------------------------------------------------------------
/roadmaptools/config/roadmaptools_config.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
21 | import fconfig.configuration
22 |
23 | from fconfig.config import Config
24 |
25 | class RoadmaptoolsConfig(Config):
26 | def __init__(self, properties: dict=None):
27 | self.map_dir = properties.get("map_dir")
28 | self.osm_source_url = properties.get("osm_source_url")
29 | self.osm_map_filename = properties.get("osm_map_filename")
30 | self.filtered_osm_filename = properties.get("filtered_osm_filename")
31 | self.geojson_file = properties.get("geojson_file")
32 | self.cleaned_geojson_file = properties.get("cleaned_geojson_file")
33 | self.sanitized_geojson_file = properties.get("sanitized_geojson_file")
34 | self.simplified_file = properties.get("simplified_file")
35 | self.simplified_file_with_speed = properties.get("simplified_file_with_speed")
36 | self.simplified_file_with_speed_and_curvature = properties.get("simplified_file_with_speed_and_curvature")
37 | self.ap_nodes_file = properties.get("ap_nodes_file")
38 | self.ap_edges_file = properties.get("ap_edges_file")
39 | self.utm_center_lon = properties.get("utm_center_lon")
40 | self.utm_center_lat = properties.get("utm_center_lat")
41 | self.shift_utm_coordinate_origin_to_utm_center = properties.get("shift_utm_coordinate_origin_to_utm_center")
42 | self.shapely_error_tolerance = properties.get("shapely_error_tolerance")
43 | self.osm_filter_params = properties.get("osm_filter_params")
44 |
45 |
46 | self.cities_envelopes = properties.get("cities_envelopes")
47 | pass
48 |
49 | config: RoadmaptoolsConfig = fconfig.configuration.load((RoadmaptoolsConfig, None))
50 |
51 |
52 |
--------------------------------------------------------------------------------
/roadmaptools/const.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import sys
21 |
22 | MIN_INTEGER = -sys.maxsize - 1
--------------------------------------------------------------------------------
/roadmaptools/download_map.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from roadmaptools.init import config
21 |
22 | import roadmaptools.inout
23 | import overpass
24 | import osm2geojson
25 |
26 | from typing import Tuple, List
27 | from roadmaptools.printer import print_info
28 |
29 |
30 | WAY_FILTER = """
31 | [highway~"(motorway|motorway_link|trunk|trunk_link|primary|primary_link|secondary|secondary_link|tertiary|tertiary_link|unclassified|unclassified_link|residential|residential_link|living_street)"]
32 | [access!="no"]
33 | """
34 |
35 |
36 | def _convert_json_to_geojson(json):
37 |
38 | # way to node list dict
39 | node_dict = {}
40 | for el in json['elements']:
41 | if el['type'] == 'way':
42 | node_dict[el['id']] = el['nodes']
43 |
44 | geojson = osm2geojson.json2geojson(json)
45 |
46 | # converting geojson to desired format
47 | for feature in geojson['features']:
48 | if feature['geometry']['type'] == 'LineString':
49 | feature['id'] = feature['properties']['id']
50 | del feature['properties']['id']
51 | for key, val in feature['properties']['tags'].items():
52 | feature['properties'][key] = val
53 | del feature['properties']['tags']
54 | feature['properties']['nodes'] = node_dict[feature['id']]
55 |
56 | return geojson
57 |
58 |
59 | def call_overpass(query: str, filepath: str):
60 | print_info("Downloading map from Overpass API")
61 | api = overpass.API(debug=True, timeout=600)
62 | out = api.get(query, verbosity='geom', responseformat="json")
63 | roadmaptools.inout.save_geojson(_convert_json_to_geojson(out), filepath)
64 |
65 |
66 | def download_cities(bounding_boxes: List[Tuple[float, float, float, float]], filepath: str):
67 | """
68 | Downloads osm map and saves it as .geojson file.
69 | :param bounding_boxes: Order of coordinates in bounding box: (min lat, min lon, max lan, max lon)
70 | :param filepath: path to output file
71 | :return:
72 | """
73 |
74 | query = '(('
75 |
76 | for bounding_box in bounding_boxes:
77 | if float(bounding_box[0]) >= float(bounding_box[2]) or float(bounding_box[1]) >= float(bounding_box[3]):
78 | raise Exception('Wrong order in: ', bounding_box)
79 | query += 'way({}){};'.format(",".join(map(str, list(bounding_box))), WAY_FILTER)
80 |
81 | query += ')->.edges;.edges >->.nodes;);'
82 | call_overpass(query, filepath)
83 |
84 |
85 | def download_by_name(name: str, filepath: str):
86 | """
87 | Downloads osm map and saves it as .geojson file.
88 | :param name: Name of the area on Open Street Map
89 | :param filepath: path to output file
90 | :return:
91 | """
92 | query = """
93 | area[name="{}"];
94 | ((
95 | way(area)
96 | {};)->.edges;.edges >->.nodes;);""".format(name, WAY_FILTER)
97 | call_overpass(query, filepath)
98 |
99 |
100 | if __name__ == '__main__':
101 | # download_cities([(49.94, 14.22, 50.17, 14.71), (49.11, 16.42, 49.30,16.72)], "test.geojson")
102 | download_cities([(envelope["south"], envelope["east"], envelope["north"], envelope["west"]) for _, envelope in config.cities_envelopes.items()] , config.geojson_file)
--------------------------------------------------------------------------------
/roadmaptools/estimate_speed_from_osm.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from typing import Tuple
21 | import geojson
22 | import codecs
23 | from roadmaptools.calculate_curvature import get_length
24 | import sys
25 | import argparse
26 | import time
27 |
28 | from geojson import Feature
29 | from roadmaptools.printer import print_info
30 | from roadmaptools.init import config
31 |
32 | MPH = 1.60934 # miles to km
33 |
34 | # https://wiki.openstreetmap.org/wiki/OSM_tags_for_routing/Maxspeed
35 | SPEED_CODE_DICT = {'CZ': {'default': 50, 'urban': 50, 'living_street': 20, 'pedestrian': 20,
36 | 'motorway': 80, 'motorway_link': 80, 'trunk': 80, 'trunk_link': 80},
37 |
38 | 'US': {'default': 30, 'living_street': 20, 'residential': 25, 'primary': 45,
39 | 'motorway': 55, 'motorway_link': 55, 'trunk': 5, 'trunk_link': 55,
40 | 'secondary': 55, 'tertiary': 55, 'unclassified': 55}}
41 |
42 | DEFAULT_UNIT = "kmh"
43 | UNIT_MAP = {'US': "mph", "CZ": "kmh"}
44 |
45 | TO_METERS_PER_SECOND = {"kmh": 3.6, "mph": 2.2369362920544}
46 |
47 |
48 | # length is computed here too!!!
49 | def estimate_posted_speed(input_filename: str, output_filename: str):
50 | print_info('Estimating travel speed')
51 | start_time = time.time()
52 |
53 | input_stream = codecs.open(input_filename, encoding='utf8')
54 | output_stream = open(output_filename, 'w')
55 |
56 | print_info("Loading file from: {}".format(input_filename))
57 | geojson_file = load_geojson(input_stream)
58 |
59 | print_info("Computing speed")
60 | geojson_out = get_geojson_with_speeds(geojson_file)
61 |
62 | print_info("Saving file to: {}".format(output_filename))
63 | save_geojson(geojson_out, output_stream)
64 | input_stream.close()
65 | output_stream.close()
66 |
67 | print_info('Speed estimation completed. (%.2f secs)' % (time.time() - start_time))
68 |
69 |
70 | def estimate_speeds(input_stream, output_stream):
71 | json_dict = load_geojson(input_stream)
72 | get_speeds(json_dict)
73 | save_geojson(json_dict, output_stream)
74 |
75 |
76 | def get_geojson_with_speeds(json_dict):
77 | get_speeds(json_dict)
78 | return json_dict
79 |
80 |
81 | def load_geojson(in_stream):
82 | json_dict = geojson.load(in_stream)
83 | return json_dict
84 |
85 |
86 | def get_speeds(json_dict):
87 | for item in json_dict['features']:
88 | item['properties']['maxspeed'] = get_posted_speed(item)
89 |
90 | item['properties']['length_gps'] = get_length(item['geometry']['coordinates'])
91 | if item['geometry']['type'].lower() == 'linestring':
92 | item['properties']['maxspeed'] = get_posted_speed(item)
93 | item['properties']['length_gps'] = get_length(item['geometry']['coordinates'])
94 |
95 |
96 | def parse_speed(speed) -> Tuple[int, str]:
97 | """
98 | Parses numeric speed data from osm.
99 |
100 | :param speed: List or string. '40', '50 mph', ['20', '30'], ['15 mph', '25 mph']
101 | :return: float value of maximum allowed speed in km/h
102 | """
103 |
104 | speed = speed[-1] if isinstance(speed, list) else speed
105 | if speed.isnumeric():
106 | return int(speed), DEFAULT_UNIT
107 |
108 | speed, unit = speed.split(' ')
109 | unit_lower = unit.lower()
110 | if unit_lower == 'mph':
111 | return int(speed.split(' ')[0]), unit_lower
112 | else:
113 | raise Exception("Unsupported unit: {}".format(unit))
114 | # return float(speed.split(' ')[0])
115 |
116 |
117 | def get_country(edge: Feature) -> str:
118 | """
119 | Returns code from coordinates.
120 | (By now only CZ or US based on latitude sign).
121 |
122 | :param edge: geojson feature
123 | :return: country code
124 | """
125 |
126 | x, y = edge['geometry']['coordinates'][0]
127 | if float(x) < 0:
128 | return 'US'
129 | else:
130 | return 'CZ'
131 |
132 |
133 | def get_speed_by_code(country_code: str, speed_tag: str) -> int:
134 | """
135 | Returns speed value from SPEED_CODE_DICT.
136 | :param country_code:
137 | :param speed_tag:
138 | :return:
139 | """
140 | speeds_for_country = SPEED_CODE_DICT[country_code]
141 | if speed_tag in speeds_for_country.keys():
142 | return speeds_for_country[speed_tag]
143 | else:
144 | return speeds_for_country['default']
145 |
146 |
147 | def get_posted_speed(edge: Feature) -> Tuple[int, str]:
148 | if 'maxspeed' in edge['properties']:
149 | speed = edge['properties']['maxspeed']
150 |
151 | if ':' in speed:
152 | # Parse speed from OSM speed code, e.g. CZ:urban
153 | country_code, code = speed.split(':')
154 | if country_code not in SPEED_CODE_DICT:
155 | raise Exception('Country code missing')
156 | return get_speed_by_code(country_code, code), UNIT_MAP[country_code]
157 | else:
158 | # Parse speed from speed string, e.g. `40` or `25 mph`
159 | return parse_speed(speed)
160 |
161 | # no maxspeed tag in source data, we use the highway tag to deetermine the max speed
162 | else:
163 | country_code = get_country(edge)
164 | if country_code not in SPEED_CODE_DICT:
165 | raise Exception('Country code missing')
166 |
167 | highway_tag = edge['properties']['highway']
168 | if highway_tag in SPEED_CODE_DICT[country_code].keys():
169 | return SPEED_CODE_DICT[country_code][highway_tag], UNIT_MAP[country_code]
170 | else:
171 | return SPEED_CODE_DICT[country_code]['default'], UNIT_MAP[country_code]
172 |
173 |
174 | def save_geojson(json_dict, out_stream):
175 | geojson.dump(json_dict, out_stream)
176 |
177 |
178 | def get_args():
179 | parser = argparse.ArgumentParser()
180 | parser.add_argument('-i', dest="input", type=str, action='store', help='input file')
181 | parser.add_argument('-o', dest="output", type=str, action='store', help='output file')
182 | return parser.parse_args()
183 |
184 |
185 | def get_speed_per_second_from_edge(edge: dict, scale: float=1, use_measured_speed: bool=False) -> float:
186 | speed = edge['properties']['measured_speed'] if use_measured_speed else edge['properties']['maxspeed']
187 | unit = edge['properties']['speed_unit']
188 | return get_speed_per_second(speed, unit, scale)
189 |
190 |
191 | def get_speed_per_second(speed: int, unit: str, scale: float = 1) -> float:
192 | return speed * scale / TO_METERS_PER_SECOND[unit]
193 |
194 |
195 | if __name__ == '__main__':
196 | args = get_args()
197 | input_stream = sys.stdin
198 | output_stream = sys.stdout
199 |
200 | if args.input is not None:
201 | input_stream = codecs.open(args.input, encoding='utf8')
202 | if args.output is not None:
203 | output_stream = codecs.open(args.output, 'w')
204 |
205 | estimate_speeds(input_stream, output_stream)
206 | input_stream.close()
207 | output_stream.close()
208 |
--------------------------------------------------------------------------------
/roadmaptools/export_nodes_and_id_maker.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import geojson
21 | import codecs
22 | from geojson import Point, Feature, FeatureCollection
23 | import networkx as nx
24 | import sys
25 | import argparse
26 | import roadmaptools.graph
27 |
28 |
29 | def create_unique_ids(input_stream, output_stream, formated):
30 | json_dict = load_geojson(input_stream)
31 | # points = export_points_to_geojson(json_dict)
32 | # save_geojson(points, output_stream, formated)
33 | get_ids(json_dict)
34 | save_geojson(json_dict, output_stream, formated)
35 |
36 |
37 | def get_geojson_with_unique_ids(json_dict):
38 | get_ids(json_dict)
39 | return json_dict
40 |
41 |
42 | def get_node(node):
43 | return (node[0], node[1]) # order lonlat
44 |
45 |
46 | def load_geojson(in_stream):
47 | json_dict = geojson.load(in_stream)
48 | return json_dict
49 |
50 |
51 | def load_graph(json_dict):
52 | g = nx.DiGraph()
53 | for item in json_dict['features']:
54 | coord = item['geometry']['coordinates']
55 | coord_u = get_node(coord[0])
56 | coord_v = get_node(coord[-1])
57 | g.add_edge(coord_u, coord_v, id=item['properties']['id'])
58 | return g
59 |
60 |
61 | def get_node_collection(json_dict):
62 | g = load_graph(json_dict)
63 | list_of_features = []
64 | index = 0
65 | for n in g.nodes():
66 | node_id = roadmaptools.graph.get_node_id(n)
67 | point = Point(n)
68 | feature = Feature(geometry=point, properties={'node_id': node_id, 'index': index})
69 | list_of_features.append(feature)
70 | index += 1
71 |
72 | json_dict_with_points = FeatureCollection(features=list_of_features)
73 |
74 | return json_dict_with_points
75 |
76 |
77 | def get_ids(json_dict):
78 | for item in json_dict['features']:
79 | # item['properties']['length'] = item['properties']['distance_best_guess']
80 | # item['properties']['speed'] = item['properties']['speed_best_guess']
81 | # del item['properties']['distance_best_guess']
82 | # if 'distance_optimistic' in item['properties']:
83 | # del item['properties']['distance_optimistic']
84 | # if 'distance_pessimistic' in item['properties']:
85 | # del item['properties']['distance_pessimistic']
86 |
87 | from_node = item['geometry']['coordinates'][0]
88 | to_node = item['geometry']['coordinates'][-1]
89 | from_node_id = roadmaptools.graph.get_node_id(from_node)
90 | to_node_id = roadmaptools.graph.get_node_id(to_node)
91 | item['properties']['from_id'] = from_node_id
92 | item['properties']['to_id'] = to_node_id
93 |
94 |
95 | def save_geojson(json_dict, out_stream, is_formated=False):
96 | if is_formated == False:
97 | geojson.dump(json_dict, out_stream)
98 | else:
99 | geojson.dump(json_dict, out_stream, indent=4, sort_keys=True)
100 |
101 |
102 | def get_args():
103 | parser = argparse.ArgumentParser()
104 | parser.add_argument('-i', dest="input", type=str, action='store', help='input file')
105 | parser.add_argument('-o', dest="output", type=str, action='store', help='output file')
106 | parser.add_argument('-formated', action='store_true', default=False, dest='formated', help='format output file')
107 | return parser.parse_args()
108 |
109 |
110 | # EXAMPLE OF USAGE
111 | if __name__ == '__main__':
112 | args = get_args()
113 | input_stream = sys.stdin
114 | output_stream = sys.stdout
115 |
116 | if args.input is not None:
117 | input_stream = codecs.open(args.input, encoding='utf8')
118 | if args.output is not None:
119 | output_stream = codecs.open(args.output, 'w')
120 |
121 | create_unique_ids(input_stream, output_stream, args.formated)
122 | input_stream.close()
123 | output_stream.close()
124 |
--------------------------------------------------------------------------------
/roadmaptools/filter.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
21 | from typing import Callable
22 | from geojson import FeatureCollection
23 |
24 |
25 | def filter_edges(edges: FeatureCollection, filter: Callable[[dict], bool]):
26 | filtered_edges = []
27 | for edge in edges['features']:
28 | if filter(edge):
29 | filtered_edges.append(edge)
30 |
31 | edges['features'] = filtered_edges
32 |
--------------------------------------------------------------------------------
/roadmaptools/generate_config.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from fconfig import configuration
21 |
22 | configuration.generate_config()
--------------------------------------------------------------------------------
/roadmaptools/geojson_linestrings.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from typing import List, Iterable
21 | from geojson import LineString, MultiLineString
22 |
23 |
24 | def merge_linestrings(linestrings: Iterable[LineString]) -> MultiLineString:
25 | linestrings_coords = []
26 | for linestring in linestrings:
27 | linestrings_coords.append(linestring["geometry"]["coordinates"])
28 |
29 | return MultiLineString(linestrings_coords)
--------------------------------------------------------------------------------
/roadmaptools/geojson_shp.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import geojson.geometry
21 |
22 | from typing import Callable, Tuple
23 | from shapely.geometry import LineString
24 |
25 |
26 | def geojson_linestring_to_shp_linestring(geojson_linestring: geojson.geometry.LineString,
27 | coordinate_convertor: Callable[[float, float], Tuple[float, float]] = None) -> LineString:
28 | points = []
29 | for point in geojson_linestring["geometry"]["coordinates"]:
30 | if coordinate_convertor:
31 | coords = coordinate_convertor(point[1], point[0])
32 | else:
33 | coords = (point[1], point[0])
34 | points.append(coords)
35 |
36 | return LineString(points)
37 |
38 |
39 |
--------------------------------------------------------------------------------
/roadmaptools/geometry.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import math
21 |
22 | from typing import Tuple, List
23 |
24 |
25 | def get_distance(from_coord: Tuple[float, float], to_coord: Tuple[float, float]) -> float:
26 | return math.sqrt(math.pow(from_coord[0] - to_coord[0], 2) + math.pow(from_coord[1] - to_coord[1], 2))
27 |
28 |
29 | def get_distance_int(from_coord: Tuple[int, int], to_coord: Tuple[int, int]) -> int:
30 | """
31 | Compute integer distance in from utm/cartesian coordinates supplied as integer.
32 | :param from_coord: UTM coord from.
33 | :param to_coord: UTM coord to.
34 | :return: Distance as integer.
35 | """
36 | return int(round(get_distance(from_coord, to_coord)))
37 |
38 |
39 | def get_length_from_coords(coords: List[Tuple[int,int]]) -> int:
40 | length = 0
41 | for i in range(0, len(coords) - 1):
42 | from_coord = coords[i]
43 | to_coord = coords[i + 1]
44 | length += get_distance_int(from_coord, to_coord)
45 |
46 | return length
47 |
48 |
--------------------------------------------------------------------------------
/roadmaptools/gmaps.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
21 | import geojson
22 | import googlemaps
23 | from datetime import datetime
24 | import networkx as nx
25 | import codecs
26 | import os.path
27 |
28 | from typing import Union, Tuple, List
29 | from roadmaptools.printer import print_info
30 |
31 |
32 | class GMapsAPI: # GET ALL DATA FOR ALL TRAFFIC MODELS
33 |
34 | traffic_models = ["best_guess", "pessimistic", "optimistic"]
35 | temp_dict = dict()
36 | json_dict = dict()
37 | g = nx.DiGraph()
38 |
39 | max_paths_in_request = 10
40 |
41 | def __init__(self, pathname=None):
42 | self.pathname = pathname
43 |
44 | def find_edge(self, id, model):
45 | duration = self.temp_dict[id]['duration_' + model]
46 | distance = self.temp_dict[id]['distance_' + model]
47 | speed = self.temp_dict[id]['speed_' + model]
48 | return [duration, distance, speed]
49 |
50 | def google_maps_find_path(self, start: Tuple[float, float], target: Tuple[float, float], time: datetime,
51 | model: str= "best_guess"):
52 | try:
53 | gmaps = googlemaps.Client(key='AIzaSyDoCeLZhFJkx2JTLH8UMcsouaVUIwbV_wY')
54 | # time = datetime(2017, 11, 14, 7, 0, 0) # 7 hours after midnight
55 | # print_info("Request sent")
56 | result = gmaps.directions(start, target, mode="driving", language="en-GB", units="metric",
57 | departure_time=time, traffic_model=model)
58 | # print_info("Response obtained")
59 | # print(self.get_velocity(result))
60 | # duration = result['rows'][0]['elements'][0]['duration_in_traffic']['value']
61 | # distance = result['rows'][0]['elements'][0]['distance']['value']
62 | return result
63 | except (googlemaps.exceptions) as error:
64 | print_info("Google maps exception: {}".format(error))
65 |
66 | def get_durations_and_distances(self, starts: Union[Tuple[float, float], List[Tuple[float, float]]],
67 | targets: Union[Tuple[float,float], List[Tuple[float,float]]], time: datetime,
68 | model: str= "best_guess"):
69 | rows = []
70 | for index, start in enumerate(starts):
71 | start_batch, target_batch = 0
72 | if index % self.max_paths_in_request == 0:
73 | if index > 0:
74 | result = self.google_maps_find_path(start_batch, target_batch, time, model)
75 | rows.extend(result['rows'])
76 | start_batch = []
77 | target_batch = []
78 |
79 | start_batch.append(start)
80 | target_batch.append(targets[index])
81 |
82 | durations = []
83 | distances = []
84 | for row in rows:
85 | durations.append(row['elements'][0]['duration_in_traffic']['value'])
86 | distances.append(row['elements'][0]['distance']['value'])
87 |
88 | return durations, distances
89 |
90 |
91 | def get_node(self, node):
92 | return (node[1], node[0]) # order is latlon
93 |
94 | def get_velocity(self, gmaps_result): # now works with optimistic time
95 | print(gmaps_result)
96 | try:
97 | time = gmaps_result['rows'][0]['elements'][0]['duration_in_traffic'][
98 | 'value'] # gmaps_result['rows'][0]['elements'][0]['duration']['value']
99 | distance = gmaps_result['rows'][0]['elements'][0]['distance']['value']
100 | velocity = distance / time
101 | return '{} {}'.format(velocity * 3.6, 'km/h')
102 | except ZeroDivisionError:
103 | print("DIVISION BY ZERO!!!")
104 |
105 | def load_file_and_graph(self):
106 | with codecs.open(self.pathname + ".geojson", encoding='utf8') as f:
107 | self.json_dict = geojson.load(f)
108 |
109 | if os.path.exists("temp_data.gpickle"):
110 | self.g = nx.read_gpickle("temp_data.gpickle")
111 | else:
112 | for item in self.json_dict['features']:
113 | coord = item['geometry']['coordinates']
114 | coord_u = self.get_node(coord[0])
115 | coord_v = self.get_node(coord[-1])
116 | self.g.add_edge(coord_u, coord_v, id=item['properties']['id'])
117 |
118 | f.close()
119 |
120 | def get_gmaps_data(self):
121 | for n, nbrsdict in self.g.adjacency_iter():
122 | for nbr, keydict in nbrsdict.items():
123 | for key, d in keydict.items():
124 | if 'duration_optimistic' not in d:
125 | for model in self.traffic_models:
126 | res = self.google_maps_find_path(n, nbr, model) # return duration, distance, speed
127 | d['duration_' + model] = res[0]
128 | d['distance_' + model] = res[1]
129 | d['speed_' + model] = res[2]
130 |
131 | nx.write_gpickle(self.g, "temp_data.gpickle") # save for sure
132 |
133 | for n, nbrsdict in self.g.adjacency_iter():
134 | for nbr, keydict in nbrsdict.items():
135 | for key, d in keydict.items():
136 | self.temp_dict[d['id']] = {}
137 | for model in self.traffic_models:
138 | self.temp_dict[d['id']]['duration_' + model] = d['duration_' + model]
139 | self.temp_dict[d['id']]['distance_' + model] = d['distance_' + model]
140 | self.temp_dict[d['id']]['speed_' + model] = d['speed_' + model]
141 |
142 | for item in self.json_dict['features']:
143 | for model in self.traffic_models:
144 | res = self.find_edge(item['properties']['id'], model) # return duration, distance, speed
145 | item['properties']['duration_' + model] = res[0]
146 | item['properties']['distance_' + model] = res[1]
147 | item['properties']['speed_' + model] = res[2]
148 |
149 | def save_file_to_geojson(self):
150 | with open(self.pathname + "-out.geojson", 'w') as outfile:
151 | geojson.dump(self.json_dict, outfile, indent=4, sort_keys=True)
152 | outfile.close()
153 |
154 |
155 | # EXAMPLE
156 | # test = Data_from_gmaps("prague_simple")
157 | # test.load_file_and_graph()
158 | # test.get_gmaps_data()
159 | # test.save_file_to_geojson()
160 |
161 | # stahnout data
162 | # osmosis
163 | # prekonvertovat do geojson
164 | # procistit
165 | # ohodnotit
166 |
--------------------------------------------------------------------------------
/roadmaptools/gpx.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from typing import Callable
21 | # from gpxpy.gpx import GPX, GPXTrackPoint, GPXTrack
22 | from gpx_lite.gpx import GPX
23 | from gpx_lite.gpxtrackpoint import GPXTrackPoint
24 | from gpx_lite.gpxtrack import GPXTrack
25 | from tqdm import tqdm
26 |
27 |
28 | def filter_gpx(gpx_content: GPX, filter: Callable[[GPXTrackPoint, GPXTrackPoint], bool]) -> GPX:
29 | removed_points_count = 0
30 | removed_segments_count = 0
31 | removed_tracks_count = 0
32 |
33 | new_tracks = []
34 | for track in tqdm(gpx_content.tracks, desc="Removing points using specified filter: {}".format(filter)):
35 | new_segments = []
36 |
37 | for segment in track.segments:
38 | new_points = []
39 | last_point = None
40 | for point in segment.points:
41 | if filter(last_point, point):
42 | new_points.append(point)
43 | last_point = point
44 |
45 | if len(new_points) < len(segment.points):
46 | removed_points_count += len(segment.points) - len(new_points)
47 | segment.points = new_points
48 |
49 | if len(new_points) != 0:
50 | new_segments.append(segment)
51 |
52 | if len(new_segments) < len(track.segments):
53 | removed_segments_count += len(track.segments) - len(new_segments)
54 | track.segments = new_segments
55 |
56 | if len(new_segments) != 0:
57 | new_tracks.append(track)
58 |
59 | if len(new_tracks) < len(gpx_content.tracks):
60 | removed_tracks_count += len(gpx_content.tracks) - len(new_tracks)
61 | gpx_content.tracks = new_tracks
62 |
63 | print("Removed points: {}".format(removed_points_count))
64 | print("Removed segments: {}".format(removed_segments_count))
65 | print("Removed tracks: {}".format(removed_tracks_count))
66 |
67 | return gpx_content
68 |
69 |
70 | def cut_trace(trace: GPXTrack, length: int) -> GPXTrack:
71 | segment = trace.segments[0]
72 | segment.points = segment.points[:length]
73 | return trace
--------------------------------------------------------------------------------
/roadmaptools/gpx_ load_test.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
21 | import roadmaptools.inout
22 |
23 | from roadmaptools.printer import print_info
24 | from roadmaptools.init import config
25 |
26 | print_info("Loading start")
27 | #oadmaptools.inout.load_gpx("/home/fido/AIC data/Shared/EXPERIMENTAL/traces/traces-raw.gpx")
28 | roadmaptools.inout.load_gpx('/home/olga/Documents/GPX/test1.gpx')
29 | print_info("Loading end")
--------------------------------------------------------------------------------
/roadmaptools/gpx_shp.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from typing import Callable, Tuple
21 | # from gpxpy.gpx import GPXTrack, GPXTrackPoint
22 | from gpx_lite.gpxtrack import GPXTrack
23 | from gpx_lite.gpxtrackpoint import GPXTrackPoint
24 | from shapely.geometry import LineString, Point
25 |
26 |
27 | def track_to_linestring(track: GPXTrack, coordinate_convertor: Callable[[float, float], Tuple[float, float]] = None) \
28 | -> LineString:
29 | points = []
30 | for segment in track.segments:
31 | for point in segment.points:
32 | if coordinate_convertor:
33 | coords = coordinate_convertor(point.latitude, point.longitude)
34 | else:
35 | coords = (point.latitude, point.longitude)
36 | points.append(coords)
37 |
38 | return LineString(points)
39 |
40 |
41 | def trackpoint_to_point(trackpoint: GPXTrackPoint,
42 | coordinate_convertor: Callable[[float, float], Tuple[float, float]] = None) -> Point:
43 | if coordinate_convertor:
44 | coords = coordinate_convertor(trackpoint.latitude, trackpoint.longitude)
45 | else:
46 | coords = (trackpoint.latitude, trackpoint.longitude)
47 | return Point(*coords)
48 |
--------------------------------------------------------------------------------
/roadmaptools/graph.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import os.path
21 | import networkx.algorithms.shortest_paths
22 | import roadmaptools.utm
23 | import roadmaptools.geometry
24 | import roadmaptools.shp
25 | import roadmaptools.inout
26 | import roadmaptools.plotting
27 |
28 | from typing import Dict, Union, Optional, List, Callable, Tuple
29 | from networkx import DiGraph
30 | from shapely.geometry import Point
31 | from scipy.spatial.kdtree import KDTree
32 | from geojson import FeatureCollection
33 | from tqdm import tqdm
34 | from roadmaptools.printer import print_info
35 | from roadmaptools.road_structures import LinestringEdge, Node
36 | from roadmaptools.utm import CoordinateConvertor
37 |
38 |
39 | def get_node_id(node) -> str:
40 | lon = int(node[0] * 10 ** 6)
41 | lat = int(node[1] * 10 ** 6)
42 | if lon < 0 and lat < 0:
43 | return "1" + str(lon)[1:] + str(lat)[1:]
44 | elif lon < 0 and lat >= 0:
45 | return "2" + str(lon)[1:] + str(lat)
46 | elif lon >= 0 and lat < 0:
47 | return "3" + str(lon) + str(lat)[1:]
48 | else:
49 | return str(lon) + str(lat)
50 |
51 |
52 | def _create_node(x: float, y: float, id: int) -> Node:
53 | return Node(x, y, id)
54 |
55 |
56 | class RoadGraph:
57 | def __init__(self, use_cache: bool = True, cache_filepath: str = "",
58 | node_creator: Callable[[float, float, int], Node] = _create_node):
59 | self.use_cache = use_cache
60 | self.cache_filepath = cache_filepath
61 | self.node_creator = node_creator
62 | self.graph: DiGraph = None
63 | self.kdtree: KDTree = None
64 | self.projection = None
65 | self.node_map: Dict[int, Node] = {}
66 |
67 | def load_from_geojson(self, geojson_filepath: str):
68 | if self.use_cache and os.path.exists(self.cache_filepath):
69 | self.graph = networkx.read_gpickle(self.cache_filepath)
70 | self.projection = roadmaptools.utm.TransposedUTM.from_zone(self.graph.graph["zone_number"],
71 | self.graph.graph["zone_letter"])
72 | CoordinateConvertor.projection = self.projection
73 |
74 | print_info("Projection loaded from the cache: {}{}".format(
75 | self.projection.origin_zone_number, self.projection.origin_zone_letter))
76 | else:
77 | geojson = roadmaptools.inout.load_geojson(geojson_filepath)
78 |
79 | # projection determination
80 | for item in geojson['features']:
81 | if item["geometry"]["type"] == "LineString":
82 | first_coord = geojson['features'][0]['geometry']['coordinates'][0]
83 | break
84 |
85 | self.projection = roadmaptools.utm.TransposedUTM.from_gps(first_coord[1], first_coord[0])
86 | print_info("Projection determined from the first coordinate: {}{}".format(
87 | self.projection.origin_zone_number, self.projection.origin_zone_letter))
88 | CoordinateConvertor.projection = self.projection
89 |
90 | self.graph = DiGraph(zone_number=self.projection.origin_zone_number,
91 | zone_letter=self.projection.origin_zone_letter)
92 |
93 | edge_counter = 0
94 |
95 | print_info("Creating networkx graph from geojson")
96 | for item in tqdm(geojson['features'], desc="processing features"):
97 | if item["geometry"]["type"] == "LineString":
98 | coords = item['geometry']['coordinates']
99 | coord_from = roadmaptools.utm.wgs84_to_utm(coords[0][1], coords[0][0], self.projection)
100 | coord_to = roadmaptools.utm.wgs84_to_utm(coords[-1][1], coords[-1][0], self.projection)
101 |
102 | node_from = self._get_node(coord_from[0], coord_from[1])
103 | node_to = self._get_node(coord_to[0], coord_to[1])
104 |
105 | edge = LinestringEdge(item, CoordinateConvertor.convert, node_from, node_to)
106 |
107 | # TODO legacy, remove after moving id from properties to id attribute
108 | edge_id = item['properties']['id'] if "id" in item['properties'] else item['id']
109 | length = item['properties']['length'] if 'length' in item['properties'] \
110 | else roadmaptools.geometry.get_distance(coord_from, coord_to)
111 |
112 | if node_from in self.graph and node_to in self.graph[node_from]:
113 | a = 1
114 |
115 | self.graph.add_edge(node_from, node_to, id=edge_id, length=length, edge=edge)
116 |
117 | edge_counter += 1
118 |
119 | if edge_counter != len(self.graph.edges):
120 | a = 1
121 |
122 | if self.use_cache:
123 | networkx.write_gpickle(self.graph, self.cache_filepath)
124 |
125 | def get_precise_path_length(self, edge_from: LinestringEdge, edge_to: LinestringEdge,
126 | point_from: Point, point_to: Point) -> Optional[float]:
127 | from_node = edge_from.node_to
128 | to_node = edge_to.node_from
129 |
130 | if edge_from == edge_to:
131 | length = roadmaptools.shp.distance_on_linestring_between_points(edge_from.linestring, point_from, point_to)
132 | else:
133 | try:
134 | length = networkx.algorithms.shortest_paths.astar_path_length(self.graph, from_node, to_node,
135 | weight="length")
136 | except networkx.exception.NetworkXNoPath:
137 | return None
138 |
139 | length += edge_from.linestring.length - edge_from.linestring.project(point_from)
140 | length += edge_to.linestring.project(point_to)
141 |
142 | return length
143 |
144 | def get_edge_path_between_edges(self, edge_from: LinestringEdge, edge_to: LinestringEdge) -> List[LinestringEdge]:
145 | edge_list = []
146 |
147 | from_node = edge_from.node_to
148 | to_node = edge_to.node_from
149 | node_list: List[Node] = networkx.algorithms.shortest_paths.astar_path(self.graph, from_node, to_node,
150 | weight="length")
151 | index = 1
152 | from_node = node_list[0]
153 | to_node = node_list[index]
154 | while True:
155 | edge = self.graph[from_node][to_node]["edge"]
156 | edge_list.append(edge)
157 |
158 | index += 1
159 | if index == len(node_list):
160 | break
161 | from_node = to_node
162 | to_node = node_list[index]
163 |
164 | return edge_list
165 |
166 | def export_for_matplotlib(self) -> Tuple[List[float], List[float]]:
167 |
168 | def iterator():
169 | for edge in self.graph.edges:
170 | yield ((edge[0].x, edge[1].x),(edge[0].y), edge[1].y)
171 |
172 | iterator = iterator()
173 |
174 | return roadmaptools.plotting.export_edges_for_matplotlib(iterator)
175 |
176 | def _get_node(self, x: float, y: float) -> Node:
177 | id = roadmaptools.utm.get_id_from_utm_coords(x, y)
178 | if id in self.node_map:
179 | return self.node_map[id]
180 | else:
181 | node = self.node_creator(x, y, id)
182 | self.node_map[id] = node
183 | return node
184 |
--------------------------------------------------------------------------------
/roadmaptools/init.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import fconfig.configuration as configuration
21 |
22 | from typing import TypeVar
23 | from typing import Type
24 | from fconfig.config import Config
25 | from roadmaptools.config.roadmaptools_config import RoadmaptoolsConfig
26 | import roadmaptools
27 | config = roadmaptools.config.roadmaptools_config.config
28 | # C = TypeVar('C', bound=Config)
29 | # CC = TypeVar('CC', bound=Config)
30 | #
31 | # config = RoadmaptoolsConfig()
32 | #
33 | #
34 | # def load_config(client_config: CC, key_in_client: str, client_local_config: str = None,
35 | # client_config_file_path: str=None):
36 | # configuration.load(config, client_config, client_config_file_path, client_local_config, key_in_client)
37 | # return config
38 |
39 |
--------------------------------------------------------------------------------
/roadmaptools/inout.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import sys
21 | # print(sys.path)
22 |
23 | # this fixes the geojson name clash, because python does not provide a really working absolute imports
24 | # current_dir = sys.path[0]
25 | # sys.path = sys.path[1:]
26 | import geojson
27 | # import geojson.feature
28 | # sys.path = [current_dir] + sys.path
29 |
30 | import urllib.request
31 | import os
32 | import bz2
33 | import sys
34 | import pickle
35 | import json
36 | import networkx as nx
37 | import csv
38 | # import gpxpy
39 | # import gpxpy.gpx
40 | import gpx_lite
41 | import pandas
42 |
43 | from typing import Iterable, Callable, Dict, Tuple, List, Union
44 | from tqdm import tqdm
45 | from gpx_lite.gpx import GPX
46 | from logging import info
47 | # from gpxpy.gpx import GPX
48 | from roadmaptools.init import config
49 | from roadmaptools.printer import print_info
50 |
51 |
52 | class Progressbar(tqdm):
53 | """Provides `update_to(n)` which uses `tqdm.update(delta_n)`."""
54 |
55 | def update_to(self, b=1, bsize=1, tsize=None):
56 | """
57 | b : int, optional
58 | Number of blocks transferred so far [default: 1].
59 | bsize : int, optional
60 | Size of each block (in tqdm units) [default: 1].
61 | tsize : int, optional
62 | Total size (in tqdm units). If [default: None] remains unchanged.
63 | """
64 |
65 | if tsize is not None:
66 | self.total = tsize
67 | self.update(b * bsize - self.n) # will also set self.n = b * bsize
68 |
69 |
70 | def download_file(url: str, file_name: str):
71 | print_info("Downloading file from {} to {}".format(url, file_name))
72 | with Progressbar(unit='B', unit_scale=True, miniters=1, desc="Downloading file") as progressbar:
73 | urllib.request.urlretrieve(url, file_name, progressbar.update_to)
74 |
75 | print_info("Download finished")
76 |
77 |
78 | def extract_file(filename: str):
79 | new_filename = filename.replace(".bz2", "")
80 | compressed_size = os.path.getsize(filename)
81 | print_info("Extracting file {} to {} (compressed size: {})".format(filename, new_filename, compressed_size))
82 |
83 | block_size = 100 * 1024
84 | uncompressed_size = 0
85 | with open(new_filename, 'wb') as new_file, bz2.BZ2File(filename, 'rb') as file:
86 | for data in iter(lambda: file.read(block_size), b''):
87 | new_file.write(data)
88 | uncompressed_size += block_size
89 | print_info("\rDecompressing - decompressed size: {:,}B".format(uncompressed_size).replace(",", " "), end='')
90 | uncompressed_size = os.path.getsize(new_filename)
91 | print_info("\nExtraction finished: uncompressed size: {:,}B".format(uncompressed_size).replace(",", " "))
92 |
93 |
94 | def get_osm_from_mapzen():
95 | print_info("Getting map from mapzen.")
96 | download_file(config.osm_source_url, config.osm_map_filename + ".bz2")
97 | extract_file(config.osm_map_filename + ".bz2")
98 | print_info("Map from mapzen ready.")
99 |
100 |
101 | def load_json(filepath: str, encoding=None) -> Union[Dict, List]:
102 | print_info("Loading json file from: {}".format(os.path.realpath(filepath)))
103 | return json.load(open(filepath, encoding=encoding))
104 |
105 |
106 | def load_geojson(filepath: str) -> geojson.feature.FeatureCollection:
107 | print_info("Loading geojson file from: {}".format(os.path.realpath(filepath)))
108 | input_stream = open(filepath, encoding='utf8')
109 | json_dict = geojson.load(input_stream)
110 | return json_dict
111 |
112 |
113 | def save_geojson(data: geojson.feature.FeatureCollection, filepath: str):
114 | print_info("Saving geojson file to: {}".format(filepath))
115 | out_stream = open(filepath, 'w')
116 | geojson.dump(data, out_stream)
117 |
118 |
119 | def load_csv(filepath: str, delimiter: str = ",") -> Iterable:
120 | print_info("Loading csv file from: {}".format(os.path.realpath(filepath)))
121 | f = open(filepath, "r")
122 | return csv.reader(f, delimiter=delimiter)
123 |
124 |
125 | def load_csv_to_pandas(filepath: str, delimiter: str = ",", header: List[str] = None) -> pandas.DataFrame:
126 | print_info("Loading csv file from: {} to dataframe".format(os.path.realpath(filepath)))
127 | if header:
128 | return pandas.read_csv(filepath, names=header)
129 | return pandas.read_csv(filepath)
130 |
131 |
132 | def save_csv(data: Iterable[Iterable[str]], filepath: str, append: bool = False, delimiter: str = ","):
133 | mode = 'a' if append else 'w'
134 | print_info("Saving csv file to: {}".format(os.path.realpath(filepath)))
135 | with open(filepath, mode, newline='') as csvfile:
136 | writer = csv.writer(csvfile, delimiter=delimiter)
137 | for row in data:
138 | writer.writerow(row)
139 |
140 |
141 | def save_gpx(data: GPX, filepath: str):
142 | print_info("Saving GPX file to: {}".format(os.path.realpath(filepath)))
143 | with open(filepath, 'w') as outfile:
144 | # outfile.write(data.to_xml())
145 | data.write_to_file(outfile)
146 | print_info("{} tracks saved".format(len(data.tracks)))
147 |
148 |
149 | def load_gpx(filepath: str) -> GPX:
150 | print_info("Loading GPX file from: {}".format(os.path.realpath(filepath)))
151 | gpx_file = open(filepath, 'r')
152 | gpx = gpx_lite.iterparse(gpx_file)
153 | print_info("{} tracks loaded".format(len(gpx.tracks)))
154 | return gpx
155 |
156 |
157 | def load_graph(data: geojson.feature.FeatureCollection,
158 | attribute_constructor: Callable[[geojson.LineString], Dict] = None,
159 | coordinate_convertor: Callable[[float, float], Tuple[float, float]] = None) -> nx.DiGraph:
160 | g = nx.DiGraph()
161 | print_info("Creating networkx graph from geojson")
162 | for item in tqdm(data['features'], desc="processing features"):
163 | coords = item['geometry']['coordinates']
164 | if coordinate_convertor:
165 | coord_from = coordinate_convertor(coords[0][1], coords[0][0])
166 | coord_to = coordinate_convertor(coords[-1][1], coords[-1][0])
167 | else:
168 | coord_from = (coords[0][1], coords[0][0])
169 | coord_to = (coords[-1][1], coords[-1][0])
170 | if attribute_constructor:
171 | g.add_edge(coord_from, coord_to, id=item['properties']['id'], attr=attribute_constructor(item))
172 | else:
173 | g.add_edge(coord_from, coord_to, id=item['properties']['id'])
174 | return g
175 |
176 |
177 | def load_graph_from_geojson(filepath: str) -> nx.DiGraph:
178 | data = load_geojson(filepath)
179 | return load_graph(data)
180 |
181 |
182 | def load_pickle(filepath: str):
183 | print_info("Loading pickle file from: {}".format(os.path.realpath(filepath)))
184 | with open(filepath, 'rb') as pickled_data:
185 | data = pickle.load(pickled_data)
186 |
187 | return data
188 |
189 |
190 | def save_pickle(data, filepath):
191 | print_info("Saving pickle file to: {}".format(os.path.realpath(filepath)))
192 | with open(filepath, 'wb') as f:
193 | pickle.dump(data, f, pickle.HIGHEST_PROTOCOL)
194 |
--------------------------------------------------------------------------------
/roadmaptools/osmfilter.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import time
21 | import platform
22 | import os
23 |
24 | from roadmaptools.printer import print_info, print_err
25 | from roadmaptools.init import config
26 | from roadmaptools.inout import download_file
27 |
28 |
29 | def filter_osm_file():
30 | """ Downloads (and compiles) osmfilter tool from web and
31 | calls that osmfilter to only filter out only the road elements.
32 | """
33 |
34 | print_info('Filtering OSM file...')
35 | start_time = time.time()
36 |
37 | if check_osmfilter():
38 | # params = '--keep="highway=motorway =motorway_link =trunk =trunk_link =primary =primary_link =secondary' \
39 | # ' =secondary_link =tertiary =tertiary_link =unclassified =unclassified_link =residential =residential_link' \
40 | # ' =living_street" --drop="access=no"'
41 | params = config.osm_filter_params
42 |
43 | command = './osmfilter' if platform.system() == 'Linux' else 'osmfilter.exe'
44 |
45 | if platform.system() == 'Linux':
46 | filter_command = '%s "%s" %s | pv > "%s"' % (command, config.osm_map_filename, params,
47 | config.filtered_osm_filename)
48 | else:
49 | filter_command = '%s "%s" %s > "%s"' % (
50 | command, config.osm_map_filename, params, config.filtered_osm_filename)
51 | print_info("Running the osmfilter: {}".format(filter_command))
52 | os.system(filter_command)
53 | else:
54 | print_info('Osmfilter not available. Exiting.')
55 | exit(1)
56 |
57 | print_info('Filtering finished. (%.2f secs)' % (time.time() - start_time))
58 |
59 |
60 | def check_osmfilter():
61 | # determine if osmfilter is installed, otherwise download it
62 |
63 | print_info("Checking if osmfilter is installed.")
64 |
65 | my_platform = platform.system() # get system info. Values: 'Linux', 'Windows'
66 | if my_platform == 'Linux': # check if osmfilter is downloaded
67 | executable = 'osmfilter'
68 |
69 | if not os.path.exists(executable):
70 | print_info('Downloading and compiling osmfilter... ')
71 | download_file('http://m.m.i24.cc/osmfilter.c', 'osmfilter.c')
72 | os.system('cc -x c osmfilter.c -O3 -o osmfilter')
73 | return True
74 |
75 | elif my_platform == 'Windows':
76 | executable = 'osmfilter.exe'
77 |
78 | if not os.path.exists(executable):
79 | print_info('Downloading and compiling osmfilter... ')
80 | download_file('http://m.m.i24.cc/osmfilter.exe', 'osmfilter.exe')
81 | return True
82 |
83 | else:
84 | print_err('OSM filtering not implemented for platform: %s. ' % my_platform)
85 | return False
86 |
--------------------------------------------------------------------------------
/roadmaptools/plotting.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import roadmaptools.utm
21 |
22 | from typing import Tuple, List, Iterable
23 | from geojson import FeatureCollection
24 |
25 |
26 | def geojson_edges_iterator(fc: FeatureCollection) -> Iterable[Tuple[Tuple[float, float], Tuple[float, float]]]:
27 | for f in fc['features']:
28 | if f["geometry"]["type"] == "LineString":
29 | from_coords = None
30 | for coordinates in f['geometry']['coordinates']:
31 | to_coords = coordinates
32 | if from_coords:
33 | yield (([from_coords[0], from_coords[1]]),([to_coords[0], to_coords[1]]))
34 | from_coords = to_coords
35 |
36 |
37 | def geojson_node_iterator(fc: FeatureCollection) -> Iterable[Tuple[float, float]]:
38 | for f in fc['features']:
39 | if f["geometry"]["type"] == "Point":
40 | coordinates = f['geometry']['coordinates']
41 | yield (coordinates[0], coordinates[1])
42 |
43 |
44 | def export_nodes_for_matplotlib(nodes_iterator: Iterable[Tuple[float, float]])\
45 | -> Tuple[List[float], List[float]]:
46 | xlist = []
47 | ylist = []
48 | projection = None
49 | for point in nodes_iterator:
50 | if not projection:
51 | projection = roadmaptools.utm.TransposedUTM.from_gps(point[0], point[1])
52 | coords = roadmaptools.utm.wgs84_to_utm(point[0], point[1], projection)
53 | xlist.append(coords[0])
54 | ylist.append(coords[1])
55 | return xlist, ylist
56 |
57 |
58 | def export_edges_for_matplotlib(edges_iterator: Iterable[Tuple[Tuple[float, float], Tuple[float, float]]])\
59 | -> Tuple[List[float], List[float]]:
60 | xlist = []
61 | ylist = []
62 | projection = None
63 | for edge in edges_iterator:
64 | if not projection:
65 | projection = roadmaptools.utm.TransposedUTM.from_gps(edge[0][1], edge[0][0])
66 | from_coords = roadmaptools.utm.wgs84_to_utm(edge[0][1], edge[0][0], projection)
67 | to_coords = roadmaptools.utm.wgs84_to_utm(edge[1][1], edge[1][0], projection)
68 | xlist.append(from_coords[0])
69 | xlist.append(to_coords[0])
70 | xlist.append(None)
71 | ylist.append(from_coords[1])
72 | ylist.append(to_coords[1])
73 | ylist.append(None)
74 | return xlist, ylist
75 |
76 |
--------------------------------------------------------------------------------
/roadmaptools/prepare_geojson_to_agentpolisdemo.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from roadmaptools.init import config
21 |
22 | from typing import Dict, List
23 | from geojson import LineString, Feature, FeatureCollection
24 | import networkx as nx
25 | import codecs
26 | import roadmaptools.inout
27 | from roadmaptools.simplify_graph import _prepare_to_saving_optimized
28 | import copy
29 | from roadmaptools.export_nodes_and_id_maker import get_node_collection, get_ids
30 | import sys
31 | import argparse
32 | import roadmaptools.sanitize
33 |
34 | temp_edges = list()
35 | temp_features = list()
36 |
37 |
38 | def traverse_and_create_graph(g, subgraph):
39 | temp_g = nx.DiGraph()
40 | for n, nbrsdict in list(g.adjacency()):
41 | if n in subgraph:
42 | for nbr, keydict in nbrsdict.items():
43 | if nbr in subgraph:
44 | for key, d in keydict.items():
45 | temp_g.add_edge(n, nbr, id=d['id'], others=d['others'], lanes=d['lanes'])
46 | return temp_g
47 |
48 |
49 | def get_nodeID(node_id): # return String
50 | lon = int(node_id[0] * 10 ** 6)
51 | lat = int(node_id[1] * 10 ** 6)
52 | return str(lat) + str(lon)
53 |
54 |
55 | def find_max_id(json_dict):
56 | max_id = -1
57 | for item in json_dict['features']:
58 | if item['properties']['id'] > max_id:
59 | max_id = item['properties']['id']
60 | return max_id
61 |
62 |
63 | def get_node_for_exporting(node):
64 | return (node[0], node[1]) # order lonlat
65 |
66 |
67 | def get_node(node):
68 | return (node[1], node[0]) # order latlon
69 |
70 |
71 | def detect_parallel_edges(g):
72 | set_of_edges = set()
73 | for n, nbrsdict in list(g.adjacency()):
74 | for nbr, keydict in nbrsdict.items():
75 | for key, d in keydict.items():
76 | if key != 0:
77 | if (n, nbr) in set_of_edges:
78 | temp_edges.append((g[n][nbr][key], get_node_reversed(n), get_node_reversed(nbr), True))
79 | else:
80 | set_of_edges.add((nbr, n)) # add the second direction to set!!
81 | temp_edges.append((g[n][nbr][key], get_node_reversed(n), get_node_reversed(nbr), False))
82 | g.remove_edge(n, nbr, key)
83 |
84 |
85 | def get_node_reversed(node):
86 | return [node[1], node[0]]
87 |
88 |
89 | def add_new_edges(json_dict, edge, new_id): # don't delete item it isn't necessary, because it's made automatically before saving
90 | for item in json_dict['features']:
91 | if item != {} and edge[0]['id'] == item['properties']['id']:
92 | if len(edge[0]['others']) > 0:
93 | if edge[3] == True:
94 | edge[0]['others'].insert(0, edge[1])
95 | line_string1 = LineString(coordinates=(edge[0]['others']))
96 | line_string2 = LineString(coordinates=(edge[0]['others'][-1], edge[2]))
97 | else:
98 | line_string1 = LineString(coordinates=(edge[1], edge[0]['others'][0]))
99 | edge[0]['others'].append(edge[2])
100 | line_string2 = LineString(coordinates=edge[0]['others'])
101 |
102 | feature1 = Feature(geometry=line_string1, properties=copy.deepcopy(item['properties']))
103 | feature1['properties']['id'] = new_id
104 | feature2 = Feature(geometry=line_string2, properties=copy.deepcopy(item['properties']))
105 | feature2['properties']['id'] = new_id + 1
106 | temp_features.append(feature1)
107 | temp_features.append(feature2)
108 | item.clear()
109 | break
110 | else:
111 | # must be added new point
112 | # y = a*x + b
113 | x = (edge[1][0] + edge[2][0]) / 2
114 | y = (edge[1][1] + edge[2][1]) / 2
115 | edge[0]['others'] = [[x, y]]
116 | if edge[3] == True:
117 | edge[0]['others'].insert(0, edge[1])
118 | line_string1 = LineString(coordinates=(edge[0]['others']))
119 | line_string2 = LineString(coordinates=(edge[0]['others'][-1], edge[2]))
120 | else:
121 | line_string1 = LineString(coordinates=(edge[1], edge[0]['others'][0]))
122 | edge[0]['others'].append(edge[2])
123 | line_string2 = LineString(coordinates=edge[0]['others'])
124 |
125 | feature1 = Feature(geometry=line_string1, properties=copy.deepcopy(item['properties']))
126 | feature1['properties']['id'] = new_id
127 | feature2 = Feature(geometry=line_string2, properties=copy.deepcopy(item['properties']))
128 | feature2['properties']['id'] = new_id + 1
129 | temp_features.append(feature1)
130 | temp_features.append(feature2)
131 | item.clear()
132 | break
133 |
134 |
135 | def load_graph(json_dict):
136 | g = nx.DiGraph()
137 | for item in json_dict['features']:
138 | coord = item['geometry']['coordinates']
139 | coord_u = get_node(coord[0])
140 | coord_v = get_node(coord[-1])
141 | if coord_u != coord_v or len(coord) != 2: # prune loops without any purpose, save loops like traffic roundabout
142 | lanes = item['properties']['lanes']
143 | data = item['geometry']['coordinates'][1:-1]
144 | if len(data) == 0:
145 | data = []
146 | g.add_edge(coord_u, coord_v, id=item['properties']['id'], others=data, lanes=lanes)
147 | return g
148 |
149 |
150 | def create_DiGraph(g):
151 | temp_gr = nx.DiGraph()
152 | for n, nbrsdict in g.adjacency():
153 | for nbr, keydict in nbrsdict.items():
154 | for key, d in keydict.items():
155 | temp_gr.add_edge(n, nbr, lanes=d['lanes'], id=d['id'], others=d['others'])
156 | return temp_gr
157 |
158 |
159 | def prepare_graph_to_agentpolisdemo():
160 | json_dict = roadmaptools.inout.load_geojson(config.simplified_file_with_speed)
161 | graph = load_graph(json_dict)
162 | # biggest_subgraph = roadmaptools.sanitize.get_biggest_component(graph)
163 | # new_graph = traverse_and_create_graph(graph, biggest_subgraph)
164 |
165 | # detect_parallel_edges(new_graph)
166 | id_iter = find_max_id(json_dict) + 1 # new id iterator
167 | for edge in temp_edges:
168 | add_new_edges(json_dict, edge, id_iter)
169 | id_iter += 2
170 | json_dict['features'] = [i for i in json_dict["features"] if i] # remove empty dicts
171 | _prepare_to_saving_optimized(graph, json_dict)
172 | json_dict['features'].extend(temp_features)
173 | get_ids(json_dict)
174 | nodes = get_node_collection(json_dict)
175 | # print len(json_dict['features'])
176 | # output_stream = open("/home/martin/MOBILITY/GITHUB/smaz/agentpolis-demo/python_scripts/data/nodes.geojson",'w')
177 | roadmaptools.inout.save_geojson(nodes, config.ap_nodes_file)
178 | # output_stream.close()
179 | # for item in json_dict['features']:
180 | # item['properties']['length']=1
181 | # item['properties']['speed']=1
182 | # output_stream = open("/home/martin/MOBILITY/GITHUB/smaz/agentpolis-demo/python_scripts/data/edges.geojson",'w')
183 | roadmaptools.inout.save_geojson(json_dict, config.ap_edges_file)
184 | # output_stream.close()
185 |
186 |
187 | def get_nodes_and_edges_for_agentpolisdemo(json_dict) -> List[FeatureCollection]:
188 | """
189 |
190 | :param json_dict:
191 | :return: [edges,nodes] list
192 | """
193 |
194 | # graph = load_graph(json_dict)
195 | # biggest_subgraph = roadmaptools.sanitize.get_biggest_component(graph)
196 | # new_graph = traverse_and_create_graph(graph, biggest_subgraph)
197 | #
198 | # detect_parallel_edges(new_graph)
199 | # id_iter = find_max_id(json_dict) + 1 # new id iterator
200 | # for edge in temp_edges:
201 | # add_new_edges(json_dict, edge, id_iter)
202 | # id_iter += 2
203 | # json_dict['features'] = [i for i in json_dict["features"] if i] # remove empty dicts
204 | # prepare_to_saving_optimized(new_graph, json_dict)
205 | # json_dict['features'].extend(temp_features)
206 | get_ids(json_dict)
207 | nodes = get_node_collection(json_dict)
208 |
209 | # gr = load_graph(json_dict) # remove duplicated edges
210 | # gr = create_DiGraph(gr)
211 | # prepare_to_saving_optimized(gr, json_dict)
212 | return [json_dict, nodes]
213 |
214 |
215 | def get_args():
216 | parser = argparse.ArgumentParser()
217 | parser.add_argument('-i', dest="input", type=str, action='store', help='input file')
218 | parser.add_argument('-o', dest="output", type=str, action='store', help='output file')
219 | return parser.parse_args()
220 |
221 |
222 | # EXAMPLE OF USAGE
223 | if __name__ == '__main__':
224 | args = get_args()
225 | input_stream = sys.stdin
226 | output_stream = sys.stdout
227 |
228 | if args.input is not None:
229 | input_stream = codecs.open(args.input, encoding='utf8')
230 | if args.output is not None:
231 | output_stream = codecs.open(args.output, 'w')
232 |
233 | prepare_graph_to_agentpolisdemo()
234 | input_stream.close()
235 | output_stream.close()
236 |
--------------------------------------------------------------------------------
/roadmaptools/printer.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import sys
21 |
22 | from typing import Union, Iterable, List
23 | from pandas import DataFrame
24 | from datetime import datetime
25 |
26 |
27 | def print_info(info, file=sys.stdout, end="\n"):
28 | print("[{0}]: {1}".format(datetime.now().strftime('%H:%M:%S'), info), file=file, end=end)
29 |
30 |
31 | def print_err(info):
32 | print_info(info, file=sys.stderr)
33 |
34 |
35 | def print_table(table: Union[DataFrame, list]):
36 | print()
37 |
38 | if isinstance(table, DataFrame):
39 | col_widths = _get_dataframe_col_widths(table)
40 | col_widths = [width + 2 for width in col_widths]
41 | _print_row(list(table.columns), col_widths=col_widths)
42 | for _, row in table.iterrows():
43 | _print_row(row, col_widths=col_widths)
44 | else:
45 | col_width = max(len(str(word)) for row in table for word in row) + 2 # padding
46 | for row in table:
47 | _print_row(row, col_width)
48 |
49 |
50 | def _print_row(row: Iterable, col_width: int = 0, col_widths: List[int] = None):
51 | if col_widths:
52 | tmp_list = [str(word).ljust(col_widths[col_i]) for col_i, word in enumerate(row)]
53 | else:
54 | tmp_list = [str(word).ljust(col_width) for word in row]
55 | print("".join(tmp_list))
56 |
57 |
58 | def _get_dataframe_col_widths(table: DataFrame) -> List[int]:
59 | widths = []
60 | for column in table.columns:
61 | width = max(table[column].map(lambda x: len(str(x))).max(), len(str(column)))
62 | widths.append(width)
63 |
64 | return widths
65 |
66 |
67 | def test_print_table():
68 | df = DataFrame([['A', 24], ['X', 62], ['F', 0]], columns=['class', 'count'])
69 | print_table(df)
--------------------------------------------------------------------------------
/roadmaptools/resources/__init__.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
--------------------------------------------------------------------------------
/roadmaptools/resources/config.cfg:
--------------------------------------------------------------------------------
1 |
2 | map_dir: 'FILL YOUR MAP DIR HERE'
3 | osm_source_url: 'https://s3.amazonaws.com/metro-extracts.mapzen.com/valencia_spain.osm.bz2'
4 | osm_map_filename: $map_dir + 'map.osm'
5 | filtered_osm_filename: $map_dir + 'map-filtered.osm'
6 | geojson_file: $map_dir + 'map.geojson'
7 | cleaned_geojson_file: $map_dir + 'map-cleaned.geojson'
8 | sanitized_geojson_file: $map_dir + 'map-sanitized.geojson'
9 | simplified_file: $map_dir + 'map-simplified.geojson'
10 | simplified_file_with_speed: $map_dir + 'map-simplified-speed.geojson'
11 | simplified_file_with_speed_and_curvature: $map_dir + 'map-simplified-speed-curvature.geojson'
12 | ap_nodes_file: $map_dir + 'nodes.geojson'
13 | ap_edges_file: $map_dir + 'edges.geojson'
14 | utm_center_lon: 50.0877506
15 | utm_center_lat: 14.4209293
16 | shift_utm_coordinate_origin_to_utm_center: false
17 | shapely_error_tolerance: 0.005
18 | osm_filter_params: '--keep="highway=motorway =motorway_link =trunk =trunk_link =primary =primary_link =secondary =secondary_link =tertiary =tertiary_link =unclassified =unclassified_link =residential =residential_link =living_street" --drop="access=no"'
19 | cities_envelopes:
20 | [
21 | {
22 | south: 49.94
23 | east: 14.22
24 | north: 50.17
25 | west: 14.71
26 | }
27 | ]
28 |
29 |
--------------------------------------------------------------------------------
/roadmaptools/road_graph_rtree.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import geojson
21 | import sys
22 | import os.path
23 |
24 | from typing import Tuple, List
25 | from rtree import index
26 | from tqdm import tqdm
27 | from networkx import DiGraph
28 | from shapely.geometry import Polygon, Point
29 | from roadmaptools.printer import print_info
30 | from roadmaptools.road_structures import LinestringEdge
31 | from roadmaptools.graph import RoadGraph
32 |
33 |
34 | class RoadGraphRtree:
35 | def __init__(self, road_graph: RoadGraph, search_size: int = 500, path: str = None):
36 | self.search_size = search_size
37 | self.index = self._build_index(road_graph, path)
38 |
39 | @staticmethod
40 | def _build_index(road_graph: RoadGraph, path: str = None):
41 | if path:
42 | cache_ready = os.path.isfile(path + ".idx")
43 | idx = index.Index(path)
44 | else:
45 | cache_ready = False
46 | idx = index.Index()
47 | if not cache_ready:
48 | print_info("Creating R-tree from geojson roadmap")
49 | for from_node, to_node, data in tqdm(road_graph.graph.edges(data=True), desc="processing edges"):
50 | edge: LinestringEdge = data["edge"]
51 | # data["attr"]["from"] = from_node
52 | # data["attr"]["to"] = to_node
53 | idx.insert(data["id"], edge.linestring.bounds, edge)
54 | if path:
55 | idx.close()
56 | idx = index.Index(path)
57 | return idx
58 |
59 | def get_nearest_edge(self, point: Point):
60 | search_bounds = Point(point).buffer(self.search_size).bounds
61 | candidates = self.index.intersection(search_bounds, objects='raw')
62 | min_distance = sys.maxsize
63 | nearest = None
64 | for candidate in candidates:
65 | edge: LinestringEdge = candidate
66 | distance = point.distance(edge.linestring)
67 | if distance < min_distance:
68 | min_distance = distance
69 | nearest = edge
70 |
71 | if not nearest:
72 | print_info("No edge found in specified distance ({} m).".format(self.search_size))
73 |
74 | envelope = Polygon(((search_bounds[0], search_bounds[3]), (search_bounds[2], search_bounds[3]),
75 | (search_bounds[2], search_bounds[1]), (search_bounds[0], search_bounds[1])))
76 | if not envelope.intersects(nearest.linestring):
77 | print_info("solution does not have to be exact")
78 |
79 | return nearest
80 |
81 | def get_edges_in_area(self, area_bounds: Polygon) -> List[LinestringEdge]:
82 | # edges whose bounding box intersects the area
83 | potential_edges_in_area = self.index.intersection(area_bounds.bounds, objects='raw')
84 |
85 | edges_in_area = []
86 | for candidate in potential_edges_in_area:
87 | if area_bounds.intersects(candidate.linestring):
88 | edges_in_area.append(candidate)
89 | return edges_in_area
90 |
--------------------------------------------------------------------------------
/roadmaptools/road_structures.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import geojson.geometry
21 | import roadmaptools.geojson_shp
22 | import roadmaptools.utm
23 |
24 | from typing import Callable, Tuple, TypeVar
25 | from shapely.geometry import Point
26 |
27 |
28 | T = TypeVar('T', int, float, str)
29 |
30 |
31 | def get_node_id(lat: T, lon: T) -> int:
32 |
33 | # convert to integer E6 format
34 | if isinstance(lat, str):
35 | try:
36 | lat = int(lat)
37 | lon = int(lon)
38 | except ValueError:
39 | lat = float(lat)
40 | lon = float(lon)
41 | if isinstance(lat, float):
42 | lat = int(lat * 10 ** 7)
43 | lon = int(lon * 10 ** 7)
44 |
45 | # compute prefix marking a sign
46 | if lon < 0 and lat < 0:
47 | prefix = 0
48 | elif lon < 0 and lat >= 0:
49 | prefix = 1
50 | elif lon >= 0 and lat < 0:
51 | prefix = 2
52 | else:
53 | prefix = 3
54 |
55 | string_id = "{}{:010}{:010}".format(prefix, abs(lat), abs(lon))
56 |
57 | return int(string_id)
58 |
59 |
60 | def get_edge_id(id_from: int, id_to: int) -> int:
61 | return int("{}{}".format(id_from, id_to))
62 |
63 |
64 | def get_edge_id_from_coords(from_coord: Tuple[T, T], to_coord: Tuple[T, T]) -> int:
65 | from_id = get_node_id(*from_coord)
66 | to_id = get_node_id(*to_coord)
67 | return get_edge_id(from_id, to_id)
68 |
69 |
70 | class Node:
71 | def __init__(self, x: float, y: float, id: int):
72 | self.id = id
73 | self.x = x
74 | self.y = y
75 | self._point = None
76 |
77 | def get_point(self) -> Point:
78 | if not self._point:
79 | self._point = Point(self.x, self.y)
80 |
81 | return self._point
82 |
83 | def __eq__(self, o: object) -> bool:
84 | if isinstance(o, self.__class__):
85 | return self.id == o.id
86 | return False
87 |
88 | def __ne__(self, o: object) -> bool:
89 | if isinstance(o, self.__class__):
90 | return self.id != o.id
91 | return False
92 |
93 | def __hash__(self) -> int:
94 | return hash(self.id)
95 |
96 |
97 | class LinestringEdge:
98 |
99 | def __init__(self, geojson_linestring: geojson.geometry.LineString,
100 | coordinate_convertor: Callable[[float, float], Tuple[float, float]], node_from: Node, node_to: Node):
101 | self.geojson_linestring = geojson_linestring
102 | self.linestring = roadmaptools.geojson_shp.geojson_linestring_to_shp_linestring(geojson_linestring,
103 | coordinate_convertor)
104 | self.node_from = node_from
105 | self.node_to = node_to
106 | self.id = get_edge_id(node_from.id, node_to.id)
107 |
108 | def __eq__(self, o: object) -> bool:
109 | if isinstance(o, self.__class__):
110 | return self.id == o.id
111 | return False
112 |
113 | def __ne__(self, o: object) -> bool:
114 | if isinstance(o, self.__class__):
115 | return self.id != o.id
116 | return False
117 |
118 | def __hash__(self) -> int:
119 | return hash(self.id)
120 |
121 |
122 |
123 |
--------------------------------------------------------------------------------
/roadmaptools/sanitize.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import networkx as nx
21 | import roadmaptools.inout
22 |
23 | from networkx import DiGraph
24 | from geojson.feature import FeatureCollection
25 | from tqdm import tqdm
26 | from roadmaptools.init import config
27 | from roadmaptools.printer import print_info
28 |
29 |
30 | def get_biggest_component(graph: DiGraph) -> set:
31 | biggest_subgraph = graph
32 |
33 | if not nx.is_strongly_connected(graph):
34 | maximum_number_of_nodes = -1
35 | for subgraph in nx.strongly_connected_components(graph):
36 | if len(subgraph) > maximum_number_of_nodes:
37 | maximum_number_of_nodes = len(subgraph)
38 | biggest_subgraph = subgraph
39 |
40 | id_dict = nx.get_edge_attributes(graph, "id")
41 |
42 | print_info("Creating edge id set")
43 | edge_ids = set()
44 | for coordinates, id in id_dict.items():
45 | if coordinates[0] in biggest_subgraph and coordinates[1] in biggest_subgraph:
46 | edge_ids.add(id)
47 |
48 | return edge_ids
49 |
50 |
51 | def filter_geojson_features_by_graph(geojson_data: FeatureCollection, edge_ids: set):
52 |
53 | print_info("Filtering geojson by edge id set")
54 | new_features = []
55 | for item in tqdm(geojson_data['features'], desc="processing features"):
56 | if item['properties']['id'] in edge_ids:
57 | new_features.append(item)
58 |
59 | geojson_data["features"] = new_features
60 |
61 |
62 | # def _detect_parallel_edges(graph):
63 | # set_of_edges = set()
64 | # for n, nbrsdict in list(graph.adjacency()):
65 | # for nbr, keydict in nbrsdict.items():
66 | # for key, d in keydict.items():
67 | # if key != 0:
68 | # if (n, nbr) in set_of_edges:
69 | # temp_edges.append((graph[n][nbr][key], get_node_reversed(n), get_node_reversed(nbr), True))
70 | # else:
71 | # set_of_edges.add((nbr, n)) # add the second direction to set!!
72 | # temp_edges.append((graph[n][nbr][key], get_node_reversed(n), get_node_reversed(nbr), False))
73 | # graph.remove_edge(n, nbr, key)
74 |
75 |
76 | def sanitize(input_filepath: str=config.cleaned_geojson_file, output_filepath: str=config.sanitized_geojson_file):
77 | """
78 | return only the biggest component from map
79 | :return:
80 | """
81 | geojson_data = roadmaptools.inout.load_geojson(input_filepath)
82 |
83 | graph = roadmaptools.inout.load_graph(geojson_data)
84 | biggest_component_set = get_biggest_component(graph)
85 | # _detect_parallel_edges(biggest_component_set)
86 |
87 | filter_geojson_features_by_graph(geojson_data, biggest_component_set)
88 |
89 | roadmaptools.inout.save_geojson(geojson_data, output_filepath)
--------------------------------------------------------------------------------
/roadmaptools/shp.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import math
21 | import shapely.ops
22 | import numpy as np
23 |
24 | from typing import List, Union, Tuple
25 | from roadmaptools.init import config
26 | from shapely.geometry import Point, LineString
27 | from shapely.geometry.base import BaseGeometry
28 |
29 |
30 | def intersects(geometry_a: BaseGeometry, geometry_b: BaseGeometry) -> bool:
31 | return geometry_a.distance(geometry_b) < config.shapely_error_tolerance
32 |
33 |
34 | def project(point: Point, linestring: LineString) -> Point:
35 | projected_point = linestring.interpolate(linestring.project(point))
36 | return projected_point
37 |
38 |
39 | def distance_on_linestring_between_points(linestring: LineString, point_a: Point, point_b: Point) -> float:
40 | return abs(linestring.project(point_a) - linestring.project(point_b))
41 |
42 |
43 | # def split(linestring: LineString, point: Point) -> List:
44 | # if linestring.coords[0] == (point.x, point.y):
45 | # return [None, linestring]
46 | # elif linestring.coords[-1] == (point.x, point.y):
47 | # return [linestring, None]
48 | # else:
49 | # parts = shapely.ops.split(linestring,point)
50 | # if len(parts) == 1:
51 | # parts = shapely.ops.split(linestring, point.buffer(config.shapely_error_tolerance))
52 | # return parts
53 |
54 | def _get_split_index(splitter_distance: float, linestring: LineString, coords) -> int:
55 |
56 | # portion_length = splitter_distance / linestring.length
57 | # index = int((len(coords) - 1) * portion_length)
58 | from_index = 0
59 | to_index = len(coords) - 1
60 | while True:
61 | index = int((to_index + from_index) / 2)
62 | index_point_distance = linestring.project(Point(coords[index]))
63 | if index_point_distance > splitter_distance:
64 | previous_point_distance = linestring.project(Point(coords[index - 1]))
65 | if previous_point_distance < splitter_distance:
66 | return index
67 | to_index = index
68 | else:
69 | next_point_distance = linestring.project(Point(coords[index + 1]))
70 | if next_point_distance > splitter_distance:
71 | return index + 1
72 | from_index = index
73 |
74 |
75 | def split(linestring: LineString, point: Point) -> Union[List[LineString], LineString]:
76 | # if linestring.distance(point) > config.shapely_error_tolerance:
77 | coords = np.array(linestring.coords, dtype=object)
78 | if linestring.distance(point) > 0.005:
79 | return linestring
80 | if coords[0][0] == point.x and coords[0][1] == point.y:
81 | return [None, linestring]
82 | elif coords[-1][0] == point.x and coords[-1][1] == point.y:
83 | return [linestring, None]
84 | else:
85 | splitter_distance = linestring.project(point)
86 | split_index = _get_split_index(splitter_distance, linestring, coords)
87 | first_part_coords = linestring.coords[:split_index]
88 | first_part_coords.append(point)
89 | second_part_coords = linestring.coords[split_index:]
90 | second_part_coords.insert(0, (point.x, point.y))
91 | return [LineString(first_part_coords), LineString(second_part_coords)]
92 |
93 |
94 | def get_remaining_linestring(linestring: LineString, point: Point) -> LineString:
95 | parts = split(linestring, point)
96 | if isinstance(parts, list):
97 | return parts[1]
98 | else:
99 | return parts
100 |
101 |
102 | def extend_line(line: LineString, distance: float) -> LineString:
103 | first_coord = line.coords[0]
104 | last_coord = line.coords[-1]
105 | extension_point = Point(extend_vector(first_coord, last_coord, distance))
106 | return LineString([Point(first_coord[0], first_coord[1]), extension_point])
107 |
108 |
109 | def extend_vector(from_coord: Tuple[float, float], to_coord: Tuple[float, float], distance: float)\
110 | -> Tuple[float, float]:
111 | length = math.sqrt(math.pow(abs(to_coord[1] - from_coord[1]), 2) + math.pow(abs(to_coord[0] - from_coord[0]), 2))
112 | sin = (to_coord[1] - from_coord[1]) / length
113 | cos = (to_coord[0] - from_coord[0]) / length
114 | new_length = length + distance
115 | new_y = from_coord[1] + sin * new_length
116 | new_x = from_coord[0] + cos * new_length
117 | extension_point = (new_x, new_y)
118 | return extension_point
119 |
120 |
121 | def get_angle_between_points(point_from: Point, point_to: Point) -> float:
122 | y = point_to.y - point_from.y
123 | x = point_to.x - point_from.x
124 |
125 | angle = math.atan2(y, x)
126 |
127 | # transformation from (-pi, pi) to (0, 2pi)
128 | angle_trans = angle if angle > 0 else 2 * math.pi - abs(angle)
129 |
130 | return angle_trans
131 |
132 |
133 | def linestring_to_points(linestring: LineString) -> List[Point]:
134 | points = []
135 | for index, x in enumerate(linestring.xy[0]):
136 | points.append(Point(x, linestring.xy[1][index]))
137 |
138 | return points
--------------------------------------------------------------------------------
/roadmaptools/simplify_graph.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from typing import Dict
21 | import geojson
22 | import networkx as nx
23 | import codecs
24 | from roadmaptools.calculate_curvature import get_curvature
25 | from geojson import LineString, Feature, FeatureCollection
26 | import argparse
27 | import sys
28 | import time
29 | import roadmaptools.inout
30 | import roadmaptools.road_structures
31 |
32 | from tqdm import tqdm
33 | from roadmaptools.init import config
34 | from roadmaptools.printer import print_info
35 |
36 | thresholds = [0, 0.01, 0.5, 1, 2]
37 |
38 |
39 | def simplify_geojson(input_file=config.sanitized_geojson_file, output_file=config.simplified_file):
40 | print_info('Simplifying geoJSON')
41 | start_time = time.time()
42 |
43 | geojson_file = roadmaptools.inout.load_geojson(input_file)
44 |
45 | print_info("Simplification process started")
46 |
47 | # l_check set True whether you don't want to simplify edges with different number of lanes
48 | # c_check set True whether you don't want to simplify edges with different curvature
49 | geojson_out = get_simplified_geojson(geojson_file, l_check=False, c_check=False)
50 |
51 | print_info('Simplification completed. (%.2f secs)' % (time.time() - start_time))
52 |
53 | roadmaptools.inout.save_geojson(geojson_out, output_file)
54 |
55 |
56 | def simplify(input_filepath, output_filepath, l_check, c_check):
57 | check_lanes = not l_check # true means don't simplify edges with different num of lanes
58 | check_curvatures = c_check
59 | json_dict = roadmaptools.inout.load_geojson(input_filepath)
60 | graph = _load_graph(json_dict)
61 | _simplify_graph(graph, check_lanes)
62 | _prepare_to_saving_optimized(graph, json_dict)
63 | if check_curvatures:
64 | _simplify_curvature(json_dict)
65 | json_dict['features'] = [i for i in json_dict["features"] if i] # remove empty dicts
66 | roadmaptools.inout.save_geojson(json_dict, output_filepath)
67 |
68 |
69 | def _is_multidigraph(g: nx.MultiDiGraph) -> bool:
70 | for node, nbrdict in g.adjacency():
71 | for nbr, attributes in nbrdict.items():
72 | if len(attributes) > 1:
73 | return True
74 | return False
75 |
76 |
77 | def _get_max_id_in_graph(g: nx.MultiDiGraph) -> int:
78 | max_id = -1
79 | for n, nbrdict in g.adjacency():
80 | for nbr, attrs in nbrdict.items():
81 | for i in range(len(attrs)):
82 | if max_id < attrs[i]['id']:
83 | max_id = attrs[i]['id']
84 | return max_id
85 |
86 |
87 | def create_digraph(graph: nx.MultiDiGraph) -> nx.DiGraph:
88 | if not _is_multidigraph(graph):
89 | return nx.DiGraph(graph)
90 |
91 | # id_counter = _get_max_id_in_graph(graph) + 1
92 | new_graph = nx.DiGraph()
93 | for node, nbrdict in graph.adjacency():
94 | for nbr, attributes in nbrdict.items():
95 | if not type(attributes) is dict:
96 | id_counter = 0
97 | for i in range(len(attributes)):
98 | properties = attributes[i]
99 | if properties['others'] == [[]]:
100 | new_graph.add_edge(node,
101 | nbr,
102 | id='{}_{}'.format(properties['id'], id_counter),
103 | lanes=properties['lanes'],
104 | others=properties['others'],
105 | from_osm_id=properties['from_osm_id'],
106 | to_osm_id = properties['to_osm_id']
107 | )
108 | id_counter += 1
109 | else:
110 | temp_nbr = get_node(properties['others'][0])
111 | new_graph.add_edge(
112 | node,
113 | temp_nbr,
114 | id='{}_{}'.format(attributes[i]['id'], id_counter),
115 | lanes=properties['lanes'],
116 | others=[[]],
117 | from_osm_id=properties['from_osm_id'],
118 | to_osm_id=properties['to_osm_id']
119 | )
120 | id_counter += 1
121 | new_graph.add_edge(
122 | temp_nbr,
123 | nbr,
124 | id='{}_{}'.format(properties['id'], id_counter),
125 | lanes=properties['lanes'],
126 | others=properties['others'][1:],
127 | from_osm_id=properties['from_osm_id'],
128 | to_osm_id=properties['to_osm_id']
129 | )
130 | id_counter += 1
131 |
132 | else:
133 | new_graph.add_edge(
134 | node,
135 | nbr,
136 | id=attributes['id'],
137 | lanes=attributes['lanes'],
138 | others=attributes['others'],
139 | from_osm_id=attributes['from_osm_id'],
140 | to_osm_id=attributes['to_osm_id']
141 | )
142 | return new_graph
143 |
144 |
145 | def get_simplified_geojson(json_dict: FeatureCollection, l_check=False, c_check=False):
146 | check_lanes = not l_check # true means don't simplify edges with different num of lanes
147 | check_curvatures = c_check
148 | graph = _load_graph(json_dict)
149 | _simplify_graph(graph, check_lanes)
150 | graph = create_digraph(graph)
151 | _prepare_to_saving_optimized(graph, json_dict)
152 | if check_curvatures:
153 | _simplify_curvature(json_dict)
154 | json_dict['features'] = [i for i in json_dict["features"] if i] # remove empty dicts
155 | return json_dict
156 |
157 |
158 | def get_node(node):
159 | return (node[1], node[0]) # order latlon
160 |
161 |
162 | def _try_find(edge_id: str, temp_dict: Dict[str, Dict]):
163 | if edge_id in temp_dict:
164 | n1 = temp_dict[edge_id]['node'][1]
165 | n2 = temp_dict[edge_id]['node'][0]
166 | nbr1 = temp_dict[edge_id]['neighbour'][1]
167 | nbr2 = temp_dict[edge_id]['neighbour'][0]
168 | coords = filter(None, temp_dict[edge_id]['coords'])
169 | ret = [False, [n1, n2]]
170 | for node in coords:
171 | ret.append(node)
172 | ret.append([nbr1, nbr2])
173 | return ret
174 | else:
175 | return [True]
176 |
177 |
178 | def _load_graph(json_dict: dict) -> nx.MultiDiGraph:
179 | graph = nx.DiGraph()
180 | for item in tqdm(json_dict['features'], desc="creating road graph"):
181 | coord = item['geometry']['coordinates']
182 | coord_u = get_node(coord[0])
183 | coord_v = get_node(coord[-1])
184 |
185 | # prune loops without any purpose, save loops like traffic roundabout
186 | if coord_u != coord_v or len(coord) != 2:
187 | props = item['properties']
188 | graph.add_edge(
189 | coord_u,
190 | coord_v,
191 | id=item['properties']['id'],
192 | others=[[]],
193 | lanes=props['lanes'],
194 | from_osm_id=props["from_osm_id"],
195 | to_osm_id=props["to_osm_id"]
196 | )
197 | return graph
198 |
199 |
200 | def _simplify_graph(g: nx.MultiDiGraph, check_lanes):
201 | for n, _ in tqdm(list(g.adjacency()), desc="simplifying oneways"):
202 | if g.out_degree(n) == 1 and g.in_degree(n) == 1: # oneways
203 | simplify_oneways(n, g, check_lanes)
204 |
205 | for n, _ in tqdm(list(g.adjacency()), desc="simplifying bidirectional"):
206 | if g.out_degree(n) == 2 and g.in_degree(n) == 2: # both directions in highway
207 | simplify_twoways(n, g, check_lanes)
208 |
209 |
210 | def get_threshold(curvature):
211 | counter = 0
212 | for i in range(0, len(thresholds) - 1):
213 | if thresholds[i] < curvature and curvature < thresholds[i + 1]:
214 | return counter
215 | counter += 1
216 | return counter # bigger then last interval
217 |
218 |
219 | def _cut_edges_off(item, id_iterator, json_dict):
220 | coords = item['geometry']['coordinates']
221 | first_edge = coords[0:3]
222 | end = False
223 | last_node = 0
224 | for i in range(3, len(coords), 2):
225 | if len(coords[i - 1:len(coords)]) < 5:
226 | second_edge = coords[i - 1:len(coords)]
227 | end = True
228 | else:
229 | second_edge = coords[i - 1:i + 2]
230 | res1 = get_curvature(first_edge)
231 | res2 = get_curvature(second_edge)
232 | u = get_threshold(res1[0])
233 | v = get_threshold(res2[0])
234 | if u != v:
235 | line_string = LineString(coordinates=coords[last_node:i])
236 | last_node = i - 1
237 | id_iterator += 1
238 | feature = Feature(geometry=line_string, id=id_iterator, properties=item['properties'])
239 | json_dict['features'].append(feature)
240 | if end == True:
241 | break
242 | else:
243 | first_edge = second_edge
244 |
245 | line_string = LineString(coordinates=coords[last_node:len(coords)])
246 | feature = Feature(geometry=line_string, id=id_iterator + 1, properties=item['properties'])
247 | return [feature, id_iterator + 1]
248 |
249 |
250 | def _simplify_curvature(json_dict):
251 | length = len(json_dict['features'])
252 | id_iterator = length + 1
253 | for i in range(0, length):
254 | if len(json_dict['features'][i]['geometry']['coordinates']) > 4:
255 | res = _cut_edges_off(json_dict['features'][i], id_iterator, json_dict)
256 | feature = res[0]
257 | id_iterator = res[1] + 1
258 | json_dict['features'].append(feature)
259 | json_dict['features'][i].clear()
260 |
261 |
262 | def simplify_oneways(node, graph, check_lanes):
263 | out_edge = list(graph.out_edges(node, data=True))[0]
264 | # out_edge_coords = list(graph.out_edges(node, data=True))[0][:2]
265 | out_edge_coords = tuple(reversed(out_edge[:2]))
266 | out_edge_props = out_edge[2]
267 | # temp = reversed(out_edge_coords)
268 | # out_edge_coords = tuple(temp)
269 | in_edge = list(graph.in_edges(node, data=True))[0]
270 | # in_edge_coords = list(graph.in_edges(node, data=True))[0][:2]
271 | in_edge_coords = in_edge[:2]
272 | in_edge_props = in_edge[2]
273 | new_id = out_edge_props['id']
274 | coords = list(filter(None, in_edge_props['others'] + [[node[1], node[0]]] + out_edge_props['others']))
275 | lanes_u = out_edge_props['lanes']
276 | lanes_v = in_edge_props['lanes']
277 | if out_edge_coords != in_edge_coords \
278 | or (hash_list_of_lists_and_compare(in_edge_props['others'], out_edge_props['others'])):
279 | # remove edges and node
280 | if lanes_u == lanes_v or lanes_u is None or lanes_v is None or check_lanes: # merge only edges with same number of lanes
281 | graph.add_edge(
282 | in_edge_coords[0],
283 | out_edge_coords[0],
284 | id=new_id,
285 | others=coords,
286 | lanes=lanes_u,
287 | from_osm_id=in_edge_props['from_osm_id'],
288 | to_osm_id=out_edge_props['to_osm_id']
289 | )
290 | graph.remove_edge(out_edge_coords[1], out_edge_coords[0])
291 | graph.remove_edge(in_edge_coords[0], in_edge_coords[1])
292 | graph.remove_node(node)
293 | # elif out_edge_coords == edge_v and hash_list_of_lists_and_compare(list(graph.in_edges(node, data=True))[0][2]['others'],
294 | # list(graph.out_edges(node, data=True))[0][2]['others']):
295 | # if lanes_u == lanes_v or lanes_u is None or lanes_v is None or check_lanes: # merge only edges with same number of lanes
296 | # graph.add_edge(edge_v[0], out_edge_coords[0], id=new_id, others=coords, lanes=lanes_u)
297 | # graph.remove_edge(out_edge_coords[1], out_edge_coords[0])
298 | # graph.remove_edge(edge_v[0], edge_v[1])
299 | # graph.remove_node(node)
300 |
301 |
302 | def hash_list_of_lists_and_compare(list1, list2):
303 | temp_hash1 = [tuple(i) for i in list1]
304 | temp_hash2 = [tuple(i) for i in list2]
305 | return set(temp_hash1) != set(temp_hash2)
306 |
307 |
308 | def simplify_twoways(node, grapg: nx.MultiDiGraph, check_lanes: bool):
309 | edge_to_1 = list(grapg.out_edges(node, data=True))[0]
310 | coords_to_1 = tuple(reversed(edge_to_1[:2]))
311 | edge_to_1_props = edge_to_1[2]
312 | edge_to_2 = list(grapg.out_edges(node, data=True))[1]
313 | coords_to_2 = tuple(reversed(edge_to_2[:2]))
314 | edge_to_2_props = edge_to_2[2]
315 | # coords_to_1 = list(grapg.out_edges(node, data=True))[0][:2]
316 | # coords_to_2 = list(grapg.out_edges(node, data=True))[1][:2]
317 | # temp1 = reversed(coords_to_1)
318 | # coords_to_1 = tuple(temp1)
319 | # temp2 = reversed(coords_to_2)
320 | # coords_to_2 = tuple(temp2)
321 |
322 | edge_from_1 = list(grapg.in_edges(node, data=True))[0]
323 | coords_from_1 = edge_from_1[:2]
324 | edge_from_1_props = edge_from_1[2]
325 | edge_from_2 = list(grapg.in_edges(node, data=True))[1]
326 | coords_from_2 = edge_from_2[:2]
327 | edge_from_2_props = edge_from_2[2]
328 | # coords_from_1 = list(grapg.in_edges(node, data=True))[0][:2]
329 | # coords_from_2 = list(grapg.in_edges(node, data=True))[1][:2]
330 |
331 | new_id_out = edge_to_1_props['id']
332 | new_id_in = edge_from_1_props['id']
333 | coords_out = list(filter(None, edge_from_2_props['others'] + [[node[1], node[0]]] + edge_to_1_props['others']))
334 | coords_in = list(reversed(coords_out))
335 |
336 | edges_u = (coords_to_1, coords_to_2)
337 | edges_v = (coords_from_1, coords_from_2)
338 | lanes_u1 = edge_to_1_props['lanes']
339 | lanes_u2 = edge_to_1_props['lanes']
340 | lanes_v1 = edge_from_1_props['lanes']
341 | lanes_v2 = edge_from_2_props['lanes']
342 | if edges_u == edges_v:
343 | # remove edges and node
344 | is_deleted = [False, False]
345 | is_loop = False # finding oneway loop (from_id == to_id)
346 | for i in edges_u:
347 | if check_oneway_loop(i):
348 | is_loop = True
349 | for i in edges_v:
350 | if check_oneway_loop(i):
351 | is_loop = True
352 | if is_loop:
353 | return
354 | if lanes_u1 == lanes_v2 or lanes_u1 is None or lanes_v2 is None or check_lanes: # merge only edges with same number of lanes
355 | grapg.remove_edge(coords_to_1[1], coords_to_1[0])
356 | grapg.remove_edge(coords_to_2[0], coords_to_2[1])
357 | grapg.add_edge(
358 | coords_from_2[0],
359 | coords_from_1[0],
360 | id=new_id_out,
361 | others=coords_out,
362 | lanes=lanes_u1,
363 | from_osm_id=edge_from_2_props['from_osm_id'],
364 | to_osm_id=edge_to_1_props['to_osm_id']
365 | )
366 | is_deleted[0] = True
367 | if lanes_u2 == lanes_v1 or lanes_u2 is None or lanes_v1 is None or check_lanes: # merge only edges with same number of lanes
368 | if coords_to_1[1] != coords_to_1[0] or coords_to_2[0] != coords_to_2[1]: # check loops
369 | grapg.remove_edge(coords_to_1[0], coords_to_1[1])
370 | grapg.remove_edge(coords_to_2[1], coords_to_2[0])
371 | grapg.add_edge(
372 | coords_from_1[0],
373 | coords_from_2[0],
374 | id=new_id_in,
375 | others=coords_in,
376 | lanes=lanes_v1,
377 | from_osm_id=edge_to_1_props['to_osm_id'],
378 | to_osm_id=edge_from_2_props['from_osm_id']
379 | )
380 | is_deleted[1] = True
381 |
382 | if is_deleted[0] and is_deleted[1] or check_lanes:
383 | grapg.remove_node(node)
384 |
385 |
386 | def check_oneway_loop(edge):
387 | return edge[0] == edge[1]
388 |
389 |
390 | def _prepare_to_saving_optimized(graph, json_dict):
391 | list_of_edges = list(graph.edges(data=True))
392 | temp_dict = dict()
393 |
394 | # map graph edges by ID
395 | for edge in list_of_edges:
396 | edge_properties = edge[2]
397 | temp_dict[edge_properties['id']] = {
398 | 'node': edge[0],
399 | 'neighbour': edge[1],
400 | 'coords': edge_properties['others'],
401 | 'from_osm_id': edge_properties['from_osm_id'],
402 | 'to_osm_id': edge_properties['to_osm_id']
403 | }
404 | # id = edge[2]['id']
405 | # temp_dict[id] = {}
406 | # temp_dict[id]['node'] = edge[0]
407 | # temp_dict[id]['neighbour'] = edge[1]
408 | # temp_dict[id]['coords'] = edge[2]['others']
409 |
410 | # map geojson edges by ID
411 | json_ids = dict()
412 | for item in json_dict['features']:
413 | if item['properties']['id'] not in json_ids:
414 | json_ids[item['properties']['id']] = [item]
415 |
416 | # processing standard edges
417 | for item in json_dict['features']:
418 | data = _try_find(item['properties']['id'], temp_dict)
419 | if data[0]:
420 | # item.clear()
421 | json_ids[item['properties']['id']].append(True)
422 | else:
423 | del item['geometry']['coordinates']
424 | item['geometry']['coordinates'] = data[1:]
425 | properties = temp_dict[item['properties']['id']]
426 | item['properties']['from_osm_id'] = properties['from_osm_id']
427 | item['properties']['to_osm_id'] = properties['to_osm_id']
428 |
429 | features = []
430 |
431 | # adding new edges with special id containing _
432 | for key, value in temp_dict.items():
433 | if isinstance(key, str) and "_" in key:
434 | data = _try_find(key, temp_dict)
435 | item = json_ids[int(key.split("_")[0])][0]
436 | item['properties']['from_osm_id'] = value["from_osm_id"]
437 | item['properties']['to_osm_id'] = value["to_osm_id"]
438 | linestring = LineString(coordinates=data[1:])
439 | # del item['geometry']['coordinates']
440 | # item['geometry']['coordinates'] = data[1:]
441 | # from_coords =
442 | # item['properties']['id'] = counter
443 | feature = Feature(geometry=linestring, id=item['id'], properties=item['properties'])
444 | features.append(feature)
445 |
446 | for key, linestring in json_ids.items():
447 | if len(linestring)>1:
448 | linestring[0].clear()
449 |
450 | json_dict['features'] = [i for i in json_dict["features"] if i] # remove empty dicts
451 | json_dict['features'].extend(features)
452 |
453 |
454 | def get_args():
455 | parser = argparse.ArgumentParser()
456 | parser.add_argument('-i', dest="input", type=str, action='store', help='input file')
457 | parser.add_argument('-o', dest="output", type=str, action='store', help='output file')
458 | parser.add_argument('-lanes', action='store_true', default=False, dest='lanes', help='simplify according lanes')
459 | parser.add_argument('-cur', action='store_true', default=False, dest='curs',
460 | help='simplify according curvatures\' thresholds')
461 | return parser.parse_args()
462 |
463 |
464 | # EXAMPLE OF USAGE
465 | if __name__ == '__main__':
466 | args = get_args()
467 | # input_stream = sys.stdin
468 | # output_stream = sys.stdout
469 | #
470 | # if args.input is not None:
471 | # input_stream = codecs.open(args.input, encoding='utf8')
472 | # if args.output is not None:
473 | # output_stream = codecs.open(args.output, 'w')
474 |
475 | # l_check set True whether you don't want to simplify edges with different number of lanes
476 | # c_check set True whether you don't want to simplify edges with different curvature
477 | simplify(args.input, args.output, l_check=args.lanes, c_check=args.curs)
478 |
--------------------------------------------------------------------------------
/roadmaptools/test/__init__.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
--------------------------------------------------------------------------------
/roadmaptools/test/angle_test.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import math
21 | import roadmaptools.shp
22 |
23 | from shapely.geometry import Point, LineString
24 |
25 | # angle = roadmaptools.shp.get_angle_between_points(Point(0, 0), Point(-1, -1))
26 | #
27 | # print(angle)
28 |
29 | mu_alpha = 50
30 | n_alpha = 3
31 |
32 | def get_angle_difference(current_point: Point, previous_point: Point, edge: LineString) -> float:
33 | trace_angle = roadmaptools.shp.get_angle_between_points(previous_point, current_point)
34 | edge_angle = roadmaptools.shp.get_angle_between_points(Point(edge.coords[0]), Point(edge.coords[-1]))
35 | angle_diff = abs(trace_angle - edge_angle)
36 | return angle_diff
37 |
38 | def _get_orientation_measure(current_point: Point, previous_point: Point, edge: LineString):
39 | angle_diff = get_angle_difference(current_point, previous_point, edge)
40 | return mu_alpha * math.pow(math.cos(angle_diff), n_alpha)
41 |
42 | a = Point(0,0)
43 | b = Point(0,1)
44 | line = LineString([(0,0), (0,1)])
45 |
46 | print(get_angle_difference(a, b, line))
47 | print(_get_orientation_measure(a, b, line))
48 |
49 |
50 |
--------------------------------------------------------------------------------
/roadmaptools/test/big_map_test.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
21 | import roadmaptools.osmfilter
22 | import roadmaptools.osmtogeojson
23 | import roadmaptools.clean_geojson
24 |
25 | # roadmaptools.osmfilter.filter_osm_file()
26 |
27 | # roadmaptools.osmtogeojson.convert_osm_to_geojson()
28 |
29 | roadmaptools.clean_geojson.clean_geojson_files()
--------------------------------------------------------------------------------
/roadmaptools/test/cut_trace.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import roadmaptools.inout
21 | import roadmaptools.gpx
22 |
23 | #trace_file = r"C:\AIC data\Shared\Map Matching Benchmark\2015 100 tracks dataset\00000000/trace-period-24.gpx"
24 | trace_file = '/home/olga/Documents/GPX/test1.gpx'
25 |
26 | gpx_content = roadmaptools.inout.load_gpx(trace_file)
27 | print(gpx_content)
28 | roadmaptools.gpx.cut_trace(gpx_content.tracks[0], 10)
29 |
30 |
31 | #roadmaptools.inout.save_gpx(gpx_content, r"C:\AIC data\Shared\Map Matching Benchmark\test traces/trace_0-period_24-first_10_points.gpx")
--------------------------------------------------------------------------------
/roadmaptools/test/distance_test.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
21 | from roadmaptools.init import config
22 |
23 | import roadmaptools.inout
24 | import roadmaptools.estimate_speed_from_osm
25 | import roadmaptools.geojson_shp
26 | import roadmaptools.utm
27 |
28 | # from roadmaptools.
29 |
30 |
31 | def coordinate_convertor(latitude: float, longitude: float):
32 | return roadmaptools.utm.wgs84_to_utm(latitude, longitude, projection)
33 |
34 |
35 | def compute_length_projected(geojson_linestring):
36 | shp_linestring = roadmaptools.geojson_shp.geojson_linestring_to_shp_linestring(geojson_linestring,
37 | coordinate_convertor)
38 |
39 | return shp_linestring.length
40 | # length = 0
41 | # for i in range(0, len(coords) - 1):
42 | # point1 = coords[i]
43 | # point2 = coords[i + 1]
44 | #
45 | #
46 | #
47 | # length += get_distance_between_coords(point1, point2)
48 | #
49 | # return length
50 |
51 |
52 | # load map
53 | map_geojson = roadmaptools.inout.load_geojson(r"C:\AIC data\Shared\amod-data\agentpolis-experiment\Prague\experiment\ridesharing_itsc_2018\maps/map-simplified-speed.geojson")
54 |
55 | # get some random edges
56 | features = map_geojson['features']
57 |
58 | edge_indexes = [0, 11, 55, 506]
59 |
60 | projection = None
61 |
62 | # for each edge
63 | for index in edge_indexes:
64 | edge = features[index]
65 |
66 | if not projection:
67 | first_coord = edge['geometry']['coordinates'][0]
68 | projection = roadmaptools.utm.TransposedUTM(first_coord[1], first_coord[0])
69 |
70 | # compute distance from gps
71 | length_gps = roadmaptools.estimate_speed_from_osm.get_length(edge['geometry']['coordinates'])
72 |
73 | # compute distance projected
74 | length_projected = compute_length_projected(edge)
75 |
76 | print("edge {} (id: {}) - length gps: {}, length projected: {}".format(index, edge["id"], length_gps, length_projected))
77 |
78 | a = 1;
--------------------------------------------------------------------------------
/roadmaptools/test/int.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import sys
21 |
22 | a =1
23 | b =123456789
24 | c =a
--------------------------------------------------------------------------------
/roadmaptools/test/networkx_pickle_test.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import networkx
21 | import roadmaptools.inout
22 | import roadmaptools.geometry
23 |
24 | from typing import Dict
25 | from geojson import FeatureCollection
26 | from tqdm import tqdm
27 | from networkx import DiGraph
28 | from roadmaptools.utm import CoordinateConvertor
29 | from roadmaptools.printer import print_info
30 | from roadmaptools.road_structures import LinestringEdge, Node
31 |
32 | node_map: Dict[int, Node] = {}
33 | graph: DiGraph = DiGraph()
34 |
35 |
36 | def load_from_geojson(geojson: FeatureCollection):
37 | # projection determination
38 | first_coord = geojson['features'][0]['geometry']['coordinates']
39 | projection = roadmaptools.utm.TransposedUTM(first_coord[1], first_coord[0])
40 | print_info("Projection determined from the first coordinate: {}{}".format(
41 | projection.origin_zone_number, projection.origin_zone_letter))
42 | CoordinateConvertor.projection = projection
43 |
44 | print_info("Creating networkx graph from geojson")
45 | for item in tqdm(geojson['features'], desc="processing features"):
46 | if item["geometry"]["type"] == "LineString":
47 | coords = item['geometry']['coordinates']
48 | coord_from = roadmaptools.utm.wgs84_to_utm(coords[0][1], coords[0][0], projection)
49 | coord_to = roadmaptools.utm.wgs84_to_utm(coords[-1][1], coords[-1][0], projection)
50 |
51 | node_from = _get_node(coord_from[0], coord_from[1])
52 | node_to = _get_node(coord_to[0], coord_to[1])
53 |
54 | edge = LinestringEdge(item, CoordinateConvertor.convert, node_from, node_to)
55 |
56 | # TODO legacy, remove after moving id from properties to id attribute
57 | edge_id = item['properties']['id'] if "id" in item['properties'] else item['id']
58 | length = item['properties']['length'] if 'length' in item['properties'] \
59 | else roadmaptools.geometry.get_distance(coord_from, coord_to)
60 | graph.add_edge(node_from, node_to, id=edge_id, length=length, edge=edge)
61 |
62 |
63 | def _get_node(x: float, y: float) -> Node:
64 | id = roadmaptools.utm.get_id_from_utm_coords(x, y)
65 | if id in node_map:
66 | return node_map[id]
67 | else:
68 | node = _create_node(x, y, id)
69 | node_map[id] = node
70 | return node
71 |
72 |
73 | def _create_node(x: float, y: float, id: int) -> Node:
74 | return Node(x, y, id)
75 |
76 | print_info("START STNDARD")
77 | map = roadmaptools.inout.load_geojson(r"C:\AIC data\Shared\Map Matching Benchmark\2015 100 tracks dataset\00000043/map.geojson")
78 | load_from_geojson(map)
79 | print_info("END STANDARD")
80 |
81 | networkx.write_gpickle(graph, "pi.pickle")
82 |
83 |
84 | print_info("START_PICKLE")
85 | networkx.read_gpickle("pi.pickle")
86 | print_info("END PICKLE")
--------------------------------------------------------------------------------
/roadmaptools/test/shapely_project_test.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | from shapely.geometry import LineString, Point
21 |
22 | line = LineString([(0,0),(0,3),(2,3),(2,0)])
23 |
24 | pointa = Point(0,3)
25 | pointb = Point(2,1)
26 |
27 |
28 | print("Point A distance along linestring: {}".format(line.project(pointa)))
29 | print("Point B distance along linestring: {}".format(line.project(pointb)))
30 |
31 |
32 | line = LineString([(0,0),(0,3),(0,0),(4,0)])
33 |
34 | pointa = Point(1,0)
35 | pointb = Point(0,2)
36 |
37 |
38 | print("Point A distance along linestring: {}".format(line.project(pointa)))
39 | print("Point B distance along linestring: {}".format(line.project(pointb)))
40 |
41 | line = LineString([(459401.4808268753, 5548751.413400644), (459416.9239635981, 5548756.833431144), (459452.9326752117, 5548768.679952354), (459471.5841274337, 5548771.740767008), (459486.558864173, 5548769.180983176), (459513.0016597167, 5548773.317026346), (459574.6251330864, 5548782.979997741), (459591.624302734, 5548787.032218366), (459598.1756266461, 5548794.41028268), (459609.8542174158, 5548792.075914356), (459675.927428248, 5548825.512556223), (459688.8854083499, 5548840.103200087), (459708.2701523154, 5548848.440909361), (459724.7171202758, 5548855.19967219), (459729.901486638, 5548857.784718673), (459744.6238166057, 5548864.46758783), (459764.0928814765, 5548872.626955694), (459790.2458894122, 5548883.982884461), (459792.5860938389, 5548885.066070594), (459800.8128602093, 5548888.895875888), (459808.1502022735, 5548892.409928492), (459809.9814406136, 5548893.352401712), (459820.8413175698, 5548898.218752398), (459818.6454109197, 5548904.906766911), (459813.970836277, 5548918.729692562), (459813.138770355, 5548921.282240469), (459809.3071075905, 5548938.745918109), (459807.8724938399, 5548954.390261103), (459808.7382261936, 5548973.430884821), (459811.9416467017, 5548990.374639822), (459817.0341712576, 5549011.2070169), (459823.7416443196, 5549038.476372919), (459826.8347908729, 5549049.327661606), (459830.7286573184, 5549062.97496253), (459832.8479212601, 5549069.886271794), (459819.8982116348, 5549076.243740443), (459798.0287004329, 5549083.591244955), (459792.8225462207, 5549084.731221056), (459782.6278883351, 5549088.366089093), (459776.1627358288, 5549094.252326386), (459770.0328825607, 5549103.81649292), (459401.4808268753, 5548751.413400644), (459330.3869575746, 5548725.079012355), (459401.4808268753, 5548751.413400644)])
42 |
43 | pointa = Point(459428.2239730014, 5548760.551026001)
44 | pointb = Point(459361.8893196194, 5548736.74802745)
45 |
46 | print("Point A distance along linestring: {}".format(line.project(pointa)))
47 | print("Point B distance along linestring: {}".format(line.project(pointb)))
48 |
49 | a = 1
--------------------------------------------------------------------------------
/roadmaptools/test/shp_test.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import roadmaptools.shp
21 |
22 | from shapely.geometry import LineString, Point
23 |
24 | point = Point(2,1)
25 | line_split = LineString([Point(1, 1), Point(3, 1)])
26 | line = LineString([(0,3),(4,3),(2,6),(2,0)])
27 | from shapely.ops import split
28 | splitted = split(line, point)
29 | for lin in splitted:
30 | print(lin)
31 |
32 | splitted = roadmaptools.shp.split(line, point)
33 |
34 | for lin in splitted:
35 | print(lin)
36 |
37 | # line = LineString([(1,1), (0, 5)])
38 | # ref_point = Point(0,10)
39 | # extended_line = roadmaptools.shp.extend_line(line, 8)
40 | # print(extended_line)
41 | # # print(LineString(line.coords.append(ref_point)))
--------------------------------------------------------------------------------
/roadmaptools/test/test_tmp.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 |
21 | import math
22 |
23 | a = math.sqrt(pow(abs(645470921 - 645459154), 2) + pow(abs(160704762- 160703505), 2))
24 |
25 | print(a)
--------------------------------------------------------------------------------
/roadmaptools/utm.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import math
21 | import utm
22 | import numpy as np
23 |
24 | from typing import Tuple
25 | from roadmaptools.init import config
26 |
27 |
28 | class TransposedUTM:
29 | def __init__(self):
30 | self.origin_easting: float = None
31 | self.origin_northing: float = None
32 | self.origin_zone_number: int = None
33 | self.origin_zone_letter: str = None
34 |
35 | @classmethod
36 | def from_gps(cls, origin_lat: float, origin_lon: float):
37 | # self.origin_lat = origin_lat
38 | # self.origin_lon = origin_lon
39 |
40 | res = utm.from_latlon(origin_lat, origin_lon)
41 |
42 | projection = cls()
43 |
44 | projection.origin_easting = res[0]
45 | projection.origin_northing = res[1]
46 | projection.origin_zone_number = res[2]
47 | projection.origin_zone_letter = res[3]
48 |
49 | return projection
50 |
51 | @classmethod
52 | def from_zone(cls, zone_number: int, zone_letter: str):
53 | projection = cls()
54 | projection.origin_zone_number = zone_number
55 | projection.origin_zone_letter = zone_letter
56 |
57 | return projection
58 |
59 | def wgs84_to_utm(self, lat, lon):
60 | easting, northing, _, _ = utm.from_latlon(lat, lon, force_zone_number=self.origin_zone_number)
61 | if config.shift_utm_coordinate_origin_to_utm_center:
62 | easting -= self.origin_easting
63 | northing -= self.origin_northing
64 | return easting, northing
65 |
66 | def utm_to_wgs84(self, easting, northing):
67 | if config.shift_utm_coordinate_origin_to_utm_center:
68 | easting += self.origin_easting
69 | northing += self.origin_northing
70 | return utm.to_latlon(easting, northing, self.origin_zone_number, self.origin_zone_letter)
71 |
72 |
73 | # Project to Euclidean plane such that the units are meters.
74 | default_projection = TransposedUTM.from_gps(config.utm_center_lon, config.utm_center_lat)
75 |
76 |
77 | # default_projection = None
78 |
79 |
80 | def np_wgs84_to_utm(latlon, projection: TransposedUTM = default_projection):
81 | easting, northing = np.vectorize(projection.wgs84_to_utm)(latlon[:, 0], latlon[:, 1])
82 | return np.column_stack([easting, northing])
83 |
84 |
85 | def np_utm_to_wgs84(latlon, projection: TransposedUTM = default_projection):
86 | lat, lon = np.vectorize(projection.utm_to_wgs84)(latlon[:, 0], latlon[:, 1])
87 | return np.column_stack([lat, lon])
88 |
89 |
90 | def wgs84_to_utm(latitude: float, longitude: float, projection: TransposedUTM = default_projection) -> Tuple[
91 | float, float]:
92 | return projection.wgs84_to_utm(latitude, longitude)
93 |
94 |
95 | def wgs84_to_utm_1E2(latitude: float, longitude: float, projection: TransposedUTM = default_projection) \
96 | -> Tuple[int, int]:
97 | """
98 | Returns utm coordinates in centimeter precision. Coresponds to the Geographtools implementation.
99 | :param latitude: Latitude WGS84
100 | :param longitude: Longitude WGS84
101 | :param projection: UTM projection
102 | :return: Projected coordinates as a centimeter precision integer.
103 | """
104 | return tuple(int(round(coord * 1E2)) for coord in wgs84_to_utm(latitude, longitude, projection))
105 |
106 |
107 | def utm_to_wgs84(latitude: float, longitude: float, projection: TransposedUTM = default_projection) -> Tuple[
108 | float, float]:
109 | return projection.utm_to_wgs84(latitude, longitude)
110 |
111 |
112 | def get_id_from_utm_coords(x: float, y: float) -> int:
113 | y_int = int(round(y, 3) * 1000)
114 | return int(round(x, 3) * 1000) * int(math.pow(10, len(str(y_int)))) + y_int
115 |
116 |
117 | class CoordinateConvertor:
118 | projection = default_projection
119 |
120 | @staticmethod
121 | def convert(latitude: float, longitude: float) -> Tuple[float, float]:
122 | return wgs84_to_utm(latitude, longitude, CoordinateConvertor.projection)
123 |
--------------------------------------------------------------------------------
/setup.cfg:
--------------------------------------------------------------------------------
1 | [metadata]
2 | description-file = README.md
3 |
--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
1 | #
2 | # Copyright (c) 2021 Czech Technical University in Prague.
3 | #
4 | # This file is part of Roadmaptools
5 | # (see https://github.com/aicenter/roadmap-processing).
6 | #
7 | # This program is free software: you can redistribute it and/or modify
8 | # it under the terms of the GNU Lesser General Public License as published by
9 | # the Free Software Foundation, either version 3 of the License, or
10 | # (at your option) any later version.
11 | #
12 | # This program is distributed in the hope that it will be useful,
13 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
14 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 | # GNU Lesser General Public License for more details.
16 | #
17 | # You should have received a copy of the GNU Lesser General Public License
18 | # along with this program. If not, see .
19 | #
20 | import setuptools
21 | from setuptools import setup
22 |
23 | setup(
24 | name='roadmaptools',
25 | version='5.0.0',
26 | description='Tools for road graph processing',
27 | author='David Fiedler, Martin Korytak',
28 | author_email='david.fiedler@agents.fel.cvut.cz',
29 | license='MIT',
30 | packages=setuptools.find_packages(),
31 | url = 'https://github.com/aicenter/roadmap-processing',
32 | # DO NOT remove the utm packege despite it is not detected by pipreqs
33 | install_requires=[
34 | 'fconfig',
35 | 'numpy',
36 | 'pandas',
37 | 'googlemaps',
38 | 'typing',
39 | 'gpx_lite',
40 | 'tqdm',
41 | 'overpass',
42 | 'shapely',
43 | 'setuptools',
44 | 'rtree',
45 | 'scipy',
46 | 'networkx>=2.0',
47 | 'geojson',
48 | 'utm'
49 | ],
50 | python_requires='>=3',
51 | package_data={'roadmaptools.resources': ['*.cfg']}
52 | )
53 |
--------------------------------------------------------------------------------