47 | load raw data. 48 |
49 | -  50 |51 | build buffer for each line. 52 |
53 | -  54 |55 | dissolve the buffers. 56 |
57 | -  58 |59 | Extract centerlines from merged buffers 60 |
61 | -  62 |63 | Extraction result. 64 |
65 | 2. road optimization: 66 | -  67 |68 | Extension of non-closed roads (in blue). 69 |
70 | -  71 |72 | Extraction of dead-end roads (points). 73 |
74 | -  75 |76 | Filtering small spikes (in red) with a length below the threshold. 77 |
78 | 3. Road Network Spatial Information Re-association:： 79 | -  80 |81 | Create buffers based on the simplified road network and load the original road network data for information assignment. 82 |
83 | 84 | -  85 |86 | For each buffer, the matching rule with the original road network is to select the most representative road within the buffer (largest overlap). 87 |
88 | 89 | -  90 |91 | eg. The most representative road within this buffer is enclosed. 92 |
93 | 4. output result 94 | -  95 | 96 | ### A Hidden Parameter: 97 | There is a hidden parameter called keep_spike in the clean_spike function in simplify.py, which is not commonly used. 98 | 99 | It may be useful when plotting data related to dead-end roads. If needed, you can manually set keep_spike to True. 100 | 101 | The data for dead-end points and dead-end lines will be output to the result path, named 'err_points.shp' and 'err_lines.shp', respectively. -------------------------------------------------------------------------------- /config/config.py: -------------------------------------------------------------------------------- 1 | import arcpy 2 | 3 | 4 | ######################################基础配置区,若不修改选中的osm_fclass / qq_subtype ，修改此处路径即可。################################# 5 | # 指定gdb文件夹路径 6 | 7 | # 建议输入完整路径，相对路径似乎有点问题,此处屏蔽本地计算机文件路径，使用相对路径 8 | FILE_ROOT = r'./data/测试过程数据库/' 9 | # 指定新建gdb名称 10 | DATABASE_NAME = 'ROAD_OPTIMIZE2.gdb' 11 | # 指定目标城市基础数据路径,文件夹内数据命名格式如下 12 | # 路网数据: 1.osm_xiangcheng.shp 2. qq_xiangcheng.shp 13 | # 边界数据: xiangcheng.shp 14 | ROAD_DATA = r'./data/测试路网数据/' 15 | BOUNDARY_DATA = r'./data/测试边界数据/' 16 | OUTPUT_PATH = r'./data/测试输出数据' 17 | ######################################配置区结束################################# 18 | 19 | # 最最最主要的道路类别 20 | 21 | # 表示道路类型的字段 22 | ROAD_TYPE_FIELD = 'roadClassi' 23 | 24 | # TPYES FOR MIXED MODE 25 | OSM_MAIN_FCLASS = ['A Road','Unknown','B Road','Unclassified','Classified Unnumbered','Not Classified'] 26 | 27 | # # TYPES FOR DIVIDED MODE 28 | # OSM_MAIN_FCLASS_NO_HIGH = ['Unknown', 'Unclassified', 'Classified Unnumbered','Not Classified'] 29 | # OSM_MAIN_FCLASS_HIGH = ['A Road', 'B Road'] 30 | 31 | # 最终数据中需要保留的字段 32 | fields_to_keep = ['OBJECTID',"localId", "SHAPE",'SHAPE_Length'] 33 | # 34 | # MERGING_MAP = { 35 | # 'A Road': 1, 36 | # 'B Road': 2, 37 | # 'Classified Unnumbered': 3, 38 | # 'Unknown': 4, 39 | # 'Not Classified': 5 40 | # } 41 | 42 | 43 | # config proj epsg code 44 | PROJ_CRS = 27700 45 | GEO_CRS = 4326 46 | # config spatial ref 47 | Project_ref = arcpy.SpatialReference(PROJ_CRS) 48 | Geographic_ref = arcpy.SpatialReference(GEO_CRS) 49 | -------------------------------------------------------------------------------- /data_convertor.py: -------------------------------------------------------------------------------- 1 | # %% 2 | import pandas as pd 3 | import geopandas as gpd 4 | from shapely import wkt 5 | import matplotlib.pyplot as plt 6 | 7 | 8 | def merge_arcpy_raw(arcpy_data_path, raw_data_path): 9 | # read the data 10 | arcpy_data = gpd.read_file(arcpy_data_path)[['localId', 'geometry']] 11 | arcpy_data['localId'] = arcpy_data['localId'].astype('int64') 12 | raw_data = gpd.read_file(raw_data_path).drop_duplicates( 13 | subset=['localId'], keep='first') 14 | # drop the rows with geom_type != LineString from raw 15 | raw_data = raw_data[raw_data['geometry'].geom_type == 'LineString'] 16 | # drop the rows with localId is null 17 | arcpy_data = arcpy_data.dropna(subset=['localId']) 18 | 19 | # drop the rows with locald == 0 20 | arcpy_data = arcpy_data[arcpy_data['localId'] != 0] 21 | 22 | # select the data with duplicated localId 23 | duplicated = arcpy_data[arcpy_data.duplicated( 24 | subset=['localId'], keep=False)].sort_values(by=['localId']) 25 | 26 | # delete the duplicated rows from arcpy_data 27 | arcpy_data_unique = arcpy_data.drop_duplicates( 28 | subset=['localId'], keep=False) 29 | 30 | # select the data with localId in duplicated from raw_data 31 | replaced_dups = raw_data[raw_data['localId'].isin(duplicated['localId'])] 32 | 33 | # merge arcpy_data_unique and raw_data on localId 34 | merged = gpd.GeoDataFrame(arcpy_data_unique.merge(raw_data, on='localId', how='left', suffixes=('_arcpy', '_raw')).rename( 35 | columns={'geometry_arcpy': 'geometry'}).drop(['geometry_raw'], axis=1), geometry='geometry') 36 | 37 | # concat the merged and replaced_dups 38 | final = pd.concat([merged, replaced_dups], ignore_index=True) 39 | return final 40 | 41 | 42 | # %% 43 | arcpy_res_path = r'C:\Users\20191\Desktop\github\road_regularization\data\arcpy_data\arcpy_res.shp' 44 | raw_data_path = r'C:\Users\20191\Desktop\github\road_regularization\data/测试路网数据/westminster_project.shp' 45 | output_path = r'C:\Users\20191\Desktop\github\road_regularization/data/clean/merged_data2.shp' 46 | 47 | merge_arcpy_raw(arcpy_res_path, raw_data_path).to_file(output_path) 48 | 49 | # %% 50 | -------------------------------------------------------------------------------- /en.md: -------------------------------------------------------------------------------- 1 | [中文版](./README.md) English 2 | # road_geo_regularization 3 | ### A universal simplification of the main road network in the city, which can be used for non-high precision land division, etc. 4 | 5 | ### Performance: 6 | #### raw road data 7 | -  8 | #### simplified road data 9 | -  10 | 11 | 12 | ### Environment: 13 | - Arcgis Pro 14 | - Python 3.x from Arcgis Pro 15 | - Arcpy from Arcgis Pro 16 | - Encoding utf-8_sig 17 | 18 | ### Required Data: 19 | - Road network data 20 | - Boundary data 21 | - Data needs to be placed in the path specified in config.py in advance, pay attention to the naming format 22 | 23 | ### User Manual: 24 | 1. Specify directories manually in config/config.py 25 | 26 | - Configuration options: 27 | - ./config/config.py needs to be manually specified: 28 | - GDB file path (automatically created if it doesn't exist), GDB will be automatically cleared if all steps are executed successfully 29 | - Road network data root directory (pay attention to the naming conventions of road network data) 30 | - Boundary data root directory (pay attention to the naming conventions) 31 | - Road types: 32 | - Currently, road types are selected mainly for the purpose of parcel division. The road network categories involved in the simplification are filtered based on the ROAD_TYPE_FIELD in config.py. You can add a new category in config.py for customized fine road network processing. 33 | 34 | 2. Execute main.py 35 | 36 | - The main function start_process in main.py accepts the following parameters: 37 | - CITY: City name, ensure the naming is the same as the suffix of the road network (see comments for details) 38 | - MODE: Simplification mode, must be 'mixed' 39 | - smooth_level: Level of simplification, the higher the value, the smoother the result (distortion) 40 | - extend_distance: Distance of extending dead-end roads, the higher the value, the more closed road networks (but may extend a road where it does not exist) 41 | - spike_keep: Threshold for cleaning small spikes, roads with a length lower than this threshold will be removed, the higher the value, the more regular the roads, but it may remove roads that are actually dead-ends 42 | 43 | ### Road Network Simplification Principle: 44 | 1. Extract CenterLines 45 | -  46 |47 | load raw data. 48 |
49 | -  50 |51 | build buffer for each line. 52 |
53 | -  54 |55 | dissolve the buffers. 56 |
57 | -  58 |59 | Extract centerlines from merged buffers 60 |
61 | -  62 |63 | Extraction result. 64 |
65 | 2. road optimization: 66 | -  67 |68 | Extension of non-closed roads (in blue). 69 |
70 | -  71 |72 | Extraction of dead-end roads (points). 73 |
74 | -  75 |76 | Filtering small spikes (in red) with a length below the threshold. 77 |
78 | 3. Road Network Spatial Information Re-association:： 79 | -  80 |81 | Create buffers based on the simplified road network and load the original road network data for information assignment. 82 |
83 | 84 | -  85 |86 | For each buffer, the matching rule with the original road network is to select the most representative road within the buffer (largest overlap). 87 |
88 | 89 | -  90 |91 | eg. The most representative road within this buffer is enclosed. 92 |
93 | 4. output result 94 | -  95 | 96 | ### A Hidden Parameter: 97 | There is a hidden parameter called keep_spike in the clean_spike function in simplify.py, which is not commonly used. 98 | 99 | It may be useful when plotting data related to dead-end roads. If needed, you can manually set keep_spike to True. 100 | 101 | The data for dead-end points and dead-end lines will be output to the result path, named 'err_points.shp' and 'err_lines.shp', respectively. -------------------------------------------------------------------------------- /img/micro_processed.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/micro_processed.jpg -------------------------------------------------------------------------------- /img/micro_raw.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/micro_raw.jpg -------------------------------------------------------------------------------- /img/process/attrToMas.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/attrToMas.jpg -------------------------------------------------------------------------------- /img/process/buffer.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/buffer.jpg -------------------------------------------------------------------------------- /img/process/dissolve.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/dissolve.jpg -------------------------------------------------------------------------------- /img/process/extend.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/extend.jpg -------------------------------------------------------------------------------- /img/process/extract.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/extract.jpg -------------------------------------------------------------------------------- /img/process/getinfo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/getinfo.jpg -------------------------------------------------------------------------------- /img/process/load.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/load.jpg -------------------------------------------------------------------------------- /img/process/res.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/res.jpg -------------------------------------------------------------------------------- /img/process/sjoinprinc.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/sjoinprinc.jpg -------------------------------------------------------------------------------- /img/process/sjoinprinc_2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/sjoinprinc_2.jpg -------------------------------------------------------------------------------- /img/process/spikep.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/spikep.jpg -------------------------------------------------------------------------------- /img/process/step1res.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/step1res.jpg -------------------------------------------------------------------------------- /img/process/threshold.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kingsley0107/road_regularization/0e9a67a9a9e2676738e8619aac2a7cc3986b491d/img/process/threshold.jpg -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | from startup.divided import * 3 | from startup.mixed import * 4 | 5 | 6 | 7 | def start_process(CITY:str,mode:str = 'divided',smooth_level:int = 30,extend_distance:int = 100,spike_keep:int = 500): 8 | """_summary_ 9 | 10 | Args: 11 | CITY (str): city to be processed, which is the key for finding the corresponding road data. eg. CITY: 'xiangcheng' -> OSM_PATH:'osm_xiangcheng.shp' 12 | mode : 'divided': split highway and non-highway when process the data while 'mixed' means mix them and process in one time. 13 | smooth_level (int, optional): The higher the smoother. The value that controls the level of simplification and smoothing of the road, the essence is the distance of the buffer that will be extracted centerlines. Defaults to 30. 14 | extend_distance (int, optional): extend_level . Defaults to 100. 15 | spike_keep (int, optional): Threshold to keep the spike. Defaults to 500. 16 | """ 17 | if mode not in ['divided','mixed']: 18 | raise ValueError("😅Invalid mode. Must be 'divided' or 'mixed'.😅") 19 | if mode == 'divided': 20 | print("🙁 type divided is underfixing 🙁") 21 | res = divided_start(CITY,smooth_level = smooth_level,extend_distance=extend_distance,spike_keep=spike_keep) 22 | elif mode == 'mixed': 23 | res = mixed_start(CITY,smooth_level = smooth_level,extend_distance=extend_distance,spike_keep=spike_keep) 24 | 25 | return res 26 | 27 | 28 | if __name__ == '__main__': 29 | # MODE: 'mixed' | 'divided' 30 | MODE ='mixed' 31 | for i in ['westminster']: 32 | print(f"😈😈😈😈😈 Begin process {i} 😈😈😈😈😈") 33 | print(f"😈😈😈😈😈 MODE: {MODE} 😈😈😈😈😈") 34 | start_time = time.time() 35 | output_path = start_process(i,MODE,smooth_level = 10,extend_distance=100,spike_keep=50) 36 | # output_path = start_process(i,'mixed',smooth_level = 30,extend_distance=100,spike_keep=500) 37 | print(f"🤭🤭🤭🤭🤭 {i} processed 🤭🤭🤭🤭🤭") 38 | print(f"🎉🎉🎉🎉🎉 time used: {time.time()- start_time} 🎉🎉🎉🎉🎉") 39 | print(f"🚀🚀🚀🚀🚀 CITY {i} MODE {MODE} result: {output_path} 🚀🚀🚀🚀🚀") -------------------------------------------------------------------------------- /process/common.py: -------------------------------------------------------------------------------- 1 | import arcpy 2 | from config.config import * 3 | 4 | def Add_filed(data,field_name,field_type,expression,expression_type): 5 | arcpy.AddField_management(data, field_name, field_type) 6 | arcpy.CalculateField_management(data, field_name, expression, expression_type) 7 | 8 | 9 | def gen_merge_expression(fclass): 10 | fclass = eval(fclass) 11 | if fclass.strip().lower() == 'primary': 12 | return MERGING_MAP['primary'] 13 | elif fclass.strip().lower() == 'secondary': 14 | return MERGING_MAP['secondary'] 15 | elif fclass.strip().lower() == 'tertiary': 16 | return MERGING_MAP['tertiary'] 17 | elif fclass.strip().lower() == 'trunk': 18 | return MERGING_MAP['trunk'] 19 | elif fclass.strip().lower() == 'motorway': 20 | return MERGING_MAP['motorway'] 21 | else: 22 | return 0 23 | 24 | def deleteGDBFile(): 25 | arcpy.Delete_management(FILE_ROOT+DATABASE_NAME) -------------------------------------------------------------------------------- /process/preprocess.py: -------------------------------------------------------------------------------- 1 | import arcpy 2 | import time 3 | from process.common import * 4 | from config.config import * 5 | 6 | def Clip_road(boundary, road): 7 | 8 | if arcpy.Exists(road): 9 | print(f'{road} exist') 10 | arcpy.Clip_analysis(road, boundary,f'cliped') 11 | # if source == 'osm': 12 | # fields = ['name','ref'] 13 | # cursor = arcpy.UpdateCursor(road) 14 | # for row in cursor: 15 | # if row.getValue(fields[0]) == " ": 16 | # row.setValue(fields[0],row.getValue(fields[1])) 17 | # cursor.updateRow(row) 18 | return f"cliped" 19 | else: 20 | raise ValueError(f"DATA {road} NOT EXIST!!!!") 21 | 22 | 23 | def Select_road(keep_list,road_path): 24 | if len(keep_list)> 1: 25 | expression = f"{ROAD_TYPE_FIELD} in {tuple(keep_list)}" 26 | else : 27 | expression = f"{ROAD_TYPE_FIELD} in ('{keep_list[0]}')" 28 | if arcpy.Exists(road_path): 29 | arcpy.FeatureClassToFeatureClass_conversion(road_path,out_name=f'road_selected',where_clause=expression) 30 | else: 31 | raise ValueError(f"DATA {road_path} NOT EXIST!!!!") 32 | arcpy.Delete_management(f"{road_path}") 33 | return f'road_selected' 34 | -------------------------------------------------------------------------------- /process/simplify.py: -------------------------------------------------------------------------------- 1 | import arcpy 2 | from config.config import * 3 | 4 | def Convert_single(osm_path,threshold): 5 | 6 | 7 | arcpy.Project_management(osm_path,'temp',Project_ref) 8 | arcpy.Buffer_analysis('temp','temp_buffered',f'{threshold} meters') 9 | # 注意 dissolve会导致路网属性丢失 10 | # 后续会sjoin回去 11 | arcpy.Dissolve_management('temp_buffered','temp_buffered_dissolved') 12 | print("################Extracting centerline") 13 | arcpy.PolygonToCenterline_topographic('temp_buffered_dissolved','road_singleLine_proj') 14 | arcpy.Delete_management(['temp','temp_buffered','temp_buffered_dissolved']) 15 | print("\n") 16 | # 返回值是该函数的结果,下一函数的输入,注意修改 17 | return 'road_singleLine_proj' 18 | 19 | def Extend_road(roads,distance): 20 | # start = time.time() 21 | # print("###########################Extending road") 22 | arcpy.FeatureToLine_management(roads,'extend_road') 23 | # 注意记得转投影坐标 24 | arcpy.ExtendLine_edit('extend_road',f'{distance} meters','EXTENSION') 25 | arcpy.Delete_management(roads) 26 | # print(f"###########################Road extended. Time used:{time.time() - start}") 27 | return 'extend_road' 28 | 29 | def Check_topo(roads): 30 | # print("###########################Strat checking topology") 31 | # start = time.time() 32 | dataset_name = 'topo' 33 | topology_name = 'topology' 34 | topo_data_name = 'road_data' 35 | # work tree : topo\topology 36 | arcpy.CreateFeatureDataset_management(out_name=dataset_name,spatial_reference=Project_ref) 37 | arcpy.FeatureClassToFeatureClass_conversion(roads,dataset_name,topo_data_name) 38 | arcpy.CreateTopology_management(dataset_name,topology_name) 39 | arcpy.AddFeatureClassToTopology_management(f"{dataset_name}/{topology_name}",f"{dataset_name}/{topo_data_name}") 40 | # print("adding topo rules...") 41 | # validate 断头点 42 | arcpy.AddRuleToTopology_management(f"{dataset_name}/{topology_name}","Must Not Have Dangles (Line)",f"{dataset_name}/{topo_data_name}") 43 | try: 44 | arcpy.ValidateTopology_management( 45 | "{0}/{1}".format(dataset_name, topology_name)) 46 | except: 47 | print('错误数量过大，但是并不影响进程，拓朴验证已修复路网') 48 | 49 | # 输出Topo监测出的断头点 50 | arcpy.ExportTopologyErrors_management("{0}/{1}".format(dataset_name, topology_name), 51 | out_basename='err') 52 | arcpy.Delete_management('err_line') 53 | arcpy.Delete_management('err_poly') 54 | # print(f"###########################Topology checked. Time used:{time.time()-start}") 55 | return "err_point" 56 | 57 | def Clean_spike(roads,pts,threshold,CITY,keep_spike = False): 58 | # start = time.time() 59 | # print("###########################Clean spike roads") 60 | arcpy.MakeFeatureLayer_management(roads, 'roads_copy') 61 | arcpy.SelectLayerByLocation_management('roads_copy', 62 | 'INTERSECT', 63 | pts) 64 | arcpy.CopyFeatures_management('roads_copy', 65 | "spike_roads") 66 | # 丸姐的断头路需求 这个参数一般不用因此藏起来了，需要的时候可以手动改此处。 67 | if keep_spike: 68 | arcpy.FeatureClassToFeatureClass_conversion(pts,OUTPUT_PATH,f"err_points_{CITY}.shp") 69 | arcpy.FeatureClassToFeatureClass_conversion("spike_roads",OUTPUT_PATH,f"err_lines_{CITY}.shp") 70 | spike = "spike_roads" 71 | if arcpy.Describe("spike_roads").spatialReference.factoryCode != PROJ_CRS: 72 | arcpy.Project_management("spike_roads","spike_roads2",Project_ref) 73 | spike = "spike_roads2" 74 | # LENGTH_GEODESIC 带有z值的距离 75 | arcpy.AddGeometryAttributes_management(spike, 76 | 'LENGTH_GEODESIC', 'METERS') 77 | expression = f"LENGTH_GEO < {threshold}" 78 | arcpy.FeatureClassToFeatureClass_conversion(spike,out_name='to_be_cut',where_clause=expression) 79 | arcpy.Erase_analysis(roads,'to_be_cut','road_master') 80 | arcpy.Delete_management('to_be_cut') 81 | arcpy.Delete_management('spike_roads') 82 | arcpy.Delete_management('tbc_buffer') 83 | arcpy.Delete_management('err_point') 84 | # print(f"###########################Spike roads cleaned. Time used:{time.time()-start}") 85 | return 'road_master' 86 | 87 | def Join_attributes(origin,master): 88 | # start = time.time() 89 | # print('###########################Start interrupted_and_spatial_join ...') 90 | if arcpy.Describe(origin).spatialReference.factoryCode != PROJ_CRS: 91 | arcpy.Project_management(origin,"origin_proj",Project_ref) 92 | origin = "origin_proj" 93 | 94 | arcpy.Buffer_analysis(master,'road_master_buffered',"7 meters") 95 | 96 | 97 | 98 | field_mappings = arcpy.FieldMappings() 99 | 100 | # 创建 FieldMap 对象并添加输入字段和输出字段 101 | field_map = arcpy.FieldMap() 102 | field_map.addInputField(origin, "localId") 103 | out_field = field_map.outputField 104 | out_field.name = "localId" 105 | out_field.aliasName = "localId" 106 | field_map.outputField = out_field 107 | 108 | # 将 FieldMap 对象添加到 FieldMappings 对象中 109 | field_mappings.addFieldMap(field_map) 110 | 111 | arcpy.SpatialJoin_analysis("road_master_buffered", origin, 112 | "road_master_buffered_with_attr", 113 | join_operation="JOIN_ONE_TO_ONE", match_option="LARGEST_OVERLAP",join_type='KEEP_ALL',field_mapping=field_mappings) 114 | 115 | arcpy.SpatialJoin_analysis("road_master","road_master_buffered_with_attr","road_master_with_attr","JOIN_ONE_TO_ONE",match_option="WITHIN",join_type='KEEP_ALL',field_mapping=field_mappings) 116 | 117 | 118 | return "road_master_with_attr" 119 | 120 | def Delete_fields(roads,CITY): 121 | # print("###########################Delete Fields") 122 | # start = time.time() 123 | all_fields = [f.name for f in arcpy.ListFields(roads)] 124 | 125 | fields_to_delete = [ 126 | f for f in all_fields if f not in fields_to_keep] 127 | 128 | arcpy.DeleteField_management(roads, fields_to_delete) 129 | arcpy.CopyFeatures_management(roads, "result") 130 | arcpy.Delete_management(roads) 131 | flag = "result" 132 | if arcpy.Describe("result").spatialReference.factoryCode != PROJ_CRS: 133 | arcpy.Project_management("result","result2",Project_ref) 134 | arcpy.Delete_management("result") 135 | flag = "result2" 136 | arcpy.AddGeometryAttributes_management(flag,"LENGTH_GEODESIC","METERS") 137 | # arcpy.Project_management(flag,f"road_result_{CITY}",Geographic_ref) 138 | # arcpy.Delete_management(flag) 139 | arcpy.Delete_management(['topo','extend_road','road_selected']) 140 | # arcpy.FeatureClassToFeatureClass_conversion(flag,OUTPUT_PATH,f'road_result_{CITY}.shp') 141 | # print(f"###########################Field deleted. Time used:{time.time()-start}") 142 | 143 | return f"{flag}" 144 | -------------------------------------------------------------------------------- /startup/divided.py: -------------------------------------------------------------------------------- 1 | import arcpy 2 | from config.config import * 3 | from process.preprocess import * 4 | from process.simplify import * 5 | from process.common import * 6 | from startup.schedule import process_body 7 | def start_process_divided(road_type:str,osm_fclass,qq_subtype,CITY:str,smooth_level:int = 30,extend_distance:int = 100,spike_keep:int = 500): 8 | if road_type not in ['highway', 'no_highway']: 9 | raise ValueError("Invalid road_type. Must be 'highway' or 'no_highway'.") 10 | res = process_body(osm_fclass,qq_subtype,CITY,smooth_level= smooth_level, extend_distance=extend_distance, spike_keep = spike_keep) 11 | arcpy.FeatureClassToFeatureClass_conversion(res,OUTPUT_PATH,f'road_result_{CITY}_{road_type}.shp') 12 | return f"{OUTPUT_PATH}/road_result_{CITY}_{road_type}.shp" 13 | 14 | def divided_start(i,smooth_level:int = 30,extend_distance:int = 100,spike_keep:int = 500): 15 | res1 = start_process_divided('no_highway',OSM_MAIN_FCLASS_NO_HIGH,QQ_MAIN_SUBTYPE_NO_HIGH,i,smooth_level = smooth_level,extend_distance=extend_distance,spike_keep=spike_keep) 16 | arcpy.Delete_management(f"{OUTPUT_PATH}/road_result_{i}.shp") 17 | res2 = start_process_divided('highway',OSM_MAIN_FCLASS_HIGH,QQ_MAIN_SUBTYPE_HIGH,i,smooth_level = smooth_level,extend_distance=extend_distance,spike_keep=spike_keep) 18 | arcpy.Delete_management(f"{OUTPUT_PATH}/road_result_{i}.shp") 19 | arcpy.Merge_management([res1,res2],f"{OUTPUT_PATH}/road_result_{i}_merge.shp") 20 | arcpy.FeatureToLine_management(f"{OUTPUT_PATH}/road_result_{i}_merge.shp",f"{OUTPUT_PATH}/road_result_{i}_merged.shp") 21 | arcpy.Delete_management([res1,res2,f"{OUTPUT_PATH}/road_result_{i}_merge.shp"]) 22 | arcpy.DeleteField_management(f"{OUTPUT_PATH}/road_result_{i}_merged.shp",['LENGTH_GEO']) 23 | deleteGDBFile() 24 | return f"{OUTPUT_PATH}/road_result_{i}_merged.shp" -------------------------------------------------------------------------------- /startup/mixed.py: -------------------------------------------------------------------------------- 1 | import arcpy 2 | from config.config import * 3 | from process.preprocess import * 4 | from process.simplify import * 5 | from startup.schedule import process_body 6 | from process.common import * 7 | import time 8 | def start_process_mixed(road_type:str,osm_fclass,CITY:str,smooth_level:int = 30,extend_distance:int = 100,spike_keep:int = 500): 9 | if road_type not in ['mixed']: 10 | raise ValueError("Invalid road_type. Must be 'mixed'.") 11 | res = process_body(osm_fclass,CITY,smooth_level= smooth_level, extend_distance=extend_distance, spike_keep = spike_keep) 12 | # arcpy.FeatureClassToFeatureClass_conversion(res,OUTPUT_PATH,f'road_result_{CITY}_{road_type}.shp') 13 | # arcpy.Delete_management([res]) 14 | return res 15 | 16 | def mixed_start(i,smooth_level:int = 30,extend_distance:int = 100,spike_keep:int = 500): 17 | res = start_process_mixed('mixed',OSM_MAIN_FCLASS,i,smooth_level,extend_distance,spike_keep) 18 | # deleteGDBFile() 19 | return res 20 | 21 | -------------------------------------------------------------------------------- /startup/schedule.py: -------------------------------------------------------------------------------- 1 | import arcpy 2 | from config.config import * 3 | from process.preprocess import * 4 | from process.simplify import * 5 | from process.common import * 6 | 7 | def process_body(osm_fclass,CITY:str,smooth_level:int = 30,extend_distance:int = 100,spike_keep:int = 500): 8 | # 创建GDB 9 | try: 10 | arcpy.CreateFileGDB_management(FILE_ROOT, DATABASE_NAME) 11 | except: 12 | print("gdb already exist") 13 | arcpy.env.workspace = FILE_ROOT+DATABASE_NAME 14 | arcpy.env.outputMFlag = "DISABLE_M_VALUE" # 禁用M值输出 15 | arcpy.env.overwriteOutput = True 16 | print(f"config workspace : {FILE_ROOT+DATABASE_NAME}") 17 | OSM_PATH = ROAD_DATA + f'{CITY}.shp' 18 | BOUNDARY_PATH = BOUNDARY_DATA + f'{CITY}.shp' 19 | # use arcpy to check the factory code: 20 | if not arcpy.Describe(OSM_PATH).spatialReference.factoryCode == PROJ_CRS: 21 | arcpy.Project_management(OSM_PATH,ROAD_DATA + f'{CITY}_project.shp',Project_ref) 22 | OSM_PATH = ROAD_DATA + f'{CITY}_project.shp' 23 | if not arcpy.Describe(BOUNDARY_PATH).spatialReference.factoryCode == PROJ_CRS: 24 | arcpy.Project_management(BOUNDARY_PATH,BOUNDARY_DATA + f'{CITY}_project.shp',Project_ref) 25 | BOUNDARY_PATH = BOUNDARY_DATA + f'{CITY}_project.shp' 26 | 27 | osm_cliped_path = Clip_road(BOUNDARY_PATH,OSM_PATH) 28 | select_osm = Select_road(osm_fclass,osm_cliped_path) 29 | raw_road = select_osm 30 | # convert to : 3857 31 | singlelines = Convert_single(select_osm,smooth_level) 32 | print(f"Convert_single Finish:{singlelines}") 33 | extendedlines = Extend_road(singlelines,extend_distance) 34 | print("Extend roads Finish") 35 | err_point = Check_topo(extendedlines) 36 | print("Check Topo Finish") 37 | master_road = Clean_spike(extendedlines,err_point,spike_keep,CITY=CITY,keep_spike=False) 38 | print("Clean Spike Finish") 39 | master_road = Join_attributes(raw_road,master_road) 40 | # conver to : 4326 41 | res = Delete_fields(master_road,CITY) 42 | return res 43 | -------------------------------------------------------------------------------- /zh-cn.md: -------------------------------------------------------------------------------- 1 | 中文版 [English](./README.md) 2 | # road_geo_regularization 3 | ### 通用的城市主要道路路网简化，可用于非高精度的地块划分等 4 | 5 | ### Performance: 6 | #### 原始路网 7 | -  8 | #### 路网简化结果 9 | -  10 | 11 | 12 | ### 运行环境: 13 | - Arcgis Pro 14 | - Python 3.x from Arcgis Pro 15 | - Arcpy from Arcgis Pro 16 | - Encoding utf-8_sig 17 | 18 | ### 所需数据： 19 | - 路网数据 20 | - boundary边界数据 21 | - 数据需要事先按照config.py路径放置，注意命名格式 22 | 23 | ### 说明书： 24 | 1. 先在config/config.py里手动指定目录 25 | - 配置项： 26 | - ./config/config.py 需手动指定： 27 | - GDB文件路径(不存在会自动创建),所有步骤成功执行会自动清空GDB 28 | - 路网数据根目录(注意路网数据命名规则) 29 | - boundary数据根目录(注意命名规则) 30 | - 道路类别： 31 | - 目前道路类型选择主要以划分地块为目的，参与简化的路网类别根据config.py中的ROAD_TYPE_FIELD进行筛选，后续定制化精细路网处理时在config.py添加新类即可 32 | 33 | 2. 执行main.py 34 | - main.py启动函数start_process主要参数： 35 | - CITY: 城市名，注意命名需要与路网后缀相同(详见注释) 36 | - MODE：简化方式，必须为'mixed' 37 | - smooth_level： 简化程度，值越高结果越平滑(失真) 38 | - extend_distance： 断头路延伸的距离，值越高，闭合的路网越多，(但会在不存在道路的地方伸出一条道路) 39 | - spike_keep： 清理毛细断头道路的阈值，道路长度低于这个threshold的都会被清除掉，值越高，道路越规整，但会使一些现实是断头路的道路消失 40 | 41 | ### 路网简化原理： 42 | 1. Extract CenterLines 43 | -  44 |45 | 加载原始数据. 46 |
47 | -  48 |49 | 对每条道路建立缓冲区. 50 |
51 | -  52 |53 | 将重叠缓冲区合并(dissolve). 54 |
55 | -  56 |57 | 提取合并后缓冲区的中心线. 58 |
59 | -  60 |61 | 提取结果. 62 |
63 | 2. 路网优化: 64 | -  65 |66 | 非闭合道路延展(蓝色). 67 |
68 | -  69 |70 | 提取断头路(points). 71 |
72 | -  73 |74 | 过滤长度低于threshold的毛刺道路(红色部分). 75 |
76 | 3. 路网空间信息重关联： 77 | -  78 |79 | 以简化后的道路建立缓冲区，加载原属路网数据准备进行信息赋值. 80 |
81 | 82 | -  83 |84 | 对于某个缓冲区，匹配原路网的规则为选取缓冲区内最具代表性的道路(largest overlap). 85 |
86 | 87 | -  88 |89 | eg.该缓冲区内最具代表性道路为圈出道路. 90 |
91 | 4. 输出结果 92 | -  93 | 94 | 95 | ### 一个隐藏参数： 96 | 在simplify.py的clean_spike函数中有一个keep_spike参数被隐藏起来了，不太常用。 97 | 98 | 可能会在作图时需要用到断头路数据，若需要可手动将keep_spike改为True. 99 | 100 | 断头点、断头线数据将会输出到结果路径中，分别命名'err_points.shp'及'err_lines.shp' --------------------------------------------------------------------------------