├── README.md
└── sketch_synthesis
    ├── __pycache__
        └── components.cpython-310.pyc
    ├── components.py
    ├── input_sketch_json
        ├── P004_0_0_GCN_Armor_cat.json
        └── P021_8_1_SAH_210.json
    ├── model
        ├── N_mlp_extrinsic.sav
        ├── N_mlp_intrinsic.sav
        ├── P_mlp_extrinsic.sav
        ├── P_mlp_intrinsic.sav
        └── mlp_curveNoise.sav
    ├── prepare_data
        ├── getCurveNoiseData.py
        ├── getExtrinsicData.py
        └── getIntrinsicData.py
    ├── requirements.txt
    ├── results
        ├── N
        │   └── P021_8_1_SAH_210_in=0.30_ex=0.30.png
        └── P
        │   └── P021_8_1_SAH_210_in=0.30_ex=0.30.png
    ├── test.py
    ├── train_curveNoise.py
    ├── train_extrinsic.py
    └── train_intrinsic.py


/README.md:
--------------------------------------------------------------------------------
 1 | # DifferSketching: How Differently Do People Sketch 3D Objects?
 2 | > [Chufeng Xiao](https://scholar.google.com/citations?user=2HLwZGYAAAAJ&hl=en), [Wanchao Su](https://ansire.github.io/), [Jing Liao](https://liaojing.github.io/html/), [Zhouhui Lian](https://www.icst.pku.edu.cn/zlian/), [Yi-Zhe Song](http://personal.ee.surrey.ac.uk/Personal/Y.Song/), [Hongbo Fu](https://sweb.cityu.edu.hk/hongbofu/)
 3 | > 
 4 | > [[Project Page]](https://chufengxiao.github.io/DifferSketching/) [[Paper]](https://arxiv.org/abs/2209.08791) [[Dataset]](https://chufengxiao.github.io/DifferSketching/#dataset) [[Supplemental Material]](https://github.com/chufengxiao/DifferSketching/tree/project-page/Supplemental_Material)
 5 | >
 6 | > Accepted by [SIGGRAPH Asia 2022](https://sa2022.siggraph.org/) (Journal Track)
 7 | 
 8 | ## To-do List
 9 |   - [x] [Dataset](https://chufengxiao.github.io/DifferSketching/#dataset)
10 |   - [ ] Multi-level Registration Method
11 |   - [x] Freehand-style Sketch Synthesis
12 |     - [x] Pre-trained models and inference code
13 |     - [x] Training code and training dataset
14 | 
15 | ## Freehand-style Sketch Synthesis
16 | ### Quick Test
17 | Please run the below commands to visualize the pipeline of our method for sketch synthesis. The pre-trained models of our method are located at the directory `./sketch_synthesis/model/`. There are two examples in the directory `./sketch_synthesis/input_sketch_json/` for testing, and you can also pick up other data from `<category>/reg_json/` under the release dataset directory. The visualization result will be save in the directory `./sketch_synthesis/results/`.
18 | 
19 | ```bash
20 | cd ./sketch_synthesis
21 | pip install -r requirements.txt
22 | python test.py
23 | ```
24 | 
25 | 
26 | 
27 | ### Training Dataset for Sketch Synthesis
28 | 
29 | Please download the latest version of our DifferSketching Dataset (updated in 8 May 2025) via [Google Drive](https://drive.google.com/file/d/1A_3RVc8Y4YdI7nhyM7tb-q7dQw4zTcCO/view) and put it at the root directory. Run the below commands to prepare data for training three MLP disturbers:
30 | ```bash
31 | cd ./sketch_synthesis
32 | pip install -r requirements.txt
33 | 
34 | # The sketch dataset should be located at root_dir="../DifferSketching_Dataset"
35 | 
36 | python ./prepare_data/getExtrinsicData.py # data for training extrinsic disturber
37 | python ./prepare_data/getIntrinsicData.py # data for training intrinsic disturber
38 | python ./prepare_data/getCurveNoiseData.py # data for training point disturber
39 | 
40 | # The extracted training data will be save at ./data
41 | ```
42 | Please check the codes to switch the dataset from novices or professionals via the variable `NP='N'` or `NP='P'`. 
43 | 
44 | ### Training
45 | You can train three MLP disturbers using the corresponding data via the below commands:
46 | ```bash
47 | cd ./sketch_synthesis
48 | 
49 | python train_extrinsic.py # train extrinsic disturber
50 | python train_intrinsic.py # train intrinsic disturber
51 | python train_curveNoise.py # train point disturber
52 | 
53 | # The trained models will be saved at ./train_models. Please check more details in the codes.
54 | ```
55 | 
56 | ## Citation
57 | If you find this repository useful in your project, please give us a star and cite the following work. Thanks :)
58 | ```
59 | @article{xiao2022differsketching,
60 |   title={Differsketching: How differently do people sketch 3d objects?},
61 |   author={Xiao, Chufeng and Su, Wanchao and Liao, Jing and Lian, Zhouhui and Song, Yi-Zhe and Fu, Hongbo},
62 |   journal={ACM Transactions on Graphics (TOG)},
63 |   volume={41},
64 |   number={6},
65 |   pages={1--16},
66 |   year={2022},
67 |   publisher={ACM New York, NY, USA}
68 | }
69 | ```
70 | 


--------------------------------------------------------------------------------
/sketch_synthesis/__pycache__/components.cpython-310.pyc:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chufengxiao/DifferSketching/305ce2d66a8fd63d325145a90a9378591e2d754f/sketch_synthesis/__pycache__/components.cpython-310.pyc


--------------------------------------------------------------------------------
/sketch_synthesis/components.py:
--------------------------------------------------------------------------------
  1 | import numpy as np
  2 | import json,cv2
  3 | 
  4 | from scipy.special import comb
  5 | import matplotlib.pyplot as plt
  6 | 
  7 | 
  8 | def readStrokes(path):
  9 |     file = open(path,"r")
 10 |     json_str = file.read()
 11 |     strokes = json.loads(json_str)
 12 |     return strokes
 13 | 
 14 | def drawPath(gt=None,pred=None,input=None):
 15 |     def draw(canvas,path,color):
 16 |         path = path.astype('int')
 17 |         for i in range(len(path)-1):
 18 |             cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),color,2)
 19 |         return canvas
 20 |     canvas = np.ones((800,800,3),dtype="uint8")*255
 21 |     if input is not None:
 22 |         canvas = draw(canvas,input,0)
 23 |     if gt is not None:
 24 |         canvas = draw(canvas,gt,(0,255,0))
 25 |     if pred is not None:
 26 |         canvas = draw(canvas,pred,(255,0,0))
 27 |     
 28 |     plt.imshow(canvas)
 29 |     plt.show()
 30 |     return canvas
 31 | 
 32 | def bezier_curve(points, nTimes=500):
 33 | 
 34 |     def bernstein_poly(i, n, t):
 35 |         """
 36 |         The Bernstein polynomial of n, i as a function of t
 37 |         """
 38 |         return comb(n, i) * ( t**(n-i) ) * (1 - t)**i
 39 |     points = points.reshape(-1,2)
 40 |     nPoints = len(points)
 41 |     xPoints = np.array([p[0] for p in points])
 42 |     yPoints = np.array([p[1] for p in points])
 43 | 
 44 |     t = np.linspace(0.0, 1.0, nTimes)
 45 | 
 46 |     polynomial_array = np.array([ bernstein_poly(i, nPoints-1, t) for i in range(0, nPoints)   ])
 47 | 
 48 |     xvals = np.dot(xPoints, polynomial_array)
 49 |     yvals = np.dot(yPoints, polynomial_array)
 50 |     path = np.array([xvals,yvals]).T.astype('int')
 51 | 
 52 |     return path
 53 | 
 54 | def writeJSON(json_path,data):
 55 |     with open(json_path,'w+') as f:
 56 |         f.write(json.dumps(data))
 57 | 
 58 | def get_bezier_parameters(X, Y, degree=3):
 59 |     if degree < 1:
 60 |         raise ValueError('degree must be 1 or greater.')
 61 | 
 62 |     if len(X) != len(Y):
 63 |         raise ValueError('X and Y must be of the same length.')
 64 | 
 65 |     if len(X) < degree + 1:
 66 |         raise ValueError(f'There must be at least {degree + 1} points to '
 67 |                          f'determine the parameters of a degree {degree} curve. '
 68 |                          f'Got only {len(X)} points.')
 69 | 
 70 |     def bpoly(n, t, k):
 71 |         """ Bernstein polynomial when a = 0 and b = 1. """
 72 |         return t ** k * (1 - t) ** (n - k) * comb(n, k)
 73 |         #return comb(n, i) * ( t**(n-i) ) * (1 - t)**i
 74 | 
 75 |     def bmatrix(T):
 76 |         """ Bernstein matrix for Bézier curves. """
 77 |         return np.matrix([[bpoly(degree, t, k) for k in range(degree + 1)] for t in T])
 78 | 
 79 |     def least_square_fit(points, M):
 80 |         M_ = np.linalg.pinv(M)
 81 |         return M_ * points
 82 | 
 83 |     T = np.linspace(0, 1, len(X))
 84 |     M = bmatrix(T)
 85 |     points = np.array(list(zip(X, Y)))
 86 |     
 87 |     final = least_square_fit(points, M).tolist()
 88 |     final[0] = [X[0], Y[0]]
 89 |     final[len(final)-1] = [X[len(X)-1], Y[len(Y)-1]]
 90 |     return np.array(final)
 91 | 
 92 | def getStrokeBezier(control_num,srcStrokes):
 93 |     
 94 |     in_curves = []
 95 | 
 96 |     for i,stroke in enumerate(srcStrokes['strokes']):
 97 |         if 'draw_type' in stroke and stroke['draw_type'] == 1:
 98 |             continue
 99 |         src_p = np.array(stroke['path'])
100 |         if len(src_p) < control_num:
101 |             continue
102 | 
103 |         inputs = get_bezier_parameters(src_p[:,0], src_p[:,1], degree=control_num-1)
104 |         inputs = np.round(inputs).astype('int').flatten()/799
105 |         
106 |         in_curves.append(inputs)
107 |     return np.array(in_curves)
108 | 
109 | def renderSketch(strokes):
110 |     canvas = np.ones((800,800),dtype="uint8")*255
111 | 
112 |     for stroke in strokes['strokes']:
113 |             
114 |         if 'draw_type' in stroke and stroke['draw_type'] == 1:
115 |             continue
116 | 
117 |         path = stroke['path']
118 |         path = np.array(path).astype("int")
119 |         for i in range(len(path)-1):
120 |             cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),0,3)
121 | 
122 |     return canvas
123 | 
124 | def saveJson(py_data,save_path):
125 |     json_str = json.dumps(py_data)
126 |     file = open(save_path,"w+")
127 |     file.write(json_str)
128 |     file.close()
129 | 


--------------------------------------------------------------------------------
/sketch_synthesis/input_sketch_json/P021_8_1_SAH_210.json:
--------------------------------------------------------------------------------
1 | {"strokes": [{"id": 1, "draw_type": 0, "path": [[287, 216], [287, 216], [287, 215], [287, 215], [287, 215], [287, 215], [287, 215], [287, 215], [286, 215], [286, 215], [286, 217], [286, 215], [285, 220], [284, 221], [283, 219], [282, 223], [279, 223], [278, 224], [276, 227], [274, 228], [271, 230], [270, 232], [268, 235], [267, 236], [266, 239], [266, 241], [265, 243], [264, 246], [262, 250], [262, 251], [261, 252], [261, 254], [260, 257], [260, 259], [258, 261], [257, 264], [256, 267], [255, 269], [255, 271], [253, 274], [252, 276], [250, 279], [250, 283], [249, 285], [248, 287], [247, 289], [245, 293], [244, 296], [244, 298], [243, 300], [243, 303], [243, 305], [242, 306], [242, 307], [240, 311], [239, 312], [239, 313], [237, 315], [236, 317], [234, 320], [233, 321], [232, 323], [232, 323], [232, 323], [232, 324], [231, 325], [230, 326], [231, 326], [230, 328], [229, 331], [222, 338], [222, 341], [220, 343], [219, 344], [217, 346], [216, 347], [215, 349], [214, 351], [212, 351], [213, 354], [211, 354], [210, 356], [209, 358], [210, 360], [205, 362], [204, 364], [203, 366], [200, 370], [197, 373], [196, 375]], "use_pressure": true, "pressure": [0.094, 0.131, 0.14, 0.156, 0.175, 0.19, 0.207, 0.221, 0.233, 0.243, 0.254, 0.259, 0.268, 0.277, 0.284, 0.289, 0.292, 0.295, 0.296, 0.296, 0.297, 0.297, 0.297, 0.297, 0.297, 0.297, 0.297, 0.297, 0.297, 0.297, 0.297, 0.299, 0.302, 0.306, 0.308, 0.311, 0.312, 0.314, 0.315, 0.315, 0.316, 0.316, 0.316, 0.318, 0.319, 0.319, 0.32, 0.32, 0.32, 0.322, 0.325, 0.331, 0.338, 0.348, 0.356, 0.365, 0.371, 0.376, 0.378, 0.381, 0.382, 0.382, 0.383, 0.385, 0.39, 0.398, 0.401, 0.402, 0.402, 0.402, 0.402, 0.404, 0.405, 0.408, 0.409, 0.409, 0.41, 0.41, 0.41, 0.41, 0.41, 0.41, 0.41, 0.41, 0.401, 0.391, 0.33, 0.248, 0.208, 0.188], "s_time": 1638002978294, "e_time": 1638002978964, "width": 4}, {"id": 2, "draw_type": 0, "path": [[189, 384], [189, 385], [188, 387], [188, 388], [186, 390], [186, 390], [185, 391], [184, 394], [183, 395], [182, 396], [181, 398], [181, 398], [181, 401], [180, 402], [181, 403], [180, 405], [179, 409], [178, 410], [178, 411], [176, 415], [176, 416], [175, 419], [175, 421], [175, 423], [174, 426], [174, 427], [173, 430], [174, 431], [174, 433], [174, 434], [174, 435], [174, 438], [176, 442], [176, 444], [177, 446], [180, 448], [181, 451], [183, 453], [183, 456], [185, 457], [185, 458], [187, 461], [189, 463], [192, 466], [195, 469], [196, 471], [198, 473], [200, 474], [202, 477], [203, 479], [206, 482], [207, 485], [210, 489], [213, 492], [214, 494], [216, 496], [218, 498], [220, 501], [224, 503], [223, 506], [227, 508], [228, 512], [228, 514], [232, 517], [235, 524], [239, 527], [242, 528], [243, 531]], "use_pressure": true, "pressure": [0.094, 0.131, 0.14, 0.156, 0.175, 0.19, 0.207, 0.223, 0.241, 0.253, 0.266, 0.271, 0.276, 0.283, 0.289, 0.294, 0.3, 0.307, 0.314, 0.324, 0.331, 0.336, 0.339, 0.342, 0.345, 0.349, 0.351, 0.351, 0.352, 0.354, 0.354, 0.354, 0.355, 0.357, 0.358, 0.358, 0.359, 0.361, 0.36, 0.359, 0.359, 0.361, 0.36, 0.359, 0.359, 0.359, 0.359, 0.359, 0.359, 0.361, 0.36, 0.359, 0.359, 0.361, 0.36, 0.359, 0.357, 0.355, 0.355, 0.355, 0.355, 0.355, 0.351, 0.345, 0.322, 0.295, 0.281, 0.274], "s_time": 1638002979505, "e_time": 1638002980009, "width": 4}, {"id": 4, "draw_type": 0, "path": [[248, 537], [248, 537], [248, 537], [248, 536], [248, 536], [248, 536], [249, 538], [249, 538], [249, 538], [249, 539], [250, 540], [250, 540], [252, 542], [252, 543], [253, 543], [253, 544], [255, 546], [256, 547], [256, 549], [258, 551], [259, 553], [260, 555], [262, 556], [263, 556], [267, 562], [269, 564], [270, 566], [272, 568], [273, 570], [275, 573], [275, 572], [275, 573], [279, 576], [279, 577], [281, 579], [283, 581], [283, 581], [285, 582], [287, 582], [288, 583], [290, 583], [294, 583], [299, 583], [302, 583], [303, 583], [306, 580], [309, 577], [311, 574], [313, 570], [315, 565], [317, 561], [318, 558], [321, 553], [322, 548], [325, 542], [329, 534], [333, 528], [335, 525]], "use_pressure": true, "pressure": [0.121, 0.169, 0.18, 0.191, 0.204, 0.214, 0.228, 0.243, 0.261, 0.269, 0.283, 0.302, 0.319, 0.338, 0.353, 0.368, 0.379, 0.391, 0.394, 0.395, 0.393, 0.391, 0.393, 0.397, 0.4, 0.401, 0.402, 0.404, 0.405, 0.405, 0.406, 0.406, 0.408, 0.411, 0.415, 0.419, 0.423, 0.429, 0.432, 0.433, 0.436, 0.438, 0.44, 0.44, 0.443, 0.446, 0.45, 0.454, 0.456, 0.459, 0.458, 0.457, 0.45, 0.443, 0.42, 0.39, 0.376, 0.368], "s_time": 1638002984470, "e_time": 1638002984907, "width": 4}, {"id": 5, "draw_type": 0, "path": [[261, 413], [260, 415], [260, 415], [260, 416], [259, 417], [260, 418], [260, 419], [260, 422], [263, 424], [263, 426], [262, 428], [263, 430], [263, 431], [263, 434], [264, 435], [264, 438], [266, 442], [267, 445], [269, 447], [270, 450], [271, 454], [272, 455], [273, 459], [273, 462], [275, 465], [277, 474], [278, 476], [279, 478]], "use_pressure": true, "pressure": [0.098, 0.135, 0.144, 0.16, 0.179, 0.188, 0.195, 0.198, 0.198, 0.199, 0.199, 0.199, 0.201, 0.204, 0.208, 0.212, 0.218, 0.225, 0.23, 0.235, 0.237, 0.24, 0.239, 0.236, 0.229, 0.221, 0.215, 0.212], "s_time": 1638002986335, "e_time": 1638002986524, "width": 4}, {"id": 6, "draw_type": 0, "path": [[280, 482], [281, 484], [283, 484], [283, 487], [286, 487], [286, 488], [287, 491], [290, 492], [294, 494], [295, 496], [299, 500], [300, 501], [302, 502], [308, 504], [310, 506], [313, 507], [317, 509], [320, 509], [323, 510], [325, 512], [330, 514], [331, 513], [334, 514], [340, 517], [341, 516], [343, 516], [345, 516], [347, 517], [350, 517], [353, 516], [356, 515], [360, 514], [364, 513], [368, 512], [373, 511], [376, 511], [380, 511], [383, 511], [388, 511], [392, 511], [394, 511], [400, 513], [405, 513], [410, 515], [414, 515], [420, 517], [423, 517], [428, 519], [431, 521], [435, 523], [439, 525], [441, 526], [445, 528], [449, 529], [451, 531], [455, 533], [459, 534], [463, 535], [464, 536], [469, 538], [476, 539], [474, 541], [483, 542], [489, 542], [492, 542], [490, 547], [493, 551], [494, 554], [497, 558], [501, 560], [503, 564], [506, 568], [509, 574], [512, 577], [512, 580], [515, 582]], "use_pressure": true, "pressure": [0.125, 0.173, 0.184, 0.204, 0.229, 0.254, 0.281, 0.298, 0.31, 0.314, 0.317, 0.321, 0.325, 0.331, 0.336, 0.343, 0.348, 0.353, 0.354, 0.357, 0.36, 0.364, 0.368, 0.372, 0.38, 0.393, 0.4, 0.408, 0.414, 0.421, 0.426, 0.431, 0.433, 0.436, 0.437, 0.439, 0.442, 0.446, 0.448, 0.451, 0.454, 0.46, 0.465, 0.47, 0.472, 0.475, 0.476, 0.476, 0.477, 0.477, 0.474, 0.473, 0.475, 0.479, 0.484, 0.491, 0.496, 0.501, 0.503, 0.503, 0.504, 0.504, 0.504, 0.506, 0.507, 0.507, 0.508, 0.508, 0.505, 0.502, 0.485, 0.463, 0.402, 0.325, 0.288, 0.27], "s_time": 1638002987686, "e_time": 1638002988253, "width": 4}, {"id": 7, "draw_type": 0, "path": [[516, 587], [517, 589], [517, 589], [518, 593], [516, 589], [519, 596], [519, 596], [519, 596], [520, 599], [522, 600], [523, 602], [525, 606], [527, 608], [530, 610], [529, 610], [533, 612], [535, 613], [537, 614], [541, 613], [543, 615], [544, 615], [546, 615], [548, 616], [553, 617], [554, 617], [558, 616], [560, 616], [563, 615], [566, 616], [568, 615], [569, 616], [570, 615], [573, 617], [575, 617], [580, 616]], "use_pressure": true, "pressure": [0.176, 0.242, 0.258, 0.281, 0.307, 0.323, 0.341, 0.35, 0.357, 0.361, 0.364, 0.37, 0.375, 0.382, 0.387, 0.394, 0.4, 0.41, 0.419, 0.428, 0.436, 0.443, 0.449, 0.454, 0.456, 0.459, 0.458, 0.457, 0.455, 0.451, 0.435, 0.412, 0.334, 0.234, 0.186], "s_time": 1638002989314, "e_time": 1638002989576, "width": 4}, {"id": 8, "draw_type": 0, "path": [[523, 535], [523, 534], [523, 534], [523, 537], [524, 540], [524, 543], [524, 544], [525, 547], [525, 549], [526, 555], [527, 558], [528, 561], [529, 564], [532, 568], [532, 569], [535, 572], [536, 574], [538, 575], [539, 578], [542, 582], [543, 584], [545, 586], [547, 589], [550, 591], [553, 596], [555, 598], [558, 600], [559, 601], [560, 601], [560, 602], [563, 604], [563, 604], [566, 605], [566, 605], [569, 607], [570, 607], [572, 608], [573, 608], [576, 609], [577, 609], [579, 612], [581, 612], [582, 611], [584, 615], [585, 612], [585, 617], [588, 616], [591, 615], [590, 617], [593, 615], [596, 615], [597, 616], [600, 615], [601, 615], [602, 615], [603, 614], [605, 615], [606, 614], [606, 614], [607, 615]], "use_pressure": true, "pressure": [0.105, 0.148, 0.157, 0.182, 0.21, 0.237, 0.268, 0.287, 0.305, 0.312, 0.319, 0.326, 0.332, 0.337, 0.343, 0.35, 0.355, 0.36, 0.362, 0.362, 0.363, 0.365, 0.368, 0.374, 0.379, 0.386, 0.391, 0.397, 0.4, 0.403, 0.405, 0.408, 0.409, 0.409, 0.41, 0.412, 0.413, 0.416, 0.415, 0.414, 0.412, 0.41, 0.41, 0.41, 0.41, 0.412, 0.413, 0.416, 0.417, 0.417, 0.413, 0.409, 0.392, 0.371, 0.338, 0.301, 0.257, 0.214, 0.192, 0.183], "s_time": 1638002990609, "e_time": 1638002991059, "width": 4}, {"id": 9, "draw_type": 0, "path": [[540, 470], [537, 473], [537, 473], [537, 476], [536, 476], [537, 479], [537, 481], [537, 485], [538, 488], [538, 492], [539, 497], [540, 499], [541, 502], [543, 506], [543, 509], [546, 516], [549, 522], [554, 535]], "use_pressure": true, "pressure": [0.156, 0.215, 0.228, 0.246, 0.265, 0.283, 0.305, 0.318, 0.331, 0.337, 0.341, 0.345, 0.347, 0.347, 0.311, 0.258, 0.229, 0.215], "s_time": 1638002992414, "e_time": 1638002992549, "width": 4}, {"id": 10, "draw_type": 0, "path": [[559, 489], [562, 485], [562, 485], [562, 485], [563, 485], [563, 485], [563, 483], [563, 483], [563, 483], [566, 483], [566, 483], [566, 483], [567, 483], [565, 485], [567, 485], [568, 484], [568, 484], [568, 484], [568, 485], [569, 485], [569, 485], [569, 487], [569, 487], [568, 487], [569, 488], [569, 488], [568, 490], [568, 490], [568, 491], [569, 492], [569, 494], [569, 494], [569, 494], [569, 497], [568, 498], [569, 499], [570, 501], [570, 503], [569, 506], [568, 507], [569, 508], [568, 509], [569, 510], [569, 510], [569, 510], [568, 510], [568, 510]], "use_pressure": true, "pressure": [0.133, 0.184, 0.194, 0.207, 0.22, 0.224, 0.228, 0.229, 0.232, 0.233, 0.236, 0.239, 0.247, 0.258, 0.267, 0.275, 0.279, 0.282, 0.286, 0.292, 0.299, 0.309, 0.317, 0.328, 0.337, 0.346, 0.352, 0.358, 0.363, 0.368, 0.372, 0.376, 0.38, 0.386, 0.391, 0.396, 0.397, 0.4, 0.399, 0.396, 0.389, 0.379, 0.354, 0.326, 0.243, 0.14, 0.089], "s_time": 1638002992894, "e_time": 1638002993251, "width": 4}, {"id": 11, "draw_type": 0, "path": [[566, 482], [563, 482], [563, 482], [563, 483], [563, 483], [563, 483], [563, 483], [562, 485], [562, 485], [559, 489], [559, 489], [558, 489], [557, 490], [557, 490], [558, 491], [557, 492], [556, 492], [556, 492], [557, 493], [556, 495], [557, 495], [556, 497], [558, 497], [558, 497], [560, 500], [561, 504], [561, 506], [563, 508], [564, 510], [564, 511], [567, 512], [568, 514], [570, 515]], "use_pressure": true, "pressure": [0.137, 0.19, 0.202, 0.217, 0.23, 0.246, 0.264, 0.278, 0.294, 0.301, 0.31, 0.32, 0.327, 0.335, 0.34, 0.347, 0.352, 0.356, 0.358, 0.361, 0.364, 0.368, 0.372, 0.378, 0.383, 0.388, 0.39, 0.393, 0.394, 0.394, 0.295, 0.151, 0.08], "s_time": 1638002993728, "e_time": 1638002993974, "width": 4}, {"id": 12, "draw_type": 0, "path": [[628, 445], [628, 445], [628, 445], [627, 445], [627, 445], [627, 445], [629, 447], [631, 450], [631, 450], [632, 451], [632, 452], [633, 453], [635, 455], [637, 459], [639, 460], [639, 462], [640, 464], [641, 466], [642, 467], [643, 468], [645, 471], [645, 474], [645, 475]], "use_pressure": true, "pressure": [0.09, 0.124, 0.132, 0.148, 0.169, 0.188, 0.208, 0.226, 0.244, 0.257, 0.267, 0.271, 0.272, 0.275, 0.276, 0.279, 0.278, 0.277, 0.267, 0.253, 0.216, 0.169, 0.146], "s_time": 1638002995062, "e_time": 1638002995235, "width": 4}, {"id": 13, "draw_type": 0, "path": [[646, 494], [646, 494], [646, 495], [646, 495], [646, 495], [646, 495], [646, 496], [646, 496], [646, 497], [646, 497], [646, 497], [647, 498], [647, 498], [647, 498], [647, 498], [647, 499], [647, 499], [647, 500], [647, 500], [647, 501], [648, 502], [648, 502], [648, 504], [648, 505], [648, 505], [648, 505], [648, 507], [648, 509], [648, 510], [649, 511], [649, 511], [649, 512], [649, 513], [650, 514], [650, 515], [651, 516], [651, 518], [651, 518], [651, 518], [652, 521], [652, 521], [652, 522], [653, 524], [653, 525], [654, 526], [653, 526], [653, 527], [654, 528], [654, 528], [653, 531], [654, 531], [653, 533], [653, 534], [653, 535], [653, 535], [652, 536], [652, 536], [651, 538], [651, 539], [650, 541], [650, 542], [650, 543], [649, 544], [649, 544], [649, 544], [649, 544], [649, 544], [649, 546], [649, 546], [647, 548], [649, 546], [649, 546], [647, 549]], "use_pressure": true, "pressure": [0.105, 0.148, 0.157, 0.169, 0.181, 0.188, 0.199, 0.208, 0.217, 0.225, 0.232, 0.236, 0.244, 0.254, 0.263, 0.271, 0.277, 0.282, 0.286, 0.29, 0.294, 0.298, 0.304, 0.311, 0.316, 0.323, 0.328, 0.333, 0.335, 0.335, 0.336, 0.336, 0.338, 0.343, 0.348, 0.353, 0.356, 0.36, 0.362, 0.365, 0.366, 0.366, 0.367, 0.369, 0.37, 0.373, 0.374, 0.377, 0.378, 0.378, 0.379, 0.381, 0.382, 0.385, 0.384, 0.383, 0.383, 0.383, 0.383, 0.383, 0.38, 0.379, 0.374, 0.37, 0.366, 0.361, 0.35, 0.333, 0.304, 0.269, 0.213, 0.151, 0.122], "s_time": 1638002995614, "e_time": 1638002996163, "width": 4}, {"id": 14, "draw_type": 0, "path": [[647, 549], [647, 549], [647, 549], [647, 549], [647, 549], [647, 549], [647, 549], [647, 549], [647, 549], [647, 549], [647, 549], [647, 549], [646, 552], [646, 552], [646, 552], [646, 552], [647, 550], [646, 553], [646, 553], [646, 553], [646, 553], [644, 555], [645, 555], [645, 557], [645, 557], [644, 557], [644, 557], [644, 558], [644, 559], [644, 559], [643, 560], [643, 562], [643, 562], [642, 563], [642, 564], [640, 567], [641, 567], [641, 569], [640, 570], [639, 571], [638, 576], [638, 578], [638, 577], [636, 580], [637, 580], [637, 582], [637, 584], [636, 585], [635, 586], [635, 587], [635, 589], [634, 589], [633, 591], [633, 591], [633, 592], [632, 594], [632, 596], [630, 598], [630, 599], [630, 599], [630, 601], [630, 601], [629, 602], [628, 603], [628, 603], [627, 604], [627, 604], [623, 605], [623, 606], [623, 607], [623, 608], [623, 608], [622, 609], [622, 609], [620, 610], [620, 610], [620, 610], [620, 610], [620, 610], [620, 610], [620, 610], [619, 611], [619, 611], [619, 611], [619, 611], [619, 611], [616, 612], [616, 612], [616, 612], [615, 611], [615, 611], [615, 611], [614, 612], [614, 613], [611, 614], [611, 614], [610, 614], [608, 615], [608, 615], [608, 615], [608, 615], [607, 614]], "use_pressure": true, "pressure": [0.09, 0.124, 0.132, 0.145, 0.156, 0.167, 0.181, 0.193, 0.208, 0.221, 0.233, 0.241, 0.249, 0.252, 0.26, 0.27, 0.28, 0.294, 0.31, 0.324, 0.336, 0.346, 0.35, 0.353, 0.354, 0.357, 0.358, 0.361, 0.366, 0.375, 0.386, 0.397, 0.409, 0.421, 0.431, 0.439, 0.443, 0.444, 0.445, 0.445, 0.447, 0.45, 0.456, 0.465, 0.47, 0.474, 0.478, 0.481, 0.485, 0.491, 0.494, 0.497, 0.501, 0.507, 0.51, 0.511, 0.514, 0.517, 0.519, 0.519, 0.52, 0.52, 0.517, 0.516, 0.516, 0.516, 0.516, 0.518, 0.519, 0.519, 0.517, 0.516, 0.511, 0.507, 0.496, 0.484, 0.471, 0.456, 0.446, 0.442, 0.441, 0.441, 0.441, 0.441, 0.438, 0.438, 0.439, 0.442, 0.444, 0.444, 0.447, 0.45, 0.454, 0.46, 0.474, 0.49, 0.503, 0.513, 0.493, 0.461, 0.444, 0.436], "s_time": 1638002996192, "e_time": 1638002996945, "width": 4}, {"id": 15, "draw_type": 0, "path": [[645, 475], [645, 476], [645, 476], [645, 476], [645, 476], [645, 476], [645, 476], [645, 476], [646, 477], [646, 477], [646, 477], [646, 480], [645, 481], [646, 482], [646, 483], [646, 484], [646, 485], [646, 486], [645, 487], [646, 488], [645, 490], [646, 492], [646, 493], [646, 494], [646, 495], [646, 496]], "use_pressure": true, "pressure": [0.129, 0.177, 0.188, 0.204, 0.22, 0.232, 0.245, 0.255, 0.266, 0.271, 0.281, 0.296, 0.312, 0.33, 0.343, 0.355, 0.362, 0.367, 0.372, 0.376, 0.378, 0.378, 0.379, 0.381, 0.309, 0.201], "s_time": 1638002997427, "e_time": 1638002997637, "width": 4}, {"id": 16, "draw_type": 0, "path": [[521, 325], [522, 328], [524, 328], [524, 330], [524, 331], [523, 333], [524, 334], [525, 335], [527, 338], [530, 343], [532, 349], [535, 354], [537, 357], [542, 361], [546, 366], [550, 369], [556, 374], [560, 377], [564, 382], [569, 387], [572, 390], [574, 392], [578, 396], [583, 399], [586, 403], [590, 406], [591, 409], [593, 411], [595, 413], [599, 416], [603, 420], [604, 422], [608, 426], [611, 427], [614, 429], [615, 431], [617, 434], [619, 437], [621, 439], [623, 440]], "use_pressure": true, "pressure": [0.078, 0.11, 0.117, 0.133, 0.153, 0.177, 0.203, 0.23, 0.258, 0.278, 0.299, 0.308, 0.317, 0.33, 0.342, 0.354, 0.364, 0.373, 0.381, 0.389, 0.395, 0.401, 0.408, 0.418, 0.427, 0.436, 0.441, 0.446, 0.45, 0.454, 0.456, 0.456, 0.457, 0.457, 0.45, 0.44, 0.376, 0.286, 0.242, 0.22], "s_time": 1638002999355, "e_time": 1638002999646, "width": 4}, {"id": 17, "draw_type": 0, "path": [[533, 299], [533, 299], [533, 300], [533, 300], [533, 301], [533, 301], [533, 302], [534, 303], [534, 304], [533, 305], [534, 309], [534, 310], [534, 312], [533, 316], [534, 318], [534, 326], [534, 327], [533, 329], [532, 332], [532, 334], [532, 335], [531, 337], [531, 339], [531, 341], [533, 345], [533, 347]], "use_pressure": true, "pressure": [0.051, 0.069, 0.073, 0.076, 0.078, 0.093, 0.114, 0.141, 0.17, 0.194, 0.216, 0.231, 0.247, 0.256, 0.264, 0.268, 0.273, 0.279, 0.285, 0.292, 0.295, 0.296, 0.262, 0.212, 0.188, 0.175], "s_time": 1638003001299, "e_time": 1638003001492, "width": 4}, {"id": 18, "draw_type": 0, "path": [[504, 272], [504, 271], [504, 271], [504, 271], [504, 271], [504, 271], [505, 271], [505, 271], [505, 273], [507, 273], [509, 273], [511, 275], [512, 276], [515, 279], [521, 282], [524, 285], [526, 286], [529, 290], [530, 291], [531, 291], [533, 294], [534, 297], [535, 300], [536, 302], [536, 305], [536, 306], [537, 309], [536, 311], [536, 314], [536, 315], [536, 316], [536, 318], [536, 323]], "use_pressure": true, "pressure": [0.117, 0.162, 0.173, 0.192, 0.215, 0.235, 0.255, 0.271, 0.286, 0.293, 0.302, 0.312, 0.321, 0.33, 0.338, 0.346, 0.354, 0.361, 0.369, 0.377, 0.383, 0.388, 0.39, 0.393, 0.394, 0.394, 0.396, 0.399, 0.401, 0.401, 0.396, 0.389, 0.385], "s_time": 1638003002726, "e_time": 1638003002981, "width": 4}, {"id": 19, "draw_type": 0, "path": [[365, 297], [368, 297], [368, 297], [370, 298], [371, 298], [371, 298], [372, 298], [372, 297], [373, 297], [373, 297], [375, 297], [376, 297], [377, 297], [377, 297], [378, 296], [378, 296], [379, 295], [380, 295], [382, 294], [383, 293], [386, 291]], "use_pressure": true, "pressure": [0.113, 0.156, 0.165, 0.175, 0.181, 0.183, 0.186, 0.188, 0.192, 0.202, 0.218, 0.228, 0.236, 0.24, 0.241, 0.242, 0.237, 0.233, 0.21, 0.18, 0.165], "s_time": 1638003004919, "e_time": 1638003005089, "width": 4}, {"id": 20, "draw_type": 0, "path": [[434, 295], [435, 295], [435, 295], [435, 296], [435, 296], [434, 296], [434, 296], [434, 296], [434, 296], [434, 296], [434, 297], [434, 297], [434, 296], [434, 296], [434, 296], [434, 296], [434, 296], [435, 296], [438, 298]], "use_pressure": true, "pressure": [0.117, 0.162, 0.173, 0.187, 0.201, 0.213, 0.226, 0.239, 0.257, 0.265, 0.274, 0.287, 0.295, 0.301, 0.304, 0.304, 0.257, 0.19, 0.157], "s_time": 1638003005157, "e_time": 1638003005311, "width": 4}, {"id": 21, "draw_type": 0, "path": [[361, 305], [361, 305], [361, 305], [362, 305], [362, 305], [362, 305], [362, 305], [367, 304], [367, 304], [367, 304], [368, 303], [368, 303], [368, 303], [372, 303], [372, 303], [375, 303], [377, 302], [379, 303], [380, 303], [382, 303], [383, 302], [385, 302], [389, 302], [392, 301], [398, 298], [397, 300], [399, 301], [398, 303], [404, 299], [404, 300], [404, 301], [405, 302], [407, 304], [407, 305], [410, 305], [410, 306]], "use_pressure": true, "pressure": [0.113, 0.156, 0.165, 0.179, 0.193, 0.21, 0.23, 0.255, 0.281, 0.294, 0.308, 0.321, 0.332, 0.342, 0.346, 0.347, 0.348, 0.35, 0.351, 0.354, 0.353, 0.352, 0.352, 0.352, 0.352, 0.354, 0.354, 0.354, 0.355, 0.357, 0.358, 0.358, 0.351, 0.338, 0.332, 0.328], "s_time": 1638003006443, "e_time": 1638003006702, "width": 4}, {"id": 22, "draw_type": 0, "path": [[376, 301], [376, 301], [376, 301], [376, 301], [376, 301], [376, 301], [375, 303], [375, 303], [374, 304], [374, 305], [376, 308], [374, 310], [373, 311], [372, 314], [376, 318], [376, 318], [373, 320], [374, 321], [376, 322], [376, 324], [377, 324], [381, 325], [382, 327], [386, 328], [389, 328], [392, 330], [395, 331], [402, 332], [408, 336]], "use_pressure": true, "pressure": [0.156, 0.215, 0.228, 0.242, 0.252, 0.262, 0.272, 0.282, 0.291, 0.295, 0.303, 0.312, 0.321, 0.332, 0.339, 0.347, 0.352, 0.356, 0.358, 0.361, 0.362, 0.362, 0.363, 0.365, 0.355, 0.343, 0.267, 0.166, 0.116], "s_time": 1638003008049, "e_time": 1638003008271, "width": 4}, {"id": 23, "draw_type": 0, "path": [[410, 309], [408, 309], [406, 312], [409, 310], [409, 310], [409, 310], [406, 313], [406, 314], [402, 317], [403, 318], [403, 319], [399, 323], [402, 324], [402, 326], [402, 326], [405, 329], [407, 331], [407, 331]], "use_pressure": true, "pressure": [0.125, 0.173, 0.184, 0.191, 0.195, 0.201, 0.213, 0.231, 0.255, 0.277, 0.298, 0.308, 0.315, 0.322, 0.326, 0.327, 0.328, 0.328], "s_time": 1638003008519, "e_time": 1638003008655, "width": 4}, {"id": 24, "draw_type": 0, "path": [[356, 321], [356, 321], [355, 321], [356, 323], [356, 323], [357, 323], [358, 324], [356, 325], [357, 326], [357, 326], [358, 328], [360, 329], [362, 330], [365, 332], [369, 333], [371, 335], [373, 337], [378, 339], [381, 342], [397, 346]], "use_pressure": true, "pressure": [0.105, 0.146, 0.154, 0.165, 0.174, 0.182, 0.191, 0.205, 0.22, 0.229, 0.236, 0.24, 0.246, 0.252, 0.256, 0.257, 0.249, 0.236, 0.23, 0.227], "s_time": 1638003010080, "e_time": 1638003010234, "width": 4}, {"id": 25, "draw_type": 0, "path": [[258, 262], [258, 263], [257, 265], [256, 268], [256, 268], [256, 269], [256, 269], [254, 272], [252, 276], [252, 277], [251, 280], [250, 282], [249, 287], [247, 290], [244, 297], [244, 301], [243, 305], [242, 308], [239, 315], [237, 319], [236, 321], [235, 323], [236, 324], [236, 325], [237, 326], [240, 330]], "use_pressure": true, "pressure": [0.109, 0.152, 0.161, 0.188, 0.219, 0.241, 0.261, 0.275, 0.29, 0.297, 0.304, 0.313, 0.319, 0.327, 0.332, 0.339, 0.344, 0.351, 0.355, 0.362, 0.363, 0.363, 0.358, 0.354, 0.351, 0.349], "s_time": 1638003011972, "e_time": 1638003012166, "width": 4}, {"id": 26, "draw_type": 0, "path": [[279, 108], [279, 108], [278, 110], [277, 111], [277, 111], [277, 111], [277, 113], [277, 114], [277, 116], [277, 118], [277, 120], [276, 125], [276, 129], [276, 133], [275, 141], [276, 150], [277, 157], [279, 164], [280, 170], [281, 178], [282, 186], [285, 197], [287, 204], [286, 208], [286, 210], [284, 216], [284, 222], [284, 228], [284, 230], [285, 235]], "use_pressure": true, "pressure": [0.105, 0.148, 0.157, 0.167, 0.175, 0.18, 0.185, 0.192, 0.203, 0.214, 0.228, 0.24, 0.253, 0.259, 0.263, 0.269, 0.27, 0.27, 0.27, 0.27, 0.27, 0.271, 0.272, 0.272, 0.273, 0.273, 0.269, 0.263, 0.26, 0.26], "s_time": 1638003013820, "e_time": 1638003014052, "width": 4}, {"id": 27, "draw_type": 0, "path": [[280, 106], [280, 105], [280, 105], [279, 104], [278, 105], [279, 103], [280, 102], [279, 101], [280, 100], [281, 98], [282, 97], [282, 97], [282, 96], [282, 96], [283, 95], [284, 94], [284, 94], [284, 94], [285, 94], [287, 93]], "use_pressure": true, "pressure": [0.074, 0.104, 0.109, 0.125, 0.146, 0.169, 0.195, 0.216, 0.236, 0.249, 0.262, 0.267, 0.271, 0.274, 0.273, 0.271, 0.249, 0.219, 0.204, 0.196], "s_time": 1638003015476, "e_time": 1638003015625, "width": 4}, {"id": 28, "draw_type": 0, "path": [[288, 91], [288, 91], [289, 92], [288, 93], [288, 93], [288, 93], [288, 93], [289, 94], [289, 94], [289, 94], [291, 94], [289, 95], [290, 96], [291, 98], [290, 99], [293, 99], [291, 101], [293, 102], [293, 104], [294, 105], [295, 106], [295, 106]], "use_pressure": true, "pressure": [0.055, 0.078, 0.084, 0.111, 0.149, 0.183, 0.216, 0.236, 0.252, 0.263, 0.273, 0.278, 0.282, 0.288, 0.291, 0.294, 0.296, 0.296, 0.288, 0.277, 0.272, 0.27], "s_time": 1638003015792, "e_time": 1638003015956, "width": 4}, {"id": 29, "draw_type": 0, "path": [[296, 107], [296, 107], [296, 107], [296, 107], [296, 107], [296, 107], [296, 107], [296, 107], [296, 107], [296, 108], [296, 108], [296, 108], [296, 107], [296, 107], [296, 108], [296, 108], [296, 108], [296, 108], [296, 108], [297, 108], [297, 108], [296, 110], [296, 110], [297, 112], [297, 112], [297, 112], [298, 113], [298, 114], [298, 114], [299, 114], [299, 114], [300, 115], [300, 115], [300, 115], [302, 116], [302, 116], [303, 117], [304, 117], [305, 118], [306, 119], [307, 121], [308, 122], [309, 122], [311, 123], [312, 125], [313, 126], [314, 126], [316, 128], [317, 129], [317, 129], [319, 131], [321, 133], [321, 134], [323, 136], [327, 138], [327, 140], [330, 144], [330, 147], [331, 149], [333, 154], [336, 160], [339, 166], [343, 172], [345, 177], [347, 186], [349, 193], [347, 197], [341, 202]], "use_pressure": true, "pressure": [0.062, 0.086, 0.092, 0.096, 0.1, 0.101, 0.104, 0.11, 0.123, 0.142, 0.165, 0.186, 0.203, 0.213, 0.221, 0.225, 0.228, 0.229, 0.232, 0.239, 0.25, 0.261, 0.272, 0.282, 0.291, 0.295, 0.298, 0.3, 0.303, 0.302, 0.301, 0.301, 0.303, 0.304, 0.304, 0.305, 0.307, 0.308, 0.311, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.312, 0.31, 0.309, 0.304, 0.3, 0.291, 0.283, 0.272, 0.262, 0.203, 0.124, 0.086], "s_time": 1638003016256, "e_time": 1638003016777, "width": 4}, {"id": 30, "draw_type": 0, "path": [[285, 124], [286, 130], [285, 130], [285, 131], [284, 135], [284, 137], [284, 138], [285, 144], [284, 147], [286, 150], [286, 154], [288, 158], [287, 165], [288, 171], [290, 176], [290, 183], [291, 192], [294, 201], [296, 205], [296, 208], [291, 212], [288, 216], [286, 223], [287, 225], [288, 228], [288, 229]], "use_pressure": true, "pressure": [0.105, 0.146, 0.154, 0.174, 0.195, 0.218, 0.243, 0.261, 0.278, 0.285, 0.292, 0.299, 0.305, 0.312, 0.314, 0.317, 0.321, 0.325, 0.324, 0.324, 0.324, 0.326, 0.318, 0.307, 0.301, 0.297], "s_time": 1638003017235, "e_time": 1638003017421, "width": 4}, {"id": 31, "draw_type": 0, "path": [[353, 201], [353, 201], [354, 202], [356, 203], [357, 202], [359, 204], [362, 204], [365, 208], [367, 208], [370, 204], [370, 210], [374, 210], [376, 215], [379, 215], [383, 207], [384, 220], [390, 215], [395, 218], [398, 219], [400, 217]], "use_pressure": true, "pressure": [0.094, 0.131, 0.14, 0.154, 0.169, 0.184, 0.202, 0.216, 0.229, 0.239, 0.248, 0.252, 0.258, 0.267, 0.272, 0.277, 0.275, 0.272, 0.271, 0.27], "s_time": 1638003018856, "e_time": 1638003019004, "width": 4}, {"id": 32, "draw_type": 0, "path": [[350, 178], [349, 180], [349, 180], [351, 179], [351, 179], [351, 181], [351, 181], [352, 181], [354, 184], [355, 183], [358, 183], [357, 181], [358, 179], [358, 177], [359, 176], [359, 175], [361, 173], [366, 172], [370, 172], [374, 174], [376, 175]], "use_pressure": true, "pressure": [0.102, 0.142, 0.15, 0.168, 0.187, 0.207, 0.229, 0.251, 0.271, 0.28, 0.286, 0.292, 0.295, 0.298, 0.3, 0.3, 0.283, 0.261, 0.226, 0.186, 0.167], "s_time": 1638003019459, "e_time": 1638003019611, "width": 4}, {"id": 33, "draw_type": 0, "path": [[399, 186], [399, 186], [399, 186], [399, 186], [399, 186], [399, 186], [399, 186], [399, 186], [395, 184], [395, 184], [395, 184], [395, 184], [395, 184], [395, 184], [395, 184], [399, 185], [399, 185], [399, 185], [401, 186], [401, 186], [401, 186], [401, 187], [402, 187], [404, 188], [405, 189], [406, 189], [409, 191], [411, 193], [414, 195], [415, 196], [419, 197], [421, 200], [423, 204], [425, 205], [426, 208], [427, 210], [430, 213], [433, 215], [435, 217], [436, 220], [437, 220], [437, 223], [438, 223], [439, 223], [440, 224], [440, 224], [441, 225], [440, 225]], "use_pressure": true, "pressure": [0.102, 0.142, 0.15, 0.152, 0.151, 0.145, 0.138, 0.135, 0.133, 0.147, 0.172, 0.194, 0.216, 0.233, 0.252, 0.265, 0.274, 0.279, 0.28, 0.283, 0.286, 0.292, 0.301, 0.312, 0.323, 0.337, 0.348, 0.357, 0.363, 0.368, 0.374, 0.383, 0.394, 0.405, 0.411, 0.415, 0.417, 0.42, 0.419, 0.418, 0.416, 0.412, 0.402, 0.391, 0.338, 0.267, 0.232, 0.215], "s_time": 1638003019775, "e_time": 1638003020127, "width": 4}, {"id": 34, "draw_type": 0, "path": [[407, 219], [410, 220], [410, 220], [410, 220], [412, 221], [412, 221], [412, 222], [413, 221], [417, 222], [419, 223], [419, 223], [421, 224], [422, 225], [422, 225], [424, 225], [428, 225], [428, 225], [429, 226]], "use_pressure": true, "pressure": [0.055, 0.076, 0.08, 0.097, 0.117, 0.142, 0.17, 0.192, 0.211, 0.221, 0.229, 0.232, 0.235, 0.237, 0.231, 0.222, 0.218, 0.215], "s_time": 1638003021329, "e_time": 1638003021467, "width": 4}, {"id": 35, "draw_type": 0, "path": [[431, 225], [431, 225], [431, 225], [431, 225], [431, 225], [431, 225], [431, 225], [432, 224], [432, 224], [434, 224], [434, 224], [435, 224], [436, 223], [437, 223], [437, 223], [438, 223], [438, 223], [438, 223], [439, 223]], "use_pressure": true, "pressure": [0.121, 0.166, 0.177, 0.188, 0.197, 0.212, 0.229, 0.25, 0.272, 0.283, 0.298, 0.317, 0.328, 0.336, 0.336, 0.336, 0.27, 0.172, 0.124], "s_time": 1638003021526, "e_time": 1638003021675, "width": 4}, {"id": 36, "draw_type": 0, "path": [[402, 174], [402, 174], [398, 176]], "use_pressure": true, "pressure": [0.059, 0.082, 0.088], "s_time": 1638003022755, "e_time": 1638003022783, "width": 4}, {"id": 37, "draw_type": 0, "path": [[388, 180], [389, 181], [389, 181], [389, 181], [389, 181], [389, 181], [389, 181], [389, 181], [389, 181], [389, 181], [389, 181], [389, 181], [390, 180], [389, 179], [389, 178], [391, 177], [392, 175], [396, 173], [397, 169], [399, 167], [402, 164], [404, 162], [406, 160], [410, 158], [412, 156], [415, 156], [417, 154], [420, 152], [423, 150], [425, 148], [428, 145], [431, 143], [436, 142], [439, 139], [442, 138], [444, 136], [446, 133], [451, 132]], "use_pressure": true, "pressure": [0.078, 0.11, 0.117, 0.144, 0.178, 0.21, 0.243, 0.269, 0.29, 0.3, 0.312, 0.325, 0.338, 0.353, 0.365, 0.38, 0.396, 0.41, 0.422, 0.432, 0.436, 0.438, 0.44, 0.443, 0.442, 0.441, 0.441, 0.443, 0.442, 0.441, 0.433, 0.422, 0.406, 0.388, 0.358, 0.325, 0.31, 0.302], "s_time": 1638003022886, "e_time": 1638003023174, "width": 4}, {"id": 38, "draw_type": 0, "path": [[453, 133], [453, 133], [453, 133], [457, 132], [457, 132], [457, 132], [453, 130], [457, 127], [459, 128], [463, 127], [463, 126], [463, 125], [463, 124], [463, 124], [463, 122], [463, 122]], "use_pressure": true, "pressure": [0.188, 0.26, 0.275, 0.303, 0.332, 0.353, 0.373, 0.384, 0.391, 0.394, 0.394, 0.395, 0.381, 0.362, 0.354, 0.349], "s_time": 1638003024229, "e_time": 1638003024349, "width": 4}, {"id": 39, "draw_type": 0, "path": [[421, 198], [421, 198], [421, 198], [421, 198], [421, 198], [421, 198], [421, 198], [421, 198], [421, 198], [421, 198], [421, 198], [425, 198], [425, 198], [425, 197], [425, 196], [426, 196], [427, 195], [428, 193], [428, 190], [429, 188], [430, 185], [432, 181], [434, 179], [435, 175], [436, 173], [439, 167], [441, 161], [444, 158], [445, 154], [448, 149], [447, 145], [448, 143], [452, 141], [456, 140]], "use_pressure": true, "pressure": [0.152, 0.208, 0.221, 0.234, 0.244, 0.256, 0.272, 0.289, 0.307, 0.315, 0.327, 0.343, 0.359, 0.375, 0.387, 0.396, 0.402, 0.409, 0.412, 0.415, 0.417, 0.42, 0.421, 0.424, 0.425, 0.428, 0.427, 0.426, 0.421, 0.415, 0.396, 0.373, 0.359, 0.354], "s_time": 1638003024678, "e_time": 1638003024936, "width": 4}, {"id": 40, "draw_type": 0, "path": [[396, 181], [396, 181], [398, 180], [398, 180], [398, 180], [396, 179], [396, 179], [396, 179], [399, 179], [399, 179], [401, 177], [401, 177], [401, 176], [402, 175], [402, 175], [406, 173], [405, 172], [406, 171], [410, 167], [410, 167], [413, 167], [414, 166], [418, 163], [421, 161], [423, 158], [426, 155], [427, 153], [431, 150], [438, 147], [444, 144]], "use_pressure": true, "pressure": [0.105, 0.148, 0.157, 0.167, 0.175, 0.182, 0.189, 0.204, 0.223, 0.243, 0.263, 0.28, 0.299, 0.308, 0.317, 0.33, 0.342, 0.356, 0.365, 0.371, 0.374, 0.377, 0.378, 0.378, 0.376, 0.375, 0.357, 0.335, 0.323, 0.317], "s_time": 1638003025453, "e_time": 1638003025679, "width": 4}, {"id": 41, "draw_type": 0, "path": [[425, 204], [425, 204], [425, 203], [425, 203], [425, 203], [425, 204], [426, 205], [426, 205], [426, 206], [426, 206], [427, 208], [428, 209], [429, 210], [433, 213], [435, 214], [436, 217], [438, 219], [440, 222], [442, 222], [445, 224], [448, 228], [454, 230], [460, 236], [464, 237], [469, 243], [474, 247], [477, 250], [482, 253], [488, 257], [492, 261], [496, 264], [499, 266]], "use_pressure": true, "pressure": [0.16, 0.222, 0.235, 0.247, 0.255, 0.259, 0.263, 0.271, 0.283, 0.289, 0.296, 0.306, 0.312, 0.319, 0.324, 0.329, 0.333, 0.337, 0.336, 0.336, 0.336, 0.338, 0.339, 0.342, 0.343, 0.346, 0.347, 0.347, 0.337, 0.323, 0.316, 0.312], "s_time": 1638003026992, "e_time": 1638003027232, "width": 4}, {"id": 42, "draw_type": 0, "path": [[343, 169], [343, 169], [343, 169], [345, 172], [345, 172], [345, 172], [345, 172], [345, 172], [345, 172], [345, 173], [347, 173], [347, 172], [350, 174], [352, 173], [353, 173], [356, 172], [357, 172], [357, 172], [358, 172], [359, 171], [361, 171], [363, 171], [363, 171], [364, 171], [364, 171], [365, 171], [366, 172], [368, 172], [369, 172], [369, 172], [372, 173], [372, 173], [375, 174], [377, 175], [379, 176], [379, 176], [382, 177], [383, 179], [383, 179], [387, 179]], "use_pressure": true, "pressure": [0.113, 0.156, 0.165, 0.179, 0.191, 0.202, 0.216, 0.233, 0.252, 0.261, 0.271, 0.283, 0.289, 0.294, 0.296, 0.296, 0.297, 0.297, 0.297, 0.299, 0.302, 0.306, 0.312, 0.32, 0.327, 0.335, 0.338, 0.341, 0.345, 0.349, 0.348, 0.348, 0.35, 0.353, 0.356, 0.362, 0.363, 0.361, 0.36, 0.359], "s_time": 1638003029004, "e_time": 1638003029300, "width": 4}, {"id": 45, "draw_type": 0, "path": [[182, 318], [182, 318], [182, 318], [183, 318], [183, 318], [183, 317], [184, 317], [183, 316], [183, 316], [184, 314], [185, 311], [185, 312], [186, 310], [186, 308], [189, 305], [189, 302], [191, 300], [191, 299], [193, 295], [193, 293], [195, 291], [197, 288], [198, 287], [200, 284], [201, 283], [202, 281], [203, 279], [205, 277], [206, 272], [208, 270], [210, 268], [211, 266], [214, 263], [214, 260], [215, 257], [216, 255]], "use_pressure": true, "pressure": [0.141, 0.194, 0.206, 0.226, 0.245, 0.257, 0.267, 0.277, 0.288, 0.294, 0.305, 0.319, 0.336, 0.352, 0.363, 0.375, 0.382, 0.39, 0.395, 0.399, 0.405, 0.414, 0.421, 0.429, 0.432, 0.435, 0.437, 0.439, 0.44, 0.44, 0.433, 0.422, 0.394, 0.356, 0.339, 0.33], "s_time": 1638003035992, "e_time": 1638003036264, "width": 4}, {"id": 46, "draw_type": 0, "path": [[217, 256], [217, 256], [217, 256], [217, 255], [217, 255], [217, 255], [217, 255], [217, 255], [218, 254], [218, 254], [218, 253], [218, 253], [218, 253], [218, 252], [218, 252], [218, 252], [219, 251], [219, 251], [219, 251], [219, 250], [219, 250], [219, 250], [219, 250], [220, 249], [220, 249], [221, 248], [221, 248], [222, 247], [222, 247], [223, 246], [224, 246], [224, 246], [225, 245], [225, 244], [225, 244], [227, 243], [227, 241], [227, 241], [228, 238], [229, 238], [229, 238], [228, 237], [229, 237], [230, 236], [231, 236], [231, 236], [234, 236], [233, 234], [233, 234], [233, 234], [235, 231], [235, 231], [235, 231], [235, 231], [235, 230], [236, 230], [238, 229], [238, 229], [239, 229], [239, 229], [240, 229], [240, 228], [241, 228], [245, 226], [247, 225], [249, 225], [250, 223], [253, 222], [256, 220], [258, 220], [259, 218], [262, 216], [265, 215], [268, 215], [272, 212], [272, 209], [275, 208], [276, 205], [278, 204], [280, 202], [280, 199], [281, 197], [282, 195], [283, 194]], "use_pressure": true, "pressure": [0.086, 0.118, 0.125, 0.143, 0.166, 0.188, 0.208, 0.222, 0.231, 0.242, 0.253, 0.259, 0.265, 0.274, 0.28, 0.288, 0.291, 0.294, 0.3, 0.307, 0.316, 0.327, 0.335, 0.34, 0.345, 0.349, 0.353, 0.358, 0.361, 0.362, 0.363, 0.365, 0.368, 0.372, 0.376, 0.38, 0.384, 0.39, 0.395, 0.399, 0.401, 0.404, 0.405, 0.408, 0.409, 0.412, 0.413, 0.413, 0.414, 0.414, 0.414, 0.416, 0.417, 0.417, 0.418, 0.418, 0.418, 0.418, 0.418, 0.418, 0.418, 0.42, 0.419, 0.418, 0.418, 0.418, 0.415, 0.414, 0.414, 0.416, 0.417, 0.42, 0.419, 0.418, 0.407, 0.394, 0.389, 0.389, 0.383, 0.374, 0.353, 0.325, 0.312, 0.306], "s_time": 1638003036777, "e_time": 1638003037405, "width": 4}, {"id": 47, "draw_type": 0, "path": [[181, 322], [181, 322], [181, 322], [182, 323], [182, 323], [182, 324], [182, 324], [182, 323], [182, 324], [182, 324], [183, 325], [184, 326], [186, 326], [188, 327], [189, 328], [192, 327], [193, 327], [195, 328], [196, 327], [198, 327], [203, 328], [203, 327], [204, 327], [207, 327], [209, 327], [211, 327], [214, 327], [217, 327], [217, 327], [219, 327], [219, 327]], "use_pressure": true, "pressure": [0.113, 0.158, 0.169, 0.178, 0.185, 0.192, 0.205, 0.217, 0.229, 0.241, 0.255, 0.262, 0.269, 0.275, 0.279, 0.28, 0.281, 0.281, 0.281, 0.283, 0.284, 0.284, 0.285, 0.287, 0.288, 0.288, 0.282, 0.272, 0.217, 0.14, 0.102], "s_time": 1638003038640, "e_time": 1638003038877, "width": 4}, {"id": 48, "draw_type": 0, "path": [[224, 328], [224, 328], [224, 328], [225, 328], [225, 328], [225, 328], [226, 328], [225, 329], [225, 330], [224, 331], [224, 332], [222, 336], [222, 337], [223, 340], [223, 340], [223, 340]], "use_pressure": true, "pressure": [0.133, 0.184, 0.194, 0.218, 0.243, 0.259, 0.272, 0.282, 0.291, 0.295, 0.296, 0.299, 0.298, 0.297, 0.297, 0.299], "s_time": 1638003038985, "e_time": 1638003039105, "width": 4}, {"id": 49, "draw_type": 0, "path": [[342, 271], [341, 271], [341, 271], [339, 275], [339, 275], [339, 275], [339, 277], [338, 282], [337, 284], [336, 286], [335, 289], [335, 292], [336, 294], [334, 299], [334, 303], [332, 307], [331, 313], [331, 322], [330, 327], [323, 335], [319, 344], [317, 349], [316, 352], [314, 355], [313, 361]], "use_pressure": true, "pressure": [0.117, 0.162, 0.173, 0.187, 0.199, 0.208, 0.217, 0.229, 0.245, 0.253, 0.27, 0.289, 0.307, 0.326, 0.339, 0.349, 0.354, 0.356, 0.358, 0.361, 0.357, 0.354, 0.275, 0.161, 0.105], "s_time": 1638003040640, "e_time": 1638003040833, "width": 4}, {"id": 50, "draw_type": 0, "path": [[304, 413], [303, 413], [303, 413], [303, 413], [303, 413], [303, 413], [303, 412], [303, 412], [303, 412], [304, 411], [304, 411], [304, 411], [304, 411], [304, 411], [304, 411], [305, 414]], "use_pressure": true, "pressure": [0.113, 0.158, 0.169, 0.178, 0.185, 0.188, 0.194, 0.203, 0.214, 0.226, 0.239, 0.246, 0.247, 0.246, 0.246, 0.246], "s_time": 1638003042488, "e_time": 1638003042608, "width": 4}, {"id": 51, "draw_type": 0, "path": [[283, 234], [284, 234], [284, 234], [284, 234], [284, 234], [284, 234], [284, 234], [284, 234], [284, 234], [283, 234], [283, 234], [283, 235], [283, 236], [283, 237], [281, 236], [279, 236], [276, 237], [272, 238], [269, 243], [265, 250], [262, 256], [261, 259], [259, 263], [257, 266], [257, 267], [256, 269], [255, 271], [255, 273], [255, 275], [255, 276]], "use_pressure": true, "pressure": [0.055, 0.078, 0.084, 0.099, 0.118, 0.144, 0.176, 0.201, 0.226, 0.238, 0.246, 0.249, 0.249, 0.252, 0.257, 0.264, 0.273, 0.284, 0.294, 0.305, 0.313, 0.322, 0.326, 0.329, 0.331, 0.334, 0.333, 0.33, 0.329, 0.328], "s_time": 1638003045498, "e_time": 1638003045710, "width": 4}, {"id": 52, "draw_type": 0, "path": [[246, 378], [246, 378], [246, 378], [246, 378], [246, 378], [246, 378], [246, 378], [246, 378], [246, 378], [246, 379], [246, 379], [246, 380], [246, 380], [246, 381], [246, 381], [246, 381], [247, 381], [247, 382], [247, 383], [248, 384], [249, 384], [249, 384], [249, 385], [249, 385], [249, 386], [249, 386], [249, 387]], "use_pressure": true, "pressure": [0.145, 0.198, 0.21, 0.24, 0.275, 0.295, 0.312, 0.319, 0.327, 0.33, 0.331, 0.334, 0.333, 0.332, 0.332, 0.334, 0.335, 0.338, 0.339, 0.342, 0.341, 0.338, 0.326, 0.31, 0.237, 0.142, 0.095], "s_time": 1638003048144, "e_time": 1638003048355, "width": 4}, {"id": 53, "draw_type": 0, "path": [[250, 388], [250, 388], [250, 388], [250, 388], [250, 390], [251, 391], [251, 392], [251, 393], [252, 393], [253, 395], [253, 398], [255, 400], [255, 402], [257, 406], [258, 407], [258, 407], [258, 407], [256, 408]], "use_pressure": true, "pressure": [0.145, 0.198, 0.21, 0.229, 0.249, 0.261, 0.273, 0.28, 0.288, 0.291, 0.294, 0.298, 0.3, 0.3, 0.294, 0.287, 0.283, 0.281], "s_time": 1638003048474, "e_time": 1638003048609, "width": 4}, {"id": 54, "draw_type": 0, "path": [[196, 375], [195, 375], [196, 375], [195, 376], [195, 376], [196, 377], [194, 378], [193, 380], [192, 380], [192, 381], [190, 383], [189, 385], [188, 388], [186, 390], [186, 390]], "use_pressure": true, "pressure": [0.156, 0.215, 0.228, 0.246, 0.265, 0.276, 0.289, 0.298, 0.307, 0.311, 0.313, 0.317, 0.28, 0.225, 0.199], "s_time": 1638003050119, "e_time": 1638003050249, "width": 4}, {"id": 55, "draw_type": 0, "path": [[538, 463], [539, 464], [538, 464], [538, 464], [539, 464], [539, 464], [539, 464], [539, 465], [539, 465], [539, 465], [539, 465], [539, 466], [539, 467], [539, 468], [539, 469], [539, 469], [538, 470], [537, 471], [537, 473], [536, 474], [535, 477]], "use_pressure": true, "pressure": [0.137, 0.19, 0.202, 0.222, 0.241, 0.26, 0.279, 0.292, 0.305, 0.31, 0.315, 0.322, 0.328, 0.335, 0.338, 0.341, 0.343, 0.343, 0.283, 0.198, 0.155], "s_time": 1638003052107, "e_time": 1638003052265, "width": 4}, {"id": 58, "draw_type": 0, "path": [[626, 585], [626, 585], [625, 585], [625, 585], [625, 585], [625, 585], [625, 585], [625, 585], [625, 585], [625, 585], [625, 585], [625, 585], [625, 585], [625, 585], [626, 585], [626, 585]], "use_pressure": true, "pressure": [0.215, 0.298, 0.316, 0.332, 0.343, 0.348, 0.353, 0.362, 0.376, 0.383, 0.392, 0.402, 0.407, 0.411, 0.413, 0.413], "s_time": 1638003058830, "e_time": 1638003058951, "width": 4}], "undo_count": 5, "rm_stroke_count": 0}


--------------------------------------------------------------------------------
/sketch_synthesis/model/N_mlp_extrinsic.sav:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chufengxiao/DifferSketching/305ce2d66a8fd63d325145a90a9378591e2d754f/sketch_synthesis/model/N_mlp_extrinsic.sav


--------------------------------------------------------------------------------
/sketch_synthesis/model/N_mlp_intrinsic.sav:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chufengxiao/DifferSketching/305ce2d66a8fd63d325145a90a9378591e2d754f/sketch_synthesis/model/N_mlp_intrinsic.sav


--------------------------------------------------------------------------------
/sketch_synthesis/model/P_mlp_extrinsic.sav:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chufengxiao/DifferSketching/305ce2d66a8fd63d325145a90a9378591e2d754f/sketch_synthesis/model/P_mlp_extrinsic.sav


--------------------------------------------------------------------------------
/sketch_synthesis/model/P_mlp_intrinsic.sav:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chufengxiao/DifferSketching/305ce2d66a8fd63d325145a90a9378591e2d754f/sketch_synthesis/model/P_mlp_intrinsic.sav


--------------------------------------------------------------------------------
/sketch_synthesis/model/mlp_curveNoise.sav:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chufengxiao/DifferSketching/305ce2d66a8fd63d325145a90a9378591e2d754f/sketch_synthesis/model/mlp_curveNoise.sav


--------------------------------------------------------------------------------
/sketch_synthesis/prepare_data/getCurveNoiseData.py:
--------------------------------------------------------------------------------
  1 | import numpy as np
  2 | from scipy.special import comb
  3 | import matplotlib.pyplot as plt
  4 | import os,cv2,json,copy
  5 | from natsort import natsorted
  6 | from scipy.ndimage.morphology import distance_transform_edt
  7 | from scipy import spatial
  8 | 
  9 | def readStrokes(path):
 10 |     file = open(path,"r")
 11 |     json_str = file.read()
 12 |     strokes = json.loads(json_str)
 13 |     return strokes
 14 | 
 15 | def renderSketch(strokes):
 16 |     canvas = np.ones((800,800),dtype="uint8")*255
 17 | 
 18 |     for stroke in strokes['strokes']:
 19 |             
 20 |         if stroke['draw_type'] == 1:
 21 |             continue
 22 | 
 23 |         path = stroke['path']
 24 |         path = np.array(path).astype("int")
 25 |         for i in range(len(path)-1):
 26 |             cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),0,3)
 27 | 
 28 |     return canvas
 29 | 
 30 | def get_bezier_parameters(X, Y, degree=3):
 31 |     """ Least square qbezier fit using penrose pseudoinverse.
 32 | 
 33 |     Parameters:
 34 | 
 35 |     X: array of x data.
 36 |     Y: array of y data. Y[0] is the y point for X[0].
 37 |     degree: degree of the Bézier curve. 2 for quadratic, 3 for cubic.
 38 | 
 39 |     Based on https://stackoverflow.com/questions/12643079/b%C3%A9zier-curve-fitting-with-scipy
 40 |     and probably on the 1998 thesis by Tim Andrew Pastva, "Bézier Curve Fitting".
 41 |     """
 42 |     if degree < 1:
 43 |         raise ValueError('degree must be 1 or greater.')
 44 | 
 45 |     if len(X) != len(Y):
 46 |         raise ValueError('X and Y must be of the same length.')
 47 | 
 48 |     if len(X) < degree + 1:
 49 |         raise ValueError(f'There must be at least {degree + 1} points to '
 50 |                          f'determine the parameters of a degree {degree} curve. '
 51 |                          f'Got only {len(X)} points.')
 52 | 
 53 |     def bpoly(n, t, k):
 54 |         """ Bernstein polynomial when a = 0 and b = 1. """
 55 |         return t ** k * (1 - t) ** (n - k) * comb(n, k)
 56 |         #return comb(n, i) * ( t**(n-i) ) * (1 - t)**i
 57 | 
 58 |     def bmatrix(T):
 59 |         """ Bernstein matrix for Bézier curves. """
 60 |         return np.matrix([[bpoly(degree, t, k) for k in range(degree + 1)] for t in T])
 61 | 
 62 |     def least_square_fit(points, M):
 63 |         M_ = np.linalg.pinv(M)
 64 |         return M_ * points
 65 | 
 66 |     T = np.linspace(0, 1, len(X))
 67 |     M = bmatrix(T)
 68 |     points = np.array(list(zip(X, Y)))
 69 |     
 70 |     final = least_square_fit(points, M).tolist()
 71 |     final[0] = [X[0], Y[0]]
 72 |     final[len(final)-1] = [X[len(X)-1], Y[len(Y)-1]]
 73 |     return np.array(final)
 74 | 
 75 | def bernstein_poly(i, n, t):
 76 |     """
 77 |      The Bernstein polynomial of n, i as a function of t
 78 |     """
 79 |     return comb(n, i) * ( t**(n-i) ) * (1 - t)**i
 80 | 
 81 | 
 82 | def bezier_curve(points, nTimes=50):
 83 |     """
 84 |        Given a set of control points, return the
 85 |        bezier curve defined by the control points.
 86 | 
 87 |        points should be a list of lists, or list of tuples
 88 |        such as [ [1,1], 
 89 |                  [2,3], 
 90 |                  [4,5], ..[Xn, Yn] ]
 91 |         nTimes is the number of time steps, defaults to 1000
 92 | 
 93 |         See http://processingjs.nihongoresources.com/bezierinfo/
 94 |     """
 95 | 
 96 |     nPoints = len(points)
 97 |     xPoints = np.array([p[0] for p in points])
 98 |     yPoints = np.array([p[1] for p in points])
 99 | 
100 |     t = np.linspace(0.0, 1.0, nTimes)
101 | 
102 |     polynomial_array = np.array([ bernstein_poly(i, nPoints-1, t) for i in range(0, nPoints)   ])
103 | 
104 |     xvals = np.dot(xPoints, polynomial_array)
105 |     yvals = np.dot(yPoints, polynomial_array)
106 | 
107 |     return xvals, yvals
108 | 
109 | def drawPath(path,color,canvas=None):
110 |     if canvas is None:
111 |         canvas = np.ones((800,800,3),dtype="uint8")*255
112 |     path = path.astype('int')
113 | 
114 |     for i in range(len(path)-1):
115 |         cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),color,3)
116 |     return canvas
117 | 
118 | 
119 | def showBezier(points,control_num):
120 |     data = get_bezier_parameters(points[:,0], points[:,1], degree=control_num-1).tolist()
121 | 
122 |     xvals, yvals = bezier_curve(data, nTimes=1000)
123 | 
124 |     # # Plot the control points
125 |     # data = np.array(data)
126 |     # x_val = data[:,0]
127 |     # y_val = data[:,1]
128 |     # plt.plot(points[:,0], points[:,1], "ro",label='Original Points')
129 |     # plt.plot(x_val,y_val,'k--o', label='Control Points')
130 |     # # Plot the resulting Bezier curve
131 |     # plt.plot(xvals, yvals, 'b-', label='B Curve')
132 |     # plt.legend()
133 |     # plt.show()
134 |     return xvals,yvals
135 | 
136 | def visualization(control_num,json_path):
137 |     strokes = readStrokes(json_path)
138 |     curves = copy.deepcopy(strokes)
139 | 
140 |     for i,stroke in enumerate(strokes['strokes']):
141 |         if stroke['draw_type'] == 1:
142 |             continue
143 |         points = np.array(stroke['path'])
144 |         if len(points) < control_num:
145 |             continue
146 |         canvas = drawPath(points,0)
147 |         x,y = showBezier(points,control_num)
148 |         curve = np.array([x,y]).T
149 |         curves['strokes'][i]['path'] = curve.astype('int')
150 |     
151 |     origin = renderSketch(strokes)
152 |     bezier = renderSketch(curves)
153 | 
154 |     plt.subplot(121)
155 |     plt.imshow(origin,'gray')
156 |     plt.subplot(122)
157 |     plt.imshow(bezier,'gray')
158 |     plt.show()
159 | 
160 | def getStrokeBezier1(control_num,srcStrokes):
161 |     in_curves = []
162 |     out = []
163 |     srcStrokes = srcStrokes['strokes']
164 |     
165 |     for i,stroke in enumerate(srcStrokes):
166 |         if stroke['draw_type'] == 1:
167 |             continue
168 | 
169 |         src_p = np.array(stroke['path'])
170 |         if len(src_p) < control_num:
171 |             continue
172 | 
173 |         inputs = get_bezier_parameters(src_p[:,0], src_p[:,1], degree=control_num-1)
174 |         
175 |         b_points = bezier_curve(inputs.tolist(), nTimes=len(src_p))
176 |         b_points = np.array(b_points).T
177 | 
178 |         tree = spatial.cKDTree(src_p)
179 |         mindist, minid = tree.query(b_points)
180 |         src_p = src_p[minid]
181 |         offset = src_p - b_points
182 |         outputs = (np.clip(offset,-20,20)+20)/40
183 | 
184 |         idx = np.arange(0,len(offset),1)/len(offset)
185 |         idx = idx[:,np.newaxis]
186 |         inputs_loc = np.array(inputs).flatten()[np.newaxis,:]/799
187 | 
188 |         inputs_loc = np.repeat(inputs_loc,repeats=len(offset),axis=0)
189 |         inputs_all = np.concatenate((inputs_loc,idx),axis=1)
190 |         
191 | 
192 |         # canvas = drawPath(b_points,0)
193 |         # canvas = drawPath(src_p,(255,0,0),canvas)
194 |         # plt.imshow(canvas)
195 |         # plt.show()
196 |         
197 |         in_curves += inputs_all.tolist()
198 |         out += outputs.tolist()
199 |     print(len(in_curves),len(out))
200 |     return in_curves,out
201 | 
202 | def getOffset(mu1,std1,mu2,std2,b_points):
203 |     mu1,std1 = mu1*20-10, std1*5
204 |     mu2,std2 = mu2*20-10, std2*5
205 | 
206 |     rdn_x = np.random.normal(mu1,std1,len(b_points))[:,np.newaxis]
207 |     rdn_y = np.random.normal(mu2,std2,len(b_points))[:,np.newaxis]
208 |     rdn_offset = np.concatenate((rdn_x,rdn_y),axis=1)
209 |     canvas = drawPath(b_points+rdn_offset,0)
210 |     plt.imshow(canvas)
211 |     plt.show()
212 | 
213 | def getStrokeBezier(control_num,srcStrokes):
214 |     in_curves = []
215 |     out = []
216 |     srcStrokes = srcStrokes['strokes']
217 |     
218 |     for i,stroke in enumerate(srcStrokes):
219 |         if stroke['draw_type'] == 1:
220 |             continue
221 | 
222 |         src_p = np.array(stroke['path'])
223 |         if len(src_p) < control_num:
224 |             continue
225 | 
226 |         inputs = get_bezier_parameters(src_p[:,0], src_p[:,1], degree=control_num-1)
227 |         
228 |         b_points = bezier_curve(inputs.tolist(), nTimes=len(src_p))
229 |         b_points = np.array(b_points).T
230 | 
231 |         tree = spatial.cKDTree(src_p)
232 |         mindist, minid = tree.query(b_points)
233 |         src_p = src_p[minid]
234 |         offset = src_p - b_points
235 |         mu1,std1 = np.mean(offset[:,0]),np.std(offset[:,0])
236 |         mu2,std2 = np.mean(offset[:,1]),np.std(offset[:,1])
237 | 
238 |         mu1,std1 = (np.clip(mu1,-10,10)+10)/20, np.clip(std1,0,5)/5
239 |         mu2,std2 = (np.clip(mu2,-10,10)+10)/20, np.clip(std2,0,5)/5
240 |         outputs = np.array([mu1,std1,mu2,std2])
241 |         inputs_loc = np.array(inputs).flatten()/799
242 | 
243 |         # print(inputs_loc)
244 |         # print(outputs)
245 |         # getOffset(mu1,std1,mu2,std2,b_points)
246 | 
247 |         in_curves.append(inputs_loc.tolist())
248 |         out.append(outputs.tolist())
249 | 
250 |     return in_curves,out
251 |     
252 | if __name__ == "__main__":
253 |     
254 |     cg_list = ['Primitive','Chair','Lamp','Industrial_Component','Shoe','Animal','Animal_Head','Vehicle','Human_Face']
255 | 
256 |     control_num = 6
257 | 
258 |     root_dir = "../DifferSketching_Dataset" # the sketch of DifferSketching Dataset
259 |     
260 |     save_folder = "./data/curve_noise/"
261 |     os.makedirs(save_folder,exist_ok=True)
262 |     
263 |     total_in_curves, total_out = [], []
264 |     for cg in cg_list:
265 | 
266 |         src_dir = os.path.join(root_dir,"%s/global_json/"%cg)
267 |         
268 |         json_list = natsorted(os.listdir(src_dir))
269 |         for i,json_path in enumerate(json_list):
270 |             print(cg,i,json_path,end='\r')
271 | 
272 |             srcStrokes = readStrokes(os.path.join(src_dir,json_path))
273 | 
274 |             in_cur,output = getStrokeBezier(control_num,srcStrokes)
275 | 
276 |             total_in_curves += in_cur
277 |             total_out += output
278 | 
279 |         print('\n len:',len(total_in_curves))
280 | 
281 |         
282 |     np.savetxt(os.path.join(save_folder,'in_all.txt'),total_in_curves)
283 |     np.savetxt(os.path.join(save_folder,'out_all.txt'),total_out)
284 | 
285 | 


--------------------------------------------------------------------------------
/sketch_synthesis/prepare_data/getExtrinsicData.py:
--------------------------------------------------------------------------------
  1 | import numpy as np
  2 | from scipy.special import comb
  3 | import matplotlib.pyplot as plt
  4 | import os,cv2,json,copy
  5 | from natsort import natsorted
  6 | from numpy.linalg import det
  7 | import math
  8 | 
  9 | def readStrokes(path):
 10 |     file = open(path,"r")
 11 |     json_str = file.read()
 12 |     strokes = json.loads(json_str)
 13 |     return strokes
 14 | 
 15 | def renderSketch(strokes):
 16 |     canvas = np.ones((800,800),dtype="uint8")*255
 17 | 
 18 |     for stroke in strokes['strokes']:
 19 |             
 20 |         if stroke['draw_type'] == 1:
 21 |             continue
 22 | 
 23 |         path = stroke['path']
 24 |         path = np.array(path).astype("int")
 25 |         for i in range(len(path)-1):
 26 |             cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),0,3)
 27 | 
 28 |     return canvas
 29 | 
 30 | def get_bezier_parameters(X, Y, degree=3):
 31 |     """ Least square qbezier fit using penrose pseudoinverse.
 32 | 
 33 |     Parameters:
 34 | 
 35 |     X: array of x data.
 36 |     Y: array of y data. Y[0] is the y point for X[0].
 37 |     degree: degree of the Bézier curve. 2 for quadratic, 3 for cubic.
 38 | 
 39 |     Based on https://stackoverflow.com/questions/12643079/b%C3%A9zier-curve-fitting-with-scipy
 40 |     and probably on the 1998 thesis by Tim Andrew Pastva, "Bézier Curve Fitting".
 41 |     """
 42 |     if degree < 1:
 43 |         raise ValueError('degree must be 1 or greater.')
 44 | 
 45 |     if len(X) != len(Y):
 46 |         raise ValueError('X and Y must be of the same length.')
 47 | 
 48 |     if len(X) < degree + 1:
 49 |         raise ValueError(f'There must be at least {degree + 1} points to '
 50 |                          f'determine the parameters of a degree {degree} curve. '
 51 |                          f'Got only {len(X)} points.')
 52 | 
 53 |     def bpoly(n, t, k):
 54 |         """ Bernstein polynomial when a = 0 and b = 1. """
 55 |         return t ** k * (1 - t) ** (n - k) * comb(n, k)
 56 |         #return comb(n, i) * ( t**(n-i) ) * (1 - t)**i
 57 | 
 58 |     def bmatrix(T):
 59 |         """ Bernstein matrix for Bézier curves. """
 60 |         return np.matrix([[bpoly(degree, t, k) for k in range(degree + 1)] for t in T])
 61 | 
 62 |     def least_square_fit(points, M):
 63 |         M_ = np.linalg.pinv(M)
 64 |         return M_ * points
 65 | 
 66 |     T = np.linspace(0, 1, len(X))
 67 |     M = bmatrix(T)
 68 |     points = np.array(list(zip(X, Y)))
 69 |     
 70 |     final = least_square_fit(points, M).tolist()
 71 |     final[0] = [X[0], Y[0]]
 72 |     final[len(final)-1] = [X[len(X)-1], Y[len(Y)-1]]
 73 |     return np.array(final)
 74 | 
 75 | def bernstein_poly(i, n, t):
 76 |     """
 77 |      The Bernstein polynomial of n, i as a function of t
 78 |     """
 79 |     return comb(n, i) * ( t**(n-i) ) * (1 - t)**i
 80 | 
 81 | 
 82 | def bezier_curve(points, nTimes=50):
 83 |     """
 84 |        Given a set of control points, return the
 85 |        bezier curve defined by the control points.
 86 | 
 87 |        points should be a list of lists, or list of tuples
 88 |        such as [ [1,1], 
 89 |                  [2,3], 
 90 |                  [4,5], ..[Xn, Yn] ]
 91 |         nTimes is the number of time steps, defaults to 1000
 92 | 
 93 |         See http://processingjs.nihongoresources.com/bezierinfo/
 94 |     """
 95 | 
 96 |     nPoints = len(points)
 97 |     xPoints = np.array([p[0] for p in points])
 98 |     yPoints = np.array([p[1] for p in points])
 99 | 
100 |     t = np.linspace(0.0, 1.0, nTimes)
101 | 
102 |     polynomial_array = np.array([ bernstein_poly(i, nPoints-1, t) for i in range(0, nPoints)   ])
103 | 
104 |     xvals = np.dot(xPoints, polynomial_array)
105 |     yvals = np.dot(yPoints, polynomial_array)
106 | 
107 |     return xvals, yvals
108 | 
109 | def drawPath(path,color,canvas=None):
110 |     if canvas is None:
111 |         canvas = np.ones((800,800,3),dtype="uint8")*255
112 |     path = path.astype('int')
113 | 
114 |     for i in range(len(path)-1):
115 |         cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),color,2)
116 |     return canvas
117 | 
118 | 
119 | def showBezier(points,control_num):
120 |     data = get_bezier_parameters(points[:,0], points[:,1], degree=control_num-1).tolist()
121 | 
122 |     xvals, yvals = bezier_curve(data, nTimes=500)
123 | 
124 |     # # Plot the control points
125 |     # data = np.array(data)
126 |     # x_val = data[:,0]
127 |     # y_val = data[:,1]
128 |     # plt.plot(points[:,0], points[:,1], "ro",label='Original Points')
129 |     # plt.plot(x_val,y_val,'k--o', label='Control Points')
130 |     # # Plot the resulting Bezier curve
131 |     # plt.plot(xvals, yvals, 'b-', label='B Curve')
132 |     # plt.legend()
133 |     # plt.show()
134 |     return xvals,yvals
135 | 
136 | def visualization(control_num,json_path):
137 |     strokes = readStrokes(json_path)
138 |     curves = copy.deepcopy(strokes)
139 | 
140 |     for i,stroke in enumerate(strokes['strokes']):
141 |         if stroke['draw_type'] == 1:
142 |             continue
143 |         points = np.array(stroke['path'])
144 |         if len(points) < control_num:
145 |             continue
146 |         canvas = drawPath(points,0)
147 |         x,y = showBezier(points,control_num)
148 |         curve = np.array([x,y]).T
149 |         curves['strokes'][i]['path'] = curve.astype('int')
150 |     
151 |     origin = renderSketch(strokes)
152 |     bezier = renderSketch(curves)
153 | 
154 |     plt.subplot(121)
155 |     plt.imshow(origin,'gray')
156 |     plt.subplot(122)
157 |     plt.imshow(bezier,'gray')
158 |     plt.show()
159 | 
160 | # get normalized R,T,S
161 | def getRST(M):
162 |     R_s = M[:2,:2]
163 |     T = M[:,-1].T
164 |     if (det(R_s)**0.5) < 0.0000001:
165 |         return None
166 |     R = R_s / (det(R_s)**0.5)
167 | 
168 |     if R[0,0] < 0.0000001:
169 |         return None
170 | 
171 |     if abs(R[0,0]-1) < 0.00001:
172 |         R[0,0] = 0.999
173 |         R[1,1] = 0.999
174 | 
175 |     theta = math.asin(R[0,1]) * (180/np.pi)
176 | 
177 |     scale = R_s[0,0] / R[0,0]
178 | 
179 |     theta = (np.clip(theta,-45,45)+45)/90
180 |     T = (np.clip(T,-800,800)+800)/1600
181 |     scale = (np.clip(scale,0.1,2.1)-0.1)/2
182 | 
183 |     return theta,scale,T
184 | 
185 | # get transform matrix from R,T,S
186 | def RST2M(R,S,T):
187 |     print(R,S,T)
188 |     S = S * 2+0.1
189 |     R = (R * 90 -45) * (np.pi/180)
190 |     T = T * 1600 - 800
191 |     alpha,beta = math.cos(R)*S, math.sin(R)*S
192 |     M = [[alpha,beta,T[0]],\
193 |          [-beta,alpha,T[1]]]
194 |     return np.array(M,dtype='float')
195 |     
196 | 
197 | def getStrokeData(control_num,srcStrokes,dstStrokes,strokePrec):
198 |     in_curves = []
199 |     out_RST = []
200 |     srcStrokes = srcStrokes['strokes']
201 |     dstStrokes = dstStrokes['strokes']
202 |     canvas = np.ones((800,800,3),dtype='uint8')*255
203 |     for i,stroke in enumerate(srcStrokes):
204 | 
205 |         if stroke['draw_type'] == 1:
206 |             continue
207 |         if strokePrec[i] < 0.7:
208 |             continue
209 |         src_p = np.array(stroke['path']).astype('int')
210 |         dst_p = np.array(dstStrokes[i]['path']).astype('int')
211 |         if len(src_p) < control_num:
212 |             continue
213 |         assert src_p.shape[0] == dst_p.shape[0]
214 |         
215 |         # M = cv2.estimateRigidTransform(src_p, dst_p,fullAffine=False)
216 |         M = cv2.estimateAffinePartial2D(src_p,dst_p)[0]
217 | 
218 |         if M is None:
219 |             continue
220 | 
221 |         RST = getRST(M)
222 | 
223 |         if RST is None:
224 |             continue
225 |         r,s,t = RST
226 | 
227 |         inputs = get_bezier_parameters(src_p[:,0], src_p[:,1], degree=control_num-1)
228 |         
229 |         # new_M = RST2M(r,s,t)
230 |         # reg_path = cv2.transform(src_p[:,np.newaxis,:], new_M).squeeze()
231 |         # src_points = bezier_curve(inputs.tolist(), nTimes=500)
232 |         # src_points = np.array(src_points).T
233 |         # canvas = drawPath(src_p,0,canvas)
234 |         # canvas = drawPath(dst_p,(255,0,0),canvas)
235 |         # canvas = drawPath(reg_path,(0,255,0),canvas)
236 |         # plt.imshow(canvas)
237 |         # plt.show()
238 | 
239 |         inputs = np.round(inputs).astype('int').flatten()
240 |         inputs = inputs.tolist()
241 | 
242 |         dist = abs(r-0.5)+abs(s-0.5)+np.abs(np.array(t)-0.5).mean()
243 |         inputs.append(dist)
244 | 
245 |         outputs = [r,s,t[0],t[1]]
246 | 
247 |         in_curves.append(inputs)
248 |         out_RST.append(outputs)
249 |         
250 |     return in_curves,out_RST
251 | 
252 | if __name__ == "__main__":
253 |     NP = 'P'  # switch between novice and professional sketches using 'N' or 'P'
254 | 
255 |     cg_list = ['Primitive','Chair','Lamp','Industrial_Component','Shoe','Animal','Animal_Head','Vehicle','Human_Face']
256 | 
257 |     control_num = 6 # number of control points for bezier curves
258 |     root_dir = "../DifferSketching_Dataset" # the sketch of DifferSketching Dataset
259 |     fail_list = open(os.path.join(root_dir,'eval_lower_1.2.txt')).read().split('\n') # filter out the failure cases of registration
260 | 
261 |     save_folder = "./data/extrinsic/"
262 |     os.makedirs(save_folder,exist_ok=True)
263 |     
264 |     total_in_curves, total_out_M = [], []
265 |     for cg in cg_list:
266 |         src_dir = os.path.join(root_dir,"%s/stroke_json/"%cg)
267 |         dst_dir = os.path.join(root_dir,"%s/global_json/"%cg)
268 | 
269 |         strokePrec_list = readStrokes(os.path.join(root_dir,'%s/regPrec_dict.json'%cg))
270 | 
271 |         json_list = natsorted(os.listdir(dst_dir))
272 |         for i,json_path in enumerate(json_list):
273 |             if NP != '' and not json_path.startswith(NP):
274 |                 continue
275 |             if json_path.split('.')[0] in fail_list:
276 |                 continue
277 | 
278 |             print(cg,i,json_path,end='\r')
279 |             strokePrec = strokePrec_list[json_path]
280 |             srcStrokes = readStrokes(os.path.join(src_dir,json_path))
281 |             dstStrokes = readStrokes(os.path.join(dst_dir,json_path))
282 | 
283 |             in_cur, out_rst = getStrokeData(control_num,srcStrokes,dstStrokes,strokePrec)
284 |             
285 |             total_in_curves += in_cur
286 |             total_out_M += out_rst
287 |             # total_out_M += out_curves
288 |             # visualization(control_num,os.path.join(src_dir,json_path))
289 | 
290 |         print('\n len:',len(total_in_curves))
291 | 
292 |     np.savetxt(os.path.join(save_folder,'in_curves_%d_%s.txt'%(control_num,NP)),total_in_curves)
293 |     np.savetxt(os.path.join(save_folder,'out_%d_%s.txt'%(control_num,NP)),total_out_M)
294 | 
295 | 


--------------------------------------------------------------------------------
/sketch_synthesis/prepare_data/getIntrinsicData.py:
--------------------------------------------------------------------------------
  1 | import numpy as np
  2 | from scipy.special import comb
  3 | import matplotlib.pyplot as plt
  4 | import os,cv2,json,copy
  5 | from natsort import natsorted
  6 | 
  7 | def readStrokes(path):
  8 |     file = open(path,"r")
  9 |     json_str = file.read()
 10 |     strokes = json.loads(json_str)
 11 |     return strokes
 12 | 
 13 | def renderSketch(strokes):
 14 |     canvas = np.ones((800,800),dtype="uint8")*255
 15 | 
 16 |     for stroke in strokes['strokes']:
 17 |             
 18 |         if stroke['draw_type'] == 1:
 19 |             continue
 20 | 
 21 |         path = stroke['path']
 22 |         path = np.array(path).astype("int")
 23 |         for i in range(len(path)-1):
 24 |             cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),0,3)
 25 | 
 26 |     return canvas
 27 | 
 28 | def get_bezier_parameters(X, Y, degree=3):
 29 |     """ Least square qbezier fit using penrose pseudoinverse.
 30 | 
 31 |     Parameters:
 32 | 
 33 |     X: array of x data.
 34 |     Y: array of y data. Y[0] is the y point for X[0].
 35 |     degree: degree of the Bézier curve. 2 for quadratic, 3 for cubic.
 36 | 
 37 |     Based on https://stackoverflow.com/questions/12643079/b%C3%A9zier-curve-fitting-with-scipy
 38 |     and probably on the 1998 thesis by Tim Andrew Pastva, "Bézier Curve Fitting".
 39 |     """
 40 |     if degree < 1:
 41 |         raise ValueError('degree must be 1 or greater.')
 42 | 
 43 |     if len(X) != len(Y):
 44 |         raise ValueError('X and Y must be of the same length.')
 45 | 
 46 |     if len(X) < degree + 1:
 47 |         raise ValueError(f'There must be at least {degree + 1} points to '
 48 |                          f'determine the parameters of a degree {degree} curve. '
 49 |                          f'Got only {len(X)} points.')
 50 | 
 51 |     def bpoly(n, t, k):
 52 |         """ Bernstein polynomial when a = 0 and b = 1. """
 53 |         return t ** k * (1 - t) ** (n - k) * comb(n, k)
 54 |         #return comb(n, i) * ( t**(n-i) ) * (1 - t)**i
 55 | 
 56 |     def bmatrix(T):
 57 |         """ Bernstein matrix for Bézier curves. """
 58 |         return np.matrix([[bpoly(degree, t, k) for k in range(degree + 1)] for t in T])
 59 | 
 60 |     def least_square_fit(points, M):
 61 |         M_ = np.linalg.pinv(M)
 62 |         return M_ * points
 63 | 
 64 |     T = np.linspace(0, 1, len(X))
 65 |     M = bmatrix(T)
 66 |     points = np.array(list(zip(X, Y)))
 67 |     
 68 |     final = least_square_fit(points, M).tolist()
 69 |     final[0] = [X[0], Y[0]]
 70 |     final[len(final)-1] = [X[len(X)-1], Y[len(Y)-1]]
 71 |     return np.array(final)
 72 | 
 73 | def bernstein_poly(i, n, t):
 74 |     """
 75 |      The Bernstein polynomial of n, i as a function of t
 76 |     """
 77 |     return comb(n, i) * ( t**(n-i) ) * (1 - t)**i
 78 | 
 79 | 
 80 | def bezier_curve(points, nTimes=50):
 81 |     """
 82 |        Given a set of control points, return the
 83 |        bezier curve defined by the control points.
 84 | 
 85 |        points should be a list of lists, or list of tuples
 86 |        such as [ [1,1], 
 87 |                  [2,3], 
 88 |                  [4,5], ..[Xn, Yn] ]
 89 |         nTimes is the number of time steps, defaults to 1000
 90 | 
 91 |         See http://processingjs.nihongoresources.com/bezierinfo/
 92 |     """
 93 | 
 94 |     nPoints = len(points)
 95 |     xPoints = np.array([p[0] for p in points])
 96 |     yPoints = np.array([p[1] for p in points])
 97 | 
 98 |     t = np.linspace(0.0, 1.0, nTimes)
 99 | 
100 |     polynomial_array = np.array([ bernstein_poly(i, nPoints-1, t) for i in range(0, nPoints)   ])
101 | 
102 |     xvals = np.dot(xPoints, polynomial_array)
103 |     yvals = np.dot(yPoints, polynomial_array)
104 | 
105 |     return xvals, yvals
106 | 
107 | def drawPath(path,color,canvas=None):
108 |     if canvas is None:
109 |         canvas = np.ones((800,800,3),dtype="uint8")*255
110 |     path = path.astype('int')
111 | 
112 |     for i in range(len(path)-1):
113 |         cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),color,2)
114 |     return canvas
115 | 
116 | 
117 | def showBezier(points,control_num):
118 |     data = get_bezier_parameters(points[:,0], points[:,1], degree=control_num-1).tolist()
119 | 
120 |     xvals, yvals = bezier_curve(data, nTimes=500)
121 | 
122 |     # # Plot the control points
123 |     # data = np.array(data)
124 |     # x_val = data[:,0]
125 |     # y_val = data[:,1]
126 |     # plt.plot(points[:,0], points[:,1], "ro",label='Original Points')
127 |     # plt.plot(x_val,y_val,'k--o', label='Control Points')
128 |     # # Plot the resulting Bezier curve
129 |     # plt.plot(xvals, yvals, 'b-', label='B Curve')
130 |     # plt.legend()
131 |     # plt.show()
132 |     return xvals,yvals
133 | 
134 | def visualization(control_num,json_path):
135 |     strokes = readStrokes(json_path)
136 |     curves = copy.deepcopy(strokes)
137 | 
138 |     for i,stroke in enumerate(strokes['strokes']):
139 |         if stroke['draw_type'] == 1:
140 |             continue
141 |         points = np.array(stroke['path'])
142 |         if len(points) < control_num:
143 |             continue
144 |         canvas = drawPath(points,0)
145 |         x,y = showBezier(points,control_num)
146 |         curve = np.array([x,y]).T
147 |         curves['strokes'][i]['path'] = curve.astype('int')
148 |     
149 |     origin = renderSketch(strokes)
150 |     bezier = renderSketch(curves)
151 | 
152 |     plt.subplot(121)
153 |     plt.imshow(origin,'gray')
154 |     plt.subplot(122)
155 |     plt.imshow(bezier,'gray')
156 |     plt.show()
157 | 
158 | def getStrokeBezier(control_num,srcStrokes,dstStrokes,strokePrec):
159 |     in_curves = []
160 |     out_curves = []
161 |     srcStrokes = srcStrokes['strokes']
162 |     dstStrokes = dstStrokes['strokes']
163 |     
164 |     for i,stroke in enumerate(srcStrokes):
165 |         if stroke['draw_type'] == 1:
166 |             continue
167 |         if strokePrec[i] < 0.7:
168 |             continue
169 |         src_p = np.array(stroke['path'])
170 |         if len(src_p) < control_num:
171 |             continue
172 |         dst_p = np.array(dstStrokes[i]['path'])
173 | 
174 |         outputs = get_bezier_parameters(dst_p[:,0], dst_p[:,1], degree=control_num-1)
175 |         inputs = get_bezier_parameters(src_p[:,0], src_p[:,1], degree=control_num-1)
176 | 
177 |         avg_dist = np.round(np.mean(np.sum((outputs-inputs)**2,axis=1)**0.5)).astype('int')
178 |         
179 |         # src_points = bezier_curve(inputs.tolist(), nTimes=500)
180 |         # src_points = np.array(src_points).T
181 |         # canvas = drawPath(src_points,0)
182 |         # dst_points = bezier_curve(outputs.tolist(), nTimes=500)
183 |         # dst_points = np.array(dst_points).T
184 |         # canvas = drawPath(dst_points,(255,0,0),canvas)
185 |         # plt.imshow(canvas)
186 |         # plt.show()
187 | 
188 |         inputs = np.round(inputs).astype('int').flatten()
189 |         outputs = np.round(outputs).astype('int').flatten()
190 |         
191 |         inputs = inputs.tolist()
192 |         inputs.append(avg_dist)
193 |         in_curves.append(inputs)
194 |         out_curves.append(outputs.tolist())
195 |         
196 |     return in_curves,out_curves
197 |     
198 | if __name__ == "__main__":
199 |     NP = 'P'
200 |     # NP = '_N' # switch between novice and professional sketches
201 | 
202 |     cg_list = ['Primitive','Chair','Lamp','Industrial_Component','Shoe','Animal','Animal_Head','Vehicle','Human_Face']
203 | 
204 |     control_num = 6 # number of control points for bezier curves
205 | 
206 |     root_dir = "../DifferSketching_Dataset" # the sketch of DifferSketching Dataset
207 |     fail_list = open(os.path.join(root_dir,'eval_lower_1.2.txt')).read().split('\n') # filter out the failure cases of registration
208 | 
209 |     save_folder = "./data/intrinsic/"
210 |     os.makedirs(save_folder,exist_ok=True)
211 |     
212 |     total_in_curves, total_out_curves = [], []
213 |     for cg in cg_list:
214 |         in_curves, out_curves = [], []
215 |         src_dir = os.path.join(root_dir,"%s/reg_json/"%cg)
216 |         dst_dir = os.path.join(root_dir,"%s/stroke_json/"%cg)
217 |         strokePrec_list = readStrokes(os.path.join(root_dir,'%s/regPrec_dict.json'%cg)) # filter out the strokes with low registration precision
218 | 
219 |         json_list = natsorted(os.listdir(dst_dir))
220 |         for i,json_path in enumerate(json_list):
221 |             if NP != '' and not json_path.startswith(NP):
222 |                 continue
223 |             if json_path.split('.')[0] in fail_list:
224 |                 continue
225 |             print(cg,i,json_path,end='\r')
226 | 
227 |             
228 |             strokePrec = strokePrec_list[json_path]
229 |             srcStrokes = readStrokes(os.path.join(src_dir,json_path))
230 |             dstStrokes = readStrokes(os.path.join(dst_dir,json_path))
231 | 
232 |             in_cur, out_cur  = getStrokeBezier(control_num,srcStrokes,dstStrokes,strokePrec)
233 |             in_curves += in_cur
234 |             out_curves += out_cur
235 | 
236 |             # visualization(control_num,os.path.join(src_dir,json_path))
237 |         print('\n len:',len(in_curves))
238 | 
239 |         total_in_curves += in_curves
240 |         total_out_curves += out_curves
241 | 
242 |     np.savetxt(os.path.join(save_folder,'in_curves_%d_%s.txt'%(control_num,NP)),total_in_curves,'%d')
243 |     np.savetxt(os.path.join(save_folder,'out_curves_%d_%s.txt'%(control_num,NP)),total_out_curves,'%d')
244 | 
245 | 


--------------------------------------------------------------------------------
/sketch_synthesis/requirements.txt:
--------------------------------------------------------------------------------
1 | matplotlib==3.10.1
2 | natsort==8.4.0
3 | numpy==2.2.5
4 | opencv_python==4.11.0.86
5 | scikit_learn==1.6.1
6 | scipy==1.15.2
7 | 


--------------------------------------------------------------------------------
/sketch_synthesis/results/N/P021_8_1_SAH_210_in=0.30_ex=0.30.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chufengxiao/DifferSketching/305ce2d66a8fd63d325145a90a9378591e2d754f/sketch_synthesis/results/N/P021_8_1_SAH_210_in=0.30_ex=0.30.png


--------------------------------------------------------------------------------
/sketch_synthesis/results/P/P021_8_1_SAH_210_in=0.30_ex=0.30.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/chufengxiao/DifferSketching/305ce2d66a8fd63d325145a90a9378591e2d754f/sketch_synthesis/results/P/P021_8_1_SAH_210_in=0.30_ex=0.30.png


--------------------------------------------------------------------------------
/sketch_synthesis/test.py:
--------------------------------------------------------------------------------
  1 | #%%
  2 | import numpy as np
  3 | import os,json,cv2,math
  4 | import matplotlib.pyplot as plt
  5 | import pickle,copy
  6 | 
  7 | from components import readStrokes, get_bezier_parameters,bezier_curve
  8 | 
  9 | from scipy import spatial
 10 | from scipy.special import softmax
 11 | from scipy.optimize import minimize,Bounds
 12 | 
 13 | def drawCanvas(path,canvas,color=0):
 14 |     canvas = np.array(canvas)
 15 |     path = np.array(path).astype('int')
 16 |     for i in range(len(path)-1):
 17 |         cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),color,3)
 18 | 
 19 |     return canvas
 20 | 
 21 | def optimize(path,target_list,idx_list):
 22 |     def L2(src,dst):
 23 |         item = np.sum((src-dst)**2,axis=1)**0.5
 24 | 
 25 |         return np.sum(item)
 26 | 
 27 |     def shapeF(path):
 28 |         valid_p = path[1:-1:5]
 29 |         pre_p = path[:-2:5]
 30 |         post_p = path[2::5]
 31 | 
 32 |         shape_feat = valid_p - 0.5*(pre_p+post_p)
 33 |         return shape_feat
 34 | 
 35 |     def smoothTerm(path,fix_idx):
 36 |         error = 0
 37 |         idx_list = np.where(fix_idx==1)[0]
 38 |         for idx in idx_list:
 39 |             front = max(0,idx-10)
 40 |             back = min(len(path),idx+10)
 41 |             
 42 |             pre_p = path[front:back-1]
 43 |             post_p = path[front+1:back]
 44 |   
 45 |             error += L2(pre_p,post_p)
 46 |         return error
 47 | 
 48 |     def objf(pred_path,origin_path,fix_points,fix_idx):
 49 |         pred_path = pred_path.reshape((2,-1)).T
 50 | 
 51 |         ## position term
 52 |         fix_error = L2(pred_path*fix_idx,fix_points*fix_idx)
 53 |         # rest_error = L2(pred_path*(1-fix_idx),origin_path*(1-fix_idx))
 54 |         P_term = fix_error*2000 #+ rest_error
 55 | 
 56 |         ## shape term
 57 |         gt_shape = shapeF(origin_path)
 58 |         pred_shape = shapeF(pred_path)
 59 |         S_term = L2(gt_shape,pred_shape)
 60 | 
 61 |         M_term = 500*smoothTerm(pred_path,fix_idx)
 62 |         # print(P_term,S_term,M_term)
 63 | 
 64 | 
 65 |         return P_term + M_term + S_term
 66 | 
 67 |     # canvas = np.ones((800,800,3),dtype="uint8")*255
 68 |     # canvas = drawCanvas(path,canvas,0)
 69 |     # path = np.round(path).astype('int')
 70 |     # for idx in range(len(idx_list)):
 71 |     #     canvas = cv2.circle(canvas,tuple(target_list[idx]),4,(255,0,0),3)
 72 |     #     canvas = cv2.circle(canvas,tuple(path[idx_list[idx]]),3,(0,255,0),3)
 73 |     #     cv2.line(canvas,tuple(target_list[idx]),tuple(path[idx_list[idx]]),(127,127,127),2)
 74 |     # plt.subplot(121)
 75 |     # plt.imshow(canvas)
 76 | 
 77 |     target_list = np.array(target_list)
 78 | 
 79 |     f_path = np.array(path).T.flatten()
 80 |     fix_points = np.array(path)
 81 |     fix_points[idx_list] = target_list
 82 |     fix_idx = np.zeros((len(fix_points),1),dtype='int')
 83 |     fix_idx[idx_list] = 1
 84 | 
 85 |     result = minimize(objf,f_path,args=(path,fix_points,fix_idx),method='L-BFGS-B',bounds=Bounds(0,799))
 86 |     # 'SLSQP'
 87 |     output = np.round(result.x).astype('int')
 88 |     output = output.reshape((2,-1)).T
 89 | 
 90 |     # canvas = drawCanvas(output,canvas,(255,0,0))
 91 |     # plt.subplot(122)
 92 |     # plt.imshow(canvas)
 93 |     # plt.show()
 94 |     return output
 95 | 
 96 | def getValidStrokes(strokes):
 97 |     validStrokes = []
 98 |     for stroke in strokes['strokes']:
 99 |         if 'draw_type' in stroke and (stroke['draw_type'] != 0 or len(stroke['path']) < 6):
100 |             continue
101 |         path = np.array(stroke['path'])
102 | 
103 |         _, idx = np.unique(path,axis=0,return_index=True)
104 | 
105 |         if len(idx) > 6:
106 |             path = path[sorted(idx)]
107 | 
108 |         validStrokes.append(path)
109 |     return validStrokes
110 | 
111 | def getStrokeBezier(control_num,srcStrokes):
112 |     
113 |     in_curves = []
114 | 
115 |     for i,src_p in enumerate(srcStrokes):
116 |         if len(src_p) < control_num:
117 |             continue
118 |         src_p = np.array(src_p)
119 |         inputs = get_bezier_parameters(src_p[:,0], src_p[:,1], degree=control_num-1)
120 |         inputs = np.round(inputs).astype('int').flatten()/799
121 |         
122 |         in_curves.append(inputs)
123 |     return np.array(in_curves)
124 | 
125 | def renderSketch(strokes):
126 |     canvas = np.ones((800,800,3),dtype="uint8")*255
127 |     cmap = plt.cm.jet
128 | 
129 |     colors = cmap(np.linspace(0, 1, len(strokes)))
130 |     colors = np.array(colors[:,:3]*255,dtype="uint8").tolist()
131 | 
132 |     s_num = 0
133 |     for j,stroke in enumerate(strokes):
134 |         
135 |         path = np.array(stroke).astype("int")
136 |         
137 |         for i in range(len(path)-1):
138 |             cv2.line(canvas,(path[i][0],path[i][1]),(path[i+1][0],path[i+1][1]),colors[s_num],3)
139 |         s_num += 1
140 | 
141 |     return canvas
142 | 
143 | def writeJSON(strokes,json_path):
144 |     saveStrokes = []
145 |     for stroke in strokes:
146 |         saveStrokes.append({'path':stroke.tolist()})
147 |     saveStrokes = {'strokes':saveStrokes}
148 |     with open(json_path,'w+') as f:
149 |         f.write(json.dumps(saveStrokes))
150 |         print('write',json_path)
151 | 
152 | class Synthesis:
153 |     def __init__(self,NP='N'):
154 |         
155 |         self.mlp1 = pickle.load(open('./model/%s_mlp_intrinsic.sav'%NP, 'rb'))
156 |         self.mlp2 = pickle.load(open('./model/%s_mlp_extrinsic.sav'%NP, 'rb'))
157 |         self.mlp3 = pickle.load(open('./model/mlp_curveNoise.sav', 'rb'))
158 |         self.cp = 6
159 |         self.srcStrokes = None
160 |         
161 | 
162 |     def load(self,srcStrokes):
163 |         self.srcStrokes = srcStrokes
164 |         b_curves = getStrokeBezier(self.cp,self.srcStrokes)
165 |         srcCurve = np.round(b_curves*799).astype('int')
166 |         self.srcCurve = self.renderCurves(srcCurve)
167 | 
168 |         # src_img = renderSketch(self.srcCurve)
169 |         # plt.imshow(src_img)
170 |         # plt.show()
171 | 
172 | 
173 |     def getInCurve(self,strokes,n1,n2):
174 |         in_cur = getStrokeBezier(self.cp,strokes)
175 |         dist1 = np.ones((len(in_cur),1))*n1
176 |         dist2 = np.ones((len(in_cur),1))*n2
177 |         self.input1 = np.concatenate((in_cur,dist1),axis=1)
178 |         self.input2 = np.concatenate((in_cur,dist2),axis=1)
179 |         
180 |         return in_cur
181 | 
182 |     def predict1(self,strokes,noise):
183 |         in_cur = getStrokeBezier(self.cp,strokes)
184 |         dist = np.ones((len(in_cur),1))*noise
185 |         input = np.concatenate((in_cur,dist),axis=1)
186 |         output = self.mlp1.predict(input)
187 |         output = np.round(output*799).astype('int')
188 |         output = self.renderCurves(output)
189 |         return output
190 |     
191 |     def predict2(self,strokes,noise):
192 |         in_cur = getStrokeBezier(self.cp,strokes)
193 |         dist = np.ones((len(in_cur),1))*noise
194 |         input = np.concatenate((in_cur,dist),axis=1)
195 |         pred = self.mlp2.predict(input)
196 |         output = self.transPath(pred,strokes)
197 |         visual = self.transPath(pred,self.srcCurve)
198 |         return output,visual
199 |     
200 |     def predict3(self,strokes):
201 |         input = getStrokeBezier(self.cp,strokes)
202 |         output = self.mlp3.predict(input)
203 |         output = self.localNoising(output,strokes)
204 |         return output
205 |     
206 |     def localNoising(self,preds,strokes):
207 |         noiseStrokes = []
208 |         for i,pred in enumerate(preds):
209 |             mu1,std1,mu2,std2 = pred
210 |             mu1,std1 = mu1*20-10, std1*5
211 |             mu2,std2 = mu2*20-10, std2*5
212 | 
213 |             rdn_x = np.random.normal(mu1,std1,len(strokes[i]))[:,np.newaxis]
214 |             rdn_y = np.random.normal(mu2,std2,len(strokes[i]))[:,np.newaxis]
215 |             rdn_offset = np.concatenate((rdn_x,rdn_y),axis=1)
216 |             rdn_offset = np.round(rdn_offset)
217 |             noiseStrokes.append(strokes[i]+rdn_offset)
218 |         return noiseStrokes
219 | 
220 |     def transOptimize(self,src,dst,i):
221 | 
222 |         offset_list = []
223 |         dist_list = []
224 | 
225 |         target_list = []
226 |         idx_list = []
227 | 
228 |         for j in range(i):
229 |             tree = spatial.cKDTree(src[j])
230 |             mindist, minid = tree.query(src[i])
231 |             b_idx = np.argmin(mindist)
232 |             a_idx = minid[b_idx]
233 | 
234 |             offset_src = src[i][b_idx] - src[j][a_idx]
235 |             target = offset_src + dst[j][a_idx]
236 |             offset_dst = target - dst[i][b_idx]
237 |             target = np.round(target).astype('int')
238 |             if mindist[b_idx] < 10:
239 |                 target_list.append(target)
240 |                 idx_list.append(b_idx)
241 |             
242 |             offset_list.append(offset_dst)
243 |             dist_list.append(mindist[b_idx])
244 | 
245 |         dist_list = np.array(dist_list)
246 |         weights = softmax(1/(dist_list+1))
247 |         trans = np.average(offset_list,axis=0,weights=weights)
248 |         trans = np.round(trans).astype('int')
249 |         initTrans_path = dst[i] + trans
250 |         needOptim = False
251 | 
252 |         for num,idx in enumerate(idx_list):
253 |             i_dist = np.sum((initTrans_path[idx] - target_list[num])**2)**0.5
254 | 
255 |             if i_dist > 5:
256 |                 needOptim = True
257 |                 break
258 |         if needOptim:
259 |             final_path = optimize(initTrans_path,target_list,idx_list)
260 |         else:
261 |             final_path = initTrans_path
262 |         return final_path
263 | 
264 |     def transPath(self,preds,strokes):
265 |         transStrokes = []
266 | 
267 |         for i,stroke in enumerate(strokes):
268 |             
269 |             R,S,t0,t1 = preds[i]
270 |             T = np.array([t0,t1])
271 | 
272 |             S = S * 2+0.1
273 |             R = (R * 90 -45) * (np.pi/180)
274 |             T = T * 1600 - 800
275 |             alpha,beta = math.cos(R)*S, math.sin(R)*S
276 |             
277 |             M = [[alpha,beta,T[0]],\
278 |                 [-beta,alpha,T[1]]]
279 |             M = np.array(M,dtype='float')
280 |             trans_path = cv2.transform(stroke[:,np.newaxis,:], M).squeeze()
281 |             transStrokes.append(trans_path)
282 |         return transStrokes
283 | 
284 |     def renderCurves(self,bezierPoints):
285 |         curves = []
286 |         for i,curve in enumerate(bezierPoints):
287 |             path = bezier_curve(curve,len(self.srcStrokes[i])).astype("int")
288 |             curves.append(path)
289 |         return curves
290 | 
291 |     def layout_refine(self,strokes,transMethod):
292 | 
293 |         # strokes = np.array(strokes)
294 |         strokes = copy.deepcopy(strokes)
295 |         
296 |         x_min,y_min,x_max,y_max = 9999,9999,-9999,-9999
297 |         for i in range(len(strokes)):
298 |             if i > 0:
299 |                 strokes[i] = transMethod(self.srcCurve,strokes,i)
300 |                 # strokes[i] = self.getTrans(self.srcCurve,strokes,i)
301 |                 # strokes[i] = self.transOptimize(self.srcCurve,strokes,i)
302 |                 sx_min,sy_min = np.min(strokes[i],axis=0)
303 |                 sx_max,sy_max = np.max(strokes[i],axis=0)
304 |                 x_min,y_min = min(sx_min,x_min),min(sy_min,y_min)
305 |                 x_max,y_max = max(sx_max,x_max),max(sy_max,y_max),
306 |         
307 |                 
308 |         m_x, m_y = int(400-(x_min+x_max)/2), int(400-(y_min+y_max)/2)
309 | 
310 |         for i in range(len(strokes)):
311 |             strokes[i] += [m_x,m_y]
312 |         return strokes
313 | 
314 |     def getTrans(self,src,dst,i):
315 |         offset_list = []
316 |         dist_list = []
317 | 
318 |         for j in range(i):
319 |             tree = spatial.cKDTree(src[j])
320 |             mindist, minid = tree.query(src[i])
321 |             b_idx = np.argmin(mindist)
322 |             a_idx = minid[b_idx]
323 | 
324 |             offset_src = src[i][b_idx] - src[j][a_idx]
325 |             target = offset_src + dst[j][a_idx]
326 |             offset_dst = target - dst[i][b_idx]
327 |             
328 |             offset_list.append(offset_dst)
329 |             dist_list.append(mindist[b_idx])
330 |         
331 |         weights = softmax(1/(np.array(dist_list)+1))
332 | 
333 |         trans = np.average(offset_list,axis=0,weights=weights)
334 |         trans = np.round(trans).astype('int')
335 |         trans_stroke = dst[i] + trans
336 |         return trans_stroke
337 | 
338 |     def pipeline(self,in_noise,ex_noise,getVisual=False):
339 | 
340 |         print("Sketch Synthesis =======")
341 |         print("Step 1: extrinsic disturbing ...")
342 |         step1, step1_visual = model.predict2(self.srcStrokes,ex_noise)
343 |         
344 |         print("Step 2: intrinsic disturbing ...")
345 |         step2 = model.predict1(step1,in_noise) # intrinsic disturbing
346 | 
347 |         print("Step 3: point disturbing ...")
348 |         step3 = model.predict3(step2) # point disturbing
349 | 
350 |         print("Step 4: layout initialization and optimization ...")
351 |         step42 = model.layout_refine(step3,self.transOptimize) # layout init + layout optimization
352 | 
353 |         pipeline_visual = None
354 | 
355 |         if getVisual:
356 |             step41 = model.layout_refine(step3,self.getTrans) # layout init
357 |             src_img = renderSketch(self.srcCurve)
358 |             img1 = renderSketch(step1_visual)
359 |             img2 = renderSketch(step2)
360 |             img3 = renderSketch(step3)
361 |             img41 = renderSketch(step41)
362 |             img42 = renderSketch(step42)
363 |             pipeline_visual = np.concatenate((src_img,img1,img2,img3,img41,img42),axis=1)
364 |             
365 |             return pipeline_visual # step-by-step result
366 | 
367 | 
368 |         return step42 # final result
369 | 
370 | 
371 | if __name__ == "__main__":
372 |     N_or_P = 'N' # for the model trained on novices data or professionals data
373 |     
374 |     model = Synthesis(NP=N_or_P)
375 |     
376 |     ex_noise = 0.3 # extrinsic noise level
377 |     in_noise = 0.3 # intrinsic noise level
378 |     
379 |     src_dir = "./sketch_json/"
380 |     
381 |     json_path = "P021_8_1_SAH_210.json" # horse expample
382 |     
383 |     # json_path = "P004_0_0_GCN_Armor_cat.json" # cat example
384 |     
385 |     save_dir = os.path.join('./results',N_or_P)
386 |     
387 |     srcStrokes = readStrokes(os.path.join(src_dir,json_path))
388 |     srcStrokes = getValidStrokes(srcStrokes)
389 |     model.load(srcStrokes)
390 |     
391 |     pipeline_visual = model.pipeline(in_noise=in_noise,ex_noise=ex_noise,getVisual=True)
392 | 
393 |     plt.imshow(pipeline_visual)
394 |     plt.show()
395 |     
396 |     os.makedirs(save_dir,exist_ok=True)
397 |     cv2.imwrite(os.path.join(save_dir,"%s_in=%.2f_ex=%.2f.png"%(json_path.split('.')[0],in_noise,ex_noise)),cv2.cvtColor(pipeline_visual,cv2.COLOR_RGB2BGR))
398 |     
399 | 
400 | 


--------------------------------------------------------------------------------
/sketch_synthesis/train_curveNoise.py:
--------------------------------------------------------------------------------
 1 | #%%
 2 | import numpy as np
 3 | import os,json,cv2,math
 4 | from sklearn import preprocessing
 5 | from sklearn.model_selection import train_test_split
 6 | from sklearn.neural_network import MLPRegressor
 7 | from sklearn.metrics import mean_squared_error
 8 | from scipy.special import comb
 9 | import matplotlib.pyplot as plt
10 | import pickle
11 | 
12 | 
13 | 
14 | class MLP:
15 |     def train(self,save_model_path='./model/mlp_curveNoise.sav'):
16 |   
17 |         ## ---------------- load data
18 |         x = np.loadtxt('./data/curve_noise/in_all.txt')
19 |         y = np.loadtxt('./data/curve_noise/out_all.txt')
20 | 
21 |         ## ---------------- Split dataset
22 |         np.random.seed(233)
23 |         x_train, x_test, y_train, y_test = train_test_split(x,y,test_size = 0.2)
24 |         # print(x_train[:2],y_train[:2])
25 | 
26 |         mlp = MLPRegressor(
27 |             hidden_layer_sizes=(100,50),  activation='relu', solver='adam', alpha=0.0001, batch_size='auto',
28 |             learning_rate='constant', learning_rate_init=0.001, power_t=0.5, max_iter=5000, shuffle=True,
29 |             random_state=1, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True,
30 |             early_stopping=False,beta_1=0.9, beta_2=0.999, epsilon=1e-08)
31 | 
32 |         mlp.fit(x_train, y_train)
33 | 
34 |         pred_train = mlp.predict(x_train)
35 |         mse_1 = mean_squared_error(pred_train,y_train)
36 | 
37 |         print ("Train ERROR = ", mse_1)
38 | 
39 |         pred_test = mlp.predict(x_test)
40 |         mse_2 = mean_squared_error(pred_test,y_test)
41 |         # print(pred_test[:5],y_test[:5])
42 |         print ("Test ERROR = ", mse_2)
43 | 
44 |         self.model = mlp
45 | 
46 |         pickle.dump(mlp, open(save_model_path, 'wb'))
47 |     
48 | 
49 |     def predict(self,input):
50 |         results = self.model.predict(input)
51 |         return results
52 | 
53 | 
54 | 
55 | if __name__ == '__main__':
56 | 
57 |     save_folder = "./train_models"
58 |     os.makedirs(save_folder,exist_ok=True)
59 |     save_model_path = os.path.join(save_folder,"mlp_curveNoise.sav")
60 | 
61 |     mlp = MLP()
62 |     mlp.train(save_model_path=save_model_path)
63 | 
64 | 


--------------------------------------------------------------------------------
/sketch_synthesis/train_extrinsic.py:
--------------------------------------------------------------------------------
 1 | #%%
 2 | import numpy as np
 3 | import os,json,cv2,math
 4 | from sklearn import preprocessing
 5 | from sklearn.model_selection import train_test_split
 6 | from sklearn.neural_network import MLPRegressor
 7 | from sklearn.metrics import mean_squared_error
 8 | from scipy.special import comb
 9 | import matplotlib.pyplot as plt
10 | import pickle
11 | 
12 | 
13 | class MLP:
14 |     def train(self,save_model_path,NP='N'):
15 |         x_MinMax = preprocessing.MinMaxScaler()
16 | 
17 |         control_num = 6
18 |         
19 |         ## ---------------- load data
20 |         x = np.loadtxt('./data/extrinsic/in_curves_%d_%s.txt'%(control_num,NP))
21 |         y = np.loadtxt('./data/extrinsic/out_%d_%s.txt'%(control_num,NP))
22 |         
23 |         ## ---------------- preprocessing
24 |         x_loc = x[:,:(control_num*2)]
25 |         x_dist = x[:,control_num*2]
26 |         x_dist = x_dist.reshape(len(x_dist),-1)
27 |         x_loc = x_loc/799
28 |         x_dist_norm = x_MinMax.fit_transform(x_dist)
29 | 
30 |         x_input = np.concatenate((x_loc,x_dist_norm),1)
31 | 
32 |         ## ---------------- Split dataset
33 |         np.random.seed(233)
34 |         x_train, x_test, y_train, y_test = train_test_split(x_input,y,test_size = 0.01)
35 | 
36 |         mlp = MLPRegressor(
37 |             hidden_layer_sizes=(100,50),  activation='relu', solver='adam', alpha=0.0001, batch_size='auto',
38 |             learning_rate='constant', learning_rate_init=0.001, power_t=0.5, max_iter=10000000, shuffle=True,
39 |             random_state=1, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True,
40 |             early_stopping=False,beta_1=0.9, beta_2=0.999, epsilon=1e-08)
41 | 
42 |         mlp.fit(x_train, y_train)
43 | 
44 |         pred_train = mlp.predict(x_train)
45 |         mse_1 = mean_squared_error(pred_train,y_train)
46 | 
47 |         print ("Train ERROR = ", mse_1)
48 | 
49 |         pred_test = mlp.predict(x_test)
50 |         mse_2 = mean_squared_error(pred_test,y_test)
51 |         # print(pred_test[:5],y_test[:5])
52 |         print ("Test ERROR = ", mse_2)
53 | 
54 |         self.model = mlp
55 | 
56 |         
57 |         pickle.dump(mlp, open(save_model_path, 'wb'))
58 | 
59 |     def predict(self,input):
60 |         results = self.model.predict(input)
61 |         return results
62 | 
63 | 
64 | 
65 | if __name__ == '__main__':
66 |     NP = 'N' # switch between novice and professional sketches using 'N' or 'P'
67 |     save_folder = "./train_models"
68 |     os.makedirs(save_folder,exist_ok=True)
69 |     save_model_path = os.path.join(save_folder,f"{NP}_mlp_extrinsic.sav")
70 |     
71 |     mlp = MLP()
72 |     mlp.train(save_model_path=save_model_path,NP=NP)
73 | 
74 | 


--------------------------------------------------------------------------------
/sketch_synthesis/train_intrinsic.py:
--------------------------------------------------------------------------------
 1 | #%%
 2 | import numpy as np
 3 | import os,json,cv2
 4 | from sklearn import preprocessing
 5 | from sklearn.model_selection import train_test_split
 6 | from sklearn.neural_network import MLPRegressor
 7 | from sklearn.metrics import mean_squared_error
 8 | from scipy.special import comb
 9 | import matplotlib.pyplot as plt
10 | import pickle
11 | 
12 | class MLP:
13 |     def train(self,save_model_path,NP='N'):
14 |         
15 |         x_MinMax = preprocessing.MinMaxScaler()
16 | 
17 |         control_num = 6
18 | 
19 |         ## ---------------- load data
20 |         x = np.loadtxt('./data/intrinsic/in_curves_%d_%s.txt'%(control_num,NP))
21 |         y = np.loadtxt('./data/intrinsic/out_curves_%d_%s.txt'%(control_num,NP))
22 | 
23 |         ## ---------------- preprocessing
24 |         x_loc = x[:,:(control_num*2)]
25 |         x_dist = x[:,control_num*2]
26 |         x_dist = x_dist.reshape(len(x_dist),-1)
27 |         x_loc = x_loc/799
28 |         x_dist_norm = x_MinMax.fit_transform(x_dist)
29 | 
30 |         x_input = np.concatenate((x_loc,x_dist_norm),1)
31 |         y_output = y/799
32 | 
33 |         ## ---------------- Split dataset
34 |         np.random.seed(233)
35 |         x_train, x_test, y_train, y_test = train_test_split(x_input,y_output,test_size = 0.01)
36 | 
37 |         mlp = MLPRegressor(
38 |             hidden_layer_sizes=(100,50),  activation='relu', solver='adam', alpha=0.0001, batch_size='auto',
39 |             learning_rate='constant', learning_rate_init=0.001, power_t=0.5, max_iter=10000000, shuffle=True,
40 |             random_state=1, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True,
41 |             early_stopping=False,beta_1=0.9, beta_2=0.999, epsilon=1e-08)
42 | 
43 |         mlp.fit(x_train, y_train)
44 | 
45 |         pred_train = mlp.predict(x_train)
46 |         mse_1 = mean_squared_error(pred_train,y_train)
47 | 
48 |         print ("Train ERROR = ", mse_1)
49 | 
50 |         pred_test = mlp.predict(x_test)
51 |         mse_2 = mean_squared_error(pred_test,y_test)
52 |         print(pred_test[0],y_test[0])
53 |         print ("Test ERROR = ", mse_2)
54 | 
55 |         pickle.dump(mlp, open(save_model_path, 'wb'))
56 |     
57 |     def predict(self,input):
58 |         results = self.model.predict(input)
59 |         return results
60 | 
61 | 
62 | if __name__ == '__main__':
63 |     NP = 'N' # switch between novice and professional sketches using 'N' or 'P'
64 |     save_folder = "./train_models"
65 |     os.makedirs(save_folder,exist_ok=True)
66 |     save_model_path = os.path.join(save_folder,f"{NP}_mlp_intrinsic.sav")
67 |     mlp = MLP()
68 |     mlp.train(save_model_path=save_model_path,NP=NP)


--------------------------------------------------------------------------------