├── .env ├── .gitignore ├── Drafts ├── CNN_WSS_Predictions.pdf └── Lumen Detection.pdf ├── Images ├── pipeline.PNG └── predictions.PNG ├── README.md ├── Templates ├── analyze_data.py ├── config.py └── train.py ├── Training_Examples ├── VelocityFieldPrediction │ ├── config.py │ ├── test.py │ └── train.py └── WSSPrediction │ ├── KFoldAblationStudies │ ├── Study1 │ │ ├── config.py │ │ ├── train.py │ │ └── view_results.py │ ├── Study2 │ │ ├── config.py │ │ └── test.py │ └── Study3 │ │ ├── config.py │ │ └── train.py │ ├── KFold_CSEFPN_U │ ├── config.py │ └── train.py │ ├── KFold_CSESEPN_U │ ├── config.py │ └── train.py │ ├── KFold_FPN_Resnet34 │ ├── config.py │ └── train.py │ ├── KFold_FPN_Resnet34B │ ├── config.py │ └── train.py │ ├── KFold_FPN_U │ ├── config.py │ └── train.py │ ├── KFold_FPN_U_Bifurcations │ ├── config.py │ ├── test.py │ └── train.py │ ├── KFold_LinkNet │ ├── config.py │ └── train.py │ ├── KFold_LinkNet_ResNet34 │ ├── config.py │ └── train.py │ ├── KFold_LinkNet_U │ ├── config.py │ └── train.py │ ├── KFold_MLP │ ├── config.py │ └── train.py │ ├── KFold_PSPNet_Resnet34 │ ├── config.py │ └── train.py │ ├── KFold_PSPNet_Resnet50 │ ├── config.py │ └── train.py │ ├── KFold_SEFPN_U │ ├── config.py │ └── train.py │ ├── KFold_Unet_Curvature │ ├── analyze_results.py │ ├── config.py │ └── train.py │ ├── KFold_Unet_Points │ ├── analyze_results.py │ ├── config.py │ └── train.py │ ├── KFold_qubel_Unet_Resnet34 │ ├── config.py │ └── train.py │ ├── Kfold_Unet_Resnet34 │ ├── config.py │ └── train.py │ └── bin │ ├── accuracy_blandtaltman_kfold.py │ ├── analyze_kfold.py │ ├── analyze_single_study.py │ ├── linear_regression_kfold.py │ ├── make_copyable_to_latex.py │ ├── print_relative_accuracy.py │ └── write_relative_accuracy_to_csv.py ├── myTorch ├── Callbacks │ ├── Callback.py │ ├── Checkpoints.py │ ├── DataAnalysis.py │ ├── LRFind.py │ ├── Resample.py │ ├── SaveIntermediate.py │ ├── Stdout.py │ ├── Tensorboard.py │ ├── Validation.py │ └── __init__.py ├── Data │ ├── DataLoaders.py │ ├── DictDataset.py │ ├── Exploration │ │ ├── __init__.py │ │ └── moving_averages.py │ ├── Transforms │ │ ├── ToGPU.py │ │ ├── TransformSequence.py │ │ ├── WithKeys.py │ │ └── __init__.py │ └── __init__.py ├── Loss │ └── __init__.py ├── Models │ ├── DilatedNet.py │ ├── EncoderBottleNeckHead.py │ ├── FCN_Eric.py │ ├── FPNNet.py │ ├── LinkNet.py │ ├── Merged.py │ ├── PSPnet.py │ ├── Utils.py │ ├── __init__.py │ ├── chris_linknet.py │ ├── chris_pspnet.py │ ├── extractors.py │ └── unet.py ├── Trainer.py ├── Utils.py ├── __init__.py ├── nn │ ├── .mypy_cache │ │ └── 3.7 │ │ │ └── @plugins_snapshot.json │ ├── ASPP.py │ ├── ConvolutionPatterns.py │ ├── Layer.py │ ├── Normalization.py │ ├── Padding.py │ ├── ResidualConvolutions.py │ ├── SequentialLayers.py │ ├── SqueezeExcitation.py │ ├── UpsampleConvolution.py │ ├── __init__.py │ ├── compare_trianing.py │ ├── convolutions.py │ ├── find_exponent.py │ ├── init.py │ ├── pspnet.py │ └── wrapped.py └── optim │ ├── FinderLRs.py │ ├── __init__.py │ ├── adabound.py │ └── lr_finder.py ├── requirements.txt └── setup.py /.env: -------------------------------------------------------------------------------- 1 | PYTHONPATH="D:\Github\myTorch" -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .vscode/ 2 | _myTorchEnv/ 3 | myTorch.egg-info/ 4 | **/__pycache__/ 5 | Training_Examples/**/fold_*/ 6 | Training_Examples/**/runs/ 7 | Training_Examples/**/test 8 | **/*.csv 9 | **/*.npy 10 | /DATA -------------------------------------------------------------------------------- /Drafts/CNN_WSS_Predictions.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Chr1sC0de/DeepLearningForWallShearStressPredictionAndImageSegmentation/dbb5e6a58b0ecfdb4ed3b05e5ca1841a321bd11b/Drafts/CNN_WSS_Predictions.pdf -------------------------------------------------------------------------------- /Drafts/Lumen Detection.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Chr1sC0de/DeepLearningForWallShearStressPredictionAndImageSegmentation/dbb5e6a58b0ecfdb4ed3b05e5ca1841a321bd11b/Drafts/Lumen Detection.pdf -------------------------------------------------------------------------------- /Images/pipeline.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Chr1sC0de/DeepLearningForWallShearStressPredictionAndImageSegmentation/dbb5e6a58b0ecfdb4ed3b05e5ca1841a321bd11b/Images/pipeline.PNG -------------------------------------------------------------------------------- /Images/predictions.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Chr1sC0de/DeepLearningForWallShearStressPredictionAndImageSegmentation/dbb5e6a58b0ecfdb4ed3b05e5ca1841a321bd11b/Images/predictions.PNG -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Convolutional Neural Networks for Wall Shear Stress Prediction 2 | 3 | Deep Convolutional Neural Networks for Real-Time Patient-Specific Wall Shear Stress 4 | Estimation, most recent draft can be found [here](https://github.com/Chr1sC0de/myTorch/tree/master/Drafts/CNN_WSS_Predictions.pdf) 5 | 6 | We outline a strategy for automatically extracting the main portion of the coronary artery from OCT [here](https://github.com/Chr1sC0de/Deep-Learning-For-Wall-Shear-Stress-Prediction/blob/master/Drafts/Lumen%20Detection.pdf) 7 | 8 | Current research involves the automatic delineation of the artery lumen as well as 9 | the analysis of artery wall shear stress using deep learning. The work is currently 10 | unpublished and is within the drafting stages. Bellow includes the paper abstract 11 | as well as several figures. Can be found in the drafts folder. 12 | 13 | Abstract—Blood flow exerts stresses on the coronary artery wall.Wall shear stress 14 | (WSS), the parallel component of stress, when abnormally low ( <1Pa) may promote the 15 | pathological development of cholesterol-laden plaque, resulting in arterial 16 | narrowing. Accurately determining WSS helps to inform clinicians about plaque 17 | progression. Nevertheless, WSS is difficult to measure directly but can be estimated 18 | accurately by solving the governing equations of fluid motion (the Navier-Stokes 19 | Equations). Solving these complex equations can be both time consuming and 20 | computationally expensive to perform (> 1hr). Deep Learning (DL) is capable of 21 | learning patterns directly from data providing an avenue for real-time prediction of 22 | WSS. Thus, in this paper, we outline a method which employs popular DL models for 23 | their use in the prediction of WSS from artery models generated from coronary 24 | angiography. The best performing model was able to predict the WSS in under 0.29s 25 | with an average coefficient of determination R2> 75% compared to high resolution CFD 26 | simulations. 27 | 28 | ![alt](./Images/pipeline.PNG) 29 | ![alt](./Images/predictions.PNG) 30 | 31 | Installation 32 | ------------ 33 | 34 | To install the requirements 35 | 36 | First install pytorch 37 | 38 | ````powershell 39 | pip install torch===1.4.0 torchvision===0.5.0 -f https://download.pytorch.org/whl/torch_stable.html 40 | ```` 41 | 42 | ````powershell 43 | python -m pip install -r requirements.txt 44 | ```` 45 | 46 | To install the main package in debug mode 47 | 48 | ````powershell 49 | python -m pip install -e . 50 | ```` 51 | 52 | To install the main package in release mode 53 | 54 | ````powershell 55 | python -m pip install . 56 | ```` 57 | 58 | to view the vtk files download and install paraview https://www.paraview.org/. 59 | 60 | Data 61 | ---- 62 | 63 | Data for training and testing is assumed to be contained in the `DATA` folder in the top directory. The data is stored in two formats, .npz and .vtk. the vtk files can be easily viewed in paraview and show the original meshes as wells as the meshes parameterized in 2D. The npz files contain the tensor data with dimensions (channels, height, width). The data files are quite large and are unable to fit on the free version of github, however they can be obtained directly from the following link, 64 | 65 | https://unimelbcloud-my.sharepoint.com/:f:/g/personal/cmamon_student_unimelb_edu_au/Epx-e_s_Zj1HtlF88MOmXF4B1MJMpbBEbQ7Zu1gVJizOLA?e=VYqjIn. 66 | 67 | Running Examples 68 | ---------------- 69 | 70 | Examples are stored in the `Training_Examples` directory. At the moment only the examples for predicting WSS are supplied. Internal field predictions will be added at a later date once the paper is completed(accuracy may vary from machine to machine). To run an example, in the terminal, 71 | 72 | ```` 73 | cd Training_Examples/WSSPrediction/ 74 | python train.py 75 | ```` 76 | -------------------------------------------------------------------------------- /Templates/analyze_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Chr1sC0de/DeepLearningForWallShearStressPredictionAndImageSegmentation/dbb5e6a58b0ecfdb4ed3b05e5ca1841a321bd11b/Templates/analyze_data.py -------------------------------------------------------------------------------- /Templates/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | 9 | current_directory = Path(__file__).parent 10 | 11 | data_folder = current_directory/r'..\__CURVATUREDATA\DATA' 12 | 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.7, 0.1, 0.2) 17 | TVS.split.kwargs = dict(random_state=1) 18 | TVS.call = dict(batch_size=1, shuffle=False) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (3, 1, 3) 22 | model.kwargs = dict(n_pool=4, base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Templates/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | 5 | if __name__ == "__main__": 6 | numpy_files = mt.Utils.glob_numpy(config.data_folder) 7 | 8 | transform = [mt.Data.Transforms.RollTensor(), ] 9 | 10 | TVSLoader = mt.Data.DataLoaders.TrainValidationTestSplit( 11 | numpy_files, transform=transform, **config.TVS.init) 12 | TVSLoader.split( 13 | *config.TVS.split.args, **config.TVS.split.kwargs) 14 | train_loader, valid_loader, test_loader = TVSLoader(**config.TVS.call) 15 | 16 | model = mt.Models.Unet(*config.model.args, **config.model.kwargs) 17 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 18 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 19 | criterion = torch.nn.L1Loss(**config.criterion) 20 | 21 | fit_callbacks = [ 22 | mt.Callbacks.Tensorboard(), 23 | mt.Callbacks.Validation(checkpoint=True, on_step=100, to_file='./valid/log'), 24 | mt.Callbacks.SavePyvistaPoints( 25 | on_step=1000, properties=['curvature'], 26 | filename='./train/data'), 27 | mt.Callbacks.LRFind(), 28 | mt.Callbacks.ReduceLROnValidPlateau() 29 | ] 30 | 31 | test_callbacks = [ 32 | mt.Callbacks.SavePyvistaPoints( 33 | on_batch=1, properties=['curvature'], 34 | filename='./test/data') 35 | ] 36 | 37 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 38 | if False: 39 | trainer.fit_on_loader( 40 | train_loader, valid_dataloader=valid_loader, cycles=[300, ], 41 | callbacks=fit_callbacks, std_to_file='./train/log') 42 | trainer.load_best_model('./Checkpoints') 43 | trainer.test_on_loader( 44 | test_loader, std_to_file='./test/log', callbacks=test_callbacks 45 | ) 46 | -------------------------------------------------------------------------------- /Training_Examples/VelocityFieldPrediction/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'mesh' 6 | target_name = 'U' 7 | 8 | ml_model = mt.Models.Unet3d 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../DATA/NewtonianSteadyInternalField' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict( 15 | input_name=input_name, target_name=target_name, 16 | suffix='.npz',dataset_method=mt.Data.DictDatasetVolume 17 | ) 18 | TVS.split = mt.Utils.AttrDict() 19 | TVS.split.args = (0.1, ) 20 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 21 | TVS.call = dict(batch_size=1, shuffle=True) 22 | 23 | model = mt.Utils.AttrDict() 24 | model.args = (3, 3, 3) 25 | model.kwargs = dict(base_channels=48, output_activation=torch.nn.functional.relu) 26 | 27 | optimizer = mt.Utils.AttrDict() 28 | optimizer.kwargs = dict(lr=1e-6, eps=1e-7) 29 | 30 | criterion = dict(reduction='sum') 31 | 32 | trainer = dict( 33 | x_key=input_name, y_key=target_name 34 | ) 35 | -------------------------------------------------------------------------------- /Training_Examples/VelocityFieldPrediction/test.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.FlipTensor()] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | if fold_number in [1]: 36 | 37 | print(f'runing fold {fold_number}/{5}') 38 | 39 | model = config.ml_model(*config.model.args, **config.model.kwargs) 40 | criterion = torch.nn.L1Loss(**config.criterion) 41 | 42 | test_callbacks = [ 43 | ] 44 | 45 | trainer = mt.Trainer(model, None, criterion, **config.trainer) 46 | 47 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 48 | 49 | start_time = time() 50 | trainer.test_on_loader( 51 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 52 | ) 53 | end_time = time() 54 | 55 | test_time = end_time-start_time 56 | 57 | del fold_number 58 | del model 59 | del criterion 60 | del test_callbacks 61 | del trainer 62 | torch.cuda.empty_cache() 63 | -------------------------------------------------------------------------------- /Training_Examples/VelocityFieldPrediction/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.FlipTensor()] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | model = config.ml_model(*config.model.args, **config.model.kwargs) 38 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 39 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 40 | criterion = torch.nn.L1Loss(**config.criterion) 41 | 42 | fit_callbacks = [ 43 | mt.Callbacks.Tensorboard(), 44 | mt.Callbacks.LRFind(), 45 | mt.Callbacks.ReduceLROnValidPlateau( 46 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 47 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 48 | ] 49 | 50 | test_callbacks = [ 51 | ] 52 | 53 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 54 | 55 | start_time = time() 56 | 57 | trainer.fit_on_loader( 58 | train_loader, valid_dataloader=valid_loader, cycles=[100], 59 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 60 | 61 | end_time = time() 62 | 63 | train_time = end_time - start_time 64 | 65 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 66 | 67 | start_time = time() 68 | trainer.test_on_loader( 69 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 70 | ) 71 | end_time = time() 72 | 73 | test_time = end_time-start_time 74 | 75 | time_log.append( 76 | (train_time, test_time) 77 | ) 78 | 79 | np.save('./time_log', time_log) 80 | 81 | del fold_number 82 | del model 83 | del optimizer 84 | del criterion 85 | del fit_callbacks 86 | del test_callbacks 87 | del trainer 88 | torch.cuda.empty_cache() 89 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFoldAblationStudies/Study1/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.CSESEVGGFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../../DATA/LADSSMNewtonianSteadyWSS' 13 | true_data = current_directory/'../../../../DATA/LADPatientNewtonianSteadyWSS' 14 | 15 | TVS = mt.Utils.AttrDict() 16 | TVS.init = dict(input_name=input_name, target_name=target_name) 17 | TVS.split = mt.Utils.AttrDict() 18 | TVS.split.args = (0.1, ) 19 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 20 | TVS.call = dict(batch_size=1, shuffle=True) 21 | 22 | model = mt.Utils.AttrDict() 23 | model.args = (4, 1, 3) 24 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 25 | 26 | optimizer = mt.Utils.AttrDict() 27 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 28 | 29 | criterion = dict(reduction='sum') 30 | 31 | trainer = dict( 32 | x_key=input_name, y_key=target_name 33 | ) 34 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFoldAblationStudies/Study1/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | if __name__ == "__main__": 9 | 10 | numpy_files_data = mt.Utils.glob_npz(config.data_folder) 11 | numpy_files_true = mt.Utils.glob_npz(config.true_data) 12 | 13 | transform = [mt.Data.Transforms.RollTensor(), ] 14 | 15 | KFoldTVSLoader_Synthetic = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 16 | numpy_files_data, transform=transform, **config.TVS.init) 17 | 18 | KFoldTVSLoader_True = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 19 | numpy_files_true, transform=transform, **config.TVS.init) 20 | 21 | KFoldTVSLoader_Synthetic.split(*config.TVS.split.args, **config.TVS.split.kwargs) 22 | KFoldTVSLoader_True.split(*config.TVS.split.args, **config.TVS.split.kwargs) 23 | 24 | split_synthetic = KFoldTVSLoader_Synthetic(**config.TVS.call) 25 | split_true = KFoldTVSLoader_True(**config.TVS.call) 26 | 27 | time_log = [] 28 | 29 | for fold_number, \ 30 | ( (train_synthetic, valid_synthetic, test_synthetic), 31 | (train_true, valid_true, test_true) ) in \ 32 | enumerate(zip(split_synthetic, split_true)): 33 | 34 | # compile the synthetic datasets 35 | train_synthetic.dataset.file_paths += test_synthetic.dataset.file_paths 36 | n_synthetic = len(train_synthetic) 37 | n_true = len(train_true) 38 | # combine the true data 39 | train_true.dataset.file_paths += valid_true.dataset.file_paths 40 | for _ in range(((n_synthetic)//2)//n_true): 41 | train_synthetic.dataset.file_paths += train_true.dataset.file_paths 42 | 43 | # now set which dataloaders will be the train, test and valid 44 | train_loader = train_synthetic 45 | test_loader = test_true 46 | # use the synthetic data for validation to avoid overfitting 47 | valid_loader = valid_synthetic 48 | 49 | if fold_number == 0: 50 | time_log.append( 51 | dict( 52 | train_steps_per_epoch=len(train_loader), 53 | test_steps_per_epoch=len(test_loader) 54 | ) 55 | ) 56 | 57 | fold_number += 1 58 | 59 | print(f'runing fold {fold_number}/{5}') 60 | 61 | 62 | model = config.ml_model(*config.model.args, **config.model.kwargs) 63 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 64 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 65 | criterion = torch.nn.L1Loss(**config.criterion) 66 | 67 | fit_callbacks = [ 68 | mt.Callbacks.Tensorboard(), 69 | mt.Callbacks.SavePyvistaPoints( 70 | on_step=1000, properties=['curvature'], 71 | filename=f'./fold_{fold_number}/train/data'), 72 | mt.Callbacks.LRFind(), 73 | mt.Callbacks.ResampleTrainingData(), 74 | mt.Callbacks.ReduceLROnValidPlateau( 75 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 76 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 77 | ] 78 | 79 | test_callbacks = [ 80 | mt.Callbacks.SavePyvistaPoints( 81 | on_batch=1, properties=['curvature'], 82 | filename=f'./fold_{fold_number}/test/data') 83 | ] 84 | 85 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 86 | 87 | start_time = time() 88 | 89 | trainer.fit_on_loader( 90 | train_loader, valid_dataloader=valid_loader, cycles=[50, 50], 91 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 92 | 93 | end_time = time() 94 | 95 | train_time = end_time - start_time 96 | 97 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 98 | 99 | start_time = time() 100 | trainer.test_on_loader( 101 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 102 | ) 103 | end_time = time() 104 | 105 | test_time = end_time-start_time 106 | 107 | time_log.append( 108 | (train_time, test_time) 109 | ) 110 | 111 | np.save('./time_log', time_log) 112 | 113 | del fold_number 114 | del model 115 | del optimizer 116 | del criterion 117 | del fit_callbacks 118 | del test_callbacks 119 | del trainer 120 | torch.cuda.empty_cache() 121 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFoldAblationStudies/Study1/view_results.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | file = r"D:\Github\myTorch\Training_Examples\WSSPrediction\KFoldAblationStudy\fold_5\test\log.npy" 4 | 5 | data = np.load(file, allow_pickle=True).item() 6 | 7 | print(data['accuracy']) 8 | 9 | pass -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFoldAblationStudies/Study2/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.CSESEVGGFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../../DATA/LADPatientNewtonianSteadyWSS' 13 | 14 | TVS = mt.Utils.AttrDict() 15 | TVS.init = dict(input_name=input_name, target_name=target_name) 16 | TVS.split = mt.Utils.AttrDict() 17 | TVS.split.args = (0.1,) 18 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 19 | TVS.call = dict(batch_size=1, shuffle=True) 20 | 21 | model = mt.Utils.AttrDict() 22 | model.args = (4, 1, 3) 23 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 24 | 25 | optimizer = mt.Utils.AttrDict() 26 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 27 | 28 | criterion = dict(reduction='sum') 29 | 30 | trainer = dict( 31 | x_key=input_name, y_key=target_name 32 | ) 33 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFoldAblationStudies/Study2/test.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | class RolledTester(mt.Trainer): 8 | 9 | def test_step(self, data_dict): 10 | 11 | self.run_extractor_hooks(data_dict) 12 | self.x, self.y_true = self.data_extractor(data_dict) 13 | self.x = self.x.to(self.device) 14 | self.y_true = self.y_true.to(self.device) 15 | roll_values = torch.linspace(0, self.x.shape[2], 10) 16 | predictions = [] 17 | for val in roll_values: 18 | rolled_tensor = torch.roll(self.x, int(val), dims=2) 19 | predictions.append(self.model(rolled_tensor)) 20 | predictions = [ 21 | torch.roll(p, -int(val), dims=2) for p, val in zip(predictions, roll_values) 22 | ] 23 | self.y_pred = torch.mean(torch.stack(predictions, dim=0), dim=0) 24 | self.loss = self.criterion(self.y_pred, self.y_true) 25 | self.accuracy = self.get_accuracy() 26 | 27 | if __name__ == "__main__": 28 | numpy_files = mt.Utils.glob_npz(config.data_folder) 29 | 30 | test_loader = torch.utils.data.DataLoader( 31 | mt.Data.DataLoaders.DictDataset( 32 | numpy_files, input_name='curvature', target_name='wss')) 33 | 34 | time_log = [] 35 | 36 | time_log.append( 37 | dict( 38 | test_steps_per_epoch=len(test_loader) 39 | ) 40 | ) 41 | 42 | model = config.ml_model(*config.model.args, **config.model.kwargs) 43 | 44 | test_callbacks = [ 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_batch=1, properties=['curvature'], 47 | filename=f'./test/data') 48 | ] 49 | 50 | criterion = torch.nn.L1Loss(**config.criterion) 51 | 52 | trainer = RolledTester(model, None, criterion, **config.trainer) 53 | 54 | trainer.load_best_model(f'D:/Github/myTorch/Training_Examples/WSSPrediction/KFold_CSESEPN_U/fold_1/Checkpoints') 55 | 56 | start_time = time() 57 | trainer.test_on_loader( 58 | test_loader, std_to_file=f'./test/log', callbacks=test_callbacks 59 | ) 60 | end_time = time() 61 | 62 | test_time = end_time-start_time 63 | 64 | time_log.append( 65 | (test_time) 66 | ) 67 | 68 | np.save('./time_log', time_log) 69 | 70 | del model 71 | del criterion 72 | del test_callbacks 73 | del trainer 74 | torch.cuda.empty_cache() 75 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFoldAblationStudies/Study3/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.CSESEVGGFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../../DATA/LADSSMNewtonianSteadyWSS' 13 | true_data = current_directory/'../../../../DATA/LADPatientNewtonianSteadyWSS' 14 | 15 | TVS = mt.Utils.AttrDict() 16 | TVS.init = dict(input_name=input_name, target_name=target_name) 17 | TVS.split = mt.Utils.AttrDict() 18 | TVS.split.args = (0.1, ) 19 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 20 | TVS.call = dict(batch_size=1, shuffle=True) 21 | 22 | model = mt.Utils.AttrDict() 23 | model.args = (4, 1, 3) 24 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 25 | 26 | optimizer = mt.Utils.AttrDict() 27 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 28 | 29 | criterion = dict(reduction='sum') 30 | 31 | trainer = dict( 32 | x_key=input_name, y_key=target_name 33 | ) 34 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFoldAblationStudies/Study3/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | if __name__ == "__main__": 9 | numpy_files_data = mt.Utils.glob_npz(config.data_folder) 10 | numpy_files_true = mt.Utils.glob_npz(config.true_data) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader_Synthetic = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files_data, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader_True = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 18 | numpy_files_true, transform=transform, **config.TVS.init) 19 | 20 | KFoldTVSLoader_Synthetic.split(*config.TVS.split.args, **config.TVS.split.kwargs) 21 | KFoldTVSLoader_True.split(*config.TVS.split.args, **config.TVS.split.kwargs) 22 | 23 | split_synthetic = KFoldTVSLoader_Synthetic(**config.TVS.call) 24 | split_true = KFoldTVSLoader_True(**config.TVS.call) 25 | 26 | time_log = [] 27 | 28 | for fold_number, \ 29 | ( (train_synthetic, valid_synthetic, test_synthetic), 30 | (train_true, valid_true, test_true) ) in \ 31 | enumerate(zip(split_synthetic, split_true)): 32 | 33 | # compile the synthetic datasets 34 | train_synthetic.dataset.file_paths += test_synthetic.dataset.file_paths 35 | n_synthetic = len(train_synthetic) 36 | n_true = len(train_true) 37 | # combine the true data 38 | train_true.dataset.file_paths += valid_true.dataset.file_paths 39 | for _ in range(((n_synthetic)//2)//n_true): 40 | train_synthetic.dataset.file_paths += train_true.dataset.file_paths 41 | 42 | # now set which dataloaders will be the train, test and valid 43 | train_loader = train_synthetic 44 | test_loader = test_true 45 | # use the synthetic data for validation to avoid overfitting 46 | valid_loader = valid_synthetic 47 | 48 | if fold_number == 0: 49 | time_log.append( 50 | dict( 51 | train_steps_per_epoch=len(train_loader), 52 | test_steps_per_epoch=len(test_loader) 53 | ) 54 | ) 55 | 56 | fold_number += 1 57 | 58 | print(f'runing fold {fold_number}/{5}') 59 | 60 | 61 | model = config.ml_model(*config.model.args, **config.model.kwargs) 62 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 63 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 64 | criterion = torch.nn.L1Loss(**config.criterion) 65 | 66 | fit_callbacks = [ 67 | mt.Callbacks.Tensorboard(), 68 | mt.Callbacks.SavePyvistaPoints( 69 | on_step=1000, properties=['curvature'], 70 | filename=f'./fold_{fold_number}/train/data'), 71 | mt.Callbacks.LRFind(), 72 | mt.Callbacks.ReduceLROnValidPlateau( 73 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 74 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 75 | ] 76 | 77 | test_callbacks = [ 78 | mt.Callbacks.SavePyvistaPoints( 79 | on_batch=1, properties=['curvature'], 80 | filename=f'./fold_{fold_number}/test/data') 81 | ] 82 | 83 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 84 | 85 | start_time = time() 86 | 87 | trainer.fit_on_loader( 88 | train_loader, valid_dataloader=valid_loader, cycles=[50, 50], 89 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 90 | 91 | end_time = time() 92 | 93 | train_time = end_time - start_time 94 | 95 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 96 | 97 | start_time = time() 98 | trainer.test_on_loader( 99 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 100 | ) 101 | end_time = time() 102 | 103 | test_time = end_time-start_time 104 | 105 | time_log.append( 106 | (train_time, test_time) 107 | ) 108 | 109 | np.save('./time_log', time_log) 110 | 111 | del fold_number 112 | del model 113 | del optimizer 114 | del criterion 115 | del fit_callbacks 116 | del test_callbacks 117 | del trainer 118 | torch.cuda.empty_cache() 119 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_CSEFPN_U/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.CSEVGGFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1,) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_CSEFPN_U/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 53 | ] 54 | 55 | test_callbacks = [ 56 | mt.Callbacks.SavePyvistaPoints( 57 | on_batch=1, properties=['curvature'], 58 | filename=f'./fold_{fold_number}/test/data') 59 | ] 60 | 61 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 62 | 63 | start_time = time() 64 | 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_CSESEPN_U/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.CSESEVGGFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1,) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_CSESEPN_U/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | if __name__ == "__main__": 9 | numpy_files = mt.Utils.glob_npz(config.data_folder) 10 | 11 | transform = [mt.Data.Transforms.RollTensor(), ] 12 | 13 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 14 | numpy_files, transform=transform, **config.TVS.init) 15 | 16 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 17 | 18 | split = KFoldTVSLoader(**config.TVS.call) 19 | 20 | time_log = [] 21 | 22 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 23 | 24 | if fold_number == 0: 25 | time_log.append( 26 | dict( 27 | train_steps_per_epoch=len(train_loader), 28 | test_steps_per_epoch=len(test_loader) 29 | ) 30 | ) 31 | 32 | fold_number += 1 33 | 34 | print(f'runing fold {fold_number}/{5}') 35 | 36 | model = config.ml_model(*config.model.args, **config.model.kwargs) 37 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 38 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 39 | criterion = torch.nn.L1Loss(**config.criterion) 40 | 41 | fit_callbacks = [ 42 | mt.Callbacks.Tensorboard(), 43 | mt.Callbacks.SavePyvistaPoints( 44 | on_step=1000, properties=['curvature'], 45 | filename=f'./fold_{fold_number}/train/data'), 46 | mt.Callbacks.LRFind(), 47 | 48 | mt.Callbacks.ReduceLROnValidPlateau( 49 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 50 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 51 | ] 52 | 53 | test_callbacks = [ 54 | mt.Callbacks.SavePyvistaPoints( 55 | on_batch=1, properties=['curvature'], 56 | filename=f'./fold_{fold_number}/test/data') 57 | ] 58 | 59 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 60 | 61 | start_time = time() 62 | 63 | trainer.fit_on_loader( 64 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 65 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 66 | 67 | end_time = time() 68 | 69 | train_time = end_time - start_time 70 | 71 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 72 | 73 | start_time = time() 74 | trainer.test_on_loader( 75 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 76 | ) 77 | end_time = time() 78 | 79 | test_time = end_time-start_time 80 | 81 | time_log.append( 82 | (train_time, test_time) 83 | ) 84 | 85 | np.save('./time_log', time_log) 86 | 87 | del fold_number 88 | del model 89 | del optimizer 90 | del criterion 91 | del fit_callbacks 92 | del test_callbacks 93 | del trainer 94 | torch.cuda.empty_cache() 95 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_Resnet34/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.ResnetFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1, ) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_Resnet34/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | if __name__ == "__main__": 8 | 9 | numpy_files = mt.Utils.glob_npz(config.data_folder) 10 | 11 | transform = [mt.Data.Transforms.RollTensor(), ] 12 | 13 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 14 | numpy_files, transform=transform, **config.TVS.init) 15 | 16 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 17 | 18 | split = KFoldTVSLoader(**config.TVS.call) 19 | 20 | time_log = [] 21 | 22 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 23 | 24 | if fold_number == 0: 25 | time_log.append( 26 | dict( 27 | train_steps_per_epoch=len(train_loader), 28 | test_steps_per_epoch=len(test_loader) 29 | ) 30 | ) 31 | 32 | fold_number += 1 33 | 34 | print(f'runing fold {fold_number}/{5}') 35 | 36 | 37 | model = config.ml_model(*config.model.args, **config.model.kwargs) 38 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 39 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 40 | criterion = torch.nn.L1Loss(**config.criterion) 41 | 42 | fit_callbacks = [ 43 | mt.Callbacks.Tensorboard(), 44 | mt.Callbacks.SavePyvistaPoints( 45 | on_step=1000, properties=['curvature'], 46 | filename=f'./fold_{fold_number}/train/data'), 47 | mt.Callbacks.LRFind(), 48 | 49 | mt.Callbacks.ReduceLROnValidPlateau( 50 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 51 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 52 | ] 53 | 54 | test_callbacks = [ 55 | mt.Callbacks.SavePyvistaPoints( 56 | on_batch=1, properties=['curvature'], 57 | filename=f'./fold_{fold_number}/test/data') 58 | ] 59 | 60 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 61 | 62 | start_time = time() 63 | 64 | trainer.fit_on_loader( 65 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 66 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 67 | 68 | end_time = time() 69 | 70 | train_time = end_time - start_time 71 | 72 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 73 | 74 | start_time = time() 75 | trainer.test_on_loader( 76 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 77 | ) 78 | end_time = time() 79 | 80 | test_time = end_time-start_time 81 | 82 | time_log.append( 83 | (train_time, test_time) 84 | ) 85 | 86 | np.save('./time_log', time_log) 87 | 88 | del fold_number 89 | del model 90 | del optimizer 91 | del criterion 92 | del fit_callbacks 93 | del test_callbacks 94 | del trainer 95 | torch.cuda.empty_cache() 96 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_Resnet34B/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.ResnetFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1, ) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=32, layers=[2, 2, 3, 5, 2],output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_Resnet34B/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 53 | ] 54 | 55 | test_callbacks = [ 56 | mt.Callbacks.SavePyvistaPoints( 57 | on_batch=1, properties=['curvature'], 58 | filename=f'./fold_{fold_number}/test/data') 59 | ] 60 | 61 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 62 | 63 | start_time = time() 64 | 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_U/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.VGGFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1,) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_U/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 53 | ] 54 | 55 | test_callbacks = [ 56 | mt.Callbacks.SavePyvistaPoints( 57 | on_batch=1, properties=['curvature'], 58 | filename=f'./fold_{fold_number}/test/data') 59 | ] 60 | 61 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 62 | 63 | start_time = time() 64 | 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_U_Bifurcations/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.VGGFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/BifurcationNewtonianSteady' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1,) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_U_Bifurcations/test.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | data = np.load(r"D:\Github\myTorch\DATA\BifurcationNewtonianSteady\base_01_inlet_2.1_bif_1.4_outlet_1.6_angle_0.npz") 4 | 5 | pass -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_FPN_U_Bifurcations/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 53 | ] 54 | 55 | test_callbacks = [ 56 | mt.Callbacks.SavePyvistaPoints( 57 | on_batch=1, properties=['curvature'], 58 | filename=f'./fold_{fold_number}/test/data') 59 | ] 60 | 61 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 62 | 63 | start_time = time() 64 | 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_LinkNet/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | 9 | current_directory = Path(__file__).parent 10 | 11 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 12 | TVS = mt.Utils.AttrDict() 13 | TVS.init = dict(input_name=input_name, target_name=target_name) 14 | TVS.split = mt.Utils.AttrDict() 15 | TVS.split.args = (0.1, ) 16 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 17 | TVS.call = dict(batch_size=1, shuffle=True) 18 | 19 | # change model params here 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_LinkNet/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | if __name__ == "__main__": 9 | numpy_files = mt.Utils.glob_npz(config.data_folder) 10 | 11 | transform = [mt.Data.Transforms.RollTensor(), ] 12 | 13 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 14 | numpy_files, transform=transform, **config.TVS.init) 15 | 16 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 17 | 18 | split = KFoldTVSLoader(**config.TVS.call) 19 | 20 | time_log = [] 21 | 22 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 23 | 24 | if fold_number == 0: 25 | time_log.append( 26 | dict( 27 | train_steps_per_epoch=len(train_loader), 28 | test_steps_per_epoch=len(test_loader) 29 | ) 30 | ) 31 | 32 | fold_number += 1 33 | 34 | print(f'runing fold {fold_number}/{5}') 35 | 36 | 37 | # model goes here !!!!!!!!!! 38 | model = mt.Models.ResNetLinkNet(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.Validation( 46 | checkpoint=True, on_step=200, to_file=f'./fold_{fold_number}/valid/log', 47 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint' 48 | ), 49 | mt.Callbacks.SavePyvistaPoints( 50 | on_step=1000, properties=['curvature'], 51 | filename=f'./fold_{fold_number}/train/data'), 52 | mt.Callbacks.LRFind(), 53 | 54 | mt.Callbacks.ReduceLROnValidPlateau() 55 | ] 56 | 57 | test_callbacks = [ 58 | mt.Callbacks.SavePyvistaPoints( 59 | on_batch=1, properties=['curvature'], 60 | filename=f'./fold_{fold_number}/test/data') 61 | ] 62 | 63 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 64 | 65 | start_time = time() 66 | trainer.fit_on_loader( 67 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 68 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_LinkNet_ResNet34/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.rn34LinkNet 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1, ) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_LinkNet_ResNet34/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 53 | ] 54 | 55 | test_callbacks = [ 56 | mt.Callbacks.SavePyvistaPoints( 57 | on_batch=1, properties=['curvature'], 58 | filename=f'./fold_{fold_number}/test/data') 59 | ] 60 | 61 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 62 | 63 | start_time = time() 64 | 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_LinkNet_U/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.VGGLinkNet 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1, ) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_LinkNet_U/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 53 | ] 54 | 55 | test_callbacks = [ 56 | mt.Callbacks.SavePyvistaPoints( 57 | on_batch=1, properties=['curvature'], 58 | filename=f'./fold_{fold_number}/test/data') 59 | ] 60 | 61 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 62 | 63 | start_time = time() 64 | 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_MLP/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | import torch 5 | 6 | input_name = 'curvature' 7 | target_name = 'wss' 8 | 9 | class Model(torch.nn.Module): 10 | def __init__(self, *args, **kwargs): 11 | super().__init__() 12 | 13 | base = 600 14 | 15 | self.lin1 = torch.nn.Linear(4, base, bias=False) 16 | self.lin2 = torch.nn.Linear(base, base * 2, bias=False) 17 | self.lin3 = torch.nn.Linear(base * 2, base * 4, bias=False) 18 | self.lin4 = torch.nn.Linear(base * 4, base * 2, bias=False) 19 | self.lin5 = torch.nn.Linear(base * 2, base * 1, bias=False) 20 | self.lin6 = torch.nn.Linear(base * 1, base // 2, bias=False) 21 | self.lin7 = torch.nn.Linear(base // 2, 1, bias=False) 22 | 23 | def forward(self, x, *args, **kwargs): 24 | x = x.transpose(1,-1) 25 | x = self.lin1(x) 26 | x = torch.nn.functional.relu(x) 27 | x = self.lin2(x) 28 | x = torch.nn.functional.relu(x) 29 | x = self.lin3(x) 30 | x = torch.nn.functional.relu(x) 31 | x = self.lin4(x) 32 | x = torch.nn.functional.relu(x) 33 | x = self.lin5(x) 34 | x = torch.nn.functional.relu(x) 35 | x = self.lin6(x) 36 | x = torch.nn.functional.relu(x) 37 | x = self.lin7(x) 38 | x = torch.nn.functional.relu(x) 39 | return x.transpose(1,-1) 40 | 41 | ml_model = Model 42 | current_directory = Path(__file__).parent 43 | 44 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 45 | TVS = mt.Utils.AttrDict() 46 | TVS.init = dict(input_name=input_name, target_name=target_name, RGB=True) 47 | TVS.split = mt.Utils.AttrDict() 48 | TVS.split.args = (0.1, ) 49 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 50 | TVS.call = dict(batch_size=1, shuffle=True) 51 | 52 | model = mt.Utils.AttrDict() 53 | model.args = (4, 1, 3) 54 | model.kwargs = dict(n_pool=4, base_channels=64, output_activation=torch.nn.functional.relu) 55 | 56 | optimizer = mt.Utils.AttrDict() 57 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 58 | 59 | criterion = dict(reduction='sum') 60 | 61 | trainer = dict( 62 | x_key=input_name, y_key=target_name 63 | ) 64 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_MLP/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | # mt.nn.ConvBase.default_padding_string = 'periodic_replication' 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | 40 | print(mt.Utils.count_parameters(model)) 41 | 42 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 43 | 44 | 45 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 46 | criterion = torch.nn.L1Loss(**config.criterion) 47 | 48 | fit_callbacks = [ 49 | mt.Callbacks.Tensorboard(), 50 | mt.Callbacks.SavePyvistaPoints( 51 | on_step=1000, properties=['curvature'], 52 | filename=f'./fold_{fold_number}/train/data'), 53 | mt.Callbacks.LRFind(), 54 | 55 | mt.Callbacks.ReduceLROnValidPlateau( 56 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 57 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 58 | ] 59 | 60 | test_callbacks = [ 61 | mt.Callbacks.SavePyvistaPoints( 62 | on_batch=1, properties=['curvature'], 63 | filename=f'./fold_{fold_number}/test/data') 64 | ] 65 | 66 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 67 | 68 | start_time = time() 69 | 70 | trainer.fit_on_loader( 71 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 72 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 73 | 74 | end_time = time() 75 | 76 | train_time = end_time - start_time 77 | 78 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 79 | 80 | start_time = time() 81 | trainer.test_on_loader( 82 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 83 | ) 84 | end_time = time() 85 | 86 | test_time = end_time-start_time 87 | 88 | time_log.append( 89 | (train_time, test_time) 90 | ) 91 | 92 | np.save('./time_log', time_log) 93 | 94 | del fold_number 95 | del model 96 | del optimizer 97 | del criterion 98 | del fit_callbacks 99 | del test_callbacks 100 | del trainer 101 | torch.cuda.empty_cache() 102 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_PSPNet_Resnet34/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | 9 | current_directory = Path(__file__).parent 10 | 11 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 12 | TVS = mt.Utils.AttrDict() 13 | TVS.init = dict(input_name=input_name, target_name=target_name) 14 | TVS.split = mt.Utils.AttrDict() 15 | TVS.split.args = (0.1, ) 16 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 17 | TVS.call = dict(batch_size=1, shuffle=False) 18 | 19 | model = mt.Utils.AttrDict() 20 | model.args = () 21 | model.kwargs = dict(n_classes=1, backend='resnet34', psp_size=512) 22 | # n_pool=4, base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_PSPNet_Resnet34/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | if __name__ == "__main__": 9 | numpy_files = mt.Utils.glob_npz(config.data_folder) 10 | 11 | transform = [mt.Data.Transforms.RollTensor(), ] 12 | 13 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 14 | numpy_files, transform=transform, **config.TVS.init) 15 | 16 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 17 | 18 | split = KFoldTVSLoader(**config.TVS.call) 19 | 20 | time_log = [] 21 | 22 | for fold_number, (train_loader, valid_loader, test_loader) in \ 23 | enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = mt.Models.PSPNet(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.Validation( 46 | checkpoint=True, on_step=200, to_file=f'./fold_{fold_number}/valid/log', 47 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint' 48 | ), 49 | mt.Callbacks.SavePyvistaPoints( 50 | on_step=1000, properties=['curvature'], 51 | filename=f'./fold_{fold_number}/train/data'), 52 | mt.Callbacks.LRFind(), 53 | 54 | mt.Callbacks.ReduceLROnValidPlateau() 55 | ] 56 | 57 | test_callbacks = [ 58 | mt.Callbacks.SavePyvistaPoints( 59 | on_batch=1, properties=['curvature'], 60 | filename=f'./fold_{fold_number}/test/data') 61 | ] 62 | 63 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 64 | 65 | start_time = time() 66 | trainer.fit_on_loader( 67 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 68 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_PSPNet_Resnet50/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | 9 | current_directory = Path(__file__).parent 10 | 11 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 12 | TVS = mt.Utils.AttrDict() 13 | TVS.init = dict(input_name=input_name, target_name=target_name) 14 | TVS.split = mt.Utils.AttrDict() 15 | TVS.split.args = (0.1, ) 16 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 17 | TVS.call = dict(batch_size=1, shuffle=False) 18 | 19 | model = mt.Utils.AttrDict() 20 | model.args = () 21 | model.kwargs = dict(n_classes=1, backend='resnet50') 22 | # n_pool=4, base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_PSPNet_Resnet50/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | if __name__ == "__main__": 9 | numpy_files = mt.Utils.glob_npz(config.data_folder) 10 | 11 | transform = [mt.Data.Transforms.RollTensor(), ] 12 | 13 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 14 | numpy_files, transform=transform, **config.TVS.init) 15 | 16 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 17 | 18 | split = KFoldTVSLoader(**config.TVS.call) 19 | 20 | time_log = [] 21 | 22 | for fold_number, (train_loader, valid_loader, test_loader) in \ 23 | enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = mt.Models.PSPNet(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.Validation( 46 | checkpoint=True, on_step=200, to_file=f'./fold_{fold_number}/valid/log', 47 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint' 48 | ), 49 | mt.Callbacks.SavePyvistaPoints( 50 | on_step=1000, properties=['curvature'], 51 | filename=f'./fold_{fold_number}/train/data'), 52 | mt.Callbacks.LRFind(), 53 | 54 | mt.Callbacks.ReduceLROnValidPlateau() 55 | ] 56 | 57 | test_callbacks = [ 58 | mt.Callbacks.SavePyvistaPoints( 59 | on_batch=1, properties=['curvature'], 60 | filename=f'./fold_{fold_number}/test/data') 61 | ] 62 | 63 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 64 | 65 | start_time = time() 66 | trainer.fit_on_loader( 67 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 68 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_SEFPN_U/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.SEVGGFPN 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1,) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (4, 1, 3) 22 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_SEFPN_U/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 53 | ] 54 | 55 | test_callbacks = [ 56 | mt.Callbacks.SavePyvistaPoints( 57 | on_batch=1, properties=['curvature'], 58 | filename=f'./fold_{fold_number}/test/data') 59 | ] 60 | 61 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 62 | 63 | start_time = time() 64 | 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_Unet_Curvature/analyze_results.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import pathlib as pt 3 | import matplotlib.pyplot as plt 4 | 5 | def relative_error(true, pred): 6 | numerator = 2*np.abs(true-pred) 7 | denominator = np.abs(true+pred+1e-7) 8 | return np.clip(numerator/denominator, 0, 1) 9 | 10 | def relative_accuracy(true, pred): 11 | return ((1-relative_error(true,pred))*100).mean() 12 | 13 | if __name__ == "__main__": 14 | 15 | cwd = pt.Path(__file__).parent 16 | 17 | fold_folders = cwd.glob("fold_*") 18 | 19 | for fold in fold_folders: 20 | test_folder = fold/'test' 21 | npz_files = list(test_folder.glob('*.npz')) 22 | 23 | all_accuracy = [] 24 | 25 | for file in npz_files: 26 | data = np.load(file) 27 | 28 | # clip the first 10 columns due to noise 29 | y_true = data['y_true'][:,:,5:] 30 | y_pred = data['y_pred'][:,:,5:] 31 | 32 | accuracy = relative_accuracy(y_true, y_pred) 33 | 34 | all_accuracy.append(accuracy) 35 | 36 | mean_acc = np.mean(all_accuracy) 37 | std_acc = np.std(all_accuracy) 38 | 39 | print(mean_acc) 40 | print(std_acc) 41 | pass 42 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_Unet_Curvature/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.Unet 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | 14 | TVS = mt.Utils.AttrDict() 15 | TVS.init = dict(input_name=input_name, target_name=target_name) 16 | TVS.split = mt.Utils.AttrDict() 17 | TVS.split.args = (0.1, ) 18 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 19 | TVS.call = dict(batch_size=1, shuffle=True) 20 | 21 | model = mt.Utils.AttrDict() 22 | model.args = (4, 1, 3) 23 | model.kwargs = dict(n_pool=4, base_channels=64, output_activation=torch.nn.functional.relu) 24 | 25 | optimizer = mt.Utils.AttrDict() 26 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 27 | 28 | criterion = dict(reduction='sum') 29 | 30 | trainer = dict( 31 | x_key=input_name, y_key=target_name 32 | ) 33 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_Unet_Curvature/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = config.ml_model(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 53 | ] 54 | 55 | test_callbacks = [ 56 | mt.Callbacks.SavePyvistaPoints( 57 | on_batch=1, properties=['curvature'], 58 | filename=f'./fold_{fold_number}/test/data') 59 | ] 60 | 61 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 62 | 63 | start_time = time() 64 | 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | 69 | end_time = time() 70 | 71 | train_time = end_time - start_time 72 | 73 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 74 | 75 | start_time = time() 76 | trainer.test_on_loader( 77 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 78 | ) 79 | end_time = time() 80 | 81 | test_time = end_time-start_time 82 | 83 | time_log.append( 84 | (train_time, test_time) 85 | ) 86 | 87 | np.save('./time_log', time_log) 88 | 89 | del fold_number 90 | del model 91 | del optimizer 92 | del criterion 93 | del fit_callbacks 94 | del test_callbacks 95 | del trainer 96 | torch.cuda.empty_cache() 97 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_Unet_Points/analyze_results.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import pathlib as pt 3 | import matplotlib.pyplot as plt 4 | 5 | def relative_error(true, pred): 6 | numerator = 2*np.abs(true-pred) 7 | denominator = np.abs(true+pred+1e-7) 8 | return np.clip(numerator/denominator, 0, 1) 9 | 10 | def relative_accuracy(true, pred): 11 | return ((1-relative_error(true,pred))*100).mean() 12 | 13 | if __name__ == "__main__": 14 | 15 | cwd = pt.Path(__file__).parent 16 | 17 | fold_folders = cwd.glob("fold_*") 18 | 19 | for fold in fold_folders: 20 | test_folder = fold/'test' 21 | npz_files = list(test_folder.glob('*.npz')) 22 | 23 | all_accuracy = [] 24 | 25 | for file in npz_files: 26 | data = np.load(file) 27 | 28 | # clip the first 10 columns due to noise 29 | y_true = data['y_true'][:,:,5:] 30 | y_pred = data['y_pred'][:,:,5:] 31 | 32 | accuracy = relative_accuracy(y_true, y_pred) 33 | 34 | all_accuracy.append(accuracy) 35 | 36 | mean_acc = np.mean(all_accuracy) 37 | std_acc = np.std(all_accuracy) 38 | 39 | print(mean_acc) 40 | 41 | pass 42 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_Unet_Points/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'points' 6 | target_name = 'wss' 7 | 8 | ml_model = mt.Models.Unet 9 | 10 | current_directory = Path(__file__).parent 11 | 12 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 13 | TVS = mt.Utils.AttrDict() 14 | TVS.init = dict(input_name=input_name, target_name=target_name) 15 | TVS.split = mt.Utils.AttrDict() 16 | TVS.split.args = (0.1, ) 17 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 18 | TVS.call = dict(batch_size=1, shuffle=True) 19 | 20 | model = mt.Utils.AttrDict() 21 | model.args = (3, 1, 3) 22 | model.kwargs = dict(n_pool=4, base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_Unet_Points/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | if __name__ == "__main__": 8 | numpy_files = mt.Utils.glob_npz(config.data_folder) 9 | 10 | transform = [mt.Data.Transforms.RollTensor(), ] 11 | 12 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 13 | numpy_files, transform=transform, **config.TVS.init) 14 | 15 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 16 | 17 | split = KFoldTVSLoader(**config.TVS.call) 18 | 19 | time_log = [] 20 | 21 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 22 | 23 | if fold_number == 0: 24 | time_log.append( 25 | dict( 26 | train_steps_per_epoch=len(train_loader), 27 | test_steps_per_epoch=len(test_loader) 28 | ) 29 | ) 30 | 31 | fold_number += 1 32 | 33 | print(f'runing fold {fold_number}/{5}') 34 | 35 | 36 | model = config.ml_model(*config.model.args, **config.model.kwargs) 37 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 38 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 39 | criterion = torch.nn.L1Loss(**config.criterion) 40 | 41 | fit_callbacks = [ 42 | mt.Callbacks.Tensorboard(), 43 | mt.Callbacks.SavePyvistaPoints( 44 | on_step=1000, properties=['curvature'], 45 | filename=f'./fold_{fold_number}/train/data'), 46 | mt.Callbacks.LRFind(), 47 | mt.Callbacks.ReduceLROnValidPlateau( 48 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 49 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 50 | ] 51 | 52 | test_callbacks = [ 53 | mt.Callbacks.SavePyvistaPoints( 54 | on_batch=1, properties=['curvature'], 55 | filename=f'./fold_{fold_number}/test/data') 56 | ] 57 | 58 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 59 | 60 | start_time = time() 61 | 62 | trainer.fit_on_loader( 63 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 64 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 65 | 66 | end_time = time() 67 | 68 | train_time = end_time - start_time 69 | 70 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 71 | 72 | start_time = time() 73 | trainer.test_on_loader( 74 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 75 | ) 76 | end_time = time() 77 | 78 | test_time = end_time-start_time 79 | 80 | time_log.append( 81 | (train_time, test_time) 82 | ) 83 | 84 | np.save('./time_log', time_log) 85 | 86 | del fold_number 87 | del model 88 | del optimizer 89 | del criterion 90 | del fit_callbacks 91 | del test_callbacks 92 | del trainer 93 | torch.cuda.empty_cache() 94 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_qubel_Unet_Resnet34/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | import segmentation_models_pytorch as smp 5 | from torch import nn 6 | 7 | input_name = 'curvature' 8 | target_name = 'wss' 9 | 10 | 11 | def model_maker(*args, **kwargs): 12 | return nn.Sequential( 13 | mt.nn.ConvNormAct2d(4, 3, 3), 14 | smp.Unet('resnet34', classes=1, activation=None) 15 | ) 16 | 17 | 18 | ml_model = model_maker 19 | 20 | current_directory = Path(__file__).parent 21 | 22 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 23 | TVS = mt.Utils.AttrDict() 24 | TVS.init = dict(input_name=input_name, target_name=target_name) 25 | TVS.split = mt.Utils.AttrDict() 26 | TVS.split.args = (0.1, ) 27 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 28 | TVS.call = dict(batch_size=8, shuffle=True) 29 | 30 | model = mt.Utils.AttrDict() 31 | model.args = (4, 1, 3) 32 | model.kwargs = dict(base_channels=64, output_activation=torch.nn.functional.relu) 33 | 34 | optimizer = mt.Utils.AttrDict() 35 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 36 | 37 | criterion = dict(reduction='sum') 38 | 39 | trainer = dict( 40 | x_key=input_name, y_key=target_name 41 | ) 42 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/KFold_qubel_Unet_Resnet34/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | running_folds = [1, 2, 3, 4] 8 | 9 | if __name__ == "__main__": 10 | numpy_files = mt.Utils.glob_npz(config.data_folder) 11 | 12 | transform = [mt.Data.Transforms.RollTensor(), ] 13 | 14 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 15 | numpy_files, transform=transform, **config.TVS.init) 16 | 17 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 18 | 19 | split = KFoldTVSLoader(**config.TVS.call) 20 | 21 | time_log = [] 22 | 23 | for fold_number, (train_loader, valid_loader, test_loader) in enumerate(split): 24 | 25 | if fold_number in running_folds: 26 | 27 | if fold_number == 0: 28 | time_log.append( 29 | dict( 30 | train_steps_per_epoch=len(train_loader), 31 | test_steps_per_epoch=len(test_loader) 32 | ) 33 | ) 34 | 35 | fold_number += 1 36 | 37 | print(f'runing fold {fold_number}/{5}') 38 | 39 | 40 | model = config.ml_model(*config.model.args, **config.model.kwargs) 41 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 42 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 43 | criterion = torch.nn.L1Loss(**config.criterion) 44 | 45 | fit_callbacks = [ 46 | mt.Callbacks.Tensorboard(), 47 | mt.Callbacks.SavePyvistaPoints( 48 | on_step=1000, properties=['curvature'], 49 | filename=f'./fold_{fold_number}/train/data'), 50 | mt.Callbacks.LRFind(), 51 | 52 | mt.Callbacks.ReduceLROnValidPlateau( 53 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 54 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint') 55 | ] 56 | 57 | test_callbacks = [ 58 | mt.Callbacks.SavePyvistaPoints( 59 | on_batch=1, properties=['curvature'], 60 | filename=f'./fold_{fold_number}/test/data') 61 | ] 62 | 63 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 64 | 65 | start_time = time() 66 | 67 | trainer.fit_on_loader( 68 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 69 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 70 | 71 | end_time = time() 72 | 73 | train_time = end_time - start_time 74 | 75 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 76 | 77 | start_time = time() 78 | trainer.test_on_loader( 79 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 80 | ) 81 | end_time = time() 82 | 83 | test_time = end_time-start_time 84 | 85 | time_log.append( 86 | (train_time, test_time) 87 | ) 88 | 89 | np.save('./time_log', time_log) 90 | 91 | del fold_number 92 | del model 93 | del optimizer 94 | del criterion 95 | del fit_callbacks 96 | del test_callbacks 97 | del trainer 98 | torch.cuda.empty_cache() 99 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/Kfold_Unet_Resnet34/config.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | import myTorch as mt 3 | import torch 4 | 5 | input_name = 'curvature' 6 | target_name = 'wss' 7 | 8 | 9 | current_directory = Path(__file__).parent 10 | 11 | data_folder = current_directory/'../../../Data/LADSSMNewtonianSteadyWSS' 12 | TVS = mt.Utils.AttrDict() 13 | TVS.init = dict(input_name=input_name, target_name=target_name) 14 | TVS.split = mt.Utils.AttrDict() 15 | TVS.split.args = (0.1, ) 16 | TVS.split.kwargs = dict(random_state=1, n_splits=5) 17 | TVS.call = dict(batch_size=1, shuffle=True) 18 | 19 | model = mt.Utils.AttrDict() 20 | model.args = (4, 1, 3) 21 | model.kwargs = dict() 22 | # n_pool=4, base_channels=64, output_activation=torch.nn.functional.relu) 23 | 24 | optimizer = mt.Utils.AttrDict() 25 | optimizer.kwargs = dict(lr=1e-7, eps=1e-7) 26 | 27 | criterion = dict(reduction='sum') 28 | 29 | trainer = dict( 30 | x_key=input_name, y_key=target_name 31 | ) 32 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/Kfold_Unet_Resnet34/train.py: -------------------------------------------------------------------------------- 1 | import myTorch as mt 2 | import torch 3 | import config 4 | import numpy as np 5 | from time import time 6 | 7 | 8 | if __name__ == "__main__": 9 | numpy_files = mt.Utils.glob_npz(config.data_folder) 10 | 11 | transform = [mt.Data.Transforms.RollTensor(), ] 12 | 13 | KFoldTVSLoader = mt.Data.DataLoaders.KFoldCrossTrainTestSplit( 14 | numpy_files, transform=transform, **config.TVS.init) 15 | 16 | KFoldTVSLoader.split(*config.TVS.split.args, **config.TVS.split.kwargs) 17 | 18 | split = KFoldTVSLoader(**config.TVS.call) 19 | 20 | time_log = [] 21 | 22 | for fold_number, (train_loader, valid_loader, test_loader) in \ 23 | enumerate(split): 24 | 25 | if fold_number == 0: 26 | time_log.append( 27 | dict( 28 | train_steps_per_epoch=len(train_loader), 29 | test_steps_per_epoch=len(test_loader) 30 | ) 31 | ) 32 | 33 | fold_number += 1 34 | 35 | print(f'runing fold {fold_number}/{5}') 36 | 37 | 38 | model = mt.Models.ResnetUnet(*config.model.args, **config.model.kwargs) 39 | model = mt.nn.init.XavierUniformWeightInitializer()(model) 40 | optimizer = torch.optim.Adam(model.parameters(), **config.optimizer.kwargs) 41 | criterion = torch.nn.L1Loss(**config.criterion) 42 | 43 | fit_callbacks = [ 44 | mt.Callbacks.Tensorboard(), 45 | mt.Callbacks.SavePyvistaPoints( 46 | on_step=1000, properties=['curvature'], 47 | filename=f'./fold_{fold_number}/train/data'), 48 | mt.Callbacks.LRFind(), 49 | 50 | mt.Callbacks.ReduceLROnValidPlateau( 51 | checkpoint=True, to_file=f'./fold_{fold_number}/valid/log', 52 | filename=f'./fold_{fold_number}/Checkpoints/checkpoint' 53 | ) 54 | ] 55 | 56 | test_callbacks = [ 57 | mt.Callbacks.SavePyvistaPoints( 58 | on_batch=1, properties=['curvature'], 59 | filename=f'./fold_{fold_number}/test/data') 60 | ] 61 | 62 | trainer = mt.Trainer(model, optimizer, criterion, **config.trainer) 63 | 64 | start_time = time() 65 | trainer.fit_on_loader( 66 | train_loader, valid_dataloader=valid_loader, cycles=[100, ], 67 | callbacks=fit_callbacks, std_to_file=f'./fold_{fold_number}/train/log') 68 | end_time = time() 69 | 70 | train_time = end_time - start_time 71 | 72 | trainer.load_best_model(f'./fold_{fold_number}/Checkpoints') 73 | 74 | start_time = time() 75 | trainer.test_on_loader( 76 | test_loader, std_to_file=f'./fold_{fold_number}/test/log', callbacks=test_callbacks 77 | ) 78 | end_time = time() 79 | 80 | test_time = end_time-start_time 81 | 82 | time_log.append( 83 | (train_time, test_time) 84 | ) 85 | 86 | np.save('./time_log', time_log) 87 | 88 | del fold_number 89 | del model 90 | del optimizer 91 | del criterion 92 | del fit_callbacks 93 | del test_callbacks 94 | del trainer 95 | torch.cuda.empty_cache() 96 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/bin/accuracy_blandtaltman_kfold.py: -------------------------------------------------------------------------------- 1 | 2 | import pathlib as pt 3 | import click 4 | from collections import defaultdict 5 | import numpy as np 6 | from numpy.random import default_rng 7 | from scipy import stats 8 | import matplotlib.pyplot as plt 9 | from matplotlib import rc 10 | import seaborn as sns 11 | import pingouin 12 | from matplotlib import transforms 13 | from scipy.stats import iqr 14 | from numpy import median 15 | 16 | rc('font', **{'family': 'serif', 'serif': ['Times New Roman']}) 17 | rc('text', usetex=True) 18 | 19 | def default_list_dict(): 20 | return defaultdict(list) 21 | 22 | def relative_error(y_true, y_pred): 23 | numerator = 2 * np.abs(y_true-y_pred) 24 | denominator = np.abs(y_true)+np.abs(y_pred) + 1e-7 25 | return numerator/denominator 26 | 27 | def relative_percentage_accuracy(y_true, y_pred): 28 | error = relative_error(y_true, y_pred) 29 | return (1-error)*100 30 | 31 | def plot_blandaltman(x, y, figsize=(5, 4), 32 | dpi=100, ax=None): 33 | # Safety check 34 | x = np.asarray(x) 35 | y = np.asarray(y) 36 | assert x.ndim == 1 and y.ndim == 1 37 | assert x.size == y.size 38 | n = x.size 39 | mean = np.vstack((x, y)).mean(0) 40 | diff = x - y 41 | 42 | # get the interquartile range 43 | q975, q025 = np.percentile(diff, 97.5), np.percentile(diff, 2.5) 44 | q75 = np.percentile(diff, 77.5) 45 | q25 = np.percentile(diff, 25) 46 | md = median(diff) 47 | 48 | # Start the plot 49 | if ax is None: 50 | fig, ax = plt.subplots(1, 1, figsize=figsize, dpi=dpi) 51 | 52 | # Plot the mean diff, limits of agreement and scatter 53 | ax.axhline(md, color='#6495ED', linestyle='--') 54 | ax.axhline(q975, color='coral', linestyle='--') 55 | ax.axhline(q025, color='coral', linestyle='--') 56 | ax.axhline(q75, color='red', linestyle='--') 57 | ax.axhline(q25, color='red', linestyle='--') 58 | ax.scatter(mean, diff, 20, alpha=0.5) 59 | 60 | offset = (min(q975-md, np.abs(md-q025)) / 100.0) * 1.5 61 | 62 | trans = transforms.blended_transform_factory(ax.transAxes, ax.transData) 63 | 64 | ax.text(1.03, md, '%.2f Median' % md, ha="left", va="center", 65 | transform=trans) 66 | 67 | ax.text(1.03, q975, 68 | '%.2f P97.5' % q975, ha="left", va="center", 69 | transform=trans) 70 | 71 | ax.text(1.03, q025, 72 | '%.2f P2.5' % q025, ha="left", va="center", 73 | transform=trans) 74 | 75 | ax.text(0.98, q25-offset-0.1, '%.2f P25'%q25, ha="right", va="top", transform=trans) 76 | ax.text(0.98, q75+offset, '%.2f P75'%q75, ha="right", va="bottom", transform=trans) 77 | 78 | # Labels and title 79 | ax.set_ylabel('Difference between methods') 80 | ax.set_xlabel('Mean of methods') 81 | ax.set_title('Bland-Altman plot') 82 | 83 | plt.locator_params(axis='y', nbins=20) 84 | plt.locator_params(axis='x', nbins=20) 85 | 86 | # Despine and trim 87 | sns.despine(trim=True, ax=ax) 88 | 89 | return ax 90 | 91 | @click.command() 92 | @click.argument('path', type=click.STRING) 93 | @click.option('--s', default='Study') 94 | @click.option('--n', default=50, type=int) 95 | def main(path, s, n): 96 | 97 | main_folder = pt.Path(path) 98 | 99 | # originally subsample the data 100 | fold_samples_dict = defaultdict(default_list_dict) 101 | 102 | for i, fold_folder in enumerate(main_folder.glob('fold*')): 103 | 104 | test_folder = fold_folder/'test' 105 | 106 | fold_number = i + 1 107 | 108 | for file in test_folder.glob('*.npz'): 109 | data = np.load(file) 110 | 111 | y_pred = data['y_pred'].flatten() 112 | y_true = data['y_true'].flatten() 113 | 114 | rng = default_rng() 115 | 116 | random_numbers = rng.choice(len(y_pred), size=n, replace=False) 117 | 118 | fold_samples_dict['fold_%d'%fold_number]['pred'].extend( 119 | [y_pred[i] for i in random_numbers] 120 | ) 121 | fold_samples_dict['fold_%d'%fold_number]['true'].extend( 122 | [y_true[i] for i in random_numbers] 123 | ) 124 | 125 | accuracy_percentage_dict = defaultdict(list) 126 | 127 | for key in fold_samples_dict.keys(): 128 | 129 | fold = fold_samples_dict[key] 130 | y_pred = np.array(fold['pred']) 131 | y_true = np.array(fold['true']) 132 | 133 | accuracy_percentage_dict['acc'].extend( 134 | relative_percentage_accuracy(y_true, y_pred)) 135 | accuracy_percentage_dict['y_true'].extend(y_true) 136 | accuracy_percentage_dict['y_pred'].extend(y_pred) 137 | 138 | acc = accuracy_percentage_dict['acc'] 139 | true = accuracy_percentage_dict['y_true'] 140 | pred = accuracy_percentage_dict['y_pred'] 141 | 142 | # ax = plot_blandaltman( 143 | # true, pred 144 | # ) 145 | ax = plot_blandaltman( 146 | true, pred 147 | ) 148 | ax.set_title( 149 | 'CFD vs CNN MWSS (Pa)' 150 | ) 151 | ax.set_xlabel( 152 | '(CFD+CNN)/2' 153 | ) 154 | ax.set_ylabel( 155 | 'CFD - CNN' 156 | ) 157 | ax.set_aspect('equal') 158 | plt.show() 159 | 160 | if __name__ == "__main__": 161 | main() -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/bin/analyze_single_study.py: -------------------------------------------------------------------------------- 1 | from analyze_kfold import * 2 | 3 | @click.command() 4 | @click.argument('path', type=click.STRING) 5 | @click.option('--s', default='Study') 6 | def main(path, s): 7 | 8 | study_folder = pt.Path(path) 9 | assert study_folder.exists(), 'the specified study path does not exist' 10 | print('analyzing', study_folder) 11 | save_file = study_folder/f'../{s}.csv' 12 | data_dict = defaultdict(list) 13 | 14 | name_method_dict = { 15 | 'relative_percentage_accuracy': relative_percentage_accuracy, 16 | 'coefficient_of_determination': get_coefficient_of_determination, 17 | 'coverage_1p00': get_coverage_1p00, 18 | 'coverage_0p75': get_coverage_0p75, 19 | 'coverage_0p50': get_coverage_0p50 20 | } 21 | name_report_dict = { 22 | 'report_1p00': get_report_1p00, 23 | 'report_0p75': get_report_0p75, 24 | 'report_0p50': get_report_0p50, 25 | } 26 | 27 | for data_file in study_folder.glob('*.npz'): 28 | 29 | data = np.load(data_file) 30 | y_true = data['y_true'].flatten() 31 | y_pred = data['y_pred'].flatten() 32 | for key, method in name_method_dict.items(): 33 | data_dict[key].append( 34 | method(y_true, y_pred)*100 35 | ) 36 | for key, method in name_report_dict.items(): 37 | 38 | limit = key.split('_')[-1] 39 | report = method(y_true, y_pred) 40 | zero_report = report['0.0'] 41 | one_report = report['1.0'] 42 | 43 | data_dict[f'f1-score_{limit}'].append(one_report['f1-score']*100) 44 | data_dict[f'precision_{limit}'].append(one_report['precision']*100) 45 | data_dict[f'sensitivity_{limit}'].append(one_report['recall']*100) 46 | data_dict[f'specificity{limit}'].append(zero_report['recall']*100) 47 | 48 | 49 | keys_items_cache = list(data_dict.items()) 50 | 51 | for key, values in keys_items_cache: 52 | data_dict[key].append( 53 | ( 54 | np.around(np.mean(values), decimals=3), 55 | np.around(np.std(values), decimals=3) 56 | ) 57 | ) 58 | 59 | df = pd.DataFrame(data_dict) 60 | df.to_csv(save_file) 61 | 62 | if __name__ == "__main__": 63 | main() -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/bin/linear_regression_kfold.py: -------------------------------------------------------------------------------- 1 | import pathlib as pt 2 | import click 3 | from collections import defaultdict 4 | import numpy as np 5 | from numpy.random import default_rng 6 | from scipy import stats 7 | import matplotlib.pyplot as plt 8 | from matplotlib import rc 9 | rc('font', **{'family': 'serif', 'serif': ['Times New Roman']}) 10 | rc('text', usetex=True) 11 | 12 | def default_list_dict(): 13 | return defaultdict(list) 14 | 15 | @click.command() 16 | @click.argument('path', type=click.STRING) 17 | @click.option('--s', default='Study') 18 | @click.option('--n', default=50, type=int) 19 | def main(path, s, n): 20 | 21 | main_folder = pt.Path(path) 22 | 23 | # originally subsample the data 24 | fold_samples_dict = defaultdict(default_list_dict) 25 | 26 | all_fold_folders = list(main_folder.glob('fold*')) 27 | 28 | for i, fold_folder in enumerate(all_fold_folders): 29 | 30 | test_folder = fold_folder/'test' 31 | 32 | fold_number = i + 1 33 | 34 | for file in test_folder.glob('*.npz'): 35 | data = np.load(file) 36 | 37 | y_pred = np.log(np.abs(data['y_pred'].flatten())) 38 | y_true = np.log(data['y_true'].flatten()) 39 | # y_pred = np.abs(data['y_pred'].flatten()) 40 | # y_true = data['y_true'].flatten() 41 | 42 | rng = default_rng() 43 | 44 | random_numbers = rng.choice(len(y_pred), size=n, replace=False) 45 | 46 | fold_samples_dict['fold_%d'%fold_number]['pred'].extend( 47 | [y_pred[v] for v in random_numbers] 48 | ) 49 | fold_samples_dict['fold_%d'%fold_number]['true'].extend( 50 | [y_true[v] for v in random_numbers] 51 | ) 52 | 53 | for fold_name in fold_samples_dict.keys(): 54 | 55 | fold = fold_samples_dict[fold_name] 56 | 57 | y_true = np.array(fold['true']) 58 | y_pred = np.array(fold['pred']) 59 | 60 | slope, intercept, r_value, p_value, std_err = \ 61 | stats.linregress(y_true, y_pred) 62 | 63 | fold_samples_dict[fold_name]['slope'] = slope 64 | fold_samples_dict[fold_name]['intercept'] = intercept 65 | fold_samples_dict[fold_name]['r_value'] = r_value 66 | fold_samples_dict[fold_name]['p_value'] = p_value 67 | fold_samples_dict[fold_name]['std_err'] = std_err 68 | 69 | mean_dict = {} 70 | 71 | mean_keys = ['slope', 'intercept', 'r_value', 'p_value', 'std_err'] 72 | 73 | for key in mean_keys: 74 | mean_dict[key] = np.mean( 75 | [fold_samples_dict[k][key] for k in fold_samples_dict.keys()] 76 | ) 77 | 78 | ax = plt.gca() 79 | m_min, m_max = -3.5, 3.5 80 | x = np.linspace(m_min, m_max, 100) 81 | 82 | for i, key in enumerate(fold_samples_dict): 83 | 84 | fold = fold_samples_dict[key] 85 | fold_id = i+1 86 | 87 | m, c, r = fold['slope'], fold['intercept'], fold['r_value'] 88 | r_2 = r**2 89 | 90 | true = fold['true'] 91 | pred = fold['pred'] 92 | 93 | ax.plot(x, m*x + c, linestyle='--', linewidth=1, alpha=1, 94 | label='fold %d, $%0.3fx %0.3f$, $r^2=%0.3f$' % 95 | (fold_id, m, c, r_2)) 96 | 97 | ax.scatter(true, pred, 10, alpha=0.5) 98 | 99 | mean_m = mean_dict['slope'] 100 | mean_c = mean_dict['intercept'] 101 | mean_r2 = mean_dict['r_value']**2 102 | 103 | ax.plot(x, mean_m*x + mean_c, color='red', linestyle='-', linewidth=1, 104 | alpha=1, label='Expected, $%0.3fx + %0.3f$, $r^2=%0.3f$' % 105 | (mean_m, mean_c, mean_r2)) 106 | 107 | ax.set_aspect('equal', 'box') 108 | ax.set_xlim(m_min, m_max) 109 | ax.set_ylim(m_min, m_max) 110 | ax.minorticks_on() 111 | ax.set_title('CFD vs CNN MWSS, Expected $r^2=%0.3f$' % (mean_r2), 112 | fontsize=12) 113 | ax.set_ylabel('Log CNN MWSS (Pa)', fontsize=10) 114 | ax.set_xlabel('Log CFD MWSS (Pa)', fontsize=10) 115 | 116 | plt.legend(fontsize=8) 117 | plt.show() 118 | 119 | if __name__ == "__main__": 120 | main() -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/bin/make_copyable_to_latex.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import pathlib as pt 3 | import click 4 | 5 | @click.command() 6 | @click.argument('path', type=click.STRING) 7 | @click.option('--s', default='latex_copyable') 8 | def main(path, s): 9 | csv_path = pt.Path(path) 10 | save_path = csv_path.parent/f"{s}.csv" 11 | assert csv_path.exists(), "the provided csv path does not exist" 12 | assert "csv" in csv_path.suffix, "file is not a csv" 13 | df = pd.read_csv(csv_path) 14 | output_string = '' 15 | 16 | per_item = False 17 | 18 | for key in df: 19 | data = df[key] 20 | if "Unnamed" not in key: 21 | output_string += key 22 | for item in data: 23 | if '(' in item: 24 | # convert the data to the correct format 25 | item = item.strip('(') 26 | item = item.strip(')') 27 | item = item.split(',') 28 | item = [float(i) for i in item] 29 | output_string += ', ' 30 | output_string += '%0.3f (%0.3f)'%(item[0], item[1]) 31 | else: 32 | item = float(item) 33 | output_string += ', ' 34 | output_string += '%0.3f'%item 35 | else: 36 | nfolds = len(data) - 1 37 | output_string += ' ' 38 | for fold_id in range(nfolds): 39 | output_string += ', fold %d'%(fold_id+1) 40 | output_string += ', average' 41 | 42 | output_string += '\n' 43 | 44 | with open(save_path, 'w') as f: 45 | f.write(output_string) 46 | 47 | if __name__ == "__main__": 48 | main() -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/bin/print_relative_accuracy.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import pathlib as pt 3 | import matplotlib.pyplot as plt 4 | import click 5 | 6 | 7 | def relative_error(true, pred): 8 | numerator = 2*np.abs(true-pred) 9 | denominator = np.abs(true+pred+1e-7) 10 | return np.clip(numerator/denominator, 0, 1) 11 | 12 | 13 | def relative_accuracy(true, pred): 14 | return ((1-relative_error(true,pred))*100).mean() 15 | 16 | 17 | def show_mean_relative_accuracy(cwd): 18 | print("showing relative accuracy for", cwd) 19 | cwd = pt.Path(cwd) 20 | fold_folders = cwd.glob("fold_*") 21 | for fold in fold_folders: 22 | test_folder = fold/'test' 23 | npz_files = list(test_folder.glob('*.npz')) 24 | all_accuracy = [] 25 | for file in npz_files: 26 | data = np.load(file) 27 | # clip the first 5 columns due to noise 28 | y_true = data['y_true'][:,:,5:] 29 | y_pred = data['y_pred'][:,:,5:] 30 | accuracy = relative_accuracy(y_true, y_pred) 31 | all_accuracy.append(accuracy) 32 | mean_acc = np.mean(all_accuracy) 33 | std_acc = np.std(all_accuracy) 34 | 35 | print("mean", mean_acc, "std", std_acc) 36 | 37 | 38 | @click.command() 39 | @click.option('--d', default='.', help='working directory') 40 | def main(d): 41 | cwd = pt.Path(d) 42 | show_mean_relative_accuracy(d) 43 | 44 | if __name__ == "__main__": 45 | main() 46 | 47 | 48 | -------------------------------------------------------------------------------- /Training_Examples/WSSPrediction/bin/write_relative_accuracy_to_csv.py: -------------------------------------------------------------------------------- 1 | from print_relative_accuracy import relative_accuracy 2 | import numpy as np 3 | import pathlib as pt 4 | import click 5 | import pandas as pd 6 | 7 | def write_relative_accuracy( 8 | cwd, write_name='output', csv_file=None, write_path=None): 9 | 10 | cwd = pt.Path(cwd) 11 | 12 | fold_folders = cwd.glob("fold_*") 13 | if csv_file is not None: 14 | df = pd.read_csv(csv_file) 15 | else: 16 | df = pd.DataFrame( 17 | { 18 | 'name':[cwd], 19 | 'accurcy_mean_fold_1':[], 20 | 'accurcy_std_fold_1':[], 21 | 'accurcy_mean_fold_2':[], 22 | 'accurcy_std_fold_2':[], 23 | 'accurcy_mean_fold_3':[], 24 | 'accurcy_std_fold_3':[], 25 | 'accurcy_mean_fold_4':[], 26 | 'accurcy_std_fold_4':[], 27 | 'accurcy_mean_fold_5':[], 28 | 'accurcy_std_fold_5':[], 29 | } 30 | ) 31 | 32 | for fold in fold_folders: 33 | test_folder = fold/'test' 34 | npz_files = list(test_folder.glob('*.npz')) 35 | all_accuracy = [] 36 | for file in npz_files: 37 | data = np.load(file) 38 | # clip the first 5 columns due to noise 39 | y_true = data['y_true'][:,:,5:] 40 | y_pred = data['y_pred'][:,:,5:] 41 | accuracy = relative_accuracy(y_true, y_pred) 42 | all_accuracy.append(accuracy) 43 | mean_acc = np.mean(all_accuracy) 44 | std_acc = np.std(all_accuracy) 45 | df = df.append( 46 | 47 | ) 48 | 49 | if __name__ == "__main__": 50 | main() 51 | 52 | 53 | -------------------------------------------------------------------------------- /myTorch/Callbacks/Callback.py: -------------------------------------------------------------------------------- 1 | class Callback: 2 | def __init__(self): 3 | pass 4 | 5 | def pre_loop(self, environment): 6 | pass 7 | 8 | def pre_cycle(self, environment): 9 | pass 10 | 11 | def pre_epoch(self, environment): 12 | pass 13 | 14 | def pre_batch(self, environment): 15 | pass 16 | 17 | def post_batch(self, environment): 18 | pass 19 | 20 | def post_epoch(self, environment): 21 | pass 22 | 23 | def post_cycle(self, environment): 24 | pass 25 | 26 | def post_loop(self, environment): 27 | pass 28 | 29 | 30 | class CallOn(Callback): 31 | 32 | def __init__( 33 | self, on_batch=False, on_epoch=False, on_cycle=False, 34 | on_step=False, on_end=True): 35 | 36 | self.on_batch = on_batch 37 | self.on_epoch = on_epoch 38 | self.on_cycle = on_cycle 39 | self.on_step = on_step 40 | self.on_end = on_end 41 | self.global_step = 0 42 | 43 | def post_batch(self, env): 44 | if self.on_step: 45 | if self.global_step % self.on_step == 0: 46 | self.method(env) 47 | if self.on_batch: 48 | if env.i_batch % self.on_batch == 0: 49 | self.method(env) 50 | self.global_step += 1 51 | 52 | def post_epoch(self, env): 53 | if self.on_epoch: 54 | if env.i_epoch % self.on_epoch == 0: 55 | self.method(env) 56 | 57 | def post_cycle(self, env): 58 | if self.on_cycle: 59 | if env.i_cycle % self.on_cycle == 0: 60 | self.method(env) 61 | 62 | def post_loop(self, env): 63 | if self.on_end: 64 | self.method(env) 65 | 66 | def method(self, env): 67 | NotImplemented 68 | -------------------------------------------------------------------------------- /myTorch/Callbacks/Checkpoints.py: -------------------------------------------------------------------------------- 1 | from . import CallOn 2 | import torch 3 | from pathlib import Path 4 | 5 | 6 | class Checkpoints(CallOn): 7 | def __init__( 8 | self, on_batch=False, on_epoch=False, on_cycle=False, on_step=False, 9 | on_end=True, filename='./Checkpoints/checkpoint', to_keep=10 10 | ): 11 | # initialize when to save state 12 | super(Checkpoints, self).__init__( 13 | on_batch=on_batch, on_epoch=on_epoch, on_cycle=on_cycle, 14 | on_step=on_step, on_end=on_end 15 | ) 16 | # initialize the directory path and the directory 17 | # name 18 | self.directory = Path(filename).parent 19 | self.filename = Path(filename).name 20 | self.to_keep = to_keep 21 | 22 | def pre_loop(self, env): 23 | if not self.directory.exists(): 24 | self.directory.mkdir() 25 | 26 | def method(self, env): 27 | self.save(env) 28 | 29 | def save(self, env): 30 | checkpoints = list(self.directory.glob( 31 | '%s*' % self.filename 32 | )) 33 | if len(checkpoints) > self.to_keep: 34 | checkpoints[0].unlink() 35 | 36 | latest_checkpoint = "%s_%05d" % (self.filename, -1) 37 | for checkpoint in checkpoints: 38 | latest_checkpoint = checkpoint.name 39 | checkpoint_number = latest_checkpoint.split('_')[-1] 40 | checkpoint_number = int(checkpoint_number) 41 | checkpoint_name = "%s_%05d" % (self.filename, checkpoint_number + 1) 42 | torch.save( 43 | self.dict_constructor(env), 44 | self.directory/checkpoint_name 45 | ) 46 | return self.directory/checkpoint_name 47 | 48 | @staticmethod 49 | def dict_constructor(env): 50 | return dict( 51 | model=env.model.state_dict(), 52 | optimizer=env.optimizer.state_dict(), 53 | loss_score=env.loss, 54 | accuracy_score=env.accuracy 55 | ) 56 | -------------------------------------------------------------------------------- /myTorch/Callbacks/DataAnalysis.py: -------------------------------------------------------------------------------- 1 | from . import Callback 2 | import matplotlib.pyplot as plt 3 | import numpy as np 4 | 5 | 6 | class DataAnalysis(Callback): 7 | 8 | def __init__( 9 | self, in_memory=False, save_folder = r'./Analysis', n_step=1, n_average = 5, 10 | **metrics): 11 | 12 | self.n_step = n_step 13 | self.n_average = n_average 14 | self.global_step = 0 15 | self.loss_log = [] 16 | self.mean_loss_accuracy = [] 17 | 18 | def post_batch(self,env): 19 | 20 | y_pred = env.y_pred.to('cpu').detach().numpy() 21 | y_true = env.y_true.to('cpu').detach().numpy() 22 | accuracy = 100 - np.abs(((y_true - y_pred)/y_true * 100).mean()) 23 | 24 | self.loss_log.append(accuracy) 25 | 26 | if self.global_step % self.n_step == 0: 27 | plt.figure() 28 | plt.xlabel('Epochs') 29 | plt.ylabel('Training Loss %') 30 | if len(self.loss_log) > self.n_average: 31 | self.mean_loss_accuracy.append( 32 | sum(self.loss_log[-self.n_average:])/self.n_average 33 | ) 34 | self.global_step += 1 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | # initialize when to save state 50 | -------------------------------------------------------------------------------- /myTorch/Callbacks/LRFind.py: -------------------------------------------------------------------------------- 1 | from . import CallOn 2 | 3 | 4 | class LRFind(CallOn): 5 | 6 | def __init__( 7 | self, on_batch=False, on_epoch=False, on_cycle=True, 8 | on_step=False, on_end=False, in_memory=False, to_file=None, 9 | on_startup=True, **lr_find_kwargs): 10 | # initialize when to save state 11 | super(LRFind, self).__init__( 12 | on_batch=on_batch, on_epoch=on_epoch, on_cycle=on_cycle, 13 | on_step=on_step, on_end=on_end 14 | ) 15 | self.on_startup = on_startup 16 | self.lr_find_kwargs = lr_find_kwargs 17 | 18 | def pre_loop(self, env): 19 | if self.on_startup: 20 | env.lr_find(env.train_dataloader, **self.lr_find_kwargs) 21 | 22 | def method(self, env): 23 | del env.lr_finder 24 | env.optimizer.param_groups[0]['lr'] = \ 25 | env.optimizer.param_groups[0]['lr']/100 26 | env.lr_find(env.train_dataloader, **self.lr_find_kwargs) 27 | -------------------------------------------------------------------------------- /myTorch/Callbacks/Resample.py: -------------------------------------------------------------------------------- 1 | from .Callback import CallOn 2 | import torch 3 | import numpy as np 4 | 5 | 6 | class ResampleTrainingData(CallOn): 7 | 8 | def __init__(self, on_batch=False, on_epoch=False, on_cycle=False, 9 | on_step=False, on_end=False, desired_accuracy=95): 10 | self.desired_accuracy = desired_accuracy 11 | super().__init__( 12 | on_batch=on_batch, on_epoch=on_epoch, 13 | on_cycle=on_cycle, on_step=on_step, 14 | on_end=on_end) 15 | 16 | def post_epoch(self, env): 17 | if self.on_epoch: 18 | if env.i_epoch > 0: 19 | if env.i_epoch % self.on_epoch == 0: 20 | self.method(env) 21 | 22 | def method(self, env): 23 | dataset = env.train_dataloader.dataset 24 | N = len(dataset) 25 | 26 | sol_list = [] 27 | 28 | with torch.no_grad(): 29 | for i in range(N): 30 | x, y = env.data_extractor(dataset[i]) 31 | if len(x.shape) ==3 : 32 | x = x.unsqueeze(0) 33 | y = y.unsqueeze(0) 34 | y_pred = env.model(x) 35 | error = torch.clamp( 36 | 2*torch.abs(y-y_pred)/(torch.abs(y+y_pred)+1e-5), 37 | 0, 38 | 1 39 | ) 40 | accuracy = (100-error*100).mean().detach().to('cpu').numpy() 41 | sol_list.append( (dataset.file_paths[i], accuracy) ) 42 | 43 | sol_list.sort(key=lambda x: x[1]) 44 | accuracies = [item[1] for item in sol_list] 45 | 46 | mean = np.mean(accuracies) 47 | std = np.std(accuracies) 48 | 49 | clipoff_range = mean-std 50 | 51 | lowest_scorering_files = [ 52 | item[0] for item in sol_list if 53 | all( 54 | [ 55 | item[1] < clipoff_range, 56 | item[1] < self.desired_accuracy 57 | ] 58 | ) 59 | ] 60 | 61 | env.train_dataloader.dataset.file_paths += lowest_scorering_files 62 | 63 | -------------------------------------------------------------------------------- /myTorch/Callbacks/Tensorboard.py: -------------------------------------------------------------------------------- 1 | from . import Callback 2 | from torch.utils.tensorboard import SummaryWriter 3 | 4 | 5 | class Tensorboard(Callback): 6 | 7 | def __init__(self, **kwargs): 8 | self.writer = SummaryWriter(**kwargs) 9 | self.graph_constructed = False 10 | 11 | def post_batch(self, environment): 12 | if not self.graph_constructed: 13 | self.writer.add_graph( 14 | environment.model, environment.x.float() 15 | ) 16 | self.graph_constructed = True 17 | 18 | def post_loop(self, environement): 19 | self.writer.close() 20 | -------------------------------------------------------------------------------- /myTorch/Callbacks/__init__.py: -------------------------------------------------------------------------------- 1 | from .Callback import Callback, CallOn 2 | from .Stdout import Stdout 3 | from .Tensorboard import Tensorboard 4 | from .Validation import Validation, ReduceLROnValidPlateau 5 | from .Checkpoints import Checkpoints 6 | from .LRFind import LRFind 7 | from .DataAnalysis import DataAnalysis 8 | from .SaveIntermediate import SaveIntermediate, SavePyvistaPoints, SaveInternalField 9 | from .Resample import ResampleTrainingData 10 | 11 | __all__ = [ 12 | 'Callback', 'CallOn', 'Stdout', 'Tensorboard', 'Validation', 13 | 'ReduceLROnValidPlateau', 'Checkpoints', 'LRFind', 'SaveIntermediate', 14 | 'SavePyvistaPoints' 15 | ] 16 | -------------------------------------------------------------------------------- /myTorch/Data/DictDataset.py: -------------------------------------------------------------------------------- 1 | import torch as _torch 2 | from pathlib import Path as _Path 3 | import numpy as _np 4 | from collections import OrderedDict 5 | try: 6 | from myTorch.Utils import ( 7 | Isinstance, IfNoneReturnDefault) 8 | from myTorch.Data.Transforms import ( 9 | TransformSequence 10 | ) 11 | except ImportError: 12 | from ..Utils import ( 13 | Isinstance, IfNoneReturnDefault) 14 | from .Transforms import ( 15 | TransformSequence) 16 | 17 | 18 | class DictDataset(_torch.utils.data.Dataset): 19 | 20 | default_names = ['input', 'target'] 21 | 22 | def __init__( 23 | self, folder_or_list, auto_gpu=True, transform=None, suffix='*.npz', 24 | input_name=None, target_name=None, RGB=False 25 | ): 26 | 27 | """__init__ 28 | 29 | class for handling datasets containing dictionaries 30 | 31 | Args: 32 | folder_or_list ([str, Path, List, Tuple]): 33 | either a folder containing all the desired file or 34 | a list of files 35 | auto_gpu (bool, optional): if true assign 36 | data to the gpu by default. Defaults to True. 37 | transform ([function, list], optional): 38 | either a list of transforms or a function. 39 | if a list of tranforms then they are compiled to a 40 | single function. Defaults to None. 41 | """ 42 | self.RGB = RGB 43 | self.input_name = input_name if input_name is not None \ 44 | else self.default_names[0] 45 | # self.target_name = target_name if target_name is not None \ 46 | # else self.default_names[1] 47 | self.target_name = target_name 48 | 49 | self.suffix = suffix 50 | 51 | if Isinstance(_Path, str)(folder_or_list): 52 | self.mainfolder = _Path(folder_or_list) 53 | self.file_paths = list( 54 | self.mainfolder.glob(self.suffix) 55 | ) 56 | else: 57 | assert Isinstance(tuple, list)(folder_or_list) 58 | self.file_paths = folder_or_list 59 | 60 | self.transform = IfNoneReturnDefault([])(transform) 61 | 62 | self.transform = TransformSequence( 63 | *self.transform, auto_gpu=auto_gpu) 64 | 65 | def __len__(self): 66 | return len(self.file_paths) 67 | 68 | def __getitem__(self, idx): 69 | self.idx = idx 70 | if _torch.is_tensor(idx): 71 | idx = idx.to_list() 72 | file_path = self.file_paths[idx] 73 | file_dict = self.load_from_file(file_path) 74 | 75 | sample_data = self.grab_from_dict(file_dict) 76 | 77 | if self.transform: 78 | sample_data = self.transform(sample_data) 79 | 80 | # sample_data["filepath"] = [self.file_paths[idx], ] 81 | 82 | return sample_data 83 | 84 | def load_from_file(self, file_path): 85 | if _Path(file_path).suffix == ".npy": 86 | return _np.load(file_path, allow_pickle=True).item() 87 | if _Path(file_path).suffix == ".npz": 88 | return _np.load(file_path) 89 | 90 | def grab_from_dict(self, file_dict): 91 | data = OrderedDict() 92 | data[self.input_name] = self.parse_tensor(file_dict[self.input_name]) 93 | if self.target_name is not None: 94 | data[self.target_name] = \ 95 | self.parse_tensor(file_dict[self.target_name]) 96 | miscallaneous_keys = [ 97 | key for key in file_dict.keys() if 98 | key not in [self.input_name, self.target_name]] 99 | for key in miscallaneous_keys: 100 | data[key] = self.parse_tensor(file_dict[key]) 101 | return data 102 | 103 | def parse_tensor(self, tensor): 104 | 105 | tensor = _torch.Tensor(tensor) 106 | 107 | if self.RGB: 108 | if len(tensor.shape) == 3: 109 | return _torch.Tensor(tensor).permute(2, 0, 1) 110 | elif len(tensor.shape) == 2: 111 | return _torch.Tensor(tensor).unsqueeze(0) 112 | else: 113 | raise Exception 114 | else: 115 | if len(tensor.shape) == 2: 116 | return _torch.Tensor(tensor).unsqueeze(0) 117 | return tensor 118 | 119 | 120 | class DictDatasetVolume(DictDataset): 121 | 122 | def __init__( 123 | self, folder_or_list, auto_gpu=True, transform=None, suffix='*.npy', 124 | input_name=None, target_name=None, 125 | ): 126 | super().__init__(folder_or_list, auto_gpu=auto_gpu, transform=transform, suffix=suffix, 127 | input_name=input_name, target_name=target_name, RGB=False) 128 | 129 | def parse_tensor(self, tensor): 130 | 131 | tensor = _torch.Tensor(tensor) 132 | 133 | if len(tensor.shape) == 3: 134 | return _torch.Tensor(tensor).unsqueeze(0) 135 | 136 | return tensor 137 | 138 | 139 | class GraphDataset(DictDataset): 140 | def parse_tensor(self, tensor): 141 | return _torch.tensor(tensor) 142 | 143 | if __name__ == "__main__": 144 | fake_files = ['1', '2'] 145 | dataset_1 = DictDataset(fake_files, transform=[1, 2, 3]) 146 | dataset_2 = DictDataset(fake_files, transform=[4, 5, 6]) 147 | print('done') 148 | -------------------------------------------------------------------------------- /myTorch/Data/Exploration/__init__.py: -------------------------------------------------------------------------------- 1 | from .moving_averages import ( 2 | MADict, MADictMulti, MovingAverage 3 | ) 4 | 5 | __all__ = [ 6 | 'MADict', 'MADictMulti', 'MovingAverage' 7 | ] 8 | -------------------------------------------------------------------------------- /myTorch/Data/Exploration/moving_averages.py: -------------------------------------------------------------------------------- 1 | try: 2 | from myTorch.Utils import load_numpy_item 3 | except ImportError: 4 | from ...Utils import load_numpy_item 5 | from pathlib import Path 6 | import matplotlib.pyplot as plt 7 | from matplotlib import rc 8 | from matplotlib.ticker import MultipleLocator 9 | rc('font', **{'family': 'serif', 'serif': ['Palatino']}) 10 | rc('text', usetex=True) 11 | 12 | 13 | def _path_or_string(obj): 14 | if isinstance(obj, Path): 15 | return True 16 | if isinstance(obj, str): 17 | return True 18 | return False 19 | 20 | 21 | _x_axis_keys = ['epoch', 'global_step'] 22 | 23 | 24 | class MovingAverage: 25 | title_defaults = dict( 26 | fontsize=10 27 | ) 28 | 29 | def __init__(self, data, ma, x_data=None): 30 | self.ma = ma 31 | self.raw_data = data 32 | self.x_data = x_data if x_data is not None else list(range(len(data))) 33 | self.ma_data = [] 34 | 35 | for i in range(len(data)): 36 | if i < ma: 37 | self.ma_data.append( 38 | sum(data[0:(i+1)])/(i+1) 39 | ) 40 | else: 41 | self.ma_data.append( 42 | sum(data[(i-ma):i])/ma 43 | ) 44 | 45 | def plot_raw(self, *args, **kwargs): 46 | plt.plot(self.x_data, self.raw_data, *args, **kwargs) 47 | self.ax = plt.gca() 48 | self.f = plt.gcf() 49 | return self 50 | 51 | def plot_ma(self, *args, **kwargs): 52 | plt.plot(self.x_data, self.ma_data, *args, **kwargs) 53 | self.ax = plt.gca() 54 | return self 55 | 56 | def log_yscale(self, **kwargs): 57 | plt.yscale('log', **kwargs) 58 | return self 59 | 60 | def set_title(self, title, *args, **kwargs): 61 | self.title_defaults.update(kwargs) 62 | if title is not None: 63 | self.ax.set_title(title, *args, **self.title_defaults) 64 | return self 65 | 66 | def minor_ticks(self, n_ticks=5): 67 | # self.ax.xaxis.set_minor_locator(MultipleLocator(n_ticks)) 68 | # self.ax.yaxis.set_minor_locator(MultipleLocator(n_ticks)) 69 | self.ax.minorticks_on() 70 | 71 | def remove_ticks(self): 72 | self.ax.set_xticks([]) 73 | self.ax.set_yticks([]) 74 | return self 75 | 76 | 77 | class MADict(MovingAverage): 78 | label_defaults = dict( 79 | fontsize=8 80 | ) 81 | 82 | def __init__(self, dictionary, ma, *, x_key=None, y_key): 83 | if _path_or_string(dictionary): 84 | dictionary = load_numpy_item(dictionary) 85 | assert isinstance(dictionary, dict), \ 86 | f'first argument must be of type {dict}' 87 | x_data = dictionary[x_key] if x_key is not None else None 88 | self.y_key = y_key 89 | self.x_key = x_key 90 | super(MADict, self).__init__( 91 | dictionary[y_key], ma, x_data=x_data 92 | ) 93 | 94 | def plot_raw(self, *args, **kwargs): 95 | if self.y_key is not None: 96 | kwargs['label'] = kwargs.get('label', self.y_key) 97 | return super(MADict, self).plot_raw( 98 | *args, **kwargs 99 | ) 100 | 101 | def plot_ma(self, *args, **kwargs): 102 | if self.y_key is not None: 103 | kwargs['label'] = \ 104 | kwargs.get('label', '%s ma=%d' % (self.y_key, self.ma)) 105 | return super(MADict, self).plot_ma( 106 | *args, **kwargs 107 | ) 108 | 109 | def label_axes(self, *args, **kwargs): 110 | self.label_defaults.update(kwargs) 111 | x_key = self.x_key 112 | y_key = self.y_key 113 | if len(args) == 1: 114 | x_key = args[0] 115 | if len(args) == 2: 116 | x_key = args[0] 117 | y_key = args[1] 118 | self.ax.set_xlabel(x_key, **self.label_defaults) 119 | self.ax.set_ylabel(y_key, **self.label_defaults) 120 | return self 121 | 122 | def set_title(self, *args, **kwargs): 123 | 124 | if len(args) == 0: 125 | return super(MADict, self).set_title( 126 | f'{self.x_key} vs {self.y_key}', 127 | *args, **kwargs) 128 | else: 129 | return super(MADict, self).set_title( 130 | *args, **kwargs) 131 | 132 | def ez_plot(self): 133 | self.plot_ma() 134 | self.plot_raw(alpha=0.3) 135 | self.label_axes() 136 | self.set_title() 137 | 138 | 139 | class MADictMulti: 140 | def __init__(self, data_dictionary, ma, *keys, x_key=None): 141 | self.keys = keys 142 | for key in keys: 143 | setattr( 144 | self, key, MADict( 145 | data_dictionary, ma, x_key=x_key, y_key=key) 146 | ) 147 | 148 | def plot_ma_all(self, *args, **kwargs): 149 | for key in self.keys: 150 | plotter = getattr(self, key) 151 | plotter.plot_ma( 152 | *args, **kwargs) 153 | 154 | def plot_raw_all(self, *args, **kwargs): 155 | for key in self.keys: 156 | plotter = getattr(self, key) 157 | plotter.plot_raw( 158 | *args, **kwargs) -------------------------------------------------------------------------------- /myTorch/Data/Transforms/ToGPU.py: -------------------------------------------------------------------------------- 1 | import torch as _torch 2 | 3 | 4 | class ToGPU: 5 | device = "cuda:0" 6 | 7 | def __call__(self, sample_dict): 8 | self.device = _torch.device( 9 | self.device if _torch.cuda.is_available else "cpu" 10 | ) 11 | 12 | for key in sample_dict.keys(): 13 | try: 14 | sample_dict[key] = sample_dict[key].to(self.device) 15 | except ValueError: 16 | print(f'{key} of {sample_dict[key].__call__} could not be placed on GPU') 17 | 18 | return sample_dict 19 | -------------------------------------------------------------------------------- /myTorch/Data/Transforms/TransformSequence.py: -------------------------------------------------------------------------------- 1 | from .ToGPU import ToGPU 2 | 3 | 4 | class TransformSequence: 5 | 6 | def __init__(self, *args, auto_gpu=True): 7 | self.transform_list = [] 8 | self.transform_list.extend(args) 9 | if auto_gpu: 10 | self.transform_list.extend([ToGPU()]) 11 | 12 | def __len__(self): 13 | return len(self.transform_list) 14 | 15 | def __call__(self, sample_list): 16 | for transform in self.transform_list: 17 | sample_list = transform(sample_list) 18 | assert sample_list is not None 19 | return sample_list 20 | -------------------------------------------------------------------------------- /myTorch/Data/Transforms/WithKeys.py: -------------------------------------------------------------------------------- 1 | import numpy as _np 2 | from ... import _callback_decriptor 3 | import torch 4 | 5 | class FromKeys: 6 | """ Methods that modifies a dictionary based on its keys 7 | """ 8 | pre_transform = _callback_decriptor('pre_transform') 9 | post_transform = _callback_decriptor('post_transform') 10 | 11 | def __init__(self, *args, **kwargs): 12 | """__init__ 13 | 14 | Transform from a set of keys. 15 | 16 | """ 17 | self.callback_list = kwargs.get('callbacks', []) 18 | 19 | def forward(self, sample_dict): 20 | NotImplementedError 21 | 22 | def __call__(self, sample_dict): 23 | self.pre_transform 24 | self.sample_dict = self.forward(sample_dict) 25 | self.post_transform 26 | return self.sample_dict 27 | 28 | 29 | class RollTensor(FromKeys): 30 | 31 | def forward(self, sample_dict): 32 | key = next(iter(sample_dict.keys())) 33 | c, h, w = sample_dict[key].shape 34 | roll_val = _np.random.randint(h) 35 | for key in sample_dict.keys(): 36 | sample_dict[key] = sample_dict[key].roll(roll_val, 1) 37 | return sample_dict 38 | 39 | 40 | class FlipTensor(FromKeys): 41 | 42 | def forward(self, sample_dict): 43 | key = next(iter(sample_dict.keys())) 44 | 45 | for key in sample_dict.keys(): 46 | if _np.random.randint(0,2): 47 | flip_dim = _np.random.randint(1,3) 48 | sample_dict[key] = torch.flip( 49 | sample_dict[key], [flip_dim] 50 | ) 51 | return sample_dict 52 | 53 | 54 | class ToDtype(FromKeys): 55 | def __init__(self, dtype, *args, **kwargs): 56 | super(ToDtype, self).__init__(*args, **kwargs) 57 | self.dtype = dtype 58 | 59 | def forward(self, sample_dict): 60 | for key in sample_dict.keys(): 61 | try: 62 | sample_dict[key] = sample_dict[key].type(self.dtype) 63 | except TypeError: 64 | print(f'could not convert {sample_dict[key]} to {self.dtype}') 65 | 66 | return sample_dict 67 | -------------------------------------------------------------------------------- /myTorch/Data/Transforms/__init__.py: -------------------------------------------------------------------------------- 1 | from .ToGPU import ToGPU 2 | from .TransformSequence import TransformSequence 3 | from .WithKeys import ( 4 | FromKeys, RollTensor, ToDtype, FlipTensor 5 | ) 6 | 7 | __all__ = [ 8 | 'ToGPU', 'TransformSequence', 'FromKeys', 'RollTensor', 9 | 'ToDtype', 'FlipTensor' 10 | ] -------------------------------------------------------------------------------- /myTorch/Data/__init__.py: -------------------------------------------------------------------------------- 1 | from .DictDataset import DictDataset, DictDatasetVolume 2 | from . import Transforms 3 | from . import DataLoaders 4 | from . import Exploration 5 | 6 | __all__ = [ 7 | 'DictDataset', 'DataLoaders', 'Transforms', 'Exploration' 8 | ] 9 | -------------------------------------------------------------------------------- /myTorch/Loss/__init__.py: -------------------------------------------------------------------------------- 1 | import warnings 2 | import torch 3 | 4 | 5 | def _relative_difference(input, target): 6 | numerator = 2.0 * torch.abs(input - target) 7 | denominator = input + target 8 | return numerator/denominator 9 | 10 | 11 | def _relative_percentage_difference(input, target): 12 | return _relative_difference(input, target) * 100 13 | 14 | 15 | def _accuracy(input, target, eps=1e-7): 16 | error = torch.clamp( 17 | torch.abs(target-input)/torch.abs(target+eps), 18 | 0,1 19 | ) 20 | return (1-error) * 100 21 | 22 | import numpy as np 23 | 24 | def _incompressibleConstantContinuity3d(pred, truth, domain): 25 | # for the internal field calcualte the distances 26 | dx = domain[:,0,1:-1,2:,1:-1] - domain[:,0,1:-1,1:-1,1:-1] + 1e-7 27 | dy = domain[:,1,2:,1:-1,1:-1] - domain[:,1,1:-1,1:-1,1:-1] + 1e-7 28 | dz = domain[:,2,1:-1,1:-1,2:] - domain[:,2,1:-1,1:-1,1:-1] + 1e-7 29 | # now solve the derivative 30 | dudx = (pred[:,0,1:-1,2:,1:-1] - pred[:,0,1:-1,1:-1,1:-1])/dx 31 | dudy = (pred[:,1,2:,1:-1,1:-1] - pred[:,1,1:-1,1:-1,1:-1])/dy 32 | dudz = (pred[:,2,1:-1,1:-1,2:] - pred[:,2,1:-1,1:-1,1:-1])/dz 33 | return torch.abs(dudx + dudy + dudz + 1e-5) 34 | 35 | def _noSlip3d(pred, truth, domain): 36 | 37 | lr = torch.abs(pred[:,:,0,:,:]) + torch.abs(pred[:,:,-1,:,:]) 38 | tb = torch.abs(pred[:,:,:,0,:]) + torch.abs(pred[:,:,:,-1,:]) 39 | 40 | return lr.mean() + tb.mean() 41 | 42 | 43 | class _loss_template: 44 | 45 | def __init__(self, function): 46 | self.function = function 47 | 48 | def __call__(self, input, target, reduction='sum'): 49 | if not(target.size() == input.size()): 50 | warnings.warn( 51 | "Using a target size ({}) that is different to the input size ({}). " 52 | "This will likely lead to incorrect results due to broadcasting. " 53 | "Please ensure they have the same size.".format( 54 | target.size(), input.size()), 55 | stacklevel=2) 56 | 57 | ret = self.function(input, target) 58 | if reduction is not None: 59 | ret = torch.mean(ret) if reduction == 'mean' else torch.sum(ret) 60 | 61 | return ret 62 | 63 | class _loss_templateCFD: 64 | 65 | def __init__(self, function): 66 | self.function = function 67 | 68 | def __call__(self, input, target, domain, reduction='sum'): 69 | if not(target.size() == input.size()): 70 | warnings.warn( 71 | "Using a target size ({}) that is different to the input size ({}). " 72 | "This will likely lead to incorrect results due to broadcasting. " 73 | "Please ensure they have the same size.".format( 74 | target.size(), input.size()), 75 | stacklevel=2) 76 | 77 | ret = self.function(input, target, domain) 78 | if reduction is not None: 79 | ret = torch.mean(ret) if reduction == 'mean' else torch.sum(ret) 80 | 81 | return ret 82 | 83 | 84 | 85 | relative_difference = _loss_template(_relative_difference) 86 | relative_percentage_difference = _loss_template( 87 | _relative_percentage_difference) 88 | accuracy = _loss_template(_accuracy) 89 | incompressibleConstantContinuity3d = _loss_templateCFD(_incompressibleConstantContinuity3d) 90 | noSlip3d = _loss_templateCFD(_noSlip3d) 91 | 92 | class _Loss(torch.nn.Module): 93 | 94 | def __init__(self, size_average=None, reduce=None, reduction='sum'): 95 | super(_Loss, self).__init__() 96 | self.reduction = reduction 97 | 98 | 99 | class RelativeDifference(_Loss): 100 | 101 | def forward(self, input, target): 102 | return relative_difference(input, target, reduction=self.reduction) 103 | 104 | 105 | class RelativePercentageDifference(RelativeDifference): 106 | 107 | def forward(self, input, target): 108 | return super(RelativePercentageDifference, self).forward( 109 | input, target 110 | ) 111 | 112 | 113 | class Accuracy(_Loss): 114 | def forward(self, input, target): 115 | return accuracy(input, target, reduction=self.reduction) 116 | 117 | 118 | class ContinuityNoSlip(_Loss): 119 | def forward(self, input, target, domain): 120 | l1 = incompressibleConstantContinuity3d(input, target, domain, reduction='mean') 121 | l2 = noSlip3d(input, target, domain, reduction='mean') 122 | l3 = torch.nn.functional.l1_loss( 123 | input, target, reduction=self.reduction 124 | ) 125 | return l1 + l2 + l3 126 | 127 | 128 | __all__ = [ 129 | 'relative_difference', 'relative_percentage_difference', 'accuracy', 130 | 'RelativeDifference', 'RelativePercentageDifference', 'Accuracy' 131 | ] -------------------------------------------------------------------------------- /myTorch/Models/DilatedNet.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Chr1sC0de/DeepLearningForWallShearStressPredictionAndImageSegmentation/dbb5e6a58b0ecfdb4ed3b05e5ca1841a321bd11b/myTorch/Models/DilatedNet.py -------------------------------------------------------------------------------- /myTorch/Models/EncoderBottleNeckHead.py: -------------------------------------------------------------------------------- 1 | try: 2 | from myTorch import nn as _nn 3 | except ImportError: 4 | from .. import nn as _nn 5 | import torch as _torch 6 | 7 | 8 | class _NameSetter: 9 | 10 | def __init__(self, name): 11 | self.name = name 12 | 13 | def __get__(self, obj, obj_type): 14 | return obj._level_prefix % (obj.i, self.name) 15 | 16 | 17 | class EncodeDecodeBase(_nn.Layer): 18 | 19 | decoder_constructor = _nn.VGGBlock 20 | upsample_constructor = _nn.UpsampleConv2d 21 | 22 | upsample_name = _NameSetter('upsample') 23 | concat_name = _NameSetter('concat') 24 | conv_name = _NameSetter('convolve') 25 | 26 | _level_prefix = 'level_%d_%s' 27 | 28 | output_activation = None 29 | 30 | def __init__(self, encoder, output_classes, kernel_size, bottleneck=None): 31 | super(EncodeDecodeBase, self).__init__() 32 | 33 | self.kernel_size = kernel_size 34 | 35 | self.encoder_callback = encoder 36 | self.encoder_callback.track_skip = True 37 | 38 | self.assign_bottleneck(bottleneck) 39 | 40 | up_channels = self.bottleneck.out_channels 41 | 42 | final_layer = self.construct_top_down_path(up_channels) 43 | 44 | self.pw_conv = _torch.nn.Conv2d( 45 | final_layer.out_channels, output_classes, 1, bias=False) 46 | 47 | def main_forward(self, x): 48 | x = self.encoder_callback(x) 49 | x = self.bottleneck_connector(x) 50 | x = self.bottleneck(x) 51 | x = self.decoder_forward(x) 52 | x = self.pw_conv(x) 53 | if self.output_activation is not None: 54 | x = self.output_activation(x) 55 | return x 56 | 57 | def iterable_layers(self): 58 | """iterable_layers 59 | 60 | Given the current state of the model and encoder extract the 61 | concatenable layers. 62 | """ 63 | return enumerate(reversed(self.encoder_callback.concatenable_layers)) 64 | 65 | def assign_bottleneck(self, bottleneck): 66 | if hasattr(self.encoder_callback, 'connector_constructor'): 67 | self.bottleneck_connector = \ 68 | self.encoder_callback.connector_constructor( 69 | self.encoder_callback) 70 | 71 | else: 72 | self.bottleneck_connector = self.bottle_neck_connector( 73 | self.encoder_callback.out_channels, 2) 74 | if bottleneck is None: 75 | self.bottleneck = \ 76 | self.encoder_callback.layer_constructor( 77 | self.bottleneck_connector, 78 | self.encoder_callback.out_channels*2, 79 | self.kernel_size, 80 | stride=1) 81 | else: 82 | self.bottleneck = bottleneck( 83 | self.bottleneck_connector, 84 | self.encoder_callback.out_channels*2, 85 | self.kernel_size, 86 | stride=1) 87 | @property 88 | def skip_layers(self): 89 | return self.encoder_callback.skip_layers 90 | -------------------------------------------------------------------------------- /myTorch/Models/FCN_Eric.py: -------------------------------------------------------------------------------- 1 | import torch as _torch 2 | import torch.nn as nn 3 | 4 | ''' 5 | https://github.com/pochih/FCN-pytorch/blob/master/python/fcn.py 6 | ''' 7 | 8 | class ConvNormActBlock(nn.Module): 9 | 10 | def __init__(self, in_channels, out_channels, kernel_size =3, 11 | stride = 1, padding = 1, dilation = 1, groups = 1, bias=False, 12 | padding_mode = 'zeros', preactivate = False): 13 | 14 | super(ConvNormActBlock, self).__init__() 15 | 16 | self.preactivate = preactivate 17 | 18 | self.conv_layer = nn.Conv2d( 19 | in_channels,out_channels,kernel_size, 20 | stride=stride,padding=padding, dilation=dilation, 21 | groups=groups, bias=bias, padding_mode=padding_mode 22 | ) 23 | 24 | if preactivate: 25 | norm_layer_channels = in_channels 26 | else: 27 | norm_layer_channels = out_channels 28 | 29 | self.norm_layer = nn.InstanceNorm2d( 30 | norm_layer_channels,eps=1e-05, momentum=0.1,affine=False, track_running_stats=False 31 | ) 32 | 33 | self.activation_layer = nn.LeakyReLU(negative_slope=0.01, inplace=False) 34 | 35 | def pre_activate_forward(self,x): 36 | x = self.activation_layer(x) 37 | x = self.norm_layer(x) 38 | x = self.conv_layer(x) 39 | return x 40 | 41 | def classic_forward(self,x): 42 | x = self.conv_layer(x) 43 | x = self.norm_layer(x) 44 | x = self.activation_layer(x) 45 | return x 46 | 47 | def forward(self,x): 48 | if self.preactivate: 49 | x = self.pre_activate_forward(x) 50 | else: 51 | x = self.classic_forward(x) 52 | return x 53 | 54 | class VGGBlock(nn.Module): 55 | 56 | def __init__(self, in_channels, out_channels, kernel_size = 3, **kwargs): 57 | super(VGGBlock, self).__init__() 58 | self.conv_layer_1 = ConvNormActBlock( 59 | in_channels, out_channels, kernel_size=kernel_size, **kwargs) 60 | self.conv_layer_2 = ConvNormActBlock( 61 | out_channels, out_channels, kernel_size=kernel_size, **kwargs) 62 | 63 | def forward(self, x): 64 | x = self.conv_layer_1(x) 65 | x = self.conv_layer_2(x) 66 | return x 67 | 68 | class FCNs(nn.Module): 69 | 70 | def __init__(self, pretrained_net, n_class): 71 | super().__init__() 72 | self.n_class = n_class 73 | self.pretrained_net = pretrained_net 74 | self.relu = nn.ReLU(inplace=True) 75 | self.deconv1 = nn.ConvTranspose2d(512, 512, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) 76 | self.bn1 = nn.BatchNorm2d(512) 77 | self.deconv2 = nn.ConvTranspose2d(512, 256, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) 78 | self.bn2 = nn.BatchNorm2d(256) 79 | self.deconv3 = nn.ConvTranspose2d(256, 128, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) 80 | self.bn3 = nn.BatchNorm2d(128) 81 | self.deconv4 = nn.ConvTranspose2d(128, 64, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) 82 | self.bn4 = nn.BatchNorm2d(64) 83 | self.deconv5 = nn.ConvTranspose2d(64, 32, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) 84 | self.bn5 = nn.BatchNorm2d(32) 85 | self.classifier = nn.Conv2d(32, n_class, kernel_size=1) 86 | 87 | def forward(self, x): 88 | output = self.pretrained_net(x) 89 | x5 = output['x5'] # size=(N, 512, x.H/32, x.W/32) 90 | x4 = output['x4'] # size=(N, 512, x.H/16, x.W/16) 91 | x3 = output['x3'] # size=(N, 256, x.H/8, x.W/8) 92 | x2 = output['x2'] # size=(N, 128, x.H/4, x.W/4) 93 | x1 = output['x1'] # size=(N, 64, x.H/2, x.W/2) 94 | 95 | score = self.bn1(self.relu(self.deconv1(x5))) # size=(N, 512, x.H/16, x.W/16) 96 | score = score + x4 # element-wise add, size=(N, 512, x.H/16, x.W/16) 97 | score = self.bn2(self.relu(self.deconv2(score))) # size=(N, 256, x.H/8, x.W/8) 98 | score = score + x3 # element-wise add, size=(N, 256, x.H/8, x.W/8) 99 | score = self.bn3(self.relu(self.deconv3(score))) # size=(N, 128, x.H/4, x.W/4) 100 | score = score + x2 # element-wise add, size=(N, 128, x.H/4, x.W/4) 101 | score = self.bn4(self.relu(self.deconv4(score))) # size=(N, 64, x.H/2, x.W/2) 102 | score = score + x1 # element-wise add, size=(N, 64, x.H/2, x.W/2) 103 | score = self.bn5(self.relu(self.deconv5(score))) # size=(N, 32, x.H, x.W) 104 | score = self.classifier(score) # size=(N, n_class, x.H/1, x.W/1) 105 | 106 | return score # size=(N, n_class, x.H/1, x.W/1) -------------------------------------------------------------------------------- /myTorch/Models/LinkNet.py: -------------------------------------------------------------------------------- 1 | import torch.nn as _nn 2 | from myTorch.Models import Utils as _Utils 3 | import torch 4 | """ 5 | https://github.com/meetshah1995/pytorch-semseg/blob/master/ptsemseg/models/linknet.py 6 | https://github.com/e-lab/pytorch-linknet 7 | https://arxiv.org/pdf/1707.03718.pdf 8 | 9 | """ 10 | 11 | 12 | class LinkNet(_nn.Module): 13 | 14 | def __init__( 15 | self, n_classes=1, in_channels=4, filters=[64,128,256,512], 16 | output_activation=_nn.LeakyReLU(inplace=True) 17 | ): 18 | super(LinkNet, self).__init__() 19 | self.conv1 = _nn.Conv2d(in_channels, filters[0], kernel_size=7,stride=1,padding=3, bias=False) 20 | self.bn1 = _nn.BatchNorm2d(filters[0]) 21 | self.relu = _nn.ReLU(inplace=True) 22 | self.maxpool = _nn.MaxPool2d(kernel_size=3,stride=2,padding=1) 23 | 24 | self.encoder1 = _Utils.Encoder(filters[0], filters[0], kernel_size=3,stride=1,padding=1) 25 | self.encoder2 = _Utils.Encoder(filters[0], filters[1], kernel_size=3,stride=2,padding=1) 26 | self.encoder3 = _Utils.Encoder(filters[1], filters[2], kernel_size=3, stride=2, padding=1) 27 | self.encoder4 = _Utils.Encoder(filters[2], filters[3], kernel_size=3, stride=2, padding=1) 28 | 29 | self.decoder1 = _Utils.Decoder(filters[0], filters[0], kernel_size=3, stride=1, padding=1, output_padding=0) 30 | self.decoder2 = _Utils.Decoder(filters[1], filters[0], kernel_size=3, stride=2, padding=1, output_padding=1) 31 | self.decoder3 = _Utils.Decoder(filters[2], filters[1], kernel_size=3, stride=2, padding=1, output_padding=1) 32 | self.decoder4 = _Utils.Decoder(filters[3], filters[2], kernel_size=3, stride=2, padding=1, output_padding=1) 33 | 34 | # Classifier 35 | self.tp_conv1 = _nn.Sequential(_nn.ConvTranspose2d(filters[0], filters[0]//2, kernel_size=3, stride=2, padding=1, output_padding=1), 36 | _nn.BatchNorm2d(filters[0]//2), 37 | _nn.ReLU(inplace=True),) 38 | self.conv2 = _nn.Sequential(_nn.Conv2d(filters[0]//2, filters[0]//2, kernel_size=3, stride=2, padding=1), 39 | _nn.BatchNorm2d(filters[0]//2), 40 | _nn.ReLU(inplace=True),) 41 | self.tp_conv2 = _nn.ConvTranspose2d(filters[0]//2, n_classes, kernel_size=2, stride=2, padding=0) 42 | self.output_activation = output_activation 43 | 44 | def forward(self, x): 45 | # Initial block 46 | x = self.conv1(x) 47 | x = self.bn1(x) 48 | x = self.relu(x) 49 | x = self.maxpool(x) 50 | 51 | # Encoder blocks 52 | e1 = self.encoder1(x) 53 | e2 = self.encoder2(e1) 54 | e3 = self.encoder3(e2) 55 | e4 = self.encoder4(e3) 56 | 57 | # Decoder blocks 58 | d4 = e3 + self.decoder4(e4) 59 | d3 = e2 + self.decoder3(d4) 60 | d2 = e1 + self.decoder2(d3) 61 | d1 = x + self.decoder1(d2) 62 | 63 | # Classifier 64 | y = self.tp_conv1(d1) 65 | y = self.conv2(y) 66 | y = self.tp_conv2(y) 67 | 68 | if self.output_activation is not None: 69 | y = self.output_activation(y) 70 | 71 | return y 72 | 73 | if __name__ == "__main__": 74 | import torch as _torch 75 | from torch.utils.tensorboard import SummaryWriter 76 | writer = SummaryWriter() 77 | 78 | input_example = torch.randn(1,3,512,512) 79 | input_example = torch.tensor(input_example,dtype = torch.float32) 80 | 81 | model = LinkNet() 82 | output = model(input_example) 83 | 84 | writer.add_graph(output, input_example) 85 | writer.close() 86 | 87 | print('done') -------------------------------------------------------------------------------- /myTorch/Models/PSPnet.py: -------------------------------------------------------------------------------- 1 | ''' 2 | https://github.com/kazuto1011/pspnet-pytorch/blob/master/libs/models/pspnet.py 3 | https://github.com/Lextal/pspnet-pytorch/blob/master/pspnet.py 4 | ''' 5 | 6 | 7 | import torch 8 | from torch import nn 9 | from torch.nn import functional as F 10 | import myTorch as mt 11 | import myTorch.nn.Wrapped 12 | from myTorch.Models import ResNetModel 13 | from torch.utils.tensorboard import SummaryWriter 14 | 15 | 16 | class PSPModule(nn.Module): 17 | def __init__(self, features, out_features=1024, sizes=(1, 2, 3, 6)): 18 | super().__init__() 19 | self.stages = [] 20 | self.stages = nn.ModuleList([self._make_stage(features, size) for size in sizes]) 21 | self.bottleneck = nn.Conv2d(features * (len(sizes) + 1), out_features, kernel_size=1) 22 | self.relu = nn.ReLU() 23 | 24 | def _make_stage(self, features, size): 25 | prior = nn.AdaptiveAvgPool2d(output_size=(size, size)) 26 | conv = nn.Conv2d(features, features, kernel_size=1, bias=False) 27 | return nn.Sequential(prior, conv) 28 | 29 | def forward(self, feats): 30 | h, w = feats.size(2), feats.size(3) 31 | priors = [F.upsample(input=stage(feats), size=(h, w), mode='bilinear') for stage in self.stages] + [feats] 32 | bottle = self.bottleneck(torch.cat(priors, 1)) 33 | return self.relu(bottle) 34 | 35 | 36 | class PSPUpsample(nn.Module): 37 | def __init__(self, in_channels, out_channels): 38 | super().__init__() 39 | self.conv = nn.Sequential( 40 | nn.Conv2d(in_channels, out_channels, 3, padding=1), 41 | nn.BatchNorm2d(out_channels), 42 | nn.ReLU() 43 | ) 44 | 45 | def forward(self, x): 46 | h, w = 2 * x.size(2), 2 * x.size(3) 47 | p = F.upsample(input=x, size=(h, w), mode='bilinear', align_corners=True) 48 | return self.conv(p) 49 | 50 | 51 | class PSPNet(nn.Module): 52 | def __init__(self, n_classes=3, sizes=(1, 2, 3, 6), psp_size=2048, deep_features_size=1024, backend='resnet34', 53 | pretrained=True): 54 | super().__init__() 55 | self.feats = getattr(ResNetModel, backend)(pretrained) 56 | self.psp = PSPModule(psp_size, 1024, sizes) 57 | # self.drop_1 = nn.Dropout2d(p=0.3) 58 | 59 | self.up_1 = PSPUpsample(1024, 256) 60 | self.up_2 = PSPUpsample(256, 64) 61 | self.up_3 = PSPUpsample(64, 64) 62 | 63 | # self.drop_2 = nn.Dropout2d(p=0.15) 64 | self.final = nn.Sequential( 65 | nn.Conv2d(64, n_classes, kernel_size=1), 66 | # nn.LogSoftmax() 67 | ) 68 | 69 | # self.classifier = nn.Sequential( 70 | # nn.Linear(deep_features_size, 256), 71 | # nn.ReLU(), 72 | # nn.Linear(256, n_classes) 73 | # ) 74 | 75 | def forward(self, x): 76 | f, class_f = self.feats(x) 77 | p = self.psp(f) 78 | # p = self.drop_1(p) 79 | 80 | p = self.up_1(p) 81 | # p = self.drop_2(p) 82 | 83 | p = self.up_2(p) 84 | # p = self.drop_2(p) 85 | 86 | p = self.up_3(p) 87 | # p = self.drop_2(p) 88 | 89 | # auxiliary = F.adaptive_max_pool2d(input=class_f, output_size=(1, 1)).view(-1, class_f.size(1)) 90 | 91 | output = self.final(p) 92 | 93 | return output 94 | 95 | if __name__ == "__main__": 96 | writer = SummaryWriter() 97 | 98 | input_example = torch.randn(1,3,512,512) 99 | input_example = torch.tensor(input_example,dtype = torch.float32) 100 | 101 | model = PSPNet() 102 | 103 | x = model(input_example) 104 | 105 | writer.add_image('input', input_example[0]) 106 | writer.add_image('output', x[0]) 107 | writer.add_graph(model, input_example) 108 | writer.close() 109 | 110 | print('done') -------------------------------------------------------------------------------- /myTorch/Models/Utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Utility functions for models 3 | 4 | """ 5 | 6 | import torch 7 | import torch.nn as _nn 8 | import numpy as numpy 9 | import torch.nn.functional as F 10 | 11 | class ResNetBlock(_nn.Module): 12 | 13 | def __init__(self, in_planes, out_planes, kernel_size, stride=1, padding=0, groups=1, bias=False): 14 | super(ResNetBlock, self).__init__() 15 | self.conv1 = _nn.Conv2d(in_planes, out_planes, kernel_size, stride, padding, groups=groups, bias=bias) 16 | self.bn1 = _nn.BatchNorm2d(out_planes) 17 | self.relu = _nn.ReLU(inplace=True) 18 | self.conv2 = _nn.Conv2d(out_planes, out_planes, kernel_size, 1, padding, groups=groups, bias=bias) 19 | self.bn2 = _nn.BatchNorm2d(out_planes) 20 | self.downsample = None 21 | if stride > 1: 22 | self.downsample = _nn.Sequential(_nn.Conv2d(in_planes, out_planes, kernel_size=1, stride=stride, bias=False), 23 | _nn.BatchNorm2d(out_planes),) 24 | 25 | def forward(self, x): 26 | 27 | residual = x 28 | 29 | out = self.conv1(x) 30 | out = self.bn1(out) 31 | out = self.relu(out) 32 | 33 | out = self.conv2(out) 34 | out = self.bn2(out) 35 | 36 | if self.downsample: 37 | residual = self.downsample(x) 38 | 39 | out += residual 40 | out = self.relu(out) 41 | 42 | return out 43 | class Encoder(_nn.Module): 44 | 45 | def __init__(self, in_planes, out_planes, kernel_size, stride=1, padding=0, groups=1, bias=False): 46 | super(Encoder, self).__init__() 47 | self.block1 = ResNetBlock(in_planes, out_planes, kernel_size, stride, padding, groups, bias) 48 | self.block2 = ResNetBlock(out_planes, out_planes, kernel_size, 1, padding, groups, bias) 49 | 50 | def forward(self, x): 51 | x = self.block1(x) 52 | x = self.block2(x) 53 | 54 | return x 55 | 56 | class Decoder(_nn.Module): 57 | def __init__(self, in_planes, out_planes, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=False): 58 | super(Decoder, self).__init__() 59 | self.conv1 = _nn.Sequential(_nn.Conv2d(in_planes, in_planes//4, 1, 1, 0, bias=bias), 60 | _nn.BatchNorm2d(in_planes//4), 61 | _nn.ReLU(inplace=True),) 62 | self.tp_conv = _nn.Sequential(_nn.ConvTranspose2d(in_planes//4, in_planes//4, kernel_size, stride, padding, output_padding, bias=bias), 63 | _nn.BatchNorm2d(in_planes//4), 64 | _nn.ReLU(inplace=True),) 65 | self.conv2 = _nn.Sequential(_nn.Conv2d(in_planes//4, out_planes, 1, 1, 0, bias=bias), 66 | _nn.BatchNorm2d(out_planes), 67 | _nn.ReLU(inplace=True),) 68 | 69 | def forward(self, x): 70 | x = self.conv1(x) 71 | x = self.tp_conv(x) 72 | x = self.conv2(x) 73 | 74 | return x 75 | 76 | -------------------------------------------------------------------------------- /myTorch/Models/__init__.py: -------------------------------------------------------------------------------- 1 | from .Merged import ( 2 | MergedFromPattern, MergedFromSequence, ResNet, VGGNet, 3 | ResNetBottleNeck, SEVGGNet, CSEVGGNet, CSESEVGGNet, VGGNet3D 4 | ) 5 | from .EncoderBottleNeckHead import EncodeDecodeBase 6 | 7 | from .unet import ( 8 | Unet, UnetBuilder, ResnetUnet, ResnetBottleneckUnet, Unet3d 9 | ) 10 | from .FPNNet import ( 11 | FPNBuilder, ResnetFPN, VGGFPN, SEVGGFPN, CSEVGGFPN, CSESEVGGFPN 12 | ) 13 | from .Utils import ( 14 | ResNetBlock, Encoder, Decoder 15 | ) 16 | from .LinkNet import ( 17 | LinkNet 18 | ) 19 | from .chris_linknet import ( 20 | ResNetLinkNet, rn34LinkNet, VGGLinkNet 21 | ) 22 | from .chris_pspnet import ( 23 | PSPNetBuilder, ResNetPSPNet, ResNetBottleNeckPSPNet, 24 | UPSPNet 25 | ) 26 | 27 | __all__ = [ 28 | 'MergedFromPattern', 'MergedFromSequence', 'ResNet', 'VGGNet', 29 | 'EncodeDecodeBase', 'Unet', 'ResNetBottleNeck', 'UnetBuilder', 30 | 'ResnetUnet', 'FPNBuilder', 'ResnetFPN', 'VGGFPN', 'ResNetBlock', 31 | 'Encoder', 'Decoder', 'LinkNet', 'PSPNet', 'ResnetBottleneckUnet', 32 | 'ResNetBottleNeckPSPNet', 'UPSPNet', 'rn34LinkNet', 'VGGLinkNet', 33 | 'VGGNet3D', 'Unet3d' 34 | ] -------------------------------------------------------------------------------- /myTorch/Models/chris_linknet.py: -------------------------------------------------------------------------------- 1 | from .EncoderBottleNeckHead import EncodeDecodeBase 2 | try: 3 | from myTorch import nn, Models 4 | except ImportError: 5 | from .. import nn, Models 6 | import torch 7 | 8 | 9 | class LinkNetBuilder(EncodeDecodeBase): 10 | decoder_constructor = nn.LinkNetDecoder 11 | upsample_constructor = torch.nn.UpsamplingBilinear2d 12 | 13 | def decoder_forward(self, x): 14 | iterable_layers = enumerate( 15 | reversed(self.skip_layers) 16 | ) 17 | for self.i, add_layer in iterable_layers: 18 | x = getattr(self, self.upsample_name)(x) 19 | x = getattr(self, self.conv_name)(x) 20 | x = x + add_layer 21 | 22 | return x 23 | 24 | def construct_top_down_path(self, in_channels): 25 | 26 | iterable_layers = list(self.iterable_layers()) 27 | 28 | self.number_of_scales = len(iterable_layers) 29 | 30 | for self.i, addable_layer in iterable_layers: 31 | up_layer = self.upsample_constructor(scale_factor=2) 32 | setattr(self, self.upsample_name, up_layer) 33 | conv_layer = self.decoder_constructor(in_channels, in_channels//2, 3) 34 | setattr(self, self.conv_name, conv_layer) 35 | in_channels = in_channels//2 36 | 37 | return conv_layer 38 | 39 | 40 | class ResNetLinkNet(LinkNetBuilder): 41 | output_activation = None 42 | 43 | def __init__( 44 | self, in_channels, num_classes, kernel_size=3, 45 | layers=[2, 1, 1, 1], 46 | base_channels=64, 47 | output_activation=None): 48 | """ Build a Resnet Unet model 49 | 50 | https://towardsdatascience.com/u-nets-with-resnet-encoders-and-cross-connections-d8ba94125a2c 51 | The model however uses instance normalization before each activation 52 | and uses Average pooling over max pooling. 53 | 54 | Args: 55 | in_channels ([int]) 56 | num_classes ([int]): the number of output channels 57 | kernel_size ([int])] 58 | Kwargs: 59 | base_channels ([int]): The total number 60 | of channels the networks is initialized with 61 | output_activation([_nn.Layer,_torch.nn.functional]): 62 | activation function of form f(x) 63 | """ 64 | encoder_layers = layers[:-1] 65 | encoder = Models.ResNet( 66 | in_channels, base_channels, kernel_size, 67 | layers=encoder_layers) 68 | n_bottleneck = layers[-1] 69 | 70 | class ResBottle(nn.RepeatedLayers): 71 | layer_constructor = encoder.layer_constructor 72 | _layer_dict = dict( 73 | stride=[[1, 1]] * n_bottleneck, 74 | ) 75 | 76 | super(ResNetLinkNet, self).__init__( 77 | encoder, num_classes, kernel_size, bottleneck=ResBottle) 78 | 79 | self.output_activation = output_activation 80 | 81 | 82 | def rn34LinkNet(in_channels, num_classes, kernel_size, **kwargs): 83 | return ResNetLinkNet( 84 | in_channels, num_classes, kernel_size=kernel_size, 85 | layers=[3, 3, 5, 2], **kwargs 86 | ) 87 | 88 | 89 | class VGGLinkNet(LinkNetBuilder): 90 | output_activation = None 91 | 92 | def __init__( 93 | self, in_channels, num_classes, kernel_size, n_pool=4, 94 | base_channels=64, output_activation=None): 95 | """__init__ Build a vanilla Unet model 96 | 97 | https://arxiv.org/abs/1505.04597 98 | The model however uses instance normalization before each activation 99 | and uses Average pooling over max pooling. 100 | 101 | Args: 102 | in_channels ([type]): [description] 103 | num_classes ([type]): [description] 104 | kernel_size ([type]): [description] 105 | n_pool (int, optional): [description]. Defaults to 4. 106 | base_channels (int, optional): [description]. Defaults to 64. 107 | output_activation ([type], optional): [description]. Defaults to None. 108 | """ 109 | layers = [1]*n_pool 110 | encoder = Models.VGGNet( 111 | in_channels, base_channels, kernel_size, 112 | layers=layers) 113 | super(VGGLinkNet, self).__init__(encoder, num_classes, kernel_size) 114 | if output_activation is not None: 115 | self.output_activation = output_activation 116 | -------------------------------------------------------------------------------- /myTorch/Models/chris_pspnet.py: -------------------------------------------------------------------------------- 1 | from .EncoderBottleNeckHead import EncodeDecodeBase 2 | try: 3 | from myTorch import nn, Models 4 | except ImportError: 5 | from .. import nn, Models 6 | import torch 7 | 8 | 9 | class PSPNetBuilder(nn.Layer): 10 | 11 | def __init__(self, in_channels, n_classes, kernel_size, layers=[3, 3, 5, 2], 12 | base_channels=64, output_activation=None,**kwargs): 13 | super(PSPNetBuilder, self).__init__() 14 | self.encoder = Models.ResNet( 15 | in_channels, base_channels, kernel_size, layers=layers, 16 | **kwargs) 17 | 18 | encoder_channels = self.encoder.out_channels 19 | 20 | self.connector = self.encoder.connector_constructor(self.encoder) 21 | 22 | self.bottleneck = nn.PyramidPool( 23 | encoder_channels, encoder_channels*2 24 | ) 25 | 26 | self.upsample = torch.nn.UpsamplingBilinear2d(scale_factor=2) 27 | 28 | self.upsample_layers = [] 29 | 30 | n_upsamples = len(layers) - 1 31 | 32 | in_channels = self.bottleneck.out_channels + self.encoder.out_channels 33 | 34 | for i in range(n_upsamples): 35 | self.upsample_layers.append( 36 | nn.UpsampleConv2d(in_channels, in_channels//2, kernel_size) 37 | ) 38 | setattr(self, 'upsample_%d' % i, self.upsample_layers[-1]) 39 | 40 | in_channels = in_channels//2 41 | 42 | self.pw_conv = torch.nn.Conv2d( 43 | self.upsample_layers[-1].out_channels, n_classes, 1, bias=False 44 | ) 45 | 46 | self.output_activation = output_activation 47 | 48 | self.out_channels = n_classes 49 | 50 | def main_forward(self, x): 51 | x_concat = self.encoder(x) 52 | x = self.connector(x_concat) 53 | x = self.bottleneck(x) 54 | x = self.upsample(x) 55 | x = torch.cat( 56 | [x, x_concat], dim=1 57 | ) 58 | for layer in self.upsample_layers: 59 | x = layer(x) 60 | 61 | x = self.pw_conv(x) 62 | if self.output_activation is not None: 63 | x = self.output_activation(x) 64 | return x 65 | 66 | class UPSPNet(PSPNetBuilder): 67 | feature_constructor = Models.VGGNet 68 | def __init__(self, *args, layers=[1, 1, 1, 1], **kwargs): 69 | super(UPSPNet, self).__init__( 70 | *args, layers=layers, **kwargs 71 | ) 72 | 73 | class ResNetPSPNet(PSPNetBuilder): 74 | feature_constructor = Models.ResNet 75 | 76 | class ResNetBottleNeckPSPNet(PSPNetBuilder): 77 | feature_constructor = Models.ResNetBottleNeck 78 | 79 | 80 | 81 | 82 | -------------------------------------------------------------------------------- /myTorch/Utils.py: -------------------------------------------------------------------------------- 1 | import pathlib 2 | import abc 3 | import copy 4 | import torch 5 | import json 6 | from . import nn 7 | import numpy as np 8 | 9 | 10 | class Cat(nn.Layer): 11 | 12 | def __init__(self, *args): 13 | # initialize concatenation layer from input layers 14 | super(Cat, self).__init__() 15 | 16 | self.in_channels = 0 17 | self.out_channels = 0 18 | 19 | if issubclass(args[0].__class__, torch.Tensor): 20 | self.__get_torch_channels(*args) 21 | else: 22 | self.__get_layer_channels(*args) 23 | 24 | self.in_channels = self.out_channels 25 | 26 | def forward(self, *args): 27 | return torch.cat(args, dim=1) 28 | 29 | def __get_torch_channels(self, *args): 30 | for arg in args: 31 | self.out_channels += args.shape[1] 32 | 33 | def __get_layer_channels(self, *args): 34 | for arg in args: 35 | self.out_channels += arg.out_channels 36 | 37 | @classmethod 38 | def from_layer(cls, layer, *args, **kwargs): 39 | raise "input layers directly into the constructor" 40 | 41 | 42 | def negative_to_positive_finder( 43 | data, window_scale=4, polyorder=3): 44 | 45 | pre_val = data[0] 46 | min_found = [] 47 | for i, val in enumerate(data[1:]): 48 | if all( 49 | [pre_val < 0, val > 0]): 50 | min_found.append(i) 51 | pre_val = val 52 | 53 | return min_found 54 | 55 | 56 | class Isinstance: 57 | def __init__(self, *args): 58 | self.instances = args 59 | 60 | def __call__(self, obj): 61 | for instance in self.instances: 62 | if isinstance(obj, instance): 63 | return True 64 | return False 65 | 66 | 67 | class GlobFiles: 68 | 69 | def __init__(self, pattern): 70 | self.pattern = pattern 71 | 72 | def __call__(self, folder): 73 | return list(pathlib.Path(folder).glob(self.pattern)) 74 | 75 | 76 | _glob_numpy = GlobFiles("*.npy") 77 | _glob_npz = GlobFiles("*.npz") 78 | 79 | 80 | def glob_numpy(folder): 81 | return list(_glob_numpy(folder)) 82 | 83 | def glob_npz(folder): 84 | return list(_glob_npz(folder)) 85 | 86 | 87 | class DefaultReturn: 88 | def __init__(self, default): 89 | self.default = default 90 | 91 | def __call__(self, value): 92 | if self.condition(value): 93 | return self.default 94 | else: 95 | return value 96 | 97 | @abc.abstractmethod 98 | def condition(self, value): 99 | pass 100 | 101 | 102 | class IfNoneReturnDefault(DefaultReturn): 103 | def condition(self, value): 104 | if value is None: 105 | return True 106 | return False 107 | 108 | 109 | class ToDevice: 110 | def __init__(self, device): 111 | self.device = device 112 | 113 | def __call__(self, *args): 114 | output_args = [] 115 | for arg in args: 116 | if isinstance(arg, (list, tuple)): 117 | output_args.append([item.to(self.device) for item in arg]) 118 | else: 119 | output_args.append(arg.to(self.device)) 120 | return output_args 121 | 122 | 123 | class StateCacher(object): 124 | 125 | def __init__(self, in_memory, cache_dir=None): 126 | self.in_memory = in_memory 127 | self.cache_dir = cache_dir 128 | 129 | if self.cache_dir is None: 130 | import tempfile 131 | self.cache_dir = tempfile.gettempdir() 132 | else: 133 | if not pathlib.Path(self.cache_dir).is_dir(): 134 | raise ValueError(f'{self.cache_dir} is not a valid directory.') 135 | else: 136 | self.cache_dir = pathlib.Path(self.cache_dir) 137 | 138 | self.cached = {} 139 | 140 | def store(self, key, state_dict): 141 | if self.in_memory: 142 | self.cached.update( 143 | {key: copy.deepcopy(state_dict)} 144 | ) 145 | else: 146 | fn = self.cache_dir/f"state_{key}_{id(self)}.pt" 147 | self.cached.update({key: fn}) 148 | torch.save(state_dict, fn) 149 | 150 | def retrieve(self, key): 151 | if key not in self.cached: 152 | raise KeyError(f"Target {key} was not in cached") 153 | 154 | if self.in_memory: 155 | return self.cached.get(key) 156 | else: 157 | fn = self.cached.get(key) 158 | if not pathlib.Path(fn).exists(): 159 | raise RuntimeError(f'Failed to load state in {fn}\ 160 | . File does not exists anymore') 161 | state_dict = torch.load( 162 | fn, map_location=lambda storage, location: storage) 163 | return state_dict 164 | 165 | 166 | class AttrDict(dict): 167 | 168 | def __init__(self, *args, **kwargs): 169 | super(AttrDict, self).__init__(*args, **kwargs) 170 | self.__dict__ = self 171 | 172 | 173 | def json_to_attrdict(path): 174 | with open(path, 'r') as f: 175 | json_dict = json.load(f) 176 | output = AttrDict() 177 | output.update(json_dict) 178 | return output 179 | 180 | 181 | def load_numpy_item(path): 182 | return np.load(path, allow_pickle=True).item() 183 | 184 | 185 | def count_parameters(model): 186 | return sum(p.numel() for p in model.parameters()) 187 | 188 | 189 | 190 | -------------------------------------------------------------------------------- /myTorch/__init__.py: -------------------------------------------------------------------------------- 1 | from . import nn 2 | from .Utils import Cat 3 | from . import Models 4 | from .Trainer import ( 5 | _callback_decriptor, Trainer, InternalFieldTrainer 6 | ) 7 | from . import Callbacks 8 | from . import Data 9 | from . import Loss 10 | from . import Utils 11 | 12 | __all__ = [ 13 | 'nn', 'Cat', 'Models', '_callback_decriptor', 'Trainer', 'Callbacks', 14 | 'Data', 'Loss', 'Utils', 'InternalFieldTrainer' 15 | ] 16 | -------------------------------------------------------------------------------- /myTorch/nn/.mypy_cache/3.7/@plugins_snapshot.json: -------------------------------------------------------------------------------- 1 | {} -------------------------------------------------------------------------------- /myTorch/nn/ASPP.py: -------------------------------------------------------------------------------- 1 | import torch as _torch 2 | 3 | try: 4 | from .. import nn as _nn 5 | except ImportError: 6 | from myTorch import nn as _nn 7 | 8 | 9 | class PyramidPool(_nn.Layer): 10 | conv_constructor = _nn.ConvNormAct2d 11 | 12 | def __init__(self, in_channels, out_channels, kernel_sizes=[1, 3, 5, 7], **kwargs): 13 | super(PyramidPool, self).__init__(in_channels, out_channels, **kwargs) 14 | assert out_channels % len(kernel_sizes) == 0, 'out_channels mut be divisible by number of kernels' 15 | self.pyramid_layers = [] 16 | n_kernels = len(kernel_sizes) 17 | 18 | for i, kernel in enumerate(kernel_sizes): 19 | self.pyramid_layers.append( 20 | self.conv_constructor(in_channels, out_channels//n_kernels, kernel, **kwargs) 21 | ) 22 | setattr( 23 | self, 'layer_%d' % i, self.pyramid_layers[-1] 24 | ) 25 | self.out_channels = out_channels 26 | 27 | def main_forward(self, x): 28 | to_concat = [] 29 | for layer in self.pyramid_layers: 30 | to_concat.append( 31 | layer(x) 32 | ) 33 | return _torch.cat(to_concat, dim=1) 34 | 35 | 36 | class AtrousPyramidPoolingBase(_nn.RepeatedLayers): 37 | layer_dict = dict( 38 | dilation=[1, 6, 12, 18], 39 | stride=[1, 1, 1, 1] 40 | ) 41 | 42 | interpolation_mode = 'bilinear' 43 | billinear_layer = None 44 | 45 | def __init__(self, *args, **kwargs): 46 | super(AtrousPyramidPoolingBase, self).__init__(*args, **kwargs) 47 | 48 | n_layers = len(next(iter(self.layer_dict.values()))) 49 | 50 | if self.billinear_layer is not None: 51 | billinear_channels = self.billinear_layer.shape[1] 52 | else: 53 | billinear_channels = 0 54 | 55 | self.pw_conv = _torch.nn.Conv2d( 56 | self.args[0]*n_layers + billinear_channels, args[1], 1, bias=False 57 | ) 58 | 59 | def main_forward(self, x): 60 | 61 | if self.billinear_layer is not None: 62 | _, _, h, w = x.shape 63 | concat_layers = [ 64 | _torch.nn.functional.interpolate( 65 | self.billinear_layer, 66 | size=(h, w), 67 | mode=self.interpolation_mode 68 | ) 69 | ] 70 | else: 71 | concat_layers = [] 72 | 73 | for name in self.layer_names: 74 | concat_layers.append( 75 | getattr(self, name)(x)) 76 | 77 | x = _torch.cat(concat_layers, dim=1) 78 | return self.pw_conv(x) 79 | 80 | 81 | class ASPP(AtrousPyramidPoolingBase): 82 | layer_constructor = _nn.ConvNormAct2d 83 | 84 | 85 | class DWASPP(AtrousPyramidPoolingBase): 86 | layer_constructor = _nn.DWConvNormAct2d 87 | 88 | 89 | class SepASPP(AtrousPyramidPoolingBase): 90 | layer_constructor = _nn.SepConvNormAct2d 91 | -------------------------------------------------------------------------------- /myTorch/nn/ConvolutionPatterns.py: -------------------------------------------------------------------------------- 1 | try: 2 | from .. import nn as _nn 3 | except ImportError: 4 | from myTorch import nn as _nn 5 | 6 | import abc as _abc 7 | 8 | operation_order = [ 9 | 'convolution_layer', 'normalization_layer', 'activation_layer'] 10 | 11 | 12 | class ConvPatternBase(_nn.Layer): 13 | 14 | reverse_order = False 15 | 16 | def __init__(self, *args, **kwargs): 17 | super(ConvPatternBase, self).__init__(*args, **kwargs) 18 | self.convolution_layer = self.convolution_constructor( 19 | *self.args, **self.kwargs) 20 | self.normalization_layer = self.normalization_constructor( 21 | self.args[self.in_norm_act] 22 | ) 23 | self.activation_layer = self.activation_constructor( 24 | self.args[self.in_norm_act] 25 | ) 26 | 27 | self.out_channels = self.convolution_layer.out_channels 28 | 29 | def main_forward(self, x): 30 | for method in self.iterable_ops: 31 | x = getattr(self, method)(x) 32 | return x 33 | 34 | @property 35 | def iterable_ops(self): 36 | if self.reverse_order: 37 | return reversed(operation_order) 38 | return operation_order 39 | 40 | @property 41 | def in_norm_act(self): 42 | if not self.reverse_order: 43 | return 1 44 | return 0 45 | 46 | @_abc.abstractmethod 47 | def convolution_constructor(self, *args, **kwargs): 48 | NotImplemented 49 | 50 | @_abc.abstractmethod 51 | def normalization_constructor(self, *args, **kwargs): 52 | NotImplemented 53 | 54 | @_abc.abstractmethod 55 | def activation_constructor(self, *args, **kwargs): 56 | NotImplemented 57 | 58 | 59 | class _ForwardConvPattern(ConvPatternBase): 60 | normalization_constructor = _nn.InstanceNorm2d 61 | activation_constructor = _nn.ReLU 62 | 63 | 64 | class ConvNormAct2d(_ForwardConvPattern): 65 | convolution_constructor = _nn.Conv2d 66 | 67 | class ConvNormAct3d(ConvPatternBase): 68 | convolution_constructor = _nn.Conv3d 69 | normalization_constructor = _nn.InstanceNorm3d 70 | activation_constructor = _nn.ReLU 71 | 72 | class DWConvNormAct2d(_ForwardConvPattern): 73 | convolution_constructor = _nn.DWConv2d 74 | 75 | 76 | class SepConvNormAct2d(_ForwardConvPattern): 77 | convolution_constructor = _nn.SepConv2d 78 | 79 | 80 | class _ReverseConvPattern(_ForwardConvPattern): 81 | reverse_order = True 82 | 83 | 84 | class ActNormConv2d(_ReverseConvPattern): 85 | convolution_constructor = _nn.Conv2d 86 | 87 | 88 | class ActNormDWConv2d(_ReverseConvPattern): 89 | convolution_constructor = _nn.DWConv2d 90 | 91 | 92 | class ActNormSepConv2d(_ReverseConvPattern): 93 | convolution_constructor = _nn.SepConv2d 94 | 95 | -------------------------------------------------------------------------------- /myTorch/nn/Layer.py: -------------------------------------------------------------------------------- 1 | import torch as _torch 2 | 3 | 4 | class LayerPartial: 5 | def __init__(self, method, **kwargs): 6 | self.method = method 7 | self.kwargs = kwargs 8 | 9 | def __call__(self, *args, **kwargs): 10 | self.kwargs.update(kwargs) 11 | return self.method(*args, **self.kwargs) 12 | 13 | 14 | class Layer(_torch.nn.Module): 15 | 16 | def __init__(self, *args, **kwargs): 17 | super(Layer, self).__init__() 18 | self.parse_arguments(*args, **kwargs) 19 | 20 | def forward(self, x, **kwargs): 21 | x = self.pre_forward(x, **kwargs) 22 | x = self.main_forward(x, **kwargs) 23 | x = self.post_forward(x, **kwargs) 24 | return x 25 | 26 | def pre_forward(self, x, **kwargs): 27 | return x 28 | 29 | def main_forward(self, x, **kwargs): 30 | return x 31 | 32 | def post_forward(self, x, **kwargs): 33 | return x 34 | 35 | def parse_arguments(self, *args, **kwargs): 36 | args = list(args) 37 | if args: 38 | if issubclass(args[0].__class__, Layer): 39 | args[0] = args[0].out_channels 40 | if issubclass(args[0].__class__, _torch.Tensor): 41 | args[0] = args[0].shape[1] 42 | if not hasattr(self, 'args'): 43 | self.args = args 44 | self.kwargs = kwargs 45 | -------------------------------------------------------------------------------- /myTorch/nn/Normalization.py: -------------------------------------------------------------------------------- 1 | import torch as _torch 2 | try: 3 | from .. import nn as _nn 4 | except ImportError: 5 | from myTorch import nn as _nn 6 | 7 | import abc as _abc 8 | 9 | 10 | class NormBase(_nn.Layer): 11 | 12 | def __init__(self, *args, **kwargs): 13 | super(NormBase, self).__init__(*args, **kwargs) 14 | self.layer = self.layer_constructor( 15 | *self.args, **self.kwargs 16 | ) 17 | self.out_channels = self.args[0] 18 | 19 | def main_forward(self, x): 20 | return self.layer(x) 21 | 22 | @_abc.abstractmethod 23 | def layer_constructor(self, *args, **kwargs): 24 | NotImplementedError 25 | 26 | 27 | class _BINormCommon(NormBase): 28 | default_kwargs = dict( 29 | track_running_stats=False, 30 | eps=0.001, momentum=0.99, affine=True) 31 | 32 | 33 | class BatchNorm1d(_BINormCommon): 34 | layer_constructor = _nn.LayerPartial( 35 | _torch.nn.BatchNorm1d, **_BINormCommon.default_kwargs) 36 | 37 | 38 | class BatchNorm2d(_BINormCommon): 39 | layer_constructor = _nn.LayerPartial( 40 | _torch.nn.BatchNorm2d, **_BINormCommon.default_kwargs) 41 | 42 | 43 | class BatchNorm3d(_BINormCommon): 44 | layer_constructor = _nn.LayerPartial( 45 | _torch.nn.BatchNorm3d, **_BINormCommon.default_kwargs) 46 | 47 | 48 | class InstanceNorm1d(_BINormCommon): 49 | layer_constructor = _nn.LayerPartial( 50 | _torch.nn.InstanceNorm1d, **_BINormCommon.default_kwargs) 51 | 52 | 53 | class InstanceNorm2d(_BINormCommon): 54 | layer_constructor = _nn.LayerPartial( 55 | _torch.nn.InstanceNorm2d, **_BINormCommon.default_kwargs) 56 | 57 | 58 | class InstanceNorm3d(_BINormCommon): 59 | layer_constructor = _nn.LayerPartial( 60 | _torch.nn.InstanceNorm3d, **_BINormCommon.default_kwargs) 61 | 62 | 63 | class GroupNorm(NormBase): 64 | layer_constructor = _torch.nn.GroupNorm 65 | 66 | def parse_arguments(self, *args, **kwargs): 67 | super(NormBase, self).parse_arguments(*args, **kwargs) 68 | 69 | if len(self.args) == 1: 70 | self.args = list(self.args) 71 | self.args.append(self.args[0]) -------------------------------------------------------------------------------- /myTorch/nn/ResidualConvolutions.py: -------------------------------------------------------------------------------- 1 | try: 2 | from .. import nn as _nn 3 | except ImportError: 4 | from myTorch import nn as _nn 5 | 6 | import torch as _torch 7 | 8 | 9 | class ResidualBase(_nn.RepeatedLayers): 10 | pool_constructor = _torch.nn.AvgPool2d 11 | 12 | def pre_forward(self, x, **kwargs): 13 | self.x_residual = x 14 | return x 15 | 16 | def post_forward(self, x): 17 | self.x_residual = self.spatial_controller(x, self.x_residual) 18 | self.x_residual = self.channel_controller(x, self.x_residual) 19 | return x + self.x_residual 20 | 21 | def spatial_controller(self, x, x_residual): 22 | _, _, x_h, _ = x.shape 23 | _, _, x_residual_h, _ = x_residual.shape 24 | 25 | if not hasattr(self, 'scaling_factor'): 26 | 27 | if x_h == x_residual_h: 28 | return x_residual 29 | 30 | if x_residual_h > x_h: 31 | x_exponent = _nn._findExponent(x_h) 32 | x_in_exponent = _nn._findExponent(x_residual_h) 33 | 34 | self.scaling_factor = x_in_exponent - x_exponent 35 | 36 | if self.scaling_factor != 0: 37 | if not hasattr(self, '_poolConcat'): 38 | self._poolConcat = self.pool_constructor(2) 39 | 40 | for _ in range(self.scaling_factor): 41 | x_residual = self._poolConcat(x_residual) 42 | 43 | return x_residual 44 | 45 | def channel_controller(self, x, x_residual): 46 | 47 | _, x_c, _, _ = x.shape 48 | 49 | _, x_residual_c, _, _ = x_residual.shape 50 | 51 | if x_residual_c == x_c: 52 | return x_residual 53 | 54 | if x_residual_c < x_c: 55 | diff = x_c - x_residual_c 56 | new_x = _torch.zeros_like(x[:, 0: diff, :, :]) 57 | new_x = _torch.cat([x_residual, new_x], dim=1) 58 | return new_x 59 | 60 | if x_residual_c > x_c: 61 | return x_residual[:, 0:x_c, :, :] 62 | 63 | 64 | class ResidualBlock(ResidualBase): 65 | layer_constructor = _nn.ActNormConv2d 66 | _layer_dict = dict( 67 | stride=[1, 1], 68 | dilation=[1, 1] 69 | ) 70 | 71 | ''' 72 | implementation of the following paper 73 | https://arxiv.org/pdf/1512.03385.pdf 74 | ''' 75 | 76 | 77 | class BottleNeckResidualBlock(ResidualBase): 78 | layer_constructor = _nn.ActNormConv2d 79 | _layer_dict = dict( 80 | stride=[1, ], 81 | dilation=[1, ] 82 | ) 83 | 84 | def __init__(self, *args, scale=4, stride=1, **kwargs): 85 | args = list(args) 86 | 87 | original_in_channels = args[0] 88 | original_out_channels = args[1] 89 | 90 | args[0] = max(original_out_channels//scale, 1) 91 | args[1] = args[0] 92 | super(BottleNeckResidualBlock, self).__init__(*args, **kwargs) 93 | 94 | self.pw_conv_A = self.layer_constructor( 95 | original_in_channels, args[0], 1, 96 | padding=0, bias=False, stride=stride) 97 | 98 | self.pw_conv_B = self.layer_constructor( 99 | args[0], original_out_channels, 100 | 1, padding=0, bias=False) 101 | 102 | self.out_channels = original_out_channels 103 | self.in_channels = original_in_channels 104 | 105 | def main_forward(self, x): 106 | x = self.pw_conv_A(x) 107 | x = super(BottleNeckResidualBlock, self).main_forward(x) 108 | x = self.pw_conv_B(x) 109 | return x 110 | 111 | 112 | class DWResidualBlock(ResidualBlock): 113 | layer_constructor = _nn.ActNormDWConv2d 114 | 115 | ''' 116 | depthwise implementation of the following paper 117 | https://arxiv.org/pdf/1512.03385.pdf 118 | ''' 119 | 120 | 121 | class SepResidualBlock(ResidualBlock): 122 | layer_constructor = _nn.ActNormSepConv2d 123 | 124 | ''' 125 | seperable implementation of the following paper 126 | https://arxiv.org/pdf/1512.03385.pdf 127 | ''' 128 | -------------------------------------------------------------------------------- /myTorch/nn/SequentialLayers.py: -------------------------------------------------------------------------------- 1 | try: 2 | from .. import nn as _nn 3 | except ImportError: 4 | from myTorch import nn as _nn 5 | import abc as _abc 6 | import itertools as _itertools 7 | 8 | 9 | class Sequential(_nn.Layer): 10 | 11 | def __init__(self, *layers): 12 | super(Sequential, self).__init__() 13 | self.n_layers = len(layers) 14 | self.layer_registry = layers 15 | self.layer_names = [] 16 | for i, layer in enumerate(self.layer_registry): 17 | name = f'layer_{i}' 18 | self.layer_names.append(name) 19 | setattr( 20 | self, name, layer 21 | ) 22 | self.out_channels = layers[-1].out_channels 23 | 24 | def main_forward(self, x, **kwargs): 25 | for layer in self.layer_registry: 26 | x = layer(x) 27 | return x 28 | 29 | 30 | class RepeatedLayers(Sequential): 31 | _layer_dict = dict() 32 | 33 | def __init__(self, *args, layer_dict={}, **kwargs): 34 | self.parse_arguments(*args, **kwargs) 35 | self.layer_registry = [] 36 | self.layer_dict = self._layer_dict.copy() 37 | self.layer_dict.update(layer_dict) 38 | self.in_channels = self.args[0] 39 | self.check_kwarg_dict_same_length() 40 | self.make_repeated_keys_n_values() 41 | self.construct_layer_registry() 42 | super(RepeatedLayers, self).__init__(*self.layer_registry) 43 | 44 | def construct_layer_registry(self): 45 | for i, (keys, zipped) in enumerate( 46 | zip(self.repeated_keys, self.zipped_values)): 47 | self.kwargs.update(dict(zip(keys, zipped))) 48 | if i == 1: 49 | self.args = list(self.args) 50 | self.args[0] = self.args[1] 51 | 52 | self.layer_registry.append( 53 | self.layer_constructor( 54 | *self.args, **self.kwargs 55 | ) 56 | ) 57 | 58 | def make_repeated_keys_n_values(self): 59 | self.repeated_keys = _itertools.repeat( 60 | self.layer_dict.keys(), self.n_repeat) 61 | self.zipped_values = zip(*self.layer_dict.values()) 62 | 63 | def check_kwarg_dict_same_length(self): 64 | assert len(self.layer_dict) 65 | 66 | key_0 = list(self.layer_dict.keys())[0] 67 | self.n_repeat = len(self.layer_dict[key_0]) 68 | for key, item in self.layer_dict.items(): 69 | assert len(item) == self.n_repeat 70 | 71 | @_abc.abstractmethod 72 | def layer_constructor(self, *args, **kwargs): 73 | NotImplemented 74 | 75 | 76 | class VGGBlock(RepeatedLayers): 77 | ''' 78 | implementation of the basic VGG block 79 | https://arxiv.org/pdf/1409.1556.pdf 80 | ''' 81 | layer_constructor = _nn.ConvNormAct2d 82 | _layer_dict = dict( 83 | stride=[1, 1], 84 | dilation=[1, 1] 85 | ) 86 | 87 | 88 | class VGGBlock3d(RepeatedLayers): 89 | ''' 90 | implementation of the basic VGG block 91 | https://arxiv.org/pdf/1409.1556.pdf 92 | ''' 93 | layer_constructor = _nn.ConvNormAct3d 94 | _layer_dict = dict( 95 | stride=[1, 1], 96 | dilation=[1, 1] 97 | ) 98 | 99 | 100 | class DWVGGBlock(VGGBlock): 101 | ''' 102 | implementation of the depthwise VGG block 103 | https://arxiv.org/pdf/1409.1556.pdf 104 | ''' 105 | layer_constructor = _nn.DWConvNormAct2d 106 | 107 | 108 | class SepVGGBlock(VGGBlock): 109 | ''' 110 | implementation of the seperable VGG block 111 | https://arxiv.org/pdf/1409.1556.pdf 112 | ''' 113 | layer_constructor = _nn.SepConvNormAct2d 114 | 115 | 116 | class LinkNetDecoder(_nn.Layer): 117 | def __init__(self, in_channels, out_channels, kernel_size, rescale_factor=4, **kwargs): 118 | super(LinkNetDecoder, self).__init__( 119 | in_channels, out_channels, kernel_size, **kwargs) 120 | scaled_in = in_channels//4 121 | self.conv1 = _nn.ConvNormAct2d(in_channels, scaled_in, 1, **kwargs) 122 | self.conv2 = _nn.ConvNormAct2d(self.conv1, scaled_in, kernel_size, **kwargs) 123 | self.conv3 = _nn.ConvNormAct2d(self.conv2, out_channels, 1, **kwargs) 124 | self.out_channels = out_channels 125 | 126 | def main_forward(self, x): 127 | x = self.conv1(x) 128 | x = self.conv2(x) 129 | x = self.conv3(x) 130 | return x -------------------------------------------------------------------------------- /myTorch/nn/SqueezeExcitation.py: -------------------------------------------------------------------------------- 1 | try: 2 | from .. import nn as _nn 3 | except ImportError: 4 | from myTorch import nn as _nn 5 | 6 | import torch as _torch 7 | 8 | 9 | class SEBlock(_nn.Layer): 10 | """SEBlock 11 | 12 | Squeeze and excitation blocks 13 | 14 | https://arxiv.org/pdf/1709.01507.pdf 15 | """ 16 | 17 | def __init__(self, *args, scale_factor=16, **kwargs): 18 | super(SEBlock, self).__init__( 19 | *args, scale_factor=scale_factor, **kwargs) 20 | 21 | 22 | self.in_channels = args[0] if isinstance(args[0], int) else args[0].out_channels 23 | 24 | self.out_channels = self.in_channels 25 | 26 | self.scaled_channels = max(self.in_channels//scale_factor, 1) 27 | 28 | self.global_pooling = _torch.nn.functional.avg_pool2d 29 | 30 | self.fc_1 = _torch.nn.Linear( 31 | self.in_channels, self.scaled_channels, bias=False) 32 | 33 | self.activation_layer = _torch.nn.ReLU() 34 | 35 | self.fc_2 = _torch.nn.Linear( 36 | self.scaled_channels, self.out_channels, bias=False) 37 | 38 | self.sigmoid_layer = _torch.nn.Sigmoid() 39 | 40 | def main_forward(self, x): 41 | _, _, *hw = x.shape 42 | x = self.global_pooling(x, hw) 43 | x = x.permute(0, 2, 3, 1) 44 | 45 | x = self.fc_1(x) 46 | x = self.activation_layer(x) 47 | x = self.fc_2(x) 48 | x = self.sigmoid_layer(x) 49 | x = x.permute(0, 3, 1, 2) 50 | 51 | return x 52 | 53 | class CSEBlock(_torch.nn.Module): 54 | """CSEBlock 55 | 56 | Channel Squeeze and spatial excitation 57 | 58 | https://arxiv.org/pdf/1803.02579.pdf 59 | """ 60 | def __init__(self, in_channels): 61 | super(CSEBlock, self).__init__() 62 | self.conv = _torch.nn.Conv2d(in_channels, 1, 1, bias=False) 63 | self.activation = _torch.nn.Sigmoid() 64 | self.out_channels = 1 65 | 66 | def forward(self, x): 67 | x = self.conv(x) 68 | x = self.activation(x) 69 | return x 70 | 71 | #spatial squeeze and channel excite 72 | 73 | class SEResBlock(_nn.ResidualBlock): 74 | 75 | def __init__(self, *args, scale_factor=16, **kwargs): 76 | super(SEResBlock, self).__init__(*args, **kwargs) 77 | self.se_block = SEBlock(self.out_channels, scale_factor=scale_factor) 78 | 79 | def post_forward(self, x): 80 | x_scale = self.se_block(x) 81 | x = x * x_scale 82 | return super(SEResBlock, self).post_forward(x) 83 | 84 | 85 | class SEVGGBlock(_nn.VGGBlock): 86 | def __init__(self, *args, scale_factor=16, **kwargs): 87 | super(SEVGGBlock, self).__init__(*args, **kwargs) 88 | self.se_block = SEBlock(self.out_channels, scale_factor=scale_factor) 89 | 90 | def post_forward(self, x): 91 | x_scale = self.se_block(x) 92 | x = x * x_scale 93 | return super(SEVGGBlock, self).post_forward(x) 94 | 95 | # channel squeeze spatial excite 96 | 97 | class CSEVGGBlock(_nn.VGGBlock): 98 | def __init__(self, *args, **kwargs): 99 | super(CSEVGGBlock, self).__init__(*args, **kwargs) 100 | self.cse_block = CSEBlock(self.out_channels) 101 | 102 | def post_forward(self, x): 103 | x_scale = self.cse_block(x) 104 | x = x * x_scale 105 | return super(CSEVGGBlock, self).post_forward(x) 106 | 107 | 108 | # concurrent 109 | 110 | class CSESEVGGBlock(_nn.VGGBlock): 111 | def __init__(self, *args, scale_factor=16, **kwargs): 112 | super(CSESEVGGBlock, self).__init__(*args, **kwargs) 113 | self.cse_block = CSEBlock(self.out_channels) 114 | self.se_block = SEBlock(self.out_channels, scale_factor=scale_factor) 115 | 116 | def post_forward(self, x): 117 | x_excite_spatial = self.cse_block(x) 118 | x_excite_channel = self.se_block(x) 119 | x = x * x_excite_spatial + x * x_excite_channel 120 | return super(CSESEVGGBlock, self).post_forward(x) 121 | 122 | 123 | 124 | -------------------------------------------------------------------------------- /myTorch/nn/UpsampleConvolution.py: -------------------------------------------------------------------------------- 1 | try: 2 | from .. import nn as _nn 3 | except ImportError: 4 | from myTorch import nn as _nn 5 | import torch as _torch 6 | 7 | 8 | class _UpsampleConv2dBase(_nn.ConvNormAct2d): 9 | upsample_constructor = _nn.LayerPartial( 10 | _torch.nn.Upsample, scale_factor=2, align_corners=True, 11 | mode='bilinear') 12 | 13 | def __init__(self, *args, scale_factor=2, **kwargs): 14 | super(_UpsampleConv2dBase, self).__init__(*args, **kwargs) 15 | self.upsampling_layer = self.upsample_constructor( 16 | scale_factor=scale_factor 17 | ) 18 | self.out_channels = self.convolution_layer.out_channels 19 | 20 | def main_forward(self, x): 21 | x = self.upsampling_layer(x) 22 | x = super(_UpsampleConv2dBase, self).main_forward(x) 23 | return x 24 | 25 | class _UpsampleConv3dBase(_nn.ConvNormAct3d): 26 | upsample_constructor = _nn.LayerPartial( 27 | _torch.nn.Upsample, scale_factor=2, align_corners=True, 28 | mode='trilinear') 29 | 30 | def __init__(self, *args, **kwargs): 31 | super(_UpsampleConv3dBase, self).__init__(*args, **kwargs) 32 | self.upsampling_layer = self.upsample_constructor() 33 | self.out_channels = self.convolution_layer.out_channels 34 | 35 | def main_forward(self, x): 36 | x = self.upsampling_layer(x) 37 | x = super(_UpsampleConv3dBase, self).main_forward(x) 38 | return x 39 | 40 | 41 | class UpsampleConv2d(_UpsampleConv2dBase): 42 | convolution_constructor = _nn.Conv2d 43 | 44 | class UpsampleConv3d(_UpsampleConv3dBase): 45 | convolution_constructor = _nn.Conv3d 46 | 47 | 48 | class DWUpsampleConv2d(_UpsampleConv2dBase): 49 | convolution_constructor = _nn.DWConv2d 50 | 51 | 52 | class SepUpsampleConv2d(_UpsampleConv2dBase): 53 | convolution_constructor = _nn.SepConv2d 54 | 55 | 56 | if __name__ == "__main__": 57 | import torch 58 | image = torch.zeros([1, 1, 25, 25]) 59 | upsampled = UpsampleConv2d(image, 10, 3, scale_factor=4)(image) 60 | print(image.shape) 61 | print(upsampled.shape) 62 | -------------------------------------------------------------------------------- /myTorch/nn/__init__.py: -------------------------------------------------------------------------------- 1 | from .find_exponent import _findExponent 2 | from .Layer import ( 3 | Layer, LayerPartial 4 | ) 5 | from .Padding import ( 6 | ZeroPad2d, ConstantPad2d, ReplicationPad2d, PeriodicPad2d, 7 | PeriodicReplication2d 8 | ) 9 | from .convolutions import ( 10 | AssignFromLayer, ConcatConv, 11 | Conv2d, ConvBase, ConvTranspose2d, DWConv2d, SepConv2d, Conv3d 12 | ) 13 | from .Normalization import ( 14 | BatchNorm1d, BatchNorm2d, BatchNorm3d, 15 | GroupNorm, InstanceNorm1d, InstanceNorm2d, InstanceNorm3d, 16 | NormBase, _BINormCommon 17 | ) 18 | from .wrapped import ( 19 | AvgPool2d, CELU, ELU, Hardshrink, Hardtanh, LeakyReLU, 20 | LogSigmoid, LogSoftmax, MaxPool2d, PReLU, ReLU, ReLU6, 21 | RReLU, SELU, Sigmoid, Softmax, Softmax2d, Softmin, 22 | Softplus, Softshrink, Softsign, Tanh, Tanhshrink, Threshold, 23 | AvgPool3d 24 | ) 25 | from .ConvolutionPatterns import ( 26 | ActNormConv2d, ActNormDWConv2d, ActNormSepConv2d, ConvNormAct2d, 27 | ConvPatternBase, DWConvNormAct2d, SepConvNormAct2d, ConvNormAct3d 28 | ) 29 | from .UpsampleConvolution import ( 30 | DWUpsampleConv2d, SepUpsampleConv2d, UpsampleConv2d, 31 | UpsampleConv3d 32 | ) 33 | from .SequentialLayers import ( 34 | DWVGGBlock, RepeatedLayers, SepVGGBlock, Sequential, 35 | VGGBlock, LinkNetDecoder, VGGBlock3d 36 | ) 37 | from .ResidualConvolutions import ( 38 | BottleNeckResidualBlock, DWResidualBlock, ResidualBase, 39 | ResidualBlock, SepResidualBlock 40 | ) 41 | from .SqueezeExcitation import ( 42 | SEBlock, SEResBlock, CSEBlock, SEVGGBlock, CSEVGGBlock, CSESEVGGBlock 43 | ) 44 | from .ASPP import ( 45 | ASPP, AtrousPyramidPoolingBase, DWASPP, SepASPP, PyramidPool 46 | ) 47 | from . import init 48 | 49 | __all__ = [ 50 | '_findExponent', 'Layer', 'LayerPartial', 'ZeroPad2d', 'ConstantPad2d', 51 | 'ReplicationPad2d', 'PeriodicPad2d', 'PeriodicReplication2d', 52 | 'AssignFromLayer', 'ConcatConv', 'Conv2d', 'ConvBase', 'ConvTranspose2d', 53 | 'DWConv2d', 'SepConv2d', 'BatchNorm1d', 'BatchNorm2d', 'BatchNorm3d', 54 | 'GroupNorm', 'InstanceNorm1d', 'InstanceNorm2d', 'InstanceNorm3d', 55 | 'NormBase', 'AvgPool2d', 'CELU', 'ELU', 'Hardshrink', 'Hardtanh', 56 | 'LeakyReLU', 'LogSigmoid', 'LogSoftmax', 'MaxPool2d', 'PReLU', 'ReLU', 57 | 'ReLU6', 'RReLU', 'SELU', 'Sigmoid', 'Softmax', 'Softmax2d', 'Softmin', 58 | 'Softplus', 'Softshrink', 'Softsign', 'Tanh', 'Tanhshrink', 'Threshold', 59 | 'ActNormConv2d', 'ActNormDWConv2d', 'ActNormSepConv2d', 'ConvNormAct2d', 60 | 'ConvPatternBase', 'DWConvNormAct2d', 'SepConvNormAct2d', 61 | 'DWUpsampleConv2d', 'SepUpsampleConv2d', 'UpsampleConv2d', 'DWVGGBlock', 62 | 'RepeatedLayers', 'SepVGGBlock', 'Sequential', 'VGGBlock', 63 | 'BottleNeckResidualBlock', 'DWResidualBlock', 'ResidualBase', 64 | 'ResidualBlock', 'SepResidualBlock', 'SEBlock', 'SEResBlock', 'ASPP', 65 | 'AtrousPyramidPoolingBase', 'DWASPP', 'SepASPP', 'init', 66 | 'CSEBlock', 'SEVGGBlock', 'CSEVGGBlock', 'CSESEVGGBlock', 'Conv3d', 67 | 'ConvNormAct3d', 'VGGBlock3d', 'AvgPool3d', 'UpsampleConv3d' 68 | ] -------------------------------------------------------------------------------- /myTorch/nn/compare_trianing.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import matplotlib.pyplot as plt 3 | 4 | if __name__ == "__main__": 5 | file1 = r'i:\CNNForCFD\KFold_LinkNet_U\fold_1\valid\log.npy' 6 | file2 = r'I:\CNNForCFD\KFold_FPN_U_AREA\fold_1\valid\log.npy' 7 | 8 | item1 = np.array(np.load(file1, allow_pickle=True).item()['accuracy']) 9 | item2 = np.array(np.load(file2, allow_pickle=True).item()['accuracy']) 10 | 11 | plt.plot(item1) 12 | plt.plot(item2) 13 | plt.show() -------------------------------------------------------------------------------- /myTorch/nn/find_exponent.py: -------------------------------------------------------------------------------- 1 | def _findExponent(in_val, start=2): 2 | i = 0 3 | val = 2**i 4 | while val < in_val: 5 | i += 1 6 | val = start**i 7 | return i 8 | -------------------------------------------------------------------------------- /myTorch/nn/init.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import pathlib as pt 3 | 4 | 5 | class ModelInitializer: 6 | 7 | def __init__(self, initializer, *args, **kwargs): 8 | self.initializer = initializer 9 | self.args = args 10 | self.kwargs = kwargs 11 | 12 | def __call__(self, model): 13 | for param in model.parameters(): 14 | if self.criterion(param): 15 | self.initializer(param, *self.args, **self.kwargs) 16 | return model 17 | 18 | def criterion(self, param): 19 | return True 20 | 21 | 22 | class WeightInitializer(ModelInitializer): 23 | def criterion(self, param): 24 | if len(param.shape) == 4: 25 | return True 26 | return False 27 | 28 | 29 | class XavierUniformWeightInitializer(WeightInitializer): 30 | def __init__(self, gain=1.0): 31 | super(XavierUniformWeightInitializer, self).__init__( 32 | torch.nn.init.xavier_uniform_, 33 | gain=gain 34 | ) 35 | 36 | 37 | class LoadBest: 38 | 39 | def __init__(self, folder, score='validation_score', model_name='model'): 40 | self.checkpoint_folder = pt.Path(folder) 41 | files = list(self.checkpoint_folder.glob('*')) 42 | self.checkpoints = [torch.load(file) for file in files] 43 | if score is not None: 44 | self.checkpoints.sort(key=lambda x: x[score]) 45 | self.checkpoint_id = -1 46 | self.model_name = model_name 47 | 48 | def __call__(self, model, checkpoint_id=None): 49 | if checkpoint_id is not None: 50 | self.checkpoint_id = checkpoint_id 51 | model.load_state_dict( 52 | self.checkpoints[self.checkpoint_id][self.model_name] 53 | ) 54 | return model 55 | 56 | 57 | 58 | 59 | -------------------------------------------------------------------------------- /myTorch/nn/pspnet.py: -------------------------------------------------------------------------------- 1 | from .EncoderBottleNeckHead import EncodeDecodeBase 2 | try: 3 | from myTorch import nn, Models 4 | except ImportError: 5 | from .. import nn, Models 6 | import torch 7 | 8 | 9 | class PSPNetBuilder(EncodeDecodeBase) -------------------------------------------------------------------------------- /myTorch/nn/wrapped.py: -------------------------------------------------------------------------------- 1 | import torch as _torch 2 | try: 3 | from . import Layer as _Layer 4 | except ImportError: 5 | from myTorch.nn import Layer as _Layer 6 | import abc as _abc 7 | 8 | 9 | class _Common(_Layer): 10 | 11 | def __init__(self, in_channels, *args, **kwargs): 12 | super(_Common, self).__init__(*args, **kwargs) 13 | if isinstance(in_channels, int): 14 | self.out_channels = in_channels 15 | else: 16 | self.out_channels = in_channels.out_channels 17 | self.layer = self.layer_constructor(*self.args, **self.kwargs) 18 | 19 | def main_forward(self, x): 20 | return self.layer(x) 21 | 22 | @_abc.abstractmethod 23 | def layer_constructor(self, *args, **kwargs): 24 | NotImplementedError 25 | 26 | 27 | class ELU(_Common): 28 | layer_constructor = _torch.nn.ELU 29 | 30 | 31 | class Hardshrink(_Common): 32 | layer_constructor = _torch.nn.Hardshrink 33 | 34 | 35 | class Hardtanh(_Common): 36 | layer_constructor = _torch.nn.Hardtanh 37 | 38 | 39 | class LeakyReLU(_Common): 40 | layer_constructor = _torch.nn.LeakyReLU 41 | 42 | 43 | class LogSigmoid(_Common): 44 | layer_constructor = _torch.nn.LogSigmoid 45 | 46 | 47 | class PReLU(_Common): 48 | layer_constructor = _torch.nn.PReLU 49 | 50 | 51 | class ReLU(_Common): 52 | layer_constructor = _torch.nn.ReLU 53 | 54 | 55 | class ReLU6(_Common): 56 | layer_constructor = _torch.nn.ReLU6 57 | 58 | 59 | class RReLU(_Common): 60 | layer_constructor = _torch.nn.RReLU 61 | 62 | 63 | class SELU(_Common): 64 | layer_constructor = _torch.nn.SELU 65 | 66 | 67 | class CELU(_Common): 68 | layer_constructor = _torch.nn.CELU 69 | 70 | 71 | class Sigmoid(_Common): 72 | layer_constructor = _torch.nn.Sigmoid 73 | 74 | 75 | class Softplus(_Common): 76 | layer_constructor = _torch.nn.Softplus 77 | 78 | 79 | class Softshrink(_Common): 80 | layer_constructor = _torch.nn.Softshrink 81 | 82 | 83 | class Softsign(_Common): 84 | layer_constructor = _torch.nn.Softsign 85 | 86 | 87 | class Tanh(_Common): 88 | layer_constructor = _torch.nn.Tanh 89 | 90 | 91 | class Tanhshrink(_Common): 92 | layer_constructor = _torch.nn.Tanhshrink 93 | 94 | 95 | class Threshold(_Common): 96 | layer_constructor = _torch.nn.Threshold 97 | 98 | 99 | class Softmin(_Common): 100 | layer_constructor = _torch.nn.Softmin 101 | 102 | 103 | class Softmax(_Common): 104 | layer_constructor = _torch.nn.Softmax 105 | 106 | 107 | class Softmax2d(_Common): 108 | layer_constructor = _torch.nn.Softmax2d 109 | 110 | 111 | class LogSoftmax(_Common): 112 | layer_constructor = _torch.nn.LogSoftmax 113 | 114 | 115 | class MaxPool2d(_Common): 116 | layer_constructor = _torch.nn.MaxPool2d 117 | 118 | 119 | class AvgPool2d(_Common): 120 | layer_constructor = _torch.nn.AvgPool2d 121 | 122 | class AvgPool3d(_Common): 123 | layer_constructor = _torch.nn.AvgPool3d 124 | -------------------------------------------------------------------------------- /myTorch/optim/FinderLRs.py: -------------------------------------------------------------------------------- 1 | from torch.optim.lr_scheduler import _LRScheduler 2 | 3 | 4 | class ExponentialLR_Finder(_LRScheduler): 5 | 6 | def __init__(self, optimizer, end_lr, num_iter, last_epoch=-1): 7 | self.end_lr = end_lr 8 | self.num_iter = num_iter 9 | super(ExponentialLR_Finder, self).__init__(optimizer, last_epoch) 10 | 11 | def get_lr(self): 12 | curr_iter = self.last_epoch + 1 13 | r = curr_iter/self.num_iter 14 | return [ 15 | base_lr * (self.end_lr / base_lr) ** r for 16 | base_lr in self.base_lrs] 17 | 18 | 19 | class LinearLR_Finder(_LRScheduler): 20 | 21 | def __init__(self, optimizer, end_lr, num_iter, last_epoch=-1): 22 | self.end_lr = end_lr 23 | self.num_iter = num_iter 24 | super(LinearLR_Finder, self).__init__(optimizer, last_epoch) 25 | 26 | def get_lr(self): 27 | curr_iter = self.last_epoch + 1 28 | r = curr_iter / self.num_iter 29 | return [base_lr + r * (self.end_lr - base_lr) for base_lr in self.base_lrs] -------------------------------------------------------------------------------- /myTorch/optim/__init__.py: -------------------------------------------------------------------------------- 1 | from .FinderLRs import ( 2 | ExponentialLR_Finder, LinearLR_Finder 3 | ) 4 | from .lr_finder import ( 5 | LRFinder, LRFinderCFD 6 | ) 7 | from .adabound import ( 8 | AdaBound 9 | ) 10 | 11 | __all__ = [ 12 | 'ExponentialLR_Finder', 'LinearLR_Finder', 'LRFinder' 13 | ] -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | absl-py==0.9.0 2 | alabaster==0.7.12 3 | appdirs==1.4.3 4 | astroid==2.3.3 5 | attrs==19.3.0 6 | Babel==2.8.0 7 | backcall==0.1.0 8 | bcrypt==3.1.7 9 | bleach==3.1.0 10 | cachetools==4.1.0 11 | certifi==2020.4.5.1 12 | cffi==1.14.0 13 | chardet==3.0.4 14 | colorama==0.4.3 15 | cryptography==2.9 16 | cycler==0.10.0 17 | decorator==4.4.1 18 | defusedxml==0.6.0 19 | docutils==0.16 20 | entrypoints==0.3 21 | flake8==3.7.9 22 | future==0.18.2 23 | google-auth==1.14.2 24 | google-auth-oauthlib==0.4.1 25 | grpcio==1.28.1 26 | idna==2.9 27 | imageio==2.6.1 28 | imagesize==1.2.0 29 | importlib-metadata==1.5.0 30 | ipykernel==5.1.4 31 | ipympl==0.4.1 32 | ipython==7.12.0 33 | ipython-genutils==0.2.0 34 | ipywidgets==7.5.1 35 | isort==4.3.21 36 | jedi==0.16.0 37 | Jinja2==2.11.1 38 | joblib==0.14.1 39 | jsonschema==3.2.0 40 | jupyter==1.0.0 41 | jupyter-client==5.3.4 42 | jupyter-console==6.1.0 43 | jupyter-core==4.6.2 44 | kiwisolver==1.1.0 45 | lazy-object-proxy==1.4.3 46 | llvmlite==0.31.0 47 | Markdown==3.2.2 48 | MarkupSafe==1.1.1 49 | matplotlib==3.1.3 50 | mccabe==0.6.1 51 | meshio==4.0.13 52 | mistune==0.8.4 53 | mpmath==1.1.0 54 | msmb-theme==1.2.0 55 | nbconvert==5.6.1 56 | nbformat==5.0.4 57 | networkx==2.4 58 | nose==1.3.7 59 | notebook==6.0.3 60 | numba==0.48.0 61 | numpy==1.18.1 62 | numpy-stl==2.11.2 63 | oauthlib==3.1.0 64 | opencv-python==4.2.0.32 65 | packaging==20.3 66 | pandas==1.0.1 67 | pandocfilters==1.4.2 68 | paramiko==2.7.1 69 | parso==0.6.1 70 | pickleshare==0.7.5 71 | Pillow==7.0.0 72 | plotly==4.6.0 73 | prometheus-client==0.7.1 74 | prompt-toolkit==3.0.3 75 | protobuf==3.11.3 76 | pyasn1==0.4.8 77 | pyasn1-modules==0.2.8 78 | pycodestyle==2.5.0 79 | pycparser==2.20 80 | pyflakes==2.1.1 81 | Pygments==2.5.2 82 | pylint==2.4.4 83 | PyNaCl==1.3.0 84 | pyparsing==2.4.6 85 | PyQt5==5.14.1 86 | PyQt5-sip==12.7.1 87 | pyrsistent==0.15.7 88 | python-dateutil==2.8.1 89 | python-utils==2.4.0 90 | pytz==2019.3 91 | pyvista==0.24.3 92 | PyWavelets==1.1.1 93 | pywin32==227 94 | pywinpty==0.5.7 95 | pyzmq==18.1.1 96 | qtconsole==4.6.0 97 | requests==2.23.0 98 | requests-oauthlib==1.3.0 99 | retrying==1.3.3 100 | rsa==4.0 101 | rstcheck==3.3.1 102 | scikit-image==0.16.2 103 | scikit-learn==0.22.2.post1 104 | scipy==1.4.1 105 | scooby==0.5.2 106 | scp==0.13.2 107 | Send2Trash==1.5.0 108 | six==1.14.0 109 | snowballstemmer==2.0.0 110 | Sphinx==3.0.0 111 | sphinx-rtd-theme==0.4.3 112 | sphinxcontrib-applehelp==1.0.2 113 | sphinxcontrib-devhelp==1.0.2 114 | sphinxcontrib-htmlhelp==1.0.3 115 | sphinxcontrib-jsmath==1.0.1 116 | sphinxcontrib-qthelp==1.0.3 117 | sphinxcontrib-serializinghtml==1.1.4 118 | sympy==1.5.1 119 | tensorboard==2.2.1 120 | tensorboard-plugin-wit==1.6.0.post3 121 | terminado==0.8.3 122 | testpath==0.4.4 123 | toml==0.10.0 124 | torch==1.5.0 125 | torchvision==0.6.0 126 | tornado==6.0.3 127 | tqdm==4.42.1 128 | traitlets==4.3.3 129 | typed-ast==1.4.1 130 | urllib3==1.25.8 131 | vtk==8.1.2 132 | wcwidth==0.1.8 133 | webencodings==0.5.1 134 | Werkzeug==1.0.1 135 | widgetsnbextension==3.5.1 136 | wrapt==1.11.2 137 | zipp==2.2.0 138 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | setup( 3 | name="myTorch", 4 | version="0.1", 5 | packages=["myTorch"] 6 | ) --------------------------------------------------------------------------------