├── .gitattributes ├── .gitignore ├── README.md ├── imgs └── explanation.png ├── pytorch_fastercnn ├── pytorch_fastercnn │ ├── __init__.py │ ├── checkpoints │ │ └── fasterrcnn_resnet50_fpn_coco-258fb6c6.pth │ └── model.py ├── setup.py ├── setup_cpu.py └── tests │ ├── pytorch_fastercnn.jpg │ ├── pytorch_fastercnn_gt.json │ └── test_pytorch_fastercnn.py └── template ├── setup.py ├── template ├── __init__.py ├── checkpoints │ └── .gitignore └── model.py └── tests └── test_model.py /.gitattributes: -------------------------------------------------------------------------------- 1 | *.pth filter=lfs diff=lfs merge=lfs -text 2 | *.json filter=lfs diff=lfs merge=lfs -text 3 | *.jpeg filter=lfs diff=lfs merge=lfs -text 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | *.py[cod] 3 | *$py.class 4 | .idea/ 5 | .vscode 6 | .nox 7 | .mypy_cache 8 | *.code-workspace -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | These are instructions on how to submit a model for listing on [Modelplace.AI](https://modelplace.ai/), the AI model marketplace. 2 | 3 | Modelplace currently supports [PyTorch](https://pytorch.org/), [Tensorflow](https://www.tensorflow.org/), [OpenVINO](https://docs.openvinotoolkit.org/latest/index.html) and [ONNXRuntime](https://www.onnxruntime.ai/) backends. 4 | 5 | # How Do I submit a model to Modelplace? 6 | 7 | To provide a model we ask you to do the following: 8 | 1. Provide [required model publishing information](#model-publishing-information) about yourself and each model you want to publish 9 | 2. Set up [environment](#evironment-setup) and try our [example model (PyTorch Faster R-CNN)](pytorch_fastercnn) 10 | 3. Implement your model in a similar way to the given [example model (PyTorch Faster R-CNN)](pytorch_fastercnn) 11 | 4. Create a Python wheel [package](#package) with your model 12 | 5. [Send your model to us](#send-your-model-to-us) with the above information and package you have created 13 | 14 | ## Model publishing information 15 | ### Required 16 | - **Model description** - The whole description of what the model does. Example: See the [example model page](https://modelplace.ai/models/2) - this is the text under the "Summary" heading. 17 | - **Model description preview short** - The text people see on the "card" for each model on the [model list page](https://modelplace.ai/models). A short (one sentence) description of your model. 18 | - **Model name (Full)** - Shown on the individual model page ([example model page](https://modelplace.ai/models/2)), e.g. `Faster region-based convolutional neural network with ResNet-50 FPN backbone` 19 | - **Model name (Short)** - A much shorter model name shown on the [model list page](https://modelplace.ai/models), e.g. `Faster R-CNN` 20 | - **Dataset name** - Specify if your model uses public data e.g. MSCOCO or Open images, or a custom dataset 21 | - **Preview image** - we will run a model on the image and will use its output for visualization. This image will be both on the [model list pages](https://modelplace.ai/models) and on the [model page](https://modelplace.ai/models/2) 22 | - **License** - License type e.g. Apache 2.0, MIT, Proprietary, etc. 23 | - **Number of classes** - For classifier models, how many distinct classes does it detect? 24 | - **Author** - Your name or organization name (e.g. University) 25 | - **Metrics** - The accuracy of your model 26 | 27 | ### Optional 28 | - **Dataset link** - The link to a dataset that you use for training and testing of your model 29 | - **Homepage link** - The link to your project page, source code or website for this model 30 | - **System requirements** - What system setup (e.g. CPU, GPU, and RAM specifications) we should use to serve your model 31 | - **Logo** - The logo will be added to model previews on listing pages and individual model pages 32 | _Note: This should be svg (png is possible) image with circle shape and size 128x128 pixels at least_ 33 | - **Avatar image** - Shown next to your name or organization name on the model listing and indivudal model pages (by default, shows OpenCV logo) 34 | _Note: This should be svg (png is possible) image with circle shape and size 128x128 pixels at least_ 35 | ![](imgs/explanation.png) 36 | 37 | ## Environment Setup 38 | - Install venv 39 | ```python3.7 -m pip install virtualenv``` 40 | - Create an empty virtual environment for python3.7 41 | ```python3.7 -m virtualenv venv``` 42 | - Activate it 43 | ```source venv/bin/activate``` 44 | - Install pytest and wheel 45 | ```python3.7 -m pip install pytest wheel``` 46 | - Install git and git-lfs 47 | ```sudo apt install git git-lfs``` 48 | - Clone the repo 49 | ```git clone https://github.com/opencv-ai/modelplace.git``` 50 | 51 | ## Example Model 52 | For package and style guidelines see the [example package (Faster R-CNN)](pytorch_fastercnn). 53 | - Change directory to the package folder (`pytorch_fastercnn`) 54 | ```cd modelplace/pytorch_fastercnn``` 55 | - Install the package 56 | ```python3.7 setup.py bdist_wheel && rm -R build/ *.egg-info && pip3 install dist/*.whl``` 57 | - Run tests 58 | ```python3.7 -m pytest``` 59 | 60 | To see how this example model is represented on the site, view its page: [https://modelplace.ai/models/2](https://modelplace.ai/models/2) 61 | 62 | ## Packaging Models for Modelplace 63 | Modelplace uses Python Wheels to simplify model serving. You should [create a wheel package](https://packaging.python.org/tutorials/packaging-projects/#creating-the-package-files) with our interfaces. See the [example (Faster R-CNN)](pytorch_fastercnn). 64 | 65 | To create a package, follow these steps: 66 | - Copy [the template folder](template) 67 | - Extend model.py as shown in [the example](pytorch_fastercnn/pytorch_fastercnn/model.py) 68 | - Update [the setup.py](template/setup.py) 69 | - Copy your checkpoints to [the template/checkpoints](template/template/checkpoints) folder 70 | - Rename "template" to your package name in all locations, including the folder name, e.g. template -> pytorch_fastercnn 71 | - Install the package locally 72 | ```python3.7 setup.py bdist_wheel && rm -R build/ *.egg-info && pip3 install dist/*.whl``` 73 | - Update tests in [test_model.py](template/tests/test_model.py) 74 | - Run tests 75 | ```python3.7 -m pytest``` 76 | - Make sure that the tests are working correctly and all tests pass 77 | 78 | ## Send Your Model To Us 79 | 80 | Once you've done all that, to submit your package simply: 81 | 82 | - Zip the folder you created in the above steps 83 | - [Send us](mailto:modelplace@opencv.ai) this zip and information about package as described above 84 | 85 | That's it! If anything is unclear, please [contact us](mailto:modelplace@opencv.ai) and we'll help you along. 86 | -------------------------------------------------------------------------------- /imgs/explanation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/opencv-ai/modelplace/22e67a27ca0df9c88aef09fb442d17db02ce8907/imgs/explanation.png -------------------------------------------------------------------------------- /pytorch_fastercnn/pytorch_fastercnn/__init__.py: -------------------------------------------------------------------------------- 1 | __version__ = "0.1.1" 2 | from pytorch_fastercnn.model import InferenceModel 3 | -------------------------------------------------------------------------------- /pytorch_fastercnn/pytorch_fastercnn/checkpoints/fasterrcnn_resnet50_fpn_coco-258fb6c6.pth: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:258fb6c638b15964ddcdd1ae0748c5eef1be9e732750120cc857feed3faac384 3 | size 167502836 4 | -------------------------------------------------------------------------------- /pytorch_fastercnn/pytorch_fastercnn/model.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | import torch 4 | import torchvision 5 | from modelplace_api import BaseModel, BBox, Device, TaskType 6 | from torchvision.transforms import functional as F 7 | 8 | 9 | class InferenceModel(BaseModel): 10 | class_names = { 11 | 0: "background", 12 | 1: "person", 13 | 2: "bicycle", 14 | 3: "car", 15 | 4: "motorcycle", 16 | 5: "airplane", 17 | 6: "bus", 18 | 7: "train", 19 | 8: "truck", 20 | 9: "boat", 21 | 10: "traffic light", 22 | 11: "fire hydrant", 23 | 13: "stop sign", 24 | 14: "parking meter", 25 | 15: "bench", 26 | 16: "bird", 27 | 17: "cat", 28 | 18: "dog", 29 | 19: "horse", 30 | 20: "sheep", 31 | 21: "cow", 32 | 22: "elephant", 33 | 23: "bear", 34 | 24: "zebra", 35 | 25: "giraffe", 36 | 27: "backpack", 37 | 28: "umbrella", 38 | 31: "handbag", 39 | 32: "tie", 40 | 33: "suitcase", 41 | 34: "frisbee", 42 | 35: "skis", 43 | 36: "snowboard", 44 | 37: "sports ball", 45 | 38: "kite", 46 | 39: "baseball bat", 47 | 40: "baseball glove", 48 | 41: "skateboard", 49 | 42: "surfboard", 50 | 43: "tennis racket", 51 | 44: "bottle", 52 | 46: "wine glass", 53 | 47: "cup", 54 | 48: "fork", 55 | 49: "knife", 56 | 50: "spoon", 57 | 51: "bowl", 58 | 52: "banana", 59 | 53: "apple", 60 | 54: "sandwich", 61 | 55: "orange", 62 | 56: "broccoli", 63 | 57: "carrot", 64 | 58: "hot dog", 65 | 59: "pizza", 66 | 60: "donut", 67 | 61: "cake", 68 | 62: "chair", 69 | 63: "couch", 70 | 64: "potted plant", 71 | 65: "bed", 72 | 67: "dining table", 73 | 70: "toilet", 74 | 72: "tv", 75 | 73: "laptop", 76 | 74: "mouse", 77 | 75: "remote", 78 | 76: "keyboard", 79 | 77: "cell phone", 80 | 78: "microwave", 81 | 79: "oven", 82 | 80: "toaster", 83 | 81: "sink", 84 | 82: "refrigerator", 85 | 84: "book", 86 | 85: "clock", 87 | 86: "vase", 88 | 87: "scissors", 89 | 88: "teddy bear", 90 | 89: "hair drier", 91 | 90: "toothbrush", 92 | } 93 | 94 | def __init__( 95 | self, 96 | model_path: str = "", 97 | model_name: str = "", 98 | model_description: str = "", 99 | threshold: float = 0.1, 100 | **kwargs, 101 | ): 102 | model_path = ( 103 | model_path 104 | if model_path != "" 105 | else os.path.join(os.path.abspath(os.path.dirname(__file__)), "checkpoints") 106 | ) 107 | super().__init__(model_path, model_name, model_description, **kwargs) 108 | self.threshold = threshold 109 | 110 | @torch.no_grad() 111 | def preprocess(self, data): 112 | preproc_data = [] 113 | for sample in data: 114 | sample = F.to_tensor(sample) 115 | sample = sample.to(device=self.device, non_blocking=True) 116 | preproc_data.append(sample) 117 | return preproc_data 118 | 119 | @torch.no_grad() 120 | def forward(self, data): 121 | result = self.model(data) 122 | return result 123 | 124 | @torch.no_grad() 125 | def postprocess(self, predictions): 126 | if not len(predictions): 127 | return [[]] 128 | predictions = [{k: v.to("cpu") for k, v in t.items()} for t in predictions] 129 | postprocessed_detections = [] 130 | for prediction in predictions: 131 | image_predictions = [] 132 | for box, score, label_id in zip( 133 | prediction["boxes"].tolist(), 134 | prediction["scores"].tolist(), 135 | prediction["labels"].tolist(), 136 | ): 137 | if score > self.threshold: 138 | image_predictions.append( 139 | BBox( 140 | class_name=self.class_names[label_id], 141 | x1=int(box[0]), 142 | y1=int(box[1]), 143 | x2=int(box[2]), 144 | y2=int(box[3]), 145 | score=score, 146 | ), 147 | ) 148 | postprocessed_detections.append(image_predictions) 149 | 150 | return postprocessed_detections 151 | 152 | def model_load(self, device): 153 | self.task_type = TaskType.detection 154 | self.model = torchvision.models.detection.fasterrcnn_resnet50_fpn( 155 | num_classes=91, pretrained=False, pretrained_backbone=False, 156 | ) 157 | 158 | self.model.load_state_dict( 159 | torch.load( 160 | os.path.join( 161 | self.model_path, "fasterrcnn_resnet50_fpn_coco-258fb6c6.pth", 162 | ), 163 | map_location="cpu", 164 | ), 165 | ) 166 | self.model.eval() 167 | if device == Device.cpu: 168 | self.device = "cpu" 169 | elif device == Device.gpu: 170 | self.device = "cuda" 171 | self.model = self.model.to(self.device) 172 | 173 | @torch.no_grad() 174 | def process_sample(self, image): 175 | data = self.preprocess([image]) 176 | output = self.forward(data) 177 | results = self.postprocess(output) 178 | return results[0] 179 | -------------------------------------------------------------------------------- /pytorch_fastercnn/setup.py: -------------------------------------------------------------------------------- 1 | import codecs 2 | import os.path 3 | 4 | from setuptools import setup 5 | 6 | 7 | def get_version(rel_path): 8 | with codecs.open( 9 | os.path.join(os.path.abspath(os.path.dirname(__file__)), rel_path), "r", 10 | ) as fp: 11 | for line in fp.read().splitlines(): 12 | if line.startswith("__version__"): 13 | delim = '"' if '"' in line else "'" 14 | return line.split(delim)[1] 15 | else: 16 | raise RuntimeError("Unable to find version string.") 17 | 18 | 19 | packages = [ 20 | "pytorch_fastercnn", 21 | ] 22 | 23 | package_data = { 24 | "": ["*"], 25 | "pytorch_fastercnn": ["checkpoints/*"], 26 | } 27 | 28 | install_requires = [ 29 | "Pillow>=8.2.0", 30 | "torch==1.5.0", 31 | "torchvision==0.6.0", 32 | "modelplace-api@https://github.com/opencv-ai/modelplace-api/archive/v0.4.12.zip", 33 | ] 34 | 35 | setup_kwargs = { 36 | "name": "pytorch-fastercnn", 37 | "version": get_version("pytorch_fastercnn/__init__.py") + "+gpu", 38 | "description": "", 39 | "long_description": None, 40 | "author": "", 41 | "author_email": "", 42 | "maintainer": "OpenCV.AI", 43 | "maintainer_email": "modelplace@opencv.ai", 44 | "url": None, 45 | "packages": packages, 46 | "package_data": package_data, 47 | "install_requires": install_requires, 48 | "python_requires": ">=3.7,<4.0", 49 | } 50 | 51 | 52 | setup(**setup_kwargs) 53 | -------------------------------------------------------------------------------- /pytorch_fastercnn/setup_cpu.py: -------------------------------------------------------------------------------- 1 | import codecs 2 | import os.path 3 | 4 | from setuptools import setup 5 | 6 | 7 | def get_version(rel_path): 8 | with codecs.open( 9 | os.path.join(os.path.abspath(os.path.dirname(__file__)), rel_path), "r", 10 | ) as fp: 11 | for line in fp.read().splitlines(): 12 | if line.startswith("__version__"): 13 | delim = '"' if '"' in line else "'" 14 | return line.split(delim)[1] 15 | else: 16 | raise RuntimeError("Unable to find version string.") 17 | 18 | 19 | packages = [ 20 | "pytorch_fastercnn", 21 | ] 22 | 23 | package_data = { 24 | "": ["*"], 25 | "pytorch_fastercnn": ["checkpoints/*"], 26 | } 27 | 28 | install_requires = [ 29 | "Pillow>=8.2.0", 30 | "torch==1.5.0+cpu", 31 | "torchvision==0.6.0+cpu", 32 | "modelplace-api@https://github.com/opencv-ai/modelplace-api/archive/v0.4.12.zip", 33 | ] 34 | 35 | setup_kwargs = { 36 | "name": "pytorch-fastercnn", 37 | "version": get_version("pytorch_fastercnn/__init__.py") + "+cpu", 38 | "description": "", 39 | "long_description": None, 40 | "author": "", 41 | "author_email": "", 42 | "maintainer": "OpenCV.AI", 43 | "maintainer_email": "modelplace@opencv.ai", 44 | "url": None, 45 | "packages": packages, 46 | "package_data": package_data, 47 | "install_requires": install_requires, 48 | "python_requires": ">=3.7,<4.0", 49 | } 50 | 51 | setup(**setup_kwargs) 52 | -------------------------------------------------------------------------------- /pytorch_fastercnn/tests/pytorch_fastercnn.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/opencv-ai/modelplace/22e67a27ca0df9c88aef09fb442d17db02ce8907/pytorch_fastercnn/tests/pytorch_fastercnn.jpg -------------------------------------------------------------------------------- /pytorch_fastercnn/tests/pytorch_fastercnn_gt.json: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:6784b44e5cd596bd6022d97894e92b00f9c435761402d8cb9670f9970ea14890 3 | size 9780 4 | -------------------------------------------------------------------------------- /pytorch_fastercnn/tests/test_pytorch_fastercnn.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | import pytest 4 | 5 | import pydantic 6 | import torch 7 | from modelplace_api import Device 8 | from modelplace_api.utils import is_equal 9 | from PIL import Image 10 | 11 | from pytorch_fastercnn import InferenceModel 12 | 13 | BACKEND = Device.gpu if torch.cuda.is_available() else Device.cpu 14 | 15 | 16 | @pytest.fixture() 17 | def prepare_data(): 18 | data_dir = os.path.abspath(os.path.dirname(__file__)) 19 | test_image_path = os.path.join(data_dir, "pytorch_fastercnn.jpg") 20 | test_result_path = os.path.join(data_dir, "pytorch_fastercnn_gt.json") 21 | test_image = Image.open(test_image_path).convert("RGB") 22 | with open(test_result_path, "r") as j_file: 23 | test_result = json.loads(j_file.read()) 24 | return test_image, test_result 25 | 26 | 27 | def test_process_sample(prepare_data): 28 | test_image, test_result = prepare_data 29 | 30 | model = InferenceModel() 31 | model.model_load(BACKEND) 32 | ret = model.process_sample(test_image) 33 | ret = [pydantic.json.pydantic_encoder(item) for item in ret] 34 | assert is_equal(ret, test_result) 35 | 36 | 37 | def test_batch_inference(prepare_data): 38 | test_image, test_result = prepare_data 39 | batch_input = [test_image, test_image, test_image, test_image] 40 | expected_output = [test_result, test_result, test_result, test_result] 41 | 42 | model = InferenceModel() 43 | model.model_load(BACKEND) 44 | preprocessed_data = model.preprocess(batch_input) 45 | forwarded_data = model.forward(preprocessed_data) 46 | postprocessed_data = model.postprocess(forwarded_data) 47 | 48 | results = [[pydantic.json.pydantic_encoder(item) for item in image_result] 49 | for image_result in postprocessed_data] 50 | assert is_equal(results, expected_output) 51 | 52 | 53 | def test_process_empty_input(): 54 | empty_input = Image.new("RGB", (1280, 720), (128, 128, 128)) 55 | 56 | model = InferenceModel() 57 | model.model_load(BACKEND) 58 | ret = model.process_sample(empty_input) 59 | assert ret == [] 60 | -------------------------------------------------------------------------------- /template/setup.py: -------------------------------------------------------------------------------- 1 | import codecs 2 | import os.path 3 | 4 | from setuptools import setup 5 | 6 | 7 | def get_version(rel_path): 8 | with codecs.open( 9 | os.path.join(os.path.abspath(os.path.dirname(__file__)), rel_path), "r", 10 | ) as fp: 11 | for line in fp.read().splitlines(): 12 | if line.startswith("__version__"): 13 | delim = '"' if '"' in line else "'" 14 | return line.split(delim)[1] 15 | else: 16 | raise RuntimeError("Unable to find version string.") 17 | 18 | 19 | packages = [ 20 | "template", 21 | ] 22 | 23 | package_data = { 24 | "": ["*"], 25 | "template": ["checkpoints/*"], 26 | } 27 | 28 | install_requires = [ 29 | "modelplace-api@https://github.com/opencv-ai/modelplace-api/archive/v0.4.7.zip", 30 | ] 31 | 32 | setup_kwargs = { 33 | "name": "template", 34 | "version": get_version("template/__init__.py"), 35 | "description": "", 36 | "long_description": None, 37 | "author": "Author Name", 38 | "author_email": "author.email@example.com", 39 | "maintainer": None, 40 | "maintainer_email": None, 41 | "url": None, 42 | "packages": packages, 43 | "package_data": package_data, 44 | "install_requires": install_requires, 45 | "python_requires": ">=3.7,<4.0", 46 | } 47 | 48 | setup(**setup_kwargs) 49 | -------------------------------------------------------------------------------- /template/template/__init__.py: -------------------------------------------------------------------------------- 1 | __version__ = "0.1.0" 2 | from template.model import InferenceModel 3 | -------------------------------------------------------------------------------- /template/template/checkpoints/.gitignore: -------------------------------------------------------------------------------- 1 | .gitignore 2 | !.gitignore -------------------------------------------------------------------------------- /template/template/model.py: -------------------------------------------------------------------------------- 1 | from typing import Any, List 2 | 3 | from modelplace_api import BaseModel 4 | from modelplace_api.objects import Device 5 | 6 | 7 | class InferenceModel(BaseModel): 8 | def model_load(self, device: Device) -> None: 9 | """ 10 | This function realise model initialization(e.g. weights). 11 | Here you have to create and load your neural network and assign it to 12 | the self.model variable. 13 | 14 | :param device: CPU or GPU (modelplace_api.Device) 15 | """ 16 | raise NotImplementedError 17 | 18 | def preprocess(self, images: List) -> List: 19 | """ 20 | This function preprocesses data for inference. 21 | Should support batch format 22 | 23 | :param images: List of RGB Pillow images 24 | :return: list of preprocessed images(tensors) 25 | """ 26 | raise NotImplementedError 27 | 28 | def forward(self, data: List) -> List: 29 | """ 30 | This function passes preprocessed data through a neural network. 31 | Should support batch format 32 | 33 | :param data: List of preprocessed images 34 | :return: List of inference results 35 | """ 36 | raise NotImplementedError 37 | 38 | def postprocess(self, predictions: List) -> List: 39 | """ 40 | This function postprocesses results. 41 | Should support batch format 42 | 43 | Returned detections and targets should correspond to API 44 | that defined in modelplace_api.objects 45 | 46 | :param predictions: List of inference results 47 | :return: list of postprocessed result 48 | """ 49 | raise NotImplementedError 50 | 51 | def process_sample(self, image: Any) -> Any: 52 | """ 53 | This function process one image through a neural network. 54 | Returned prediction should correspond to API that defined in modelplace_api 55 | 56 | :param image: RGB Pillow image 57 | :return: prediction 58 | """ 59 | raise NotImplementedError 60 | -------------------------------------------------------------------------------- /template/tests/test_model.py: -------------------------------------------------------------------------------- 1 | import pydantic 2 | from modelplace_api import Device 3 | from modelplace_api.utils import is_equal 4 | from template.model import InferenceModel 5 | 6 | test_result = None 7 | test_image = None 8 | 9 | 10 | def test_process_sample(): 11 | model = InferenceModel() 12 | model.model_load(Device.cpu) 13 | ret = model.process_sample(test_image) 14 | ret = [pydantic.json.pydantic_encoder(item) for item in ret] 15 | assert is_equal(ret, test_result) 16 | --------------------------------------------------------------------------------