├── .gitignore ├── ModelDeployment.md └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | *.html 2 | .Rproj.user 3 | .Rhistory 4 | .RData 5 | .Ruserdata 6 | -------------------------------------------------------------------------------- /ModelDeployment.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: ModelDeployment 3 | topic: Model Deployment with R 4 | maintainer: Yuan Tang, James Joseph Balamuta 5 | email: terrytangyuan@gmail.com 6 | version: 2022-08-24 7 | source: https://github.com/cran-task-views/ModelDeployment 8 | --- 9 | 10 | This CRAN task view contains a list of packages, grouped by topic, that 11 | provides functionalities to streamline the process of deploying models 12 | to various environments, such as mobile devices, edge devices, cloud, 13 | and GPUs, for scoring or inferencing on new data. It complements the 14 | related task views on `r view("HighPerformanceComputing")` and 15 | `r view("MachineLearning")`. 16 | 17 | Model deployment is often challenging due to various reasons. Some 18 | example challenges are: 19 | 20 | - It involves deploying models on heterogenous environments, e.g. edge 21 | devices, mobile devices, GPUs, etc. 22 | - It is hard to compress the model to very small size that could fit 23 | on devices with limited storage while keeping the same precision and 24 | minimizing the overhead to load the model for inference. 25 | - Deployed models sometimes need to process new data records within 26 | limited memory on small devices. 27 | - Many deployment environments have bad network connectivity so 28 | sometimes cloud solutions may not meet the requirements. 29 | - There's interest in stronger user data privacy paradigms where user 30 | data does not need to leave the mobile device. 31 | - There's growing demand to perform on-device model-based data 32 | filtering before collecting the data. 33 | 34 | Many of the areas discussed in this task view are undergoing rapid 35 | changes in industries and academia. Please send any suggestions to the 36 | maintainer via e-mail or submit an issue or pull request in the GitHub 37 | repository linked above. All suggestions and corrections by others are 38 | gratefully acknowledged. 39 | 40 | 41 | ### Deployment through different types of artifacts 42 | 43 | This section includes packages that provides functionalities to export 44 | the trained model to an artifact that could fit in small devices such as 45 | mobile devices (e.g. Android, iOS) and edge devices (Rasberri Pi). These 46 | packages are built based on different model format. 47 | 48 | - Predictive Model Markup Language (PMML) is an XML-based language 49 | which provides a way for applications to define statistical and data 50 | mining models and to share models between PMML compliant 51 | applications. The following packages are based on PMML: 52 | - The `r pkg("pmml")` package provides the main 53 | interface to PMML. 54 | - The `r pkg("pmmlTransformations")` package allows 55 | for data to be transformed before using it to construct models. 56 | Builds structures to allow functions in the PMML package to 57 | output transformation details in addition to the model in the 58 | resulting PMML file. 59 | - The `r pkg("arules")` package provides the 60 | infrastructure for representing, manipulating and analyzing 61 | transaction data and patterns (frequent itemsets and association 62 | rules). The associations can be written to disk in PMML. 63 | - The `r pkg("arulesSequences")` package is an add-on 64 | for arules to handle and mine frequent sequences. 65 | - The `r pkg("arulesCBA")` package provides a function 66 | to build an association rule-based classifier for data frames, 67 | and to classify incoming data frames using such a classifier. 68 | - Plain Old Java Object (POJO) or a Model Object, Optimized (MOJO) are 69 | intended to be easily embeddable in any Java environment. The only 70 | compilation and runtime dependency for a generated model is a 71 | h2o-genmodel.jar file produced as the build output of these 72 | packages. The `r pkg("h2o")` package provides 73 | easy-to-use interface to build a wide range of machine learning 74 | models, such as GLM, DRF, and XGBoost models based on 75 | `r pkg("xgboost")` package, which can then be exported 76 | as MOJO and POJO format. The MOJO and POJO artifacts can then be 77 | loaded by its REST interface as well as different language bindings, 78 | e.g. Java, Scala, R, and Python. 79 | - [TensorFlow](https://www.tensorflow.org/)'s 80 | [SavedModel](https://www.tensorflow.org/api_docs/python/tf/saved_model) 81 | as well as its optimized version [TensorFlow 82 | Lite](https://www.tensorflow.org/mobile/tflite/), which uses many 83 | techniques for achieving low latency such as optimizing the kernels 84 | for mobile apps, pre-fused activations, and quantized kernels that 85 | allow smaller and faster (fixed-point math) models. It enables 86 | on-device machine learning inference with low latency and small 87 | binary size. The packages listed below can produce models in this 88 | format. Note that these packages are R wrappers of their 89 | corresponding Python API based on the 90 | `r pkg("reticulate")` package. Though Python binary is 91 | required for creating the models, it's not required during 92 | inference time for deployment. 93 | - The `r pkg("tensorflow")` package provides full 94 | access to TensorFlow API for numerical computation using data 95 | flow graphs. 96 | - The `r pkg("tfestimators")` package provides 97 | high-level API to machine learning models as well as highly 98 | customized neural network architectures. 99 | - The `r pkg("keras")` package high-level API to 100 | construct different types of neural networks. 101 | - The `r pkg("onnx")` package provides the interface to 102 | [Open Neural Network Exchange (ONNX)](https://onnx.ai/) which is a 103 | standard format for models built using different frameworks (e.g. 104 | TensorFlow, MXNet, PyTorch, CNTK, etc). It defines an extensible 105 | computation graph model, as well as definitions of built-in 106 | operators and standard data types. Models trained in one framework 107 | can be easily transferred to another framework for inference. This 108 | open source format enables the interoperability between different 109 | frameworks and streamlining the path from research to production 110 | will increase the speed of innovation in the AI community. Note that 111 | this package is based on the `r pkg("reticulate")` 112 | package to interface with the original Python API so Python binary 113 | is required for deployment. 114 | - The `r pkg("xgboost")` and 115 | `r pkg("lightgbm")` packages can be used to create 116 | gradient-boosted decision tree (GBDT) models and serialize them to 117 | text and binary formats which can be used to create predictions with 118 | other technologies outside of R, including but not limited to 119 | [Apache Spark](https://spark.apache.org/), 120 | [Dask](https://dask.org/), and 121 | [treelite](https://github.com/dmlc/treelite). 122 | 123 | ### Deployment through cloud/server 124 | 125 | Many deployment environments are based on cloud/server. The following 126 | packages provides functionalities to deploy models in those types of 127 | environments: 128 | 129 | - The `r pkg("yhatr")` package allows to deploy, maintain, 130 | and invoke models via the [Yhat](https://www.yhat.com) REST API. 131 | - The `r pkg("cloudml")` package provides functionality to 132 | easily deploy models to [Google Cloud ML Engine](https://cloud.google.com/ml-engine/). 133 | - The `r pkg("tfdeploy")` package provides functions to 134 | run a local test server that supports the same REST API as CloudML 135 | and [RStudio Connect](https://www.rstudio.com/products/connect/). 136 | - The `r pkg("vetiver")` package provides tooling to version, share, 137 | deploy, and monitor a trained model. Functions handle both recording 138 | and checking the model's input data prototype, and predicting from a 139 | remote API endpoint. This package is extensible, with generics to 140 | support many kinds of models. 141 | - The `r pkg("domino")` package provides R interface to 142 | [Domino](https://www.dominodatalab.com/) CLI, a service that makes 143 | it easy to run your code on scalable hardware, with integrated 144 | version control and collaboration features designed for analytical 145 | workflows. 146 | - The `r pkg("tidypredict")` package provides 147 | functionalities to run predictions inside database. It's based on 148 | `r pkg("dplyr")` and `r pkg("dbplyr")` that 149 | could translate data manipulations written in R to database queries 150 | that can be used later to execute the data transformations and 151 | aggregations inside various types of databases. 152 | - The `r pkg("ibmdbR")` package allows many basic and 153 | complex R operations to be pushed down into the database, which 154 | removes the main memory boundary of R and allows to make full use of 155 | parallel processing in the underlying database. 156 | - The `r pkg("sparklyr")` package provides bindings to 157 | [Apache Spark](https://spark.apache.org/)'s distributed machine 158 | learning library and allows to deploy the trained models to 159 | clusters. Additionally, the `r pkg("rsparkling")` 160 | package uses `r pkg("sparklyr")` for Spark job 161 | deployment while using `r pkg("h2o")` package for 162 | regular model building. 163 | - The non-CRAN 164 | [mrsdeploy](https://docs.microsoft.com/en-us/machine-learning-server/r-reference/mrsdeploy/mrsdeploy-package) 165 | package provides functions for establishing a remote session in a 166 | console application and for publishing and managing a web service 167 | that is backed by the R code block or script you provided. 168 | - The `r pkg("opencpu")` package provides a server that 169 | exposes a simple but powerful HTTP API for RPC and data interchange 170 | with R. This provides a reliable and scalable foundation for 171 | statistical services or building R web applications. 172 | - Several general purpose server/client frameworks for R exist that 173 | could help deploy models in server based environments: 174 | - The `r pkg("Rserve")` and 175 | `r pkg("RSclient")` packages both provide server and 176 | client functionality for TCP/IP or local socket interfaces to 177 | enable access to R from many languages and systems. 178 | - The `r pkg("httpuv")` package provides a low-level 179 | socket and protocol support for handling HTTP and WebSocket 180 | requests directly within R. 181 | - Several packages offer functionality for turning R code into a web API: 182 | - The `r pkg("FastRWeb")` package provides some basic 183 | infrastructure for this. 184 | - The `r pkg("plumber")` package allows you to create 185 | a web API by merely decorating your existing R source code with 186 | special comments. 187 | - The `r pkg("RestRserve")` package is a R web API 188 | framework for building high-performance microservices and app 189 | backends based on `r pkg("Rserve")`. 190 | 191 | 192 | ### Links 193 | - Non-CRAN package: [mrsdeploy](https://docs.microsoft.com/en-us/machine-learning-server/r-reference/mrsdeploy/mrsdeploy-package) 194 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## CRAN Task View: Model Deployment with R 2 | 3 | **URL:** 4 | 5 | **Source file:** [ModelDeployment.md](ModelDeployment.md) 6 | 7 | **Contributions:** Suggestions and improvements for this task view are very 8 | welcome and can be made through issues or pull requests here on GitHub or 9 | via e-mail to the maintainer address. For further details see the 10 | [Contributing](https://github.com/cran-task-views/ctv/blob/main/Contributing.md) 11 | guide. All contributions must adhere to the 12 | [code of conduct](https://github.com/cran-task-views/ctv/blob/main/CodeOfConduct.md). 13 | --------------------------------------------------------------------------------