├── FHE ├── BFV_cn.md ├── BGV12_en.md ├── GSW13.md ├── BV11_en.md ├── Bra12_cn.md ├── BGV12_cn.md └── BV11_cn.md ├── .gitignore ├── .gitattributes ├── imgs ├── plug.png ├── res.png ├── seed.png ├── cmake.png ├── config.png ├── digit.png ├── digit2.jpg ├── digit2.png ├── digit3.png ├── select.png ├── upload.png ├── Secp256k1.png ├── atAddress.png ├── callable.png ├── compiled.png ├── cerebroMenu.png ├── confirmation.png ├── cortexlogo.png ├── double_arrow.png ├── fixedPoint.png ├── wallet_top.png ├── CortexTheseus.png ├── push-progress.png ├── testNetDeploy.png ├── torrentSetting.png ├── model-transaction.png ├── twistedEdwardsCurve.png ├── wallet-transaction.png └── push-progress-success.png ├── model-list.jpg ├── mxnetvscvm.png ├── cvm ├── demo │ ├── demo.gif │ └── apple-demo.mp4 └── executor │ ├── CVM Formalization.pdf │ └── CVM Formalization.md ├── FS Module v0.0.2 dev doc.pdf ├── quota.md ├── cvm.md ├── model-data-upload.md ├── model-list.md ├── mrt.md ├── cortex-remix.md ├── storage-layer.md ├── ctxc.md ├── CortexTheseus使用说明.md ├── endorphin.md ├── solidity-data-types.md ├── bug-bounty.md ├── cortex-intro.md ├── README.md ├── zkRollup.md ├── ai-contracts.md ├── ai-dapps.md ├── mining.md ├── CIPs └── cip-1.md ├── miner_dev └── cuckoo_2_sha3_dev.md ├── cortex-guide.md ├── clients.md ├── apple-demo.md ├── operators.md ├── hello-world-tutorial-contract.md └── model-writing-tutorial.md /FHE/BFV_cn.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /FHE/BGV12_en.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | 3 | client/ 4 | ref/ 5 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | *.mp4 filter=lfs diff=lfs merge=lfs -text 2 | -------------------------------------------------------------------------------- /imgs/plug.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/plug.png -------------------------------------------------------------------------------- /imgs/res.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/res.png -------------------------------------------------------------------------------- /imgs/seed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/seed.png -------------------------------------------------------------------------------- /imgs/cmake.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/cmake.png -------------------------------------------------------------------------------- /imgs/config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/config.png -------------------------------------------------------------------------------- /imgs/digit.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/digit.png -------------------------------------------------------------------------------- /imgs/digit2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/digit2.jpg -------------------------------------------------------------------------------- /imgs/digit2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/digit2.png -------------------------------------------------------------------------------- /imgs/digit3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/digit3.png -------------------------------------------------------------------------------- /imgs/select.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/select.png -------------------------------------------------------------------------------- /imgs/upload.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/upload.png -------------------------------------------------------------------------------- /model-list.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/model-list.jpg -------------------------------------------------------------------------------- /mxnetvscvm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/mxnetvscvm.png -------------------------------------------------------------------------------- /cvm/demo/demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/cvm/demo/demo.gif -------------------------------------------------------------------------------- /imgs/Secp256k1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/Secp256k1.png -------------------------------------------------------------------------------- /imgs/atAddress.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/atAddress.png -------------------------------------------------------------------------------- /imgs/callable.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/callable.png -------------------------------------------------------------------------------- /imgs/compiled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/compiled.png -------------------------------------------------------------------------------- /imgs/cerebroMenu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/cerebroMenu.png -------------------------------------------------------------------------------- /imgs/confirmation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/confirmation.png -------------------------------------------------------------------------------- /imgs/cortexlogo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/cortexlogo.png -------------------------------------------------------------------------------- /imgs/double_arrow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/double_arrow.png -------------------------------------------------------------------------------- /imgs/fixedPoint.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/fixedPoint.png -------------------------------------------------------------------------------- /imgs/wallet_top.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/wallet_top.png -------------------------------------------------------------------------------- /imgs/CortexTheseus.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/CortexTheseus.png -------------------------------------------------------------------------------- /imgs/push-progress.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/push-progress.png -------------------------------------------------------------------------------- /imgs/testNetDeploy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/testNetDeploy.png -------------------------------------------------------------------------------- /imgs/torrentSetting.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/torrentSetting.png -------------------------------------------------------------------------------- /imgs/model-transaction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/model-transaction.png -------------------------------------------------------------------------------- /FS Module v0.0.2 dev doc.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/FS Module v0.0.2 dev doc.pdf -------------------------------------------------------------------------------- /imgs/twistedEdwardsCurve.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/twistedEdwardsCurve.png -------------------------------------------------------------------------------- /imgs/wallet-transaction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/wallet-transaction.png -------------------------------------------------------------------------------- /imgs/push-progress-success.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/imgs/push-progress-success.png -------------------------------------------------------------------------------- /cvm/executor/CVM Formalization.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CortexFoundation/tech-doc/HEAD/cvm/executor/CVM Formalization.pdf -------------------------------------------------------------------------------- /cvm/demo/apple-demo.mp4: -------------------------------------------------------------------------------- 1 | version https://git-lfs.github.com/spec/v1 2 | oid sha256:a185afe0ec5608de3adedb0816ffcf23ea7dcfa47aa855286f9eab518a26dbe4 3 | size 50960135 4 | -------------------------------------------------------------------------------- /quota.md: -------------------------------------------------------------------------------- 1 | # Quota 2 | 3 | Quota is a kind of resource like gas. While gas is used to limit the computation consume for transactions in a block, quota is used to limit the space consume. And Quota increase by block mining and cost by uploading files to blockchain. 4 | 5 | You can find `Quota` and `Quota_used` in the block header. `Quota` describes the total quota in the blockchain. `Quota_used` describes the total used quota in the blockchain. 6 | 7 | 8 | 9 | ## Following rules: 10 | 11 | 1. **The Quota of each block increases by NewQuota.** 12 | 13 | CurrentBlock.Quota == ParentBlock.Quota + NewQuota 14 | 15 | NewQuota is set according to the network ID (BLOCK_QUOTA/Bernard_BLOCK_QUOTA/Dolores_BLOCK_QUOTA) 16 | 17 | 2. **The QuotaUsed of each block is less or equal than the Quota in the block** 18 | 19 | Block.QuotaUsed <= Block.Quota 20 | 21 | 3. **The increased QuotaUsed is computed by function ApplyTransaction** 22 | 23 | -------------------------------------------------------------------------------- /cvm.md: -------------------------------------------------------------------------------- 1 | # Cortex Virtual Machine (CVM) 2 | The Cortex Virtual Machine (CVM), is ported from the Ethereum Virtual Machine (EVM) with added support for AI inference. The CVM is backward-compatible with EVM and capable of running both traditional smart contracts and AI smart contracts. 3 | 4 | The CVM has two layers: infer instructions and deterministic inference engine. 5 | 6 | Infer instructions allows models to be called in contracts through instruction sets, including Infer (code: 0xc0), InferArray (code: 0xc1). 7 | 8 | The deterministic inference engine is called Synapse or the CVM Executor. It guarantees the consistency of AI inference results in heterogeneous computing environments, without significantly compromising performance or accuracy. Synapse proposes a model-based fixed-point execution framework and a corresponding deterministic machine learning operator library. AI developers can train and quantize their models using MRT to be executable on CVM. 9 | 10 | -------------------------------------------------------------------------------- /model-data-upload.md: -------------------------------------------------------------------------------- 1 | ## Model and Data Uploading 2 | 3 | There are 3 phases to upload ml models/data on the storage layer of the Cortex blockchain (see [storage layer](storage-layer.md) for more detail): 4 | 5 | ### Upload Phase 6 | 7 | mI models and input data are treated as a special type of smart contract on the Cortex blockchain. Creators need to send a special transaction with a function call to advanced the upload progress. Each transaction will increase the file upload progress by 512K bytes, consuming the corresponding storage quota. 8 | 9 | ### Preparation Phase 10 | 11 | After the completion of the upload phase, the file preparation phase is entered. This phase lasts for 100 blocks (about 25 minutes). 12 | 13 | ### Mature Phase 14 | 15 | After 100 blocks, the prepared files enter the mature phase. The model/data is saved on the storage layer of the Cortex blockchain. After mature phase, the file will completed broadcasting to the network to reach the entire distributed file system; otherwise, the network consensus will reject relevant contract calls. 16 | 17 | -------------------------------------------------------------------------------- /model-list.md: -------------------------------------------------------------------------------- 1 | # Machine Learning Models 2 | 3 | Cortex has 23 models trained with 4 datasets that serve 7 different purposes. All models have been quantized using MRT that is compatible to make inference in CVM, the runtime environment for smart contracts equipped with machine learning models. 4 | 5 | ## Model List 6 | 7 | ![img](model-list.jpg) 8 | 9 | ## Methods 10 | 11 | Converting the original floating-point model to our CVM representation results in approximately 4x model size reduction while the model accuracy does not decrease significantly. Besides, we only introduce minor additional computation overheads, e.g. requantization, keeping the number of operations in the same order of magnitude. All operators in the model can be further optimized using vectorization techniques, which will reduce the computation time dramatically, e.g., avx512-vnni instruction set. 12 | 13 | We apply the proposed converter on pre-trained models with ImageNet dataset from MXNet Model Zoo. The results of top-1 accuracy are shown below, 14 | 15 | ![img](mxnetvscvm.png)*Figure. 3* 16 | 17 | -------------------------------------------------------------------------------- /mrt.md: -------------------------------------------------------------------------------- 1 | # Model Representation Tool 2 | 3 | MRT, short for Model Representation Tool, is a deterministic quantization framework developed by Cortex that enables model inference in the limited-resource and strictly deterministic environment of blockchain, ushering in a new generation of AI smart contracts. MRT is designed to convert floating point models supported by nnvm into fixed-point models executable on the CVM while preventing significant loss of precision. 4 | 5 | The quantization method reduces the output number field of all layers of the model to INT8 or INT32 to simulate the floating-point network and converts the operators involved in the floating-point operation into integer operators using fuse and rewrite. Quantization ensures no overflow and guarantees the deterministic outcome of the model execution. 6 | 7 | Currently, we only support models trained by MXnet framework. 8 | 9 | ## Converting 10 | 11 | 1. Use the MXNet framework to train the floating-point model. 12 | 2. Use MRT to convert your floating-point model to fixed-point model executable on the cvm. 13 | 14 | -------------------------------------------------------------------------------- /cortex-remix.md: -------------------------------------------------------------------------------- 1 | # Cortex Remix 2 | 3 | [Cortex Remix](https://cerebro.cortexlabs.ai/remix) is a browser-based compiler and IDE using programming language Solidity. It is based on [Remix IDE](https://github.com/ethereum/remix-ide). It supports the compilation and deployment of AI smart contracts and debugging transactions. 4 | 5 | Developing on Cortex closely resembles writing Solidity contracts on Ethtereum, but with added instruction set. The instruction sets allows contracts to interact with models and data on Cortex and make inference. 6 | 7 | Cortex Remix is designed to use 8 | 9 | Cortex Remix mainly consists of two functional modules: compilation and deployment. 10 | 11 | ## Compilation 12 | 13 | The compilation module supports compilation and optimization of AI smart contracts. Complied abi, bytecode, and additional information are also displayed in this module. 14 | 15 | ## Deployment 16 | 17 | The deployment module can help deploy AI smart contracts to the Cortex network with the support of Cortex Wallet, allowing for on-chain inference. 18 | 19 | ## Recent Updates： 2022/11/29 20 | soljson has been updated to 0.8.15 . 21 | -------------------------------------------------------------------------------- /storage-layer.md: -------------------------------------------------------------------------------- 1 | # Storage Layer 2 | 3 | Cortex uses a distributed file system based on DHT (Distributed Hash Table) as a storage layer solution to reduce network load and network transmission cost. Storage quota is treated as a resource on the Cortex chain. Each mined block provides a 64K byte storage quota. Users freely bid on the use of storage quota with transaction fees. 4 | 5 | A DHT is a form of a distributed database that can store and retrieve information associated with a key in a network of peer nodes that can join and leave the network at any time. The nodes coordinate among themselves to balance and store data in the network without any central coordinating party. DHT is both fault tolerant and resilient when key/value pairs are replicated. 6 | 7 | The idea of DHT is that each node either stores this key itself, or it is acquainted. It knows some other computer which is closer to this key in terms of the distance on the circle. That way, if a request comes to any node in the network for a key, it either can find this key inside it's own storage or redirect the request to another node which is closer to this key. Thenthat node will either store the key or direct the code to the next node, which is even closer to that key. In finite number of iterations the request will come to the node that will actually stores the key. 8 | -------------------------------------------------------------------------------- /ctxc.md: -------------------------------------------------------------------------------- 1 | # CTXC 2 | 3 | CTXC is the native token used within Cortex. It is used to pay for computational resources within the CVM. CTXC can be stored in Cortex Wallet ([download through official website](https://www.cortexlabs.ai/)) or any of the supported exchanges. 4 | 5 | Cortex charges a fee, known as endorphin, per computational step that is executed in a contract or transaction to prevent deliberate attacks and abuse on the Cortex network (See [endorphine](endorphin.md)). 6 | 7 | ## Getting CTXC 8 | 9 | You can get CTXC with the following methods: 10 | 11 | - Trade through cryptocurrency exchanges ([full list here](https://www.cortexlabs.ai/#exchanges)); 12 | - Become a Cortex Miner (see [mining](mining.md)) 13 | 14 | 15 | ## Fractional Units 16 | 17 | |Value|Unit|Honoring| 18 | |------------|-------|--------------| 19 | |1 CTXC |CTXC|Cortex Coin| 20 | |1e-3 CTXC|minsky|Marvin Lee Minsky, American computer scientist, first artificial neuron network| 21 | |1e-6 CTXC|hinton|Geoffrey Everest Hinton, Canadian computer scientist, first algorithm of deep neuron network training| 22 | |1e-9 CTXC|cajal|Santiago Ramón y Cajal, Spanish neuroscientist, discovery of neuron cells| 23 | |1e-12 CTXC|maxwell|James Clerk Maxwell, Scottish physicist, discovery of electromagnetism, foundation for both real cortex and digital computing| 24 | |1e-15 CTXC|yau|Shing-Tung Yau, Chinese mathematician, discovery of the Calabi–Yau manifold, extra dimensions for the modern string theory, theory of everything| 25 | |1e-18 CTXC|turing|Alan Mathison Turing, English cryptographer, the computational complexity theory, needed by both GUT and modern AI theory| -------------------------------------------------------------------------------- /FHE/GSW13.md: -------------------------------------------------------------------------------- 1 | # Homomorphic Encryption from Learning with Errors: Conceptually-Simpler, Asymptotically-Faster, Attribute-Based 2 | 3 | ## Basic Introduction 4 | 5 | secret key: $\textbf{v} \in \mathbb{Z}_q^N$ 6 | 7 | plaintext: $\mu_i \in \{0,1\}$ 8 | 9 | ciphertext: $C_i \in \mathbb{Z}_q^{N \times N}$ with “small” entries (much smaller than q) 10 | 11 | error: $e$ 12 | 13 | $C_i \cdot \textbf{v} \approx \mu_2 \cdot \textbf{v}$ 14 | 15 | $C_i \cdot \textbf{v} = \mu_2 \cdot \textbf{v} + \textbf{e}$ 16 | 17 | $(C_1 + C_2) \cdot \textbf{v} = (\mu_1 + \mu_2) \cdot \textbf{v} + \textbf{e}_1 + \textbf{e}_2$ 18 | 19 | $(C_1 \cdot C_2) \cdot \textbf{v} = C_1 \cdot (\mu_2 \cdot \textbf{v} + \textbf{e}_2) = \mu_1 \cdot \mu_2 \cdot \textbf{v} + \mu_2 \cdot \textbf{e}_1 + C_1 \cdot \textbf{e}_2$ 20 | 21 | Assumption: $\mu_2, C_1$ are small 22 | 23 | Lattice Gadget: 24 | 25 | 1. Let $\vec{a}, \vec{b}$ be vectors of some dimension $k$ over $\mathbb{Z}_q$, $l=\lfloor log_2q \rfloor + 1$ and $N = k \cdot l$. 26 | 27 | $\textbf{BitDecomp}(\vec{a}) = (a_{1,0},...,a_{1,l-1},...,a_{k,0},...,a_{k,l-1})$, where $a_{i,j}$ is the j-th bit in $a_i$’s binary representation 28 | 29 | 2. For $\vec{a^{'}} = (a_{1,0},...,a_{1,l-1},...,a_{k,0},...,a_{k,l-1}) \in \mathbb{Z}_q^N$, let $\textbf{BitDecomp}^{-1}(\vec{a^{'}}) = \sum{2^j \cdot a_{1,j} + ... + \sum{2^j \cdot a_{k,j}}}$ 30 | 31 | 3. For N-demensional $\vec{a^{'}}$, let $\textbf{Flatten}(\vec{a^{'}}) = \textbf{BitDecomp}(\textbf{BitDecomp}^{-1}(\vec{a^{'}}))$ 32 | 33 | 4. For N-demensional $\vec{b}$, let $\textbf{Powerof2}(\vec{b}) = (b_1, 2b_1,..., 2^{l-1}b_1,...,b_k,2b_k,...2^{l-1}b_k)$ 34 | 35 | 5. 36 | 37 | 38 | 39 | ## Basic Encryption Scheme 40 | 41 | * $\textbf{Setup}(1^\lambda,1^L)$ 42 | * $\textbf{SecretKeyGen}(params)$ 43 | 44 | -------------------------------------------------------------------------------- /CortexTheseus使用说明.md: -------------------------------------------------------------------------------- 1 | # **Ubuntu** 2 | 3 | ## 基础环境 (NVIDIA GeForce GTX 1080 Ti is needed) 4 | 5 | #### `cmake 3.11.0+` 6 | 7 | ```shell 8 | wget https://cmake.org/files/v3.11/cmake-3.11.0-rc4-Linux-x86_64.tar.gz 9 | tar zxvf cmake-3.11.0-rc4-Linux-x86_64.tar.gz 10 | sudo mv cmake-3.11.0-rc4-Linux-x86_64 /opt/cmake-3.11 11 | sudo ln -sf /opt/cmake-3.11/bin/* /usr/bin/ 12 | ``` 13 | 14 | #### **`go 1.16+`** 15 | 16 | ```shell 17 | wget https://dl.google.com/go/go1.16.linux-amd64.tar.gz 18 | sudo tar -C /usr/local -xzf go1.16.linux-amd64.tar.gz 19 | echo 'export PATH="$PATH:/usr/local/go/bin"' >> ~/.bashrc 20 | source ~/.bashrc 21 | ``` 22 | 23 | #### **`gcc/g++ 5.4+`** 24 | 25 | ```shell 26 | sudo apt install gcc 27 | sudo apt install g++ 28 | ``` 29 | 30 | #### **`cuda-9.2`** 31 | 32 | [下载地址](https://developer.nvidia.com/cuda-toolkit-archive) 33 | 34 | ```shell 35 | export LD_LIBRARY_PATH=/usr/local/cuda-9.2/lib64/:/usr/local/cuda-9.2/lib64/stubs:$LD_LIBRARY_PATH 36 | export LIBRARY_PATH=/usr/local/cuda-9.2/lib64/:/usr/local/cuda-9.2/lib64/stubs:$LIBRARY_PATH 37 | ``` 38 | 39 | ![image-20210325151317036](/Users/yangchunxueya/Library/Application Support/typora-user-images/image-20210325151317036.png) 40 | 41 | #### **nvidia驱动下载** 42 | 43 | [下载地址](https://www.nvidia.cn/Download/index.aspx?lang=cn) 44 | 45 | 根据显卡类型进行下载安装。 46 | 47 | 48 | 49 | ## 代码 50 | 51 | #### 编译代码 52 | 53 | ```shell 54 | git clone --recursive https://github.com/CortexFoundation/CortexTheseus.git 55 | cd CortexTheseus 56 | make clean && make -j$(nproc) 57 | 58 | make mine 59 | ``` 60 | 61 | #### 创建账户 62 | 63 | ```shell 64 | ./build/bin/cortex account new 65 | ``` 66 | 67 | > 记录地址，下面会使用到 68 | 69 | #### 开始运行 70 | 71 | ```shell 72 | ./build/bin/cortex --mine --miner.threads=1 --miner.coinbase="{you account}" --miner.cuda 73 | ``` 74 | 75 | -------------------------------------------------------------------------------- /endorphin.md: -------------------------------------------------------------------------------- 1 | # Endorphin 2 | 3 | Endorphin is the execution fee that senders of transactions need to pay for every operation made on Cortex. Every transaction is required to include an endorphin limit and a fee that it is willing to pay per Endorphin. Miners have the choice of including the transaction and collecting the fee or not. 4 | 5 | In contrast to the traditional blockchain, in which the reward for each packaging block is paid directly to the miners, a portion of the reward goes to the model provider to motivate developers to submit richer and better models. Endorphins invoking the contract, are not only allocated to miners who help package the block on the full node, but also used to pay the model providers. 6 | 7 | ## Mechanism 8 | 9 | The proportion of fees charged will be determined by the market gaming price, similar to the Ethereum Gas mechanism. AI smart contracts having a higher max Endorphins price have a higher priority to be executed. A single execution of a contract will cause a fee of Endorphins Limit * max Endorphin Price. Endorphin is a measurement of the number of computing resources spent on a hardware level within a virtual machine when bringing a data model into a contract during inference. Generally speaking, the cost of the Endorphin is proportional to the size of the model. Cortex also sets an upper bound of 1GB on the parameter size of the model, corresponding to up to about 500 million Float32 parameters. 10 | 11 | ## Estimating transaction costs 12 | 13 | The total ether cost of a transaction is based on 2 factors: 14 | 15 | `endorphinLimit` is the total endorphin that is consumed by the transaction 16 | 17 | `endorphinPrice` price (in ctxc) of one unit of endorphin specified in the transaction 18 | 19 | Transaction fee = endorphinLimit * endorphinPrice 20 | 21 | The higher the endorphin price, the more likely and quicker miners will execute and verify the transaction. -------------------------------------------------------------------------------- /solidity-data-types.md: -------------------------------------------------------------------------------- 1 | # Data Types 2 | 3 | Solidity is a statically typed language, which means that the type of each variable (state and local) needs to be specified. Solidity provides several elementary types which can be combined to form complex types. 4 | 5 | Some of the basic data types offered in Solidity: 6 | 7 | - Boolean (bool) 8 | 9 | Boolean value, true or false, with logical operators ! (not), && (and), || (or), == (equal), and != (not equal). 10 | 11 | - Integer (int, uint) 12 | 13 | Signed (int) and unsigned (uint) integers, declared in increments of 8 bits from int8 to uint256. Without a size suffix, 256-bit quantities are used, to match the word size of the EVM. 14 | 15 | - Fixed point (fixed, ufixed) 16 | 17 | Fixed-point numbers, declared with (`u`)`fixed*M*x*N*` where *M* is the size in bits (increments of 8 up to 256) and *N* is the number of decimals after the point (up to 18); e.g., ufixed32x2. 18 | 19 | - Address 20 | 21 | A 20-byte Cortex address. The address object has many helpful member functions, the main ones being balance (returns the account balance) and `transfer` (transfers ether to the account). 22 | 23 | - Byte array (fixed) 24 | 25 | Fixed-size arrays of bytes, declared with bytes1 up to bytes32. 26 | 27 | - Byte array (dynamic) 28 | 29 | Variable-sized arrays of bytes, declared with bytes or string. 30 | 31 | - Enum 32 | 33 | User-defined type for enumerating discrete values: enum NAME {LABEL1, LABEL 2, ...}. 34 | 35 | - Arrays 36 | 37 | An array of any type, either fixed or dynamic: uint32[][5] is a fixed-size array of five dynamic arrays of unsigned integers. 38 | 39 | - Struct 40 | 41 | User-defined data containers for grouping variables: `struct NAME {TYPE1 VARIABLE1; TYPE2 VARIABLE2; ...}`. 42 | 43 | - Mapping 44 | 45 | Hash lookup tables for *key* => *value* pairs: mapping(KEY_TYPE => VALUE_TYPE) NAME. 46 | 47 | In addition to these data types, Solidity also offers a variety of value literals that can be used to calculate different units: 48 | 49 | - Time units 50 | 51 | The units seconds, minutes, hours, and days can be used as suffixes, converting to multiples of the base unit seconds. 52 | 53 | - CTXC units 54 | 55 | The [units](ctxc.md) can be used as suffixes, converting to multiples of the base unit endorphin. 56 | 57 | ## Reference 58 | 59 | - [Solidity Docs](https://solidity.readthedocs.io/en/latest/) 60 | - [Mastering Ethereum](https://github.com/ethereumbook/ethereumbook/blob/develop/07smart-contracts-solidity.asciidoc) 61 | 62 | -------------------------------------------------------------------------------- /bug-bounty.md: -------------------------------------------------------------------------------- 1 | ### Cortex Bug Bounty Program 2 | Cortex is the first public blockchain capable of executing AI algorithms and AI DApps on the blockchain. Cortex provides an AI platform for developers to upload their models on the blockchain and be incorporated into smart contracts. Instead of a black box, we can execute AI models on the blockchain in a decentralized, immutable, and transparent manner − network consensus verifies every step of the AI inference. 3 | To put it in context, blockchain started with bitcoin, a decentralized digital currency. Then entered Ethereum, which allows programming on top of the blockchain, namely the smart contract. Now Cortex builds on top of Ethereum to enable AI-powered smart contracts. 4 | We are launching a bug bounty program to call upon security experts around the world to make the Cortex ecosystem more secure! 5 | 6 | ### Program Rules 7 | Please provide detailed reproducible steps. 8 | Only the first report will be awarded in the case of duplicates. 9 | Only one bounty will be rewarded for multiple vulnerabilities caused by the same underlying issue. 10 | Bugs are rewarded according to severity levels (see below) 11 | The Cortex team reserves the right to classify a bug's severity level, determine the final awards, modify and cancel the bounty program at any time. 12 | 13 | ### Rewards 14 | Bug level is categorized by Bugcrowd Vulnerability Rating Taxonomy (https://bugcrowd.com/vulnerability-rating-taxonomy) 15 | Rewards will be paid out in CTXC. 16 | Please provide your CTXC address - make sure it is a MainNet address, especially if it is an exchange address (not recommended). 17 | If a vulnerability is found to be extremely critical, the reward may be greater than listed below on a case-by-case basis. 18 | Technical Severity Reward Range in CTXC 19 | P1 - Critical 50,000 - 100,000 20 | P2 - Severe 10,000 - 50,000 21 | P3 - Moderate 6,000 - 15,000 22 | P4 - Low 2,000 - 6,000 23 | 24 | ### Documentation 25 | - Master documentation (https://github.com/CortexFoundation/tech-doc) 26 | - Set up full nodes (https://github.com/CortexFoundation/CortexTheseus) 27 | - AI smart contract tutorial (https://github.com/.../hello-world-tutorial-contract.md) 28 | - AI model upload tutorial (https://github.com/.../blob/master/model-writing-tutorial.md) 29 | 30 | ### Disclosure Policy 31 | Please do not discuss the vulnerabilities outside of this bounty program without explicit consent from the core Cortex team. 32 | 33 | ------------------------------------------------------------------ 34 | 35 | Thank you for making Cortex more secure and pushing forward the AI on Blockchain Ecosystem! Note that this is only version 1.0 of the bounty program - upon feedback, we will look to modify or expand the bounty program. 36 | -------------------------------------------------------------------------------- /cortex-intro.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | The current challenge to execute machine learning programs on the conventional blockchain is that the virtual machine is extremely inefficient when running any nontrivial machine learning models. Therefore, most people think it is impossible to run AI on the blockchain. 4 | 5 | To the surprise of many, Cortex has built a blockchain whose virtual machine (CVM) utilizes the GPU, which can execute AI programs on-chain, and is compatible with the EVM. In other words, the Cortex chain can execute all the Ethereum Dapps, and on top of that, incorporate machine learning into these Dapps. For the first time, instead of a black box, we can run ml models on the blockchain in a decentralized, immutable, and transparent manner − network consensus verifies every step of the AI inference. 6 | 7 | ## The Cortex Ecosystem - An Open Decentralized AI Ecosystem 8 | 9 | Besides Dapp developers, users, and miners, Cortex brings a new relevant party into its blockchain ecosystem: AI developers. 10 | AI developers upload trained machine learning models onto the storage layer of the Cortex chain. These models can later be either inferred on their own or incorporated into smart contracts by Dapp developers. Each time an AI model is called, it's model uploader gets rewards in CTXCs, coming from a part of the network transaction fees. 11 | This mechanism incentivizes AI developers to train and upload better models, leading to an evolving decentralized AI ecosystem; good models are openly distributed on the Cortex blockchain instead of being monopolized by a few big corporations. 12 | 13 | #### AI Smart Contract and AI DApp 14 | 15 | Cortex adds machine learning support to smart contracts and DApps to adapt to real-world use cases. Developers can use the most popular language, Solidity, to create truly intelligent smart contracts and AI-enhanced DApps. 16 | 17 | #### AI Inference Engine - Synapse 18 | 19 | A deterministic inference engine that guarantees exactly the same result in heterogenetic computing environments. The inference result no longer comes from a third-party, eliminating the trust problem. 20 | 21 | #### Decentralizing AI Resarch 22 | 23 | Foster a world of open-source models where AI technology and the associated algorithms are made open and controlled democratically. 24 | 25 | #### Cortex Virtual Machine 26 | 27 | The CVM is EVM-compatible with added AI features that utilizes the GPU, allowing developers to use Solidity language with unique infer instructions to call the algorithms. 28 | 29 | #### Cuckoo Cycle PoW Algorithm 30 | 31 | Most promising ASIC-resistant PoW algorithm to establish a friendly mining environment. Anti-ASIC is a firm belief in Cortex in order to lower the mining requirements to lay the foundation for future scalability in AI computing demand market. 32 | 33 | #### Cortex MRT 34 | 35 | A deterministic quantization framework to convert MXNet models to fixed-point models that are executable on the Cortex blockchain. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Cortex Documentation 2 | 3 | ## Introduction 4 | 5 | The current challenge to execute machine learning programs on the conventional blockchain is that the virtual machine is extremely inefficient when running any nontrivial machine learning models. Therefore, most people think it is impossible to run AI on the blockchain. 6 | 7 | Cortex has built a blockchain whose virtual machine (CVM) utilizes the GPU, which can execute AI programs on-chain, and is compatible with the EVM. In other words, the Cortex chain can execute all the Ethereum Dapps, and on top of that, incorporate machine learning into these Dapps. For the first time, instead of a black box, we can run ml models on the blockchain in a decentralized, immutable, and transparent manner − network consensus verifies every step of the AI inference. 8 | 9 | ## Basic Concepts 10 | 11 | - [Introduction](cortex-intro.md) - Introduction to Cortex 12 | - [Details About Cortex](cortex-details.md) - Details about the Cortex Project 13 | - [Hello World Tutorial](hello-world-tutorial-contract.md) - Get Started Calling an AI Model in a Smart Contract 14 | - [Cortex Clients](clients.md) - Official golang implementation of the CortexTheseus protocol 15 | - [Mining](mining.md) - Mining instruction and PoW algorithm explained (https://www.cortexlabs.ai/forum/topic/72/radiance-cortex-gpu-nvidia-8g-miner-v1-0-0/2) 16 | - [Cortex Virtual Machine](cvm.md) - Explanation on CVM 17 | - [AI Contract](ai-contracts.md) - Instruction on the AI contract development 18 | - [Cortex Remix](cortex-remix.md) - Instruction on Cortex browser-based compiler and IDE 19 | - [Operators](operators.md) - A list of possible operators to interact with ml models 20 | - [MRT](mrt.md) - A deterministic quantization framework that enables model inference on Cortex blockchain 21 | - [Model List](model-list.md) - 13 pre-loaded models by Cortex team 22 | - [Model and Data Upload](model-data-upload.md) - Instruction on how to upload model and data 23 | - [CTXC](ctxc.md) - CTXC and fractional units explained 24 | - [Quota](quota.md) - Quota and related rules description 25 | - [AI DApp](ai-dapps.md) - Instruction and tools for AI DApp development 26 | - [Endorphin](endorphin.md) - Explanation on the fee structure and mechanism 27 | - [Storage Layer](storage-layer.md) - Explanation on storage stack and how models/data are uploaded 28 | - [Wiki](https://github.com/ghlai9665/cortex-wiki) You can find json-rpc here 29 | - [Cuckoo](https://github.com/CortexFoundation/PoolMiner/blob/dev/README.md) cuckoo cycle by cortex 30 | - [Resource list](https://github.com/chainrg/reading) some sources of blockchain 31 | 32 | ## Getting Started 33 | 34 | For information on Cortex setup, installation, bbuild, configuration, and usage, please see [Cortex Clients Guide](clients.md). To get started for mining, please see [Mining Guide](mining.md). 35 | 36 | ## Need Help? 37 | 38 | Besides enabling AI smart contracts, the other part of the Cortex project is the building of an open decentralized AI ecosystem on the blockchain. We have established a developer forum to curate a community to discuss and get helps. 39 | 40 | Visit our [Official Forum](https://www.cortexlabs.ai/forum/) to get answers to common questions. 41 | 42 | -------------------------------------------------------------------------------- /zkRollup.md: -------------------------------------------------------------------------------- 1 | # zkRollup 2 | 3 | 这个文档主要介绍zkRollup用到的密码学组件：secp256k1，baby_jubjub和alt_baby_jubjubu加密方法 4 | 5 | ## Secp256k1 6 | 7 | 非对称加密的方案都是基于有限域上对数难问题来构造的。传统的RSA加密方案采用整数有限域，而椭圆曲线加密方案采用的二维点的有限域，有限域上的所有点都满足一类特殊形式的椭圆曲线：$y^2=x^3+ax+b \mod p$。椭圆曲线方案在同样密码安全性下比RSA方案更高效。 8 | 9 | Secp256k1特指一条椭圆曲线：$y^2 = x^3 + 7$，这条椭圆曲线最早被比特币采用。 10 | 11 | 椭圆曲线里面的基本元素不是数而是点（$x,y$），因此加法和乘法也需要重新定义。形象来说，两个点的加法，就是两个点的连线，与椭圆曲线的交点关于x轴的对称点。如下图所示，因为整个曲线关于x轴对称，所以结果也在椭圆曲线上。每个点加自己就是关于该点的切线与椭圆曲线的交点关于x轴的对称点。 12 | 13 | ![Secp256k1](imgs/Secp256k1.png) 14 | 15 | 从一个点出发，不断加自己，可以遍历整个有限域，这个域就是循环域。出发点就是生成元 $G$。 16 | 17 | 椭圆曲线的密码参数 $T = (p,a,b,G,n,h)$ 18 | 19 | - $p$ 是椭圆曲线数的有限域的大小，这里是一个大素数 $p = 2^{256} - 2^{32} - 2^9 - 2^8 - 2^7 - 2^6 - 2^4 - 1$ 20 | 21 | - $a,b$ 是椭圆曲线的参数，这里 $a = 0, b = 7$ 22 | 23 | - 生成元 $G$ 的压缩形式是 $G = 02\ 79BE667E\ F9DCBBAC\ 55A06295\ CE870B07\ 029BFCDB\ 2DCE28D9\ 59F2815B\ 16F81798$， 24 | 25 | 完整形式是$G = 04\ 79BE667E\ F9DCBBAC\ 55A06295\ CE870B07\ 029BFCDB\ 2DCE28D9\ 59F2815B\ 16F81798\ 483ADA77\ 26A3C465\ 5DA4FBFC\ 0E1108A8\ FD17B448\ A6855419\ 9C47D08F\ FB10D4B8$ 26 | 27 | 区别应该就是压缩形式记录 $y$ 然后代入函数计算 $x$，完整形式直接存 $x$ 和 $y$。 28 | 29 | - $n = FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFE BAAEDCE6 AF48A03B BFD25E8C D0364141$ 30 | 31 | - $h = 01$ 32 | 33 | 34 | 35 | ## baby_jubjub 36 | 37 | Baby jubjub 是另一条椭圆曲线，[EIP-2494](https://eips.ethereum.org/EIPS/eip-2494) 提出的。 38 | 39 | zk-SNARK 通过对链下计算提供零知识证明能提升以太坊性能，并且保障隐私性。zk-SNARK协议（ [[Pinnochio\]](https://eprint.iacr.org/2013/279.pdf) 和 [[Groth16\]](https://eprint.iacr.org/2016/260.pdf)）在证明和验证的时候都需要椭圆曲线运算，需要曲线满足有限域 $F_r$，以太坊采用alt_bn128(BN254)曲线，不支持zk-SNARK的ECC计算。而 Baby jubjub 的特点在于，专门为zk-SNARK计算设计的。 40 | 41 | Baby jubjub 属于一类特殊的椭圆曲线，扭曲爱德华滋曲线：$ax^2 + y^2 = 1 + dx^2y^2$，如下图所示，这个曲线关于 x 轴和 y 轴都对称。 42 | 43 | ![twistedEdwardsCurve](imgs/twistedEdwardsCurve.png) 44 | 45 | 参数如下： 46 | 47 | - $r = 21888242871839275222246405745257275088548364400416034343698204186575808495617$ 48 | - $a = 168700$ 49 | - $d = 168696$ 50 | - $n = 21888242871839275222246405745257275088614511777268538073601725287587578984328$ 51 | - $n = h \times l$，$l$ 是素数 52 | - $h = 8$ 53 | - $l = 2736030358979909402780800718157159386076813972158567259200215660948447373041$ 54 | - $G = (x,y)$ 55 | - $x = 995203441582195749578291179787384436505546430278305826713579947235728471134$ 56 | - $y = 5472060717959818805561601436314318772137091100104008585924551046643952123905$ 57 | - $B = (x,y) = l \times G$ 58 | - $x = 5299619240641551281634865583518297030282874472190772894086521144482721001553$ 59 | - $y = 16950150798460657717958625567821834550301663161624707787222815936182638968203$ 60 | 61 | 计算： 62 | 63 | - $P1 = (x1, y1)$，$P2 = (x2, y2)$，$P1 + P2 = (x3, y3)$ 64 | 65 | - $x3 = (x1*y2 + y1*x2)/(1 + d*x1*x2*y1*y2)$ 66 | - $y3 = (y1*y2 - a*x1*x2)/(1 - d*x1*x2*y1*y2)$ 67 | 68 | 实现： 69 | 70 | - Python: https://github.com/barryWhiteHat/baby_jubjub_ecc 71 | - JavaScript: https://github.com/iden3/circomlib/blob/master/src/babyjub.js 72 | - Circuit (circom): https://github.com/iden3/circomlib/blob/master/circuits/babyjub.circom 73 | - Rust: https://github.com/arnaucube/babyjubjub-rs 74 | - Solidity: https://github.com/yondonfu/sol-baby-jubjub 75 | - Go: https://github.com/iden3/go-iden3-crypto/tree/master/babyjub 76 | 77 | ## alt_baby_jubjub 78 | 79 | 80 | 81 | -------------------------------------------------------------------------------- /ai-contracts.md: -------------------------------------------------------------------------------- 1 | # AI Contracts 2 | 3 | The idea of smart contracts is not originated by Cortex development team but by cryptographer Nick Szabo in 1994. Ethereum later implemented a nearly Turing-complete language on its blockchain and popularized the usage of smart contracts. Cortex adds machine learning operators to allow various interactions within the smart contracts that is executable on CVM without losing performance and precision. 4 | 5 | ## Solidity 6 | 7 | Cortex officially supports Solidity programming language to develop AI contracts and complie to CVM bytcode. 8 | 9 | - [Solidity Docs](https://solidity.readthedocs.org/en/latest/) - Solidity is the Smart Contract language developed by Ethereum, which compiles to CVM (Cortex Virtual Machine) codes. 10 | - [Cortex-Remix](https://cerebro.cortexlabs.ai/remix) - a browser-based compiler and IDE using programming language Solidity. 11 | 12 | ## Calling On-Chain Model 13 | 14 | Function infer(model, input, infer_output) 15 | 16 | - Parameters 17 | 18 | 1. model - address of the model 19 | 2. input - address of the data 20 | 3. infer_output - array of uint256, receive infer output 21 | 22 | - Returns 23 | None 24 | 25 | Function inferArray(model, input_data, infer_output) 26 | 27 | - Parameters 28 | 29 | 1. model - address of the model 30 | 2. input_data - storage array of uint256 31 | 3. infer_output - array of uint256, receive infer output 32 | 33 | - Returns 34 | None 35 | 36 | 37 | ## Fractional Units for Cortex 38 | 39 | Refer to [CTXC](ctxc.md) 40 | 41 | ## A Simple Smart Contract Example 42 | 43 | This contract illustrate a simple implementation of calling and inferring a model on the Cortex blockchain. 44 | 45 | ```js 46 | pragma solidity ^0.4.18; 47 | contract AIContract { 48 | uint256[] input_data; 49 | uint256[] infer_output = new uint256[](uint256((1 * 10 + 31) >> 5)); 50 | 51 | constructor() public { 52 | input_data = new uint256[]((1 * 3 * 32 * 32 + 31) >> 5); 53 | } 54 | 55 | function Infer(address model, address input) public returns (uint256) { 56 | // feed data in input to model and store the output in infer_output 57 | infer(model, input, infer_output); 58 | return infer_output[0]; 59 | } 60 | 61 | function InferArray(address model) public returns (uint256) { 62 | // feed data in input_data to model and store the output in infer_output 63 | inferArray(model, input_data, infer_output); 64 | return infer_output[0]; 65 | } 66 | } 67 | ``` 68 | 69 | The first line tells the source code is written for Solidty version 0.4.18 or newer. The `pragma` command are used by the compiler to check compatability, while the caret symbol (^) means any *minor* version above 0.4.18, e.g., 0.4.19, but not 0.5.0. 70 | 71 | In Solidity, the data types of both the function parameters and output need to be specified. In the first case, the function `infer` takes `address model`, `address input`, `infer_output`, and returns the infer result stored in infer_output[0]. While `inferArray` take uint256 storage array as input data for the model. It is to ensure enough length to read data stored on the Cortex blockchain. The size of uint256 is equal to 32 int8. 72 | 73 | `infer` and `inferArray` calls the on-chain model with paramemters model, input and output. When executed, the CVM will call the data and model on the Cortex blockchain, make inference, and set the result in the output, namely the first element of `output` array. 74 | 75 | -------------------------------------------------------------------------------- /FHE/BV11_en.md: -------------------------------------------------------------------------------- 1 | # Efficient Fully Homomorphic Encryption from (Standard) LWE 2 | 3 | ## Security Assumption 4 | 5 | LWE problem(Learning With Error) 6 | 7 | ## Scheme Brief Description 8 | 9 | Plaintext: $ m \in \{0, 1\}$ 10 | 11 | Secret key: $\textbf{s} \in \mathbb{Z}_q^n$ 12 | 13 | Random vector: $\textbf{a} \in \mathbb{Z}_q^n$ 14 | 15 | Ciphertext: $c =(\textbf{a},b) \in \mathbb{Z}_q^n \times \mathbb{Z}_q$ 16 | 17 | Encryption function: $b=\langle\textbf{a},\textbf{s}\rangle+2e+m \in \mathbb{Z}_q$ 18 | 19 | Decryption fuction: $f_{\textbf{a},b}(\textbf{x})=b-\langle{\textbf{a},\textbf{x}}\rangle (mod\ q)=b-\sum_{i=1}^n{\textbf{a}[i]\cdot\textbf{x}[i]} \in \mathbb{Z}_q$ 20 | 21 | ## Homomorphic addition 22 | 23 | $$ 24 | f_{\textbf{a}+\textbf{a}^{'},b+b^{'}}(\textbf{x})=b+b^{'}-\langle{\textbf{a}+\textbf{a}^{'},\textbf{x}}\rangle (mod\ q)=f_{\textbf{a},b}(\textbf{x})+f_{\textbf{a}^{'},b^{'}}(\textbf{x}) 25 | $$ 26 | 27 | The homomorphic addition can be computed directly. 28 | 29 | ## Homomorphic multiplication 30 | 31 | $$ 32 | f_{\textbf{a},b}(\textbf{x})\cdot f_{\textbf{a}^{'},b^{'}}(\textbf{x}) = \left(b-\sum{\textbf{a}[i]\textbf{x}[i]}\right)\cdot \left(b^{'}-\sum{\textbf{a}^{'}[i]\textbf{x}[i]}\right)\\ = h_0+\sum{h_i\cdot \textbf{x}[i]}+\sum{h_{i,j}\cdot \textbf{x}[i]\textbf{x}[j]} 33 | $$ 34 | 35 | The decryption algorithm has to know all the coefficients of this quadratic polynomial, which means that the size of the ciphertext just went up from $n+1$ elements to (roughly) $n^2/2$ 36 | 37 | ### Re-linearization technique 38 | 39 | Aim: Reduce the size of the ciphertext back down to $n+1$ 40 | 41 | Idea: Imagine that we publish “encryptions” of all the linear and quadratic terms in the secret key $s$, namely all the numbers $s[i]$ as well as $s[i]s[j]$, under a new secret key $t$. 42 | 43 | New ciphertexts: 44 | 45 | $b_{i}=\langle\textbf{a}_{i},\textbf{t}\rangle + 2e_{i} + s[i] \approx \langle\textbf{a}_{i},\textbf{t}\rangle + s[i] \in \mathbb{Z}_q$ 46 | 47 | $b_{i,j}=\langle\textbf{a}_{i,j},\textbf{t}\rangle + 2e_{i,j} + s[i]s[j] \approx \langle\textbf{a}_{i,j},\textbf{t}\rangle + s[i]s[j] \in \mathbb{Z}_q$ 48 | 49 | Now, the sum $h_0+\sum{h_i\cdot \textbf{x}[i]}+\sum{h_{i,j}\cdot \textbf{x}[i]\textbf{x}[j]}$ can be written (approximately) as 50 | 51 | $h_0+\sum_i{h_i\cdot (b_i - \langle{\textbf{a}_i},\textbf{t}\rangle)}+\sum_{i,j}{h_{i,j}\cdot (b_{i,j}-\langle{\textbf{a}_{i,j},\textbf{t}}\rangle)}$ 52 | 53 | which is a linear function of $t$ after simplification. 54 | 55 | A “chain” of L secret keys (together with encryptions of quadratic terms of one secret key using the next secret key) allows us to perform up to L levels of multiplications without blowing up the ciphertext size. 56 | 57 | #### Flatten(described in the [GSW13](GSW13.md)) 58 | 59 | Consider the binary representation of $h_{i,j}$, namely $h_{i,j} = \sum_{\tau=0}^{\lfloor{log\ q}\rfloor}h_{i,j,\tau}2^{\tau} \textbf{s}[i]\cdot \textbf{s}[j]$ 60 | 61 | For each value of $\tau$, we have a pair $(\textbf{a}_{i,j,\tau},b_{i,j,\tau})$ such that 62 | 63 | $b_{i,j,\tau} = \langle{\textbf{a}_{i,j,\tau}, \textbf{t}}\rangle + 2e_{i,j,\tau} + 2^{\tau}\textbf{s}[i]\cdot \textbf{s}[j] \approx\langle{\textbf{a}_{i,j,\tau}, \textbf{t}}\rangle + 2^{\tau}\textbf{s}[i]\cdot \textbf{s}[j] $ 64 | 65 | then $h_{i,j}\cdot \textbf{s}[i]\textbf{s}[j] = \sum_{\tau=0}^{\lfloor{log\ q}\rfloor}2^{\tau}\textbf{s}[i]\cdot \textbf{s}[j] \approx h_{i,j,\tau}(b_{i,j,\tau} - \langle \textbf{a}_{i,j,\tau}, \textbf{t} \rangle)$ 66 | 67 | ### Dimension-Modulus Reduction 68 | 69 | Aim: Taking a ciphertext with parameters $(n,log\ q)$ as above, and convert it into a ciphertext of the same message, but with parameters $(k,log\ p)$ which are much smaller than $(n,log\ q)$. 70 | 71 | -------------------------------------------------------------------------------- /ai-dapps.md: -------------------------------------------------------------------------------- 1 | # AI DApps 2 | 3 | The application built on the blockchain, known as "DApp", has many different characteristics than the traditional application. Most distinctively, a DApp is an application that enables direct interaction between end users and providers. 4 | 5 | An AI DApp adds AI features to the DApp and expands its usability. AI DApps would typically have their own suite of associated contracts with various ml models on the blockchain which they use to encode business logic and allow persistent storage of their consensus-critical state. 6 | 7 | ## AI DApp Use Cases 8 | 9 | There are various applications to utilize contracts with ml models. Below examples are only use cases thought of by the Cortex team alone. It is almost certain that the community will conceive many more and better use cases for AI on the blockchain. 10 | 11 | - DeFi: credit report, anti-fraud in decentralized exchanges, p2p financing, insurance, cryptocurrency lending 12 | 13 | - Gaming: AI judge, player agent, NPC, assistant/coaching/education 14 | 15 | - Global Climate Action & Carbon Credit Management and Trading: collects environmental data and puts on the blockchain for a transparent carbon pricing system 16 | 17 | - AI Governance: stablecoins based on machine learning, sentiment analysis, decentralized decision making, malicious behavior detection, smart resource allocation 18 | 19 | - Others: on-chain data mining, facial recognition, recommendation, chatbot, machine translation, voice synthesis, etc. 20 | 21 | ## Developer Tools 22 | 23 | AI DApp development is similar to DApp development on Ethereum. It requires an understanding of Solidity programming language, web3.js, and JSON-RPC API. Below is a list of links to get you started: 24 | 25 | - [Solidity Docs](https://solidity.readthedocs.org/en/latest/) - Solidity is the Smart Contract language developed by Ethereum, which compiles to CVM (Cortex Virtual Machine) opcodes. 26 | - [Cortex-Remix](https://cerebro.cortexlabs.ai/remix) - a browser-based compiler and IDE using programming language Solidity. 27 | - [Web3 JavaScript API](https://web3js.readthedocs.io/en/1.0/) - This is the main JavaScript SDK to use when you want to interact with a node. 28 | - [JSON RPC API](https://github.com/ethereum/wiki/wiki/JSON-RPC) - This is the low-level JSON RPC 2.0 interface to interface with a node. 29 | 30 | ## AI DApp Examples 31 | 32 | #### DeFi 33 | For example, a decentralized lending app can run an AI algorithm to determine your interest rate based on your personal credit history. The AI used to analyze your credit score is not a black box, but instead, every step of the AI inference is transparent to prevent discrimination and ensure fairness. 34 | 35 | #### Gaming 36 | CryptoKitties would be much cuter, more life-like, and unique if they incorporated AI. Imagine these kitties moving dynamically and behaving uniquely depending on your personal experience interacting with them. While Ethereum is not able to execute AI models and allow for this user experience, this is something that Cortex can uniquely enable. 37 | 38 | #### Insurance 39 | Blockchain finds many use cases in the insurance industry, where immutability, fairness, openness, and transparency are in high demand. AI can help improve underwriting decisions, better manage risks, and prevent fraud. An insurance DAO powered by on-chain AI can bring us better, cheaper, fairer, and less bureaucratic insurance. 40 | 41 | #### Anti-fake AI 42 | The emergence of deepfakes (AI-manipulated videos that are indistinguishable to the human eye) poses a significant threat to society. Social stability will inevitably suffer if video recordings can simply be dismissed as untrustworthy in court. Anti-fake AI algorithms (algorithms that detect whether a video has been tampered with) will run on the blockchain to ensure their transparency and fairness, especially if there were to be used in court. 43 | -------------------------------------------------------------------------------- /mining.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | The proof of work algorithm used is called [Cukoo Cycle](https://github.com/tromp/cuckoo), a graph theory-based algorithm that is far less energy-intensive than most other CPU, GPU or ASIC-bound PoW algorithms. The goal is to lower mining requirements, ensuring true decentralization, and laying the foundation for future scalability. 4 | 5 | Cuckoo Cycle is designed to be ASIC-resistant and optimal for GPU-based transactions processing. PoW is used to facilitate a decentralized ecosystem where no single entity controls the network. Just as in a typical PoW project, miners provide the computing power to support the Cortex network, in return are rewarded with transaction fees. The process incidentally is Cortex's way to issue more CTXCs, the native Cortex asset. 6 | 7 | The difficulty adjusts dynamically so that on average, a block is produced every 15 seconds, i.e., 15s block time. This rate ensures the synchronization of the system state while preventing double-spend and history alteration unless an attacker possesses more than 51% of the network's mining power. 8 | 9 | ## Mining Rewards 10 | 11 | Any node participating in the network can be a miner. The miner of the winning block receives: 12 | - A fixed block reward for the 'winning' block, consisting of exactly 7 CTXCs 13 | - The sender of the transaction pays for all the endorphins that carry the transaction 14 | - All of the endorphins from the transactions in the block are split between the winning miner and AI model providers 15 | 16 | 17 | ## The Algorithm 18 | 19 | Cuckoo Cycle tries to find a fixed length L ring in the Cuckoo Cycle bipartite graph randomly generated by Siphash. As the scale of the graph increases, the L value increases and becomes more difficult to find a ring that fits the length of L. Solving the solution (i.e., the process of finding a loop) itself requires a relatively large amount of memory to store the sorting of data. The Cuckoo Cycle algorithm is *resisted* to ASIC chips. The difficulty and cost to make Cuckoo Cycle compatible on ASIC are much higher than the SHA256 of Bitcoin. 20 | 21 | Cortex utilizes CuckAroo 30, a variation of Cukoo Cycle. CuckARoo replaces Siphash-2-4 with Siphash-4-8, focusing more on memory use. The solution time is bound to memory bandwidth, making it resistant to ASIC chips. Unlike other PoW algorithms that require maxing out the capacity of your hardware and consuming a lot of power in the process, a memory-bound algorithm like CuckARoo requires far less energy than most other GPU, CPU or ASIC-bound PoW algorithms. 22 | 23 | ## Requirements 24 | 25 | Below is the recommended minimum requirements for the official miner. Third party miners may be available after the Mainnet release which might be optimized to use less memory. 26 | 27 | - System: Linux Ubuntu 16.04+ 28 | - GPU: Nvidia GPU with 10.7GB VRAM 29 | - Space: 2TB (*the size of the blockchain increases over time*) 30 | - CUDA version: 9.2+ 31 | - CUDA driver: 396+ 32 | - Compiler: Go 1.10+, GCC 5.4+ 33 | 34 | ## Statistics 35 | - Quota general: 64k per block (model uploading space) 36 | - Uploading network bandwidth: 1MB/s 37 | - Model mature: 100 blocks 38 | - Model size limit: 1GB 39 | - Pre-allocation: 149792458 40 | - Total reward for mining: 150000000 41 | - Total supply: 299792458 42 | - Reward: 7 per block (half every 4 years) = 8409600 43 | - Uncle Block Reward: $(1- \frac{current - uncle} {8}) * block\_reward$ 44 | current = current block no., uncle = uncle block no., block_reward = reward 45 | 46 | 47 | ## Installation 48 | ### UBUNTU 49 | download Cortex Pool Miner 50 | ```bash 51 | git clone git@github.com:CortexFoundation/PoolMiner.git 52 | cd PoolMiner && make clean && make 53 | ``` 54 | 55 | ### Dependency Version Check 56 | Ensure you have the correct version of gcc and go 57 | gcc/g++ 5.4 or later is required. 58 | Use g++ --version or gcc --version to check which version you have. 59 | On Ubuntu, if your version is too old then you can update to gcc/g++ 5.4 as follows: 60 | 61 | ```bash 62 | sudo add-apt-repository ppa:ubuntu-toolchain-r/test 63 | sudo apt-get update 64 | sudo apt-get install g++-5.4 65 | ``` 66 | 67 | go 1.10 or later is required 68 | Use go --version to check which version you have. 69 | Update to Go 1.10 as follows: 70 | 71 | ```bash 72 | sudo add-apt-repository ppa:ubuntu-toolchain-r/test 73 | sudo apt-get update 74 | sudo apt-get install go-1.10 75 | ``` 76 | 77 | ## Start Mining 78 | 79 | MinerPool Website ：https://pool.cortexlabs.ai/ 80 | MinerPool Uri：miner-cn.cortexlabs.ai:8009 or miner.cortexlabs.ai:8009 81 | 82 | ### Options 83 | 84 | #### --account 85 | Set miner account 86 | 87 | #### --pool_uri [Necessary] 88 | Set the miner pool URI for submitting solutions. Up to 3. 89 | 90 | #### --worker 91 | Enable multiple miners to mine with the same account. Set different worker to differentiate different miner account 92 | 93 | #### --deviceids [id1,id2,id3] 94 | Set which GPU device use for mining, param is device ids concatenated by commas. 95 | 96 | #### --verbosity 97 | Set the verbosity of log printing. 98 | 99 | ### Running Bash 100 | ```Bash 101 | ./build/bin/cortex_miner -pool_uri=cuckoo.cortexlabs.ai:8008 -pool_uri_1=cuckoo.cortexlabs.ai:8008 -pool_uri_2=cuckoo.cortexlabs.ai:8008 -worker=cortex-nmg-4 -devices=0,1,2,3 -account=0xE893BA644128a0065B75d2c4f642615710802D4F 102 | ``` -------------------------------------------------------------------------------- /CIPs/cip-1.md: -------------------------------------------------------------------------------- 1 | # Synapse: Cortex Deterministic Deep Learning Inference Framework 2 | 3 | ## Abstract 4 | Simply put, the sole purpose of The Framework is to guarantee the execution of any compatible deep learning model is deterministic on any full node, while not downgrading performance or accuracy by too much. 5 | 6 | ## Summary 7 | The deterministic condition formally translates into the following criteria: 8 | 9 | 1. `Criteria-of-Closure`: No external randomness should be introduced as input or in the runtime, software introduced or hardware introduced. 10 | 2. `Criteria-of-Invariance`: Output of parallel procedures should be invariant of scheduling strategies. 11 | 12 | As far as we know, `Criteria-of-Closure` can easily be met, because pseudo-randomness suffice in deep learning, which is deterministic in nature. `Criteria-of-Invariance` is non-trivial for most models and inference frameworks, because low-level parallelism can behave differently if the order of execution is different. 13 | 14 | Cortex replaces floating point numbers to integer in The Framework to eliminate randomness of floating point calculation errors in different hardware architectures. Furthermore, Cortex limits the width and depth for different type of neural network layer to eliminate potential overflow for intermediate results, therefore implementing `Criteria-of-Invariance`. 15 | 16 | Under the new set of limitation, precision could downgrade for most models. Currently fine tuning is needed to train the new integer models to reduce precision loss. With 8-bit integer models we can have a compression rate of 25% (75% size reduction). 17 | 18 | ## Specification 19 | Parameter type: 20 | ``` 21 | 8-bit integer(int8) 22 | ``` 23 | For basic operations: 24 | 1. Convolutional layers 25 | ``` 26 | Conv1D, Conv2D, Conv3D(Not supported in testnet) 27 | Kernel size * channel number < 2^16 28 | int8 weights with int8 input 29 | int8 output features 30 | int32 intermediate results 31 | ``` 32 | 2. Normalization Layers 33 | ``` 34 | BatchNorm(scale factors must be int8) 35 | ``` 36 | 3. Non-Linear Activation Layers 37 | ``` 38 | ReLU, PReLU(p must be int8) 39 | ``` 40 | 4. Fully Connected Layers 41 | ``` 42 | int8 weights with int8 input 43 | int8 output features 44 | int32 intermediate results 45 | ``` 46 | 5. Residual Network Configuration 47 | ``` 48 | In x + cf(x), c=1 49 | ``` 50 | 6. Model constraints 51 | ``` 52 | Input image dimension less than 3 * 224 * 224 * 8-bit color 53 | Model depth <= 200 layers 54 | Channel size <= 4096 55 | ``` 56 | 57 | ## Notes 58 | - Please use Conv1D and quantization instead of RNN-like structures, e.g. GRU/LSTM. Total Accuracy will not be harmed in NLP/OCR practices. 59 | - Please use wider or deeper residual structure and grouping instead of Concat Layers. 60 | 61 | ## Roadmap 62 | - int4/int2 models for higher model compression rate 63 | - int8 models with less harm to model accuracy 64 | - int32/int16 activation for compatibility 65 | - More models beyond images 66 | 67 | ## References 68 | [Cor(2018)] Coreml document, integrate machine learning models into your app., 2018. URL https://developer.apple.com/documentation/coreml. 69 | 70 | [tvm(2018)] Open deep learning compiler stack for cpu, gpu and specialized accelerators: dmlc/tvm, 2018. URL https://github.com/dmlc/tvm. 71 | 72 | [Ba et al.(2016)Ba, Kiros, and Hinton] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016. 73 | 1 74 | 75 | [Chen et al.(2018)Chen, Moreau, Jiang, Zheng, Yan, Cowan, Shen, Wang, Hu, Ceze, Guestrin, and Krishnamurthy] Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, 76 | Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. TVM: An Automated End-to-End 77 | Optimizing Compiler for Deep Learning. arXiv preprint arXiv:1802.04799, February 2018. 78 | 79 | [He et al.(2015)He, Zhang, Ren, and Sun] K He, X Zhang, S Ren, and J Sun. Deep residual learning for image recognition. corr, vol. abs/1512.03385, 2015. 80 | 81 | [Ioffe & Szegedy(2015)Ioffe and Szegedy] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015. 82 | 83 | [Krizhevsky et al.(2012)Krizhevsky, Sutskever, and Hinton] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information process- ing systems, pp. 1097–1105, 2012. 84 | 85 | [Li & Liu(2016)Li and Liu] F Li and B Liu. Ternary weight networks.(2016). arXiv preprint arXiv:1605.04711, 2016. 86 | 87 | [NVIDIA(2018)] NVIDIA. Tensorrt document, 2018. URL https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/index.html. 88 | 89 | [Sabour et al.(2017)Sabour, Frosst, and Hinton] Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. In Advances in Neural Information Processing Systems, pp. 3856–3866, 2017. 90 | 91 | [Srivastava et al.(2015)Srivastava, Greff, and Schmidhuber] Rupesh Kumar Srivastava, Klaus Greff, and Ju ̈rgen Schmidhuber. Highway networks. arXiv preprint arXiv:1505.00387, 2015. 92 | 93 | [Szegedy et al.(2016)Szegedy, Ioffe, and Vanhoucke] C Szegedy, S Ioffe, and V Vanhoucke. Inception-v4, inception-resnet and the impact of residual connections on learning. corr abs/1602.07261. URL http://arxiv.org/abs/1602.07261, 2016. 94 | 95 | [Wu & He(2018)Wu and He] Yuxin Wu and Kaiming He. Group normalization. arXiv preprint arXiv:1803.08494, 2018. -------------------------------------------------------------------------------- /FHE/Bra12_cn.md: -------------------------------------------------------------------------------- 1 | # [Bra12]方案 - Fully Homomorphic Encryption without Modulus Switching from Classical GapSVP 2 | 3 | ## 安全基础 4 | 5 | ## 思路简介 6 | 7 | [BGV12方案](BGV12_cn.md)中提出利用模数切换来压缩噪音的规模，保证每次乘法之后噪音和上限比值的增长是线性的而非最初的双指数级，因此能进行的乘法深度变为线性。这带来一个思路，缩放虽然不能改变模数和噪音的比例，但是能改变我们看待密文的角度，从而管理噪音。 8 | 9 | 因此Bra12方案直接考虑将噪音缩放到[0,1)的区间，假设噪音的bound是 $B$，则将噪音压缩为 $B/q < 1$。这样一来，后续的所有乘法都不会导致噪音的剧烈增长，因为 $(B/q)^2 < (B/q)$ 。所以乘法后噪音的增长并不是平方级别，而是乘上一个密码参数的多项式 $p(n)$ ，经过 $L$ 层乘法，噪音从 $(B/q)$ 增长到 $(B/q) \cdot p(n)^L$ 。意味着我们的模数上限可以设置为 $q \approx B \cdot p(n)^L$，一方面降低了运算复杂度，另一方面增强了安全性。 10 | 11 | 在模数 1 的范围内进行运算这一想法在[AD97]，[Reg03]和[Reg05]中都有提及。本方案并没有采用浮点数，而是用整数域 $\mathbb{Z}_q$ 上的元素进行近似计算。 12 | 13 | 这一设计的好处有： 14 | 15 | 1. 放缩不变性，只和 $q/B$ 的比值有关，与绝对值无关，因此只要保证 $q$ 的长度符合安全，实际上可以选择 2 的幂次作为模数来加速运算。 16 | 2. 不需要模数切换的操作，噪音的增长已经控制住了 17 | 18 | 19 | 20 | ## Regev 公钥加密体系 21 | 22 | ### 私钥生成 23 | 24 | **输入**：$1^n$ 25 | 26 | **输出**：$sk \in \mathbb{Z}_q^n$ 27 | 28 | **过程**： 29 | 30 | 采样 $\textbf{s} \stackrel{$}\leftarrow \mathbb{Z}_q^n$，输出 $sk = \textbf{s}$ 31 | 32 | 对消息 $m \in \{0, 1\}$ 加密得到向量 $\textbf{c}$ 满足 $\lang \textbf{c}, \textbf{s} \rang = \lfloor \frac{q}{2} \rfloor \cdot m + e + qI$ 33 | 34 | ### 公钥生成 35 | 36 | **输入**：私钥 $\textbf{s} \in \mathbb{Z}_q^n$ 37 | 38 | **输出**：公钥 $pk = \textbf{P} \in \mathbb{Z}_q^{N \times (n+1)}$ 39 | 40 | **过程**： 41 | 42 | 令 $N \triangleq (n+1) \cdot (\log q + O(1))$，采样 $\textbf{A} \stackrel{$}\leftarrow \mathbb{Z}_q^{N \times n}$ 以及 ${\bf e} \stackrel{$}\leftarrow \chi^N$ 43 | 44 | 计算 $\textbf{b} := [\textbf{A} \cdot \textbf{s} + \textbf{e}]_q$ ，并且令 $\textbf{P} := [\textbf{b} \| -\textbf{A}] \in \mathbb{Z}_q^{N \times (n+1)}$ 45 | 46 | ### 加密 47 | 48 | **输入**：消息 $m \in \{0, 1\}$，公钥 $pk$ 49 | 50 | **输出**：密文 $\textbf{c} \in \mathbb{Z}_q^{(n+1)}$ 51 | 52 | **过程**： 53 | 54 | 采样 $\textbf{r} \in \{0, 1\}^N$ 55 | 56 | 计算 ${\bf c} := [{\bf P}^T \cdot r + \lfloor \frac{q}{2} \cdot {\bf m} \rfloor]_q \in \mathbb{Z}_q^{n+1}$ 57 | 58 | 其中 ${\bf m} \triangleq (m, 0, ..., 0) \in \{0, 1\}^{n+1}$ 59 | 60 | ### 解密 61 | 62 | **输入**：密文 ${\bf c}$，私钥 $sk$ 63 | 64 | **输出**：明文消息 $m$ 65 | 66 | **过程**：计算 67 | $$ 68 | m := [\lfloor 2 \cdot \frac{[\lang {\bf c}, (1, {\bf s}) \rang]_q}{q}\rceil]_2 69 | $$ 70 | 71 | 72 | ### 正确性证明 73 | 74 | 引理3.1证明加密带来的噪音在固定范围内，引理3.2证明密文包含在给定范围内的噪音能被正确解密 75 | 76 | #### 引理3.1 77 | 78 | 令 $q,n,N,|\chi| \leq B$ 为 ${\sf Regev}$ 的参数，令 ${\bf s} \in \mathbb{Z}^n$ 为任意向量并且 $m \in \{0, 1\}$。设置 ${\bf P} \leftarrow {\sf Regev.PublicKeygen}({\bf s})$ 以及 ${\bf c} \leftarrow {\sf Regev.Enc}_{\bf P}(m)$。则存在 $e$ 满足 $|e| \leq N \cdot B$ ，使得 79 | $$ 80 | \lang {\bf c}, (1, {\bf s}) \rang = \lfloor \frac{q}{2} \rfloor \cdot m + e \mod q 81 | $$ 82 | 证明： 83 | $$ 84 | \begin{aligned} 85 | \lang {\bf c}, (1, {\bf s}) \rang &= \lang {\bf P}^T \cdot {\bf r} + \biggl\lfloor \frac{q}{2} \biggl\rfloor \cdot m, (1, {\bf m}) \rang \mod q \\ 86 | &= \biggl\lfloor \frac{q}{2} \biggl\rfloor \cdot m + {\bf r}^T {\bf P} \cdot (1, {\bf s}) \mod q \\ 87 | &= \biggl\lfloor \frac{q}{2} \biggl\rfloor \cdot m + {\bf r}^T {\bf b} - {\bf r}^T {\bf As} \mod q \\ 88 | &= \biggl\lfloor \frac{q}{2} \biggl\rfloor \cdot m + \lang {\bf r, e} \rang \mod q \\ 89 | \end{aligned} 90 | $$ 91 | 92 | 93 | 因为 $|\lang {\bf r, e} \rang| \leq N \cdot B$，所以引理3.1成立 94 | 95 | #### 引理3.2 96 | 97 | 令 ${\bf s} \in \mathbb{Z}^n$ 为某一向量，${\bf c} \in \mathbb{Z}_q^{n+1}$ 满足 98 | $$ 99 | \lang {\bf c}, (1, {\bf s}) \rang = \lfloor \frac{q}{2} \rfloor \cdot m + e \mod q 100 | $$ 101 | 其中 $m \in \{0,1\}$ 并且 $|e| < \lfloor q / 2 \rfloor / 2$ 102 | $$ 103 | {\sf Regev.Dec_{\bf s}({\bf c})} = m 104 | $$ 105 | 106 | ## 向量拆解和密钥交换 107 | 108 | ### 向量拆解 109 | 110 | 和BV11以及BGV12中使用的技术一样，包含 ${\sf BitDecomp}$ 和 ${\sf PowersOfTwo}$ 函数 111 | 112 | ### 密钥交换 113 | 114 | 和BGV12中使用的维数切换技术一样，只实现维数的切换，不改变模数 115 | 116 | 117 | 118 | ## 缩放不变性的同态加密方案 （SI-HE, A Scale Invariant Homomorphic Encryption Scheme） 119 | 120 | ### 密钥生成 121 | 122 | **输入**：计算深度和私钥长度 $(1^L, 1^n)$ 123 | 124 | **输出**：公钥 $pk = {\bf P}_0$，切换密钥 $evk = \{{\bf P}_{(i-1):i}\}_{i \in [L]}$，私钥 $sk = {\bf s}_L$ 125 | 126 | **过程**： 127 | 128 | 采样 $L+1$ 个向量 ${\bf s}_0,...,{\bf s}_L \leftarrow {\sf Regev.SecretKeygen}(1^n)$ 129 | 130 | 计算公钥 ${\bf P}_0 \leftarrow {\sf Regev.PublicKeygen}({\bf s}_0)$ 131 | 132 | 对所有 $i \in [L]$，定义 133 | $$ 134 | \tilde{\bf s}_{i-1} := {\sf BitDecomp((1,{\bf s}_{i-1}))} \otimes {\sf BitDecomp}((1, {\bf s}_{i-1})) \in \{0,1\}^{((n+1) \lceil \log q \rceil)^2} 135 | $$ 136 | 计算 137 | $$ 138 | {\bf P}_{(i-1):i} \leftarrow {\sf SwitchKeyGen}(\tilde{\bf s}_{i-1}, {\bf s}_i) 139 | $$ 140 | 141 | ### 加密算法 142 | 143 | **输入**：密文 ${\bf c}$，私钥 $sk$ 144 | 145 | **输出**：明文消息 $m$ 146 | 147 | **过程**：直接调用 ${\bf c} \leftarrow {\sf Regev.Enc}_{pk}(m)$ 148 | 149 | ### 同态加法 150 | 151 | **输入**：密文 ${\bf c}_1,{\bf c}_2$，切换密钥 $evk$ 152 | 153 | **输出**：加法密文 ${\bf c}_{add}$ 154 | 155 | **过程**： 156 | 157 | 假设密文 ${\bf c}_1,{\bf c}_2$ 都对应密钥 ${\bf s}_{i-1}$，计算 158 | $$ 159 | \tilde{\bf c}_{add} := {\sf PowersOfTwo}({\bf c}_1 + {\bf c}_2) \otimes {\sf PowersOfTwo}((1,0,...,0)) 160 | $$ 161 | 输出 162 | $$ 163 | {\bf c}_{add} \leftarrow {\sf SwitchKey}({\bf P}_{(i-1):i}, \tilde{\bf c}_{add}) \in \mathbb{Z}_q^{n+1} 164 | $$ 165 | 166 | ### 同态乘法 167 | 168 | **输入**：密文 ${\bf c}_1,{\bf c}_2$，切换密钥 $evk$ 169 | 170 | **输出**：乘法密文 ${\bf c}_{mult}$ 171 | 172 | **过程**： 173 | 174 | 假设密文 ${\bf c}_1,{\bf c}_2$ 都对应密钥 ${\bf s}_{i-1}$，计算 175 | $$ 176 | \tilde{\bf c}_{mult} := \biggl\lfloor \frac{2}{q} \cdot \biggl({\sf PowersOfTwo}({\bf c}_1) \otimes {\sf PowersOfTwo}({\bf c}_2) \biggl) \biggl\rceil 177 | $$ 178 | 输出 179 | $$ 180 | {\bf c}_{mult} \leftarrow {\sf SwitchKey}({\bf P}_{(i-1):i}, \tilde{\bf c}_{mult}) \in \mathbb{Z}_q^{n+1} 181 | $$ 182 | 183 | ### 解密算法 184 | 185 | **输入**：密文 ${\bf c}$，私钥 $sk$ 186 | 187 | **输出**：明文消息 $m$ 188 | 189 | **过程**：不失一般性，假设密文对应 $sk = S_L$，解密等价于 190 | $$ 191 | m \leftarrow {\sf Regev.Dec}_{sk}({\bf c}) 192 | $$ 193 | -------------------------------------------------------------------------------- /miner_dev/cuckoo_2_sha3_dev.md: -------------------------------------------------------------------------------- 1 | # Target: 2 | 1. Replace The complex Mining Algorithm on CUDA with 3 | Another Classic Mining Algorithm on CPU. 4 | 2. The Mining Process Should Be Configurable, with 5 | flags to control the behaviour of mining of generating blocks. 6 | 3. The Module is better compatible with latest master branch, 7 | and should be plugable. 8 | 9 | # Mechanism: 10 | 11 | ## Cuckoo: 12 | > POW, Proof of AI work for [CortexTheseus](https://github.com/CortexFoundation/CortexTheseus). 13 | > Cuckoo controls flow of generating solutions, verifying solutions, veirifying blocks, accumalate reward, and finalizing blocks. 14 | > Based on implementation of [cvm-runtime](https://github.com/CortexFoundation/cvm-runtime), cuckoo calls plugins of CUDA env to generate and verify solutions. 15 | 16 | ## SHA3 Origin: 17 | > Zero-ahead nonce counting hashing is a classic mechanism for Crypto-Currency Mining, also a realization of Pow (power of work), which supports BlockChain Technique developing. 18 | Partly refer to [bitcoin-algorithm-explained](https://www.mycryptopedia.com/bitcoin-algorithm-explained/), SHA3 requires header hash, nonce, difficulty and timestamp to do next hashing. 19 | 20 | 21 | - Version – The version number of the Bitcoin software. 22 | - Previous block hash – A reference to the hash of the previous block that was included on the blockchain. 23 | - Merkle Root – A representative hash of all transactions that are included in the candidate block. 24 | - Timestamp – A piece of information that references the time that the block was created. 25 | - Target – The target hash threshold, this block’s header hash must be less than or equal to the target hash that has been set by the network. 26 | - Nonce – The variable that is used in the proof of work mining process. 27 | 28 | ## Difficulty Adjustment Algorithm 29 | > https://github.com/cortex/EIPs/issues/100. 30 | algorithm: 31 | diff = (parent_diff + 32 | (parent_diff / 2048 * max((2 if len(parent.uncles) else 1) - ((timestamp - parent.timestamp) // 9), -99)) 33 | ) + 2^(periodCount - 2) 34 | > the estimated interval is [9,18], the average is 13.5 35 | > using block generating timestamp as factors of difficulty adjustment, which can control 36 | the block generating interval. The Difficulty is higher with shorter interval. 37 | 38 | # Dev Doc 39 | 40 | ## SHA3 Solution(CPU) 41 | > Simple sha3 solution, Relace seal_miner "Gen Solution" 42 | > header_hash len is 32*u8, nonce len is 2*u32, 43 | > including sha3 solution Verify, Replace CuckooVerifyHeader() 44 | 45 | Function Definition1: 46 | `func (cuckoo *Cuckoo) GenSha3Solution(hash []byte, nonce uint64) (r uint32, sols [][]uint32)` 47 | 48 | Function Definition2: 49 | `func (cuckoo *Cuckoo) CuckooVerifyHeader_SHA3(hash []byte, nonce uint64, sol *types.BlockSolution, targetDiff *big.Int) bool` 50 | 51 | 52 | ## Difficulty Adjustment 53 | calcDifficultySHA3 the difficulty adjustment algorithm. It returns 54 | the difficulty that a new block should have when created at time given the 55 | parent block's time and difficulty. The calculation uses the Byzantium rules, 56 | tuning x growing faster when difficulty growing positively 57 | The estimated interval is (interval + 2*interval)/2 58 | 59 | 1. replace cuckoo CalcDifficulty with calcDifficultySHA3. 60 | 61 | 2. implement calcDifficultySHA3 as: 62 | Function Deifinition: 63 | `func calcDifficultySHA3(time uint64, parent *types.Header, interval *big.Int) *big.Int` 64 | 65 | 3. test whether difficulty successfully controls the blocking generating interval 66 | 67 | ## Command Flags 68 | Using Mining Or Cuckoo Flags to control Block generating interval and Mining Behaviour (with or without Txs) 69 | 70 | ### Block Interval 71 | `cuckoo.blockinterval`: 72 | > --cuckoo.blockinterval 10s 73 | a flag embedded into cuckoo config, and infect the calcDifficultySHA3 to adjust 74 | difficulty as the estimated average interval, which should be `1.5 * cuckoo.blockinterval` 75 | 76 | ### CarryTx 77 | `miner.carrytx`: 78 | > --miner.carrytx 79 | a flag embedded into miner config, and infect the worker to only commit work with newTxs 80 | 81 | ## Testing 82 | 83 | ### go test 84 | > sh3sol_test.go 85 | 86 | ### with some cortex node scripts: 87 | #### create new account 88 | ```bash 89 | ./build/bin/cortex --dolores --datadir .cortex2/dolores account new 90 | ``` 91 | #### list accounts 92 | ```bash 93 | ./build/bin/cortex --dolores --datadir .cortex2/dolores account list 94 | ``` 95 | #### start mining with specific account, with interval 7.5s, should carrytx 96 | ```bash 97 | ./build/bin/cortex --dolores --storage.mode lazy --datadir .cortex2/dolores --rpc --rpcapi ctxc,web3,admin,txpool --rpcaddr 0.0.0.0 --rpcport 38545 --mine --miner.coinbase 7b775e3bc393db28d515ff6ad7bdf3713018935f --cuckoo.blockinterval 5s --allow-insecure-unlock --unlock 7b775e3bc393db28d515ff6ad7bdf3713018935f --miner.carrytx 98 | ``` 99 | #### send tx with js console attach 100 | ```bash 101 | ./build/bin/cortex attach --dolores --datadir .cortex2/ 102 | ``` 103 | ```js 104 | ctxc.sendTransaction({ 105 | "from": "0x7b775e3bc393db28d515ff6ad7bdf3713018935f", 106 | "to": "0xf51d56e3e9647a23e52c8da2ae26900835ee1015", 107 | "value": 100e2, 108 | "gas": 21000, 109 | "gasPrice": 3, 110 | "nonce": 0}) 111 | ``` 112 | 113 | 114 | ### Web3 Txs Testing 115 | ```bash 116 | yarn add web3 117 | node index.ts 118 | ``` 119 | 120 | ```typescript 121 | // index.ts In Node.js use: const Web3 = require('web3'); 122 | const Web3 = require('web3'); 123 | 124 | const web3_ = new Web3(new Web3.providers.HttpProvider("http://localhost:38545")); 125 | 126 | var version = web3_.version 127 | console.log('@@@web3', web3_) 128 | console.log('web3 version', version); 129 | 130 | // cortex is compatible with eth 131 | // get node info 132 | web3_.eth.getNodeInfo().then( 133 | console.log 134 | ); 135 | 136 | // cortex is compatible with eth 137 | // get chain id, normally is 43 138 | web3_.eth.getChainId().then( 139 | console.log 140 | ); 141 | 142 | // get accounts address 143 | web3_.eth.getAccounts().then( 144 | console.log 145 | ) 146 | 147 | // get block Number 148 | web3_.eth.getBlockNumber().then( 149 | console.log 150 | ) 151 | 152 | // send Transaciton 153 | web3_.eth.sendTransaction( 154 | { 155 | "from": "0x7b775e3bc393db28d515ff6ad7bdf3713018935f", 156 | "to": "0xf51d56e3e9647a23e52c8da2ae26900835ee1015", 157 | "value": 100e2, 158 | "gas": 21000, 159 | "gasPrice": 3, 160 | "nonce": 1 161 | } 162 | ).then((rlt)=>{ 163 | console.log(rlt); 164 | // get Transaciton after success 165 | web3_.eth.getTransaction(rlt 166 | ).then( 167 | console.log 168 | ); 169 | 170 | web3_.eth.getTransactionCount( 171 | ).then( 172 | console.log 173 | ); 174 | } 175 | ).catch( 176 | console.warn 177 | ); 178 | ``` 179 | 180 | -------------------------------------------------------------------------------- /cortex-guide.md: -------------------------------------------------------------------------------- 1 | # Cortex Theseus 2 | 3 | [TOC] 4 | 5 | ## Introduction 6 | 7 | The inspiration of CortexLabs starts from the development of virtual machine in Ethereum project, which can execute simple contract logic. And further more, can it execute the AI on blockchain? The absolute answer is YES! That is the **AI on Blockchain** @CortexLabs. 8 | 9 | Firstly, welcome to visit our offical website: https://www.cortexlabs.ai. 10 | 11 | And then , we will give an total review of our foundation structure. All the whole projects of CortexLabs are arranged in the below image. You can have a quick glance at the CortexLabs' global mind map, and then find the corresponding sections in the documentation from the picture. 12 | 13 | ![XMind](imgs/CortexTheseus.png) 14 | 15 | ## Documentation 16 | 17 | The main introduction to CortexLabs project is located at [cortex documentation](README.md). 18 | 19 | CortexLabs chain-node GitHub project is named [**CortexTheseus**](https://github.com/CortexFoundation/CortexTheseus). 20 | 21 | ## Quick Demo 22 | 23 | This is an preview demo to show what CortexLabs is doing with AI, edge devices, and blockchain. Refers to [Apple Demo](apple-demo.md) to learn more. 24 | 25 | ![demo](cvm/demo/demo.gif) 26 | 27 | ## Start Up 28 | 29 | The Cortex Theseus project start-up documentation is located in this [link](clients.md). 30 | 31 | ## Cortex Release 32 | 33 | We compiled and organized many necessary binaries for people who do not want to access the source code. the Cortex Release project is located in this [link](https://github.com/CortexFoundation/Cortex_Release). 34 | 35 | ## Blockchain 36 | 37 | ### Mining 38 | 39 | Mining documentation, [mining](mining.md). 40 | 41 | - Cuckoo Cycle Algorithm 42 | 43 | Cortex Theseus Client uses [cuckoo cycle](https://github.com/CortexFoundation/PoolMiner/blob/dev/README.md) as mining algorithm. 44 | 45 | - Pool Miner 46 | 47 | We have published the mining pool source code in [GitHub](https://github.com/CortexFoundation/PoolMiner). 48 | 49 | ### Account Balance 50 | 51 | Cortex Theseus inherits numerous traits from the [Ethereum](https://github.com/ethereum/go-ethereum) project (which is inspired by [Bitcoin](https://github.com/bitcoin/bitcoin)). Hence a wallet address in the blockchain have the concept of balance that can use to transfer CTXC in between accounts. 52 | 53 | ### Currency: CTXC 54 | 55 | CTXC is the native token used within Cortex. Refer to this [link](ctxc.md) for more details. 56 | 57 | ### Endorphin / Gas 58 | 59 | Payment for transaction packed by miner in blockchain is calledEndorphin, or Gas as called in Ethereum. Refer to this [link](endorphin.md) for more details. 60 | 61 | ### JSON RPC 62 | 63 | Cortex Full Node(Cortex Theseus) can supply JSON RPC interface, its functionality is similar to the [RPC of Ethereum](https://github.com/ethereum/wiki/wiki/JSON-RPC) except that the prefix `eth_` should be replaced with `ctxc_` if method contains. 64 | 65 | You should enable the JSON RPC in the Cortex Theseus Node with commands: 66 | 67 | ```bash 68 | --rpc --rpcapi ctxc,web,admin,txpool --rpcaddr 0.0.0.0 --rpcport 38888 69 | ``` 70 | 71 | The official JSON RPC documentation is still in progress. You can have a glance through this [link](http://ec2-18-191-10-249.us-east-2.compute.amazonaws.com:5000/). 72 | 73 | ### Node Peers 74 | 75 | We have collected all the running nodes peers from global MainNet network. 76 | 77 | Refer to this [link](https://github.com/CortexFoundation/discv4-dns-lists) for more detial . 78 | 79 | ## Torrent File Storage 80 | 81 | Cortex Theseus has a distributed file system storing the model and input data. The storage layer is named Torrent FS. 82 | 83 | Rrefer to this [link](storage-layer.md) for more detail. 84 | 85 | ### Model & Input 86 | 87 | - Available Models on Chain 88 | 89 | We have supply some pre-quantized model on the Cortex blockchain including image classification and NLP category. More details [here](model-list.md). 90 | 91 | - Model & Input Upload 92 | 93 | Model & Input upload process require a few steps. Refer to this [link](model-data-upload.md) if you are interested in uploading models and on-chain AI contract call. 94 | 95 | ## Cortex Virtual Machine 96 | 97 | Cortex Virtual Machine is inherited from EVM in Ethereum project with added **important and powerful** features. Exposition of CVM is described [here](cvm.md). 98 | 99 | ### AI Contract 100 | 101 | - Contract Deployment refers to the [link](ai-contracts.md). 102 | - AI Dapps refer to the [link](ai-dapps.md). 103 | - Contract Editor: Remix refer to the [link](cortex-remix.md). 104 | - Solidity Project refers to the [link](https://github.com/CortexFoundation/ctxc-solc). 105 | 106 | ### Synapse / CVM Executor 107 | 108 | Synapse is a go-lang level wrapper of cvm-runtime. We supply portable usage for GPU-lack devices like remote inference without native AI inference. 109 | - Local Inference (To be continued) 110 | - Remote Inference (To be continued) 111 | 112 | ## CVM Runtime 113 | 114 | CVM Runtime is a deterministic AI framework like TensorFlow, Caffe, MxNet, etc. We have sperated the project from CortexTheseus as an independent repository and have completed many features AI developers want such as high-level python interface, graph model support, etc. 115 | 116 | ### Documentation 117 | 118 | - operator's OPS refer to the [link](https://github.com/CortexFoundation/cvm-runtime/blob/wlt/docs/cvm/ops.md). 119 | 120 | ### Github Project 121 | 122 | The cvm-runtime github url is https://github.com/CortexFoundation/cvm-runtime/. 123 | 124 | **Notice**: Many new features under development is in branch wlt, that will be merged into master in the future. 125 | 126 | ### Formalization 127 | 128 | We have researched the operator formalization of cvm-runtime, using `Z3-Prover` to verify the operator's deterministic process logic. All the verification source code and records have been uploaded in the [GitHub link](https://github.com/CortexFoundation/z3_prover). 129 | 130 | ## Model Representation Tools 131 | 132 | MRT is a representation toolset for transforming the floating AI model into a full integer and non-data-flow model that CVM Runtime can accept. More information can be found in this [link](mrt.md). 133 | 134 | ### Documentation 135 | 136 | - Install and Introduction, [link](https://github.com/CortexFoundation/cvm-runtime/blob/wlt/docs/mrt/README.md). 137 | - MNist train and quantize tutorial, [link](https://github.com/CortexFoundation/cvm-runtime/blob/wlt/docs/mrt/mnist_tutorial.md). 138 | - Pre-quantized Model Information, [link](https://github.com/CortexFoundation/cvm-runtime/blob/wlt/docs/mrt/model.md). 139 | - MRT Start and API, [link](https://github.com/CortexFoundation/cvm-runtime/blob/wlt/docs/mrt/mrt.md) 140 | 141 | ### Github Project 142 | 143 | The MRT code is intergral in the [cvm-runtime](https://github.com/CortexFoundation/cvm-runtime/tree/wlt) project. Mainly located at the directory `python/mrt`. 144 | -------------------------------------------------------------------------------- /clients.md: -------------------------------------------------------------------------------- 1 | # Clients 2 | 3 | The Cortex repository is a fork of [Go Ethereum](https://github.com/ethereum/go-ethereum>) which contains protocol changes to support machine learning model inference. This implements the Cortex network, which maintains a separate ledger from the Ethereum network, for several reasons, the most immediate of which are AI inference support and the deterministic inference consensus. 4 | 5 | ## Requirements 6 | 7 | To ensure that your Cortex client runs gracefully, please check your system meets the following requirements: 8 | 9 | - System: Linux Ubuntu 16.04+ 10 | - CPU: Xeon processor 16 Cores 11 | - GPU: Nvidia GPU 1060 12 | - RAM: 32GB 13 | - Space: 2TB (*the size of the blockchain increases over time*) 14 | - CUDA version: 9.2+ 15 | - CUDA driver: 396+ 16 | - Compiler: Go 1.10+, GCC 5.4+ 17 | 18 | note: Currently, we only officially support Linux (Ubuntu), but we are actively investigating development for other operating systems and platforms(e.g., macOS, Ubuntu, Windows, Fedora). 19 | 20 | Building Cortex clients requires a Go (version 1.10 or later), a C compiler, and a CUDA (version 9.2 or later). We will guide you through the Installation section. 21 | 22 | ## Installation 23 | 24 | Before we begin installing Cortex, we need to get some dependencies for your system. 25 | 26 | ### Install NVIDIA Driver 27 | 28 | Remove old driver 29 | 30 | ```bash 31 | sudo apt-get remove --purge nvidia* 32 | sudo apt-get install build-essential freeglut3-dev libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev 33 | sudo nano /etc/modprobe.d/blacklist-nouveau.conf 34 | ``` 35 | 36 | Paste the following content into the text editor and save `blacklist-nouveau.conf`, 37 | 38 | blacklist nouveau 39 | options nouveau modeset=0 40 | 41 | Download the driver - Option 1 - via Nvidia.com 42 | 43 | ```bash 44 | wget http://us.download.nvidia.com/XFree86/Linux-x86_64/410.93/NVIDIA-Linux-x86_64-410.93.run 45 | chmod +x NVIDIA-Linux-x86_64-410.93.run 46 | ``` 47 | 48 | Download the driver - Option 2 - via PPA repository 49 | 50 | ```bash 51 | sudo add-apt-repository ppa:graphics-drivers/ppa 52 | sudo apt-get update 53 | ``` 54 | 55 | 56 | Press Ctrl + Alt + F1 to enter tty1 console, 57 | 58 | ```bash 59 | sudo update-initramfs -u 60 | sudo service lightdm stop 61 | sudo ./NVIDIA-Linux-x86_64-410.93.run –no-opengl-files 62 | (do not apply xorg config here!!!!) 63 | ``` 64 | Then 65 | 66 | ```bash 67 | sudo service lightdm start 68 | ``` 69 | 70 | Press Ctrl + Alt + F7 to go back tty7 interface. 71 | 72 | ### Install CUDA (without NVIDIA Driver) 73 | 74 | Option 1 - Install CUDA 10.0 (Without NVIDIA Driver) 75 | 76 | ```bash 77 | wget https://developer.nvidia.com/compute/cuda/10.0/Prod/ local_installers/cuda_10.0.130_410.48_linux 78 | mv cuda_10.0.130_410.48_linux cuda_10.0.130_410.48_linux.run 79 | sudo sh cuda_10.0.130_410.48_linux.run 80 | sudo ldconfig /usr/local/cuda/lib64 81 | (IMPORTANT: don't install driver here!!!) 82 | 83 | // Add two lines to ~/.bashrc 84 | echo 'export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}' >> ~/.bashrc 85 | echo 'export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc 86 | source ~/.bashrc 87 | ``` 88 | 89 | Option 2 - Install CUDA 9.2 (Without NVIDIA Driver) 90 | 91 | ```bash 92 | wget https://developer.nvidia.com/compute/cuda/9.2/Prod2/local_installers/cuda_9.2.148_396.37_linux 93 | mv cuda_9.2.148_396.37_linux cuda_9.2.148_396.37_linux.run 94 | sudo sh cuda_9.2.148_396.37_linux.run 95 | sudo ldconfig /usr/local/cuda/lib64 96 | (IMPORTANT: don't install driver here!!!) 97 | 98 | // Add two lines to ~/.bashrc 99 | echo 'export PATH=/usr/local/cuda-0.2/bin${PATH:+:${PATH}}' >> ~/.bashrc 100 | echo 'export LD_LIBRARY_PATH=/usr/local/cuda-9.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc 101 | source ~/.bashrc 102 | ``` 103 | 104 | 105 | ### Install Go 106 | 107 | Optnion 1 - Install via Google.com 108 | 109 | ```bash 110 | wget https://dl.google.com/go/go1.11.5.linux-amd64.tar.gz 111 | sudo tar -C /usr/local -xzf go1.11.5.linux-amd64.tar.gz 112 | echo 'export PATH="$PATH:/usr/local/go/bin"' >> ~/.bashrc 113 | source ~/.bashrc 114 | ``` 115 | 116 | Optnion 2 - Install via package manager 117 | 118 | ```bash 119 | sudo apt install golang-1.11 120 | ``` 121 | 122 | Make sure you have installed correctly 123 | 124 | ```bash 125 | go version 126 | ``` 127 | 128 | ### Change Linux Open File Limit 129 | 130 | you will have to edit the following file: 131 | 132 | ```bash 133 | vi /etc/sysctl.conf 134 | ``` 135 | 136 | Add the following line: 137 | 138 | ```bash 139 | fs.file-max=500000 140 | ``` 141 | 142 | Of course, you can change the number per your needs. To verify the changes again use: 143 | 144 | ```bash 145 | cat /proc/sys/fs/file-max 146 | ``` 147 | 148 | Users will need to logout and login again for the changes to take effect. If you want to apply the limit immediately, you can use the following command: 149 | 150 | ```bash 151 | sysctl -p 152 | ``` 153 | 154 | #### Set User Level Open File limits in Linux 155 | 156 | For that purpose, as user root, you will need to edit the following file: 157 | 158 | ```bash 159 | vi /etc/security/limits.conf 160 | ``` 161 | 162 | The lines that you should add take the following parameters: 163 | 164 | ```bash 165 | 166 | ``` 167 | 168 | Here is an example of setting a soft and hard limits for user cortex: 169 | ```bash 170 | ## Example hard limit for max opened files 171 | cortex hard nofile 409600 172 | ## Example soft limit for max opened files 173 | cortex soft nofile 102400 174 | ``` 175 | 176 | Another example of setting a soft and hard limits for all user: 177 | ```bash 178 | ## Example hard limit for max opened files 179 | cortex hard nofile 409600 180 | ## Example soft limit for max opened files 181 | cortex. / soft nofile 102400 182 | ``` 183 | 184 | 185 | 186 | ## Building the source 187 | 188 | Clone the source (Need permission) 189 | 190 | ```bash 191 | git clone https://github.com/CortexFoundation/CortexTheseus.git --branch dev 192 | (with git accessable key) 193 | 194 | cd CortexTheseus 195 | ``` 196 | 197 | Once the dependencies are installed, run 198 | 199 | ```bash 200 | make -j cortex 201 | ``` 202 | 203 | Save the executable file 204 | 205 | ```bash 206 | sudo mkdir -p /serving/cortex-core/bin 207 | sudo chmod 777 /serving/cortex-core/bin 208 | cp build/bin/cortex /serving/cortex-core/bin/cortex 209 | ``` 210 | 211 | The compiled binary files are located in the ./build/bin 212 | 213 | ```bash 214 | ./build/bin/cortex 215 | ``` 216 | 217 | 218 | ## Run the fullnode 219 | ### Fullnode executables directly 220 | 221 | ```bash 222 | ./cortex --port 37566 --rpc --rpccorsdomain '*' --rpcport 30089 --rpcaddr 127.0.0.1 --rpcapi web3,personal,ctxc,miner,net,txpool --verbosity 4 --gcmode archive 223 | ``` 224 | 225 | ### Create bash script 226 | 227 | ```bash 228 | rm /serving/cortex-core/bin/cortex.sh 229 | sudo nano /serving/cortex-core/bin/cortex.sh 230 | ``` 231 | 232 | Create /serving/cortex-core/bin/cortex.sh 233 | 234 | ```bash 235 | #!/bin/bash 236 | export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}} 237 | export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} 238 | /serving/cortex-core/bin/cortex --port 37566 --rpc --rpccorsdomain '*' --rpcport 30089 --rpcaddr 127.0.0.1 --rpcapi web3,ctxc,miner,net,txpool,personal --verbosity 4 --gcmode archive 239 | ``` 240 | 241 | Make the script executable 242 | 243 | ```bash 244 | sudo chmod +x /serving/cortex-core/bin/cortex.sh 245 | ``` 246 | 247 | ### config 248 | 249 | [program:cortexnode] 250 | directory=/serving/cortex-core/bin/ 251 | command=bash /serving/cortex-core/bin/cortex.sh 252 | autostart=true 253 | autorestart=true 254 | startsecs=5 255 | stderr_logfile=/tmp/cortex_fullnode_stderr.log 256 | stdout_logfile=/tmp/cortex_fullnode_stdout.log 257 | 258 | #### check running status 259 | 260 | ```bash 261 | sudo supervisorctl tail cortexnode stdout 262 | ``` 263 | 264 | ### Executables 265 | 266 | ------------------ 267 | 268 | #### Command 269 | 270 | use the parameter directly. 271 | 272 | | parameter | description | 273 | | --------- | -------------------------------------------------- | 274 | | help | Print Cortex binary help statements. | 275 | | init | Init Cortex node from a genesis configuration file | 276 | 277 | #### Options 278 | 279 | use "--" prefix with the parameter. 280 | 281 | | parameter | description | 282 | | --------------- | ------------------------------------------------------------ | 283 | | storage.dir | Set the directory of Cortex storage layer. By default, $HOME/.cortex/storage the directory is the storage dir. | 284 | | datadir | Set Cortex binary data directory. By default, $HOME/.cortex/ is data directory. | 285 | | port | Set Cortex binary listening port. | 286 | | bootnodes | [Optional] Set bootnodes of Cortex blockchain. | 287 | | verbosity | Set logging level to print, by default is 3, range in [1, 5], which represent Error, Warn, Info, Debug, Trace. | 288 | -------------------------------------------------------------------------------- /apple-demo.md: -------------------------------------------------------------------------------- 1 | 2 | # Apple Demo in Jetson Nano Car 3 | 4 | *Newcomers may be confused about what real-life application can CortexLabs bring to the table. This repository provides a quick glance into what we are doing at Cortex, as indicated on our [official website](www.cortexlabs.ai) introduced: AI on Blockchain.* 5 | 6 | ## A Compressed GIF Version 7 | 8 | ![demo](cvm/demo/demo.gif) 9 | 10 | 11 | 12 | The gif above is a demo Car running Jetson Nano by NVIDIA. The inference framework used is our very own [CVM Runtime](https://github.com/CortexFoundation/cvm-runtime) by CortexLabs. 13 | 14 | The full video has been uploaded on YouTube via CortexLabs official account. Click [Here](https://youtu.be/88c-446s9JE) to watch. 15 | 16 | ## Video Guide 17 | 18 | Below is the written steps shown in the video: 19 | 20 | 1. Execute deterministic AI framework, **cvm-runtime**, in an off-chain edge device that can make AI decision in realtime. In this case, the Jetson Nano Car. 21 | 2. Upload the image data to Cortex blockchain and call the on-chain AI contract to ensure results are consistent across the Jetson Nano Car and on-chain AI contract. 22 | The purpose is to: 23 | - Strictly reproduce real-time AI decisions in future requests for analysis or audit. 24 | - Add the confidence level of AI inference by using a decentralization and credible blockchain record will. Any doubt of the last AI decision will focus on the AI inference procedure and model creator, instead of the inference process. 25 | 26 | ## Implementation Details 27 | 28 | *The demo is tested in Cortex Dolores TestNet.* 29 | 30 | ### Real-time AI Inference In Jetson Nano Car 31 | 32 | Please clone our cvm-runtime project in GitHub and compile. Refer to the project link for more detail: 33 | 34 | [cvm-runtime in GitHub](https://github.com/CortexFoundation/cvm-runtime) 35 | 36 | #### Developer API 37 | 38 | We have created a few interfaces in cvm-runtime, both python and c++ are available to invoke. We will use the python API that is shown in the video demo for demonstration. The python API is located in `{ProjectRoot}/python`. 39 | 40 | ##### cvm.runtime 41 | 42 | The package has three main inference interface: 43 | 44 | | Name | Parameters | Function | 45 | | --------------- | ------------------------------ | ---------------------------------- | 46 | | CVMAPILoadModel | (json_str, param_bytes[, ctx]) | Load model and return reference | 47 | | CVMAPIFreeModel | (model_reference) | Free the loaded model by reference | 48 | | CVMAPIInference | (model_reference, input_data) | inference input via model | 49 | 50 | A simple code usage likes 51 | 52 | ```python 53 | # symbol_path and params_path is model stored in disk path. 54 | json_str, param_bytes = cvm.utils.load_model(symbol_path, params_path) 55 | model_reference = cvm.runtime.CVMAPILoadModel(json_str, param_bytes, ctx=cvm.gpu()) 56 | # input_data is cvm.nd.NDArray format, generally create in cvm.nd.array(numpy_data) 57 | result = cvm.runtime.CVMAPIInference(model_reference, input_data) 58 | # do something process the inference result ... 59 | # free the loaded model 60 | cvm.runtime.CVMAPIFreeModel(model_reference) 61 | ``` 62 | 63 | ### Upload Image to Blockchain 64 | 65 | Image upload ton the blockchain requires many procedures. All the lower and intrinsic representation format is a transaction with different parameters. The default raw transaction JSON RPC format is: 66 | 67 | #### ctxc_sendRawTransaction 68 | 69 | *Creates new message call transaction or a contract creation for signed transactions.* 70 | 71 | ##### Parameter Format 72 | 73 | `Object` - The transaction object 74 | 75 | - `from`: `DATA`, 20 Bytes - The address the transaction is sent from. 76 | - `to`: `DATA`, 20 Bytes - (optional when creating new contract) The address the transaction is directed to. 77 | - `gas`: `QUANTITY` - (optional) Integer of the gas provided for the transaction execution. It will return unused gas. 78 | - `gasPrice`: `QUANTITY` - (optional, default: To-Be-Determined) Integer of the gasPrice used for each paid gas 79 | - `value`: `QUANTITY` - (optional) Integer of the value sent with this transaction. 80 | - `data`: `DATA` - The compiled code of a contract OR the hash of the invoked method signature and encoded parameters. 81 | - `nonce`: `QUANTITY` - (optional) Integer of a nonce. This allows to overwrite your own pending transactions that use the same nonce. 82 | 83 | 1. `DATA`, The rlp encoded data of `object` above. 84 | 85 | ##### Returns 86 | 87 | `DATA`, 32 Bytes - the transaction hash, or the zero hash if the transaction is not yet available. 88 | 89 | Use `ctxc_getTransactionReceipt` to get the contract address, after the transaction ismined, you can create a contract. 90 | 91 | After acknowledging the native transaction rules, it's time to introduce the detailed upload procedure. 92 | 93 | #### 1. Create torrent for image data 94 | 95 | You should save the image data in folder called `data`, create the torrent file and organize at the same level. Directory structure should look like this: 96 | 97 | ``` 98 | root_directory/ 99 | ├── data 100 | └── torrent 101 | ``` 102 | 103 | #### 2. Announce image uploading onto blockchain 104 | 105 | First, we should tell the blockchain that we want to upload an image, and supply the image information to the blockchain via transaction before download. The most important thing is the torrent info hash. After blockchain node received the uploaded transaction, it will start to find the torrent corresponding with the info hash from tracker and DHT network. 106 | 107 | **Be aware** Do not forget to seed the `root_directory` directory list above in your local environment. It's important for chain-node to find the right torrent, or else the chain-node will not be able to find it then refuse to download the image you want to upload. You can use a torrent seeding server such as *qbittorrent*, *libtorrent* etc to serve the `root_directory`. 108 | 109 | Image is called as `Input` in blockchain, same as contract deployment that sends a transaction of key `to` is null. However, `Input` deployment has a unique hex prefix indicator: `0002` before the contract data. 110 | 111 | The `Input` meta is a structure of `Comments, InfoHash, RawSize, Shape `, and then one uses the RLP method to encode the list above. 112 | 113 | So an announce image upload trasaction may looks like this: 114 | 115 | - `method`: ctxc_sendRawTransaction 116 | - `parameters`: 117 | - `to`: none 118 | - `value`: 0 119 | - `data`: "0x0002" + RLPEncode([Comments, InfoHash, RawSize, Shape]) 120 | 121 | #### 3. Wait for seeding block 122 | 123 | After announcing the uploaded image on the blockchain, we have set some seeding blocks left for chain-node to download the torrent from DHT network. The default seeding block number in MainNet and TestNet is 6, that is you have to wait for six blocks(about 90 sec) mined before starting the next step. 124 | 125 | #### 4. Push upload progress 126 | 127 | `Input` uploading consumes CTXC. You should transfer to the `Input` contract address with a specific gas price: 277777. One of the specific transaction should push your upload progress on-chain with 512K size. The calculated formula of the transaction number to be sent is $\lceil RawSize / 512K \rceil$. 128 | 129 | #### 5. Wait for mature block 130 | 131 | Once the push process is done, the mature block is set to wait for the chain-node to download. The MainNet mature block number is 100, but the TestNet is set to 1 to speed up the testing. 132 | 133 | ### AI Contract on Blockchain Inference 134 | 135 | If you are new to contract deployment, use this [link](ai-contracts.md) to learn how to deploy a contract. 136 | 137 | Here is the core contract inference code in the video: 138 | 139 | ```javascript 140 | function bet(address _input, address _receiveAddress) payable public { 141 | require(msg.value == betPerGame, "Invalid bet!"); // constant betting amount 142 | uint256[] memory output = new uint256[](uint256(1)); 143 | infer(defaultModelAddress, _input, output); 144 | 145 | // Note: 146 | // this part is hardcoded for this specific model 147 | uint256 result = output[0] >> 16; 148 | uint256 mask = 0xff; 149 | 150 | // grabing the second 16 bits (index 1) 151 | int8 index1 = int8((result & (mask << (28 * 8))) >> (28 * 8)); 152 | // grabing the first 16 bits (index 0) 153 | int8 temp = int8((result & (mask << (29 * 8))) >> (29 * 8)); 154 | if(temp > index1) { 155 | emit BettingResult(false, 0, index1, temp); // or make reward negative 156 | return; 157 | } 158 | for(int8 i = 27; i >= 0; --i) { 159 | temp = int8((result & (mask << uint(i * 8))) >> uint(i * 8)); 160 | if(temp > index1) { 161 | emit BettingResult(false, 0, index1, temp); // or make reward negative 162 | return; 163 | } 164 | } 165 | uint256 rewardVal = msg.value * 2; // 2x odds 166 | _receiveAddress.transfer(rewardVal); 167 | emit BettingResult(true, rewardVal, index1, temp); 168 | } 169 | ``` 170 | 171 | The `bet` function will transfer 0.02 CTXC to you if on-chain AI recognize an image as apples as well as the deterministic AI framework in the Jetson Nano Car, you'll receive the reward CTXC after calling the contract. 172 | -------------------------------------------------------------------------------- /operators.md: -------------------------------------------------------------------------------- 1 | # Interacting with machin learning models 2 | 3 | Operators are one of the fundamental buidling blocks to interact with machine learning models on the Cortex network. A operator is a set of statements that performs a task or calculates a value. To use a function, you must define it somewhere in the scope from which you wish to call it. 4 | 5 | ## CVM Operators 6 | 7 | **nn** 8 | 9 | | OP | Name | Requirement | Default | 10 | |------------|---------------|-------------------|------------------| 11 | | Conv2D | channels | | | 12 | | | kernel_size | 2-D | | 13 | | | strides | 2-D | (1, 1) | 14 | | | padding | 2-D | (0, 0) | 15 | | | dilation | 2-D | (1, 1) | 16 | | | groups | 1 or in_channels | 1 | 17 | | | layout | NCHW | NCHW | 18 | | | kernel_layout | OIHW | OIHW | 19 | | | out_layout | __undef__ or NCHW | __undef__ | 20 | | | out_dtype | -1 or 4(kInt32) | -1 | 21 | | | use_bias | | TRUE | 22 | | | input data | <= INT8 | | 23 | | Dense | units | >=1 | | 24 | | | input data | <= INT8 | | 25 | | Upsampling | use_bias | | TRUE | 26 | | | scale | >0 | | 27 | | | layout | NCHW | NCHW | 28 | | | method | NEAREST_NEIGHBOR | NEAREST_NEIGHBOR | 29 | | MaxPool2D | pool_size | 2-D | | 30 | | | strides | 2-D | (1, 1) | 31 | | | padding | 1-D or 2-D | (0, 0) | 32 | | | layout | NCHW | NCHW | 33 | | | ceil_mode | FALSE | FALSE | 34 | 35 | **reduce** 36 | 37 | | OP | Name | Requirement | Default | 38 | |--------|----------|----------------------|---------| 39 | | sum,max | axis | -ndim <= axis < ndim | () | 40 | | | keepdims | | FALSE | 41 | | | exclude | | FALSE | 42 | | | dtype | kInt32 | kInt32 | 43 | 44 | **transform** 45 | 46 | | OP | Name | Requirement | Default | 47 | |---------------|-------------|--------------------------------|---------| 48 | | expand_dims | axis | -ndim-1 <= axis <= ndim | | 49 | | | num_newaxis | >=0 | 1 | 50 | | transpose | axes | NONE or =ndim | NONE | 51 | | reshape | shape | | | 52 | | squeeze | axis | -ndim <= axis < ndim | NONE | 53 | | concatenate | axis | -ndim <= axis < ndim | 1 | 54 | | take | axis | -ndim <= axis < ndim | NONE | 55 | | strided_slice | begin | | | 56 | | | end | | | 57 | | | stride | | | 58 | | repeat | repeats | >=1 | | 59 | | | axis | -ndim <= axis < ndim | 0 | 60 | | tile | reps | | NONE | 61 | | slice_like | axis | -src_ndim <= axis <= dest_ndim | NONE | 62 | | cvm_lut | | | | 63 | | flatten | | | | 64 | 65 | **vision** 66 | 67 | | OP | Name | Requirement | Default | 68 | |---------------------|-------------------|-------------------------|---------| 69 | | get_valid_count | score_threshold | | 0 | 70 | | non_max_suppression | return_indices | FALSE | FALSE | 71 | | | iou_threshold | multiply 100 by default | 50 | 72 | | | force_suppress | | FALSE | 73 | | | top_k | | -1 | 74 | | | id_index | | 0 | 75 | | | coord_start | | 2 | 76 | | | score_index | | 1 | 77 | | | max_output_size | | -1 | 78 | | | invalid_to_bottom | TRUE | TRUE | 79 | 80 | **broadcast** 81 | 82 | | OP | Name | Requirement | Default | 83 | |---------------|------|-------------|---------| 84 | | broadcast_add | | | | 85 | | broadcast_sub | | | | 86 | | broadcast_mul | | | | 87 | | broadcast_max | | | | 88 | 89 | **elemwise** 90 | 91 | | OP | Name | Requirement | Default | 92 | |-----------------|------|-------------|---------| 93 | | abs | | | | 94 | | log2 | | | | 95 | | elemwise_add | | | | 96 | | elemwise_sub | | | | 97 | | negative | | | | 98 | | clip | | | | 99 | | cvm_clip | | | | 100 | | cvm_right_shift | | | | 101 | | cvm_left_shift | | | | 102 | 103 | ## CVM Operator Attributes 104 | 105 | **nn/convolution.cc** 106 | 107 | | Name | Precision Check | Attribute Check | 108 | | ------ | --------------- | --------------- | 109 | | conv2d | ✓ | ✓ | 110 | 111 | **nn/nn.cc** 112 | 113 | | Name | Precision Check | Attribute Check | 114 | |--------|-----------------|-----------------| 115 | | conv2d | ✓ | ✓ | 116 | | dense | ✓ | ✓ | 117 | | relu | ✓ | ✓ | 118 | 119 | **nn/pooling.cc** 120 | 121 | | Name | Precision Check | Attribute Check | 122 | | ---------- | --------------- | --------------- | 123 | | max_pool2d | ✓ | ✓ | 124 | 125 | **nn/upsampling.cc** 126 | 127 | | Name | Precision Check | Attribute Check | 128 | | ---------- | --------------- | --------------- | 129 | | upsampling | ✓ | ✓ | 130 | 131 | **nn/nms.cc** 132 | 133 | | Name | Precision Check | Attribute Check | 134 | |---------------------|-----------------|-----------------| 135 | | non_max_suppression | ✓ | ✓ | 136 | | get_valid_counts | ✓ | ✓ | 137 | 138 | **tensor/broadcast.cc** 139 | 140 | | Name | Precision Check | Attribute Check | 141 | |---------------------|-----------------|-----------------| 142 | | broadcast_add | ✓ | ✓ | 143 | | broadcast_sub | ✓ | ✓ | 144 | | broadcast_mul | ✓ | ✓ | 145 | | broadcast_max | ✓ | ✓ | 146 | 147 | **tensor/reduce.cc** 148 | 149 | | Name | Precision Check | Attribute Check | 150 | |--------|-----------------|-----------------| 151 | | sum | ✓ | ✓ | 152 | | max | ✓ | ✓ | 153 | 154 | **tensor/elemwise.cc** 155 | 156 | | Name | Precision Check | Attribute Check | 157 | |---------------------|-----------------|-----------------| 158 | | abs | ✓ | ✓ | 159 | | log2 | ✓ | ✓ | 160 | | elemwise_add | ✓ | ✓ | 161 | | elemwise_sub | ✓ | ✓ | 162 | | negative | ✓ | ✓ | 163 | | clip | ✓ | ✓ | 164 | | cvm_clip | ✓ | ✓ | 165 | | cvm_right_shift | ✓ | ✓ | 166 | | cvm_left_shift | ✓ | ✓ | 167 | 168 | **tensor/transform.cc** 169 | 170 | | Name | Precision Check | Attribute Check | 171 | |---------------------|-----------------|-----------------| 172 | | repeat | ✓ | ✓ | 173 | | tile | ✓ | ✓ | 174 | | flatten | ✓ | ✓ | 175 | | concatenate | ✓ | ✓ | 176 | | expand_dims | ✓ | ✓ | 177 | | reshape | ✓ | ✓ | 178 | | squeeze | ✓ | ✓ | 179 | | transpose | ✓ | ✓ | 180 | | slice\|strided_slice |✓ | ✓ | 181 | | take | ✓ | ✓ | 182 | | cvm_lut | ✓ | ✓ | 183 | | slice_like | ✓ | ✓ | -------------------------------------------------------------------------------- /FHE/BGV12_cn.md: -------------------------------------------------------------------------------- 1 | # [BGV12]方案 - (Leveled) Fully Homomorphic Encryption without Bootstrapping 2 | 3 | ## 安全基础 4 | 5 | BV11方案基于LWE（Learning With Error）困难问题，计算都为整数有限域 $\mathbb{Z}_q$。 6 | 7 | BGV12方案引入了RLWE（Ring Learning With Error）困难问题，在同样安全性下运算性能更快。 8 | 9 | 标量从整数变为多项式环上的元素，$R_q=\mathbb{Z}_q[x]/(x^d+1)$，表示环上的多项式都对 $x^d+1$ 取模（$d$ 是2的幂），并且每个系数都对 $q$ 取模。 10 | 11 | BGV12结合LWE和RLWE提出GLWE，其实就是允许 $d = 1$ 成立，此时退化为LWE的情况。 12 | 13 | ## 思路简介 14 | 15 | 前面介绍的[BV11方案](BV11_cn.md)中，在完成 $L$ 层运算之后运用了模数-维数压缩技术，将密文的维数和模数从 $(n, q)$ 压缩到 $(k, p)$，使得BV11方案能正确完成解密的计算。 16 | 17 | Brakerski等人发现模数压缩和维数压缩可以分开处理，并且模数压缩还同时能降低噪音的绝对值大小（并不能降低噪音的比例，但是降低噪音的绝对值大小可以降低后续计算噪音增长的速率），并且该技术可以多次使用。 18 | 19 | 简化噪音的原理在于，假设初始噪音为 $E$， 总上限为 $q = E^n$，每次乘法运算噪音变为 $E^2, E^4,...,E^{2^L}$，为了防止噪音的规模达到同一量级而影响运算，需满足 $E^{2^L} \ll q$，即 $L \ll \log n$。而每次乘法运算后采用模数压缩技术，可以将噪音规模降回 $E$，但同时模数上限降到$q/E^L$，因此 $L \ll n$，从对数级别提升到线性级别。 20 | 21 | 具体的实现方式为设计一系列满足梯度的密钥 $s_0, s_1, ..., s_L$，其中 $s_j$ 的模数长度为 $(j+1) \cdot \mu$，因此满足 $q_j / q_{j-1} = E$ 22 | 23 | 因此，BGV12方案主要在BV11方案的基础上，将重线性技术和维数-模数压缩技术结合起来，每次重线性的同时进行压缩。这一方案可以有效降低噪音增长的速率（从对数级别到线性级别），在不使用Bootstrapping的情况下完成 $L$ 层电路计算，每个门电路的复杂度为 $\tilde{O}(\lambda \cdot L^3)$。当计算层数过深时，复杂度增长剧烈，因此这一方案也不能无限进行，还是需要Bootstrapping来达成FHE。 24 | 25 | 26 | 27 | ## 引理 28 | 29 | ### 引理1 30 | 31 | 令 $p$ 和 $q$ 均为奇数模数，$\textbf{c}$ 为整数向量，定义 $\textbf{c}^{\prime}$ 为满足 $\textbf{c}^\prime = \textbf{c} \mod 2$ 中最接近 $(p/q) \cdot \textbf{c}$ 的整数向量。对任意满足 $|[\lang \textbf{c}, \textbf{s} \rang]_q| < q/2 - (q/p) \cdot \ell_1(\textbf{s})$ 的向量 $\textbf{s}$，有 32 | $$ 33 | [\lang \textbf{c}^\prime, \textbf{s} \rang]_p = [\lang \textbf{c}, \textbf{s} \rang]_q \mod 2 \ \ \textbf{and} \ \ |[\lang \textbf{c}^\prime, \textbf{s} \rang]_p| < (p/q) \cdot |[\lang \textbf{c}, \textbf{s} \rang]_q| + \ell_1(\textbf{s}) 34 | $$ 35 | 其中 $\ell_1(\textbf{s})$ 表示向量 $\textbf{s}$ 的 $\ell_1$ 范式 36 | 37 | **证明**： 38 | 39 | 存在整数 $k$，使得 $[\lang \textbf{c}, \textbf{s} \rang]_q = \lang \textbf{c}, \textbf{s} \rang - kq$。对同样的 $k$，令 $e_p = \lang \textbf{c}^\prime, \textbf{s} \rang - kp \in \mathbb{Z}$，因为 $\textbf{c}^\prime = \textbf{c} \mod 2$， $p = q = 1\mod 2$ 40 | 41 | 所以有 $e_p = \lang \textbf{c}^\prime, \textbf{s} \rang - kp \equiv \lang \textbf{c}, \textbf{s} \rang - kq = [\lang \textbf{c}, \textbf{s} \rang]_q \mod 2$ 42 | 43 | 还需要证明 $e_p = [\lang \textbf{c}^\prime, \textbf{s} \rang]_p$，并且范式值足够小 44 | 45 | 有 $e_p = (p/q)[\lang \textbf{c}, \textbf{s} \rang]_q + \lang \textbf{c}^\prime - (p/q)\textbf{c}, \textbf{s} \rang$ 46 | 47 | 因此，$|e_p| \leq (p/q)[\lang \textbf{c}, \textbf{s} \rang]_q + \ell_1(s) < p/2$，不等式的后者证明 $e_p = [\lang \textbf{c}^\prime, \textbf{s} \rang]_p$ 48 | 49 | 50 | 51 | ### 引理2 52 | 53 | 对长度相同的向量 $\textbf{c}, \textbf{s}$，有 $\lang {\sf BitDecomp}({\bf c}, q), {\sf Powersof2}({\bf s}, q) \rang = \lang {\bf c}, {\bf s} \rang \mod q$ 54 | 55 | **证明**： 56 | $$ 57 | \lang {\sf BitDecomp}({\bf c}, q), {\sf Powersof2}({\bf s}, q) \rang = \sum_{j=0}^{\lfloor \log q \rfloor} \lang {\bf u}_j,2^j \cdot {\bf s} \rang = \sum_{j=0}^{\lfloor \log q \rfloor} \lang 2^j \cdot {\bf u}_j,{\bf s} \rang = \lang \sum_{j=0}^{\lfloor \log q \rfloor} 2^j \cdot {\bf u}_j,{\bf s} \rang = \lang {\bf c}, {\bf s} \rang 58 | $$ 59 | 60 | ## 基于GLWE的基础加密方案 E 61 | 62 | ### 参数初始化 63 | 64 | **输入**： $(1^\lambda, 1^\mu, b)$ 65 | 66 | **输出**：初始化参数 $params = (q, d, n, N, \chi)$ 67 | 68 | **过程**：用 $b \in \{0,1\}$ 决定采用基于LWE的方案（$d=1$）还是基于RLWE的方案（$n=1$），选择一个 $\mu$-bit 的模数 $q$ 以及其他参数。 69 | 70 | $d = d(\lambda, \mu, b), n = n(\lambda, \mu, b), N = \lceil (2n+1) \log q \rceil, \chi = \chi(\lambda, \mu, b)$ 71 | 72 | 这些参数保障该方案针对已知攻击的安全性达到 $2^\lambda$ 73 | 74 | 令 $R = \mathbb{Z}[x]/(x^d + 1)$，以及 $params = (q, d, n, N, \chi)$ 75 | 76 | ### 私钥生成 77 | 78 | **输入**：$params = (q, d, n, N, \chi)$ 79 | 80 | **输出**：私钥 $sk \in R_q^{n+1}$ 81 | 82 | **过程**：抽样 $\textbf{s}^\prime \leftarrow \chi^n$，令 $sk = \textbf{s} \leftarrow (1, \textbf{s}^\prime[1], ..., \textbf{s}^\prime[n]) \in R_q^{n+1}$ 83 | 84 | ### 公钥生成 85 | 86 | **输入**：$params = (q, d, n, N, \chi), sk$ 87 | 88 | **输出**：公钥 $pk \in R_q^{N \times (n+1)}$ 89 | 90 | **过程**：生成均匀分布的矩阵 $\textbf{A}^\prime \leftarrow R_q^{N \times n}$ 以及向量 $\textbf{e} \leftarrow \chi^N$ 91 | 92 | 令 $\textbf{b} \leftarrow \textbf{A}^\prime \textbf{s}^\prime + 2\textbf{e}$，$\textbf{A} \leftarrow [\textbf{b}\ |\ -\textbf{A}^\prime]$，有 $\textbf{A} \cdot \textbf{s} = \textbf{b} - \textbf{A}^\prime \textbf{s}^\prime = 2\textbf{e}$ 93 | 94 | 令 $pk = \textbf{A}$ 95 | 96 | ### 加密算法 97 | 98 | **输入**：$params = (q, d, n, N, \chi), pk, m \in \{0,1\}$ 99 | 100 | **输出**：密文 $\textbf{c} \in R_q^{n+1}$ 101 | 102 | **过程**：令 $\textbf{m} \leftarrow (m, 0, ..., 0) \in R_q^{n+1}$，抽样 $\textbf{r} \leftarrow R_2^N$ 103 | 104 | 密文 $\textbf{c} \leftarrow \textbf{m} + \textbf{A}^T \textbf{r} \in R_q^{n+1}$ 105 | 106 | ### 解密算法 107 | 108 | **输入**：$params = (q, d, n, N, \chi), sk, \textbf{c}$ 109 | 110 | **输出**：明文 $m$ 111 | 112 | **过程**：$m \leftarrow [[\lang \textbf{c}, \textbf{s} \rang]_q]_2$ 113 | 114 | 115 | 116 | ## 密钥切换（维数压缩） 117 | 118 | ### 切换密钥生成 119 | 120 | **输入**：切换前后的两个维数不同的密钥 $\textbf{s}_1 \in R_q^{n_1}$，$\textbf{s_2} \in R_q^{n_2}$ 121 | 122 | **输出**：切换矩阵 $\tau_{\textbf{s}_1 \rightarrow \textbf{s}_2}$ 123 | 124 | **过程**： 125 | 126 | 1. 计算 $\textbf{A} \leftarrow {\sf E.PublicKeyGen}(\textbf{s}_2, N)$，其中 $N = n_1 \cdot \lceil \log q \rceil$ 127 | 2. 令 $\textbf{B} \leftarrow \textbf{A} + {\sf Powerof2}(\textbf{s}_1)$，加到矩阵 $\textbf{A}$ 的第一列，输出 $\tau_{\textbf{s}_1 \rightarrow \textbf{s}_2} = \textbf{B}$ 128 | 129 | ### 密钥切换（实际是对密文的切换，保障切换后的密文能用新密钥正常解密） 130 | 131 | **输入**：切换矩阵及切换前的密文 $\tau_{\textbf{s}_1 \rightarrow \textbf{s}_2} = \textbf{B}, \textbf{c}_1$ 132 | 133 | **输出**：切换后的密文 $\textbf{c}_2$ 134 | 135 | **过程**：计算 $\textbf{c}_2 = {\sf BitDecomp(\textbf{c}_1)}^T \cdot \textbf{B} \in R_q^{n_2}$ 136 | 137 | ### 引理3 【正确性】 138 | 139 | 令 $\textbf{s}_1, \textbf{s}_2, q, n_1, n_2, \textbf{A}, \textbf{B} = \tau_{\textbf{s}_1 \rightarrow \textbf{s}_2} \leftarrow {\sf SwitchKeyGen}(\textbf{s}_1, \textbf{s}_2)$，$\textbf{A} \cdot \textbf{s}_2 = 2 \textbf{e}_2 \in R_q^N$，$\textbf{c}_1 \in R_q^{n_1}$ 以及 ${\bf c}_2 \leftarrow {\sf SwitchKey}(\tau_{{\bf s}_1 \rightarrow {\bf s}_2}, {\bf c}_1)$ 140 | 141 | 有 $\lang {\bf c}_2, {\bf s}_2 \rang = 2 \lang {\sf BitDecomp}({\bf c}_1), {\bf e}_2 \rang + \lang {\bf c}_1, {\bf s}_1 \rang \mod q$ 142 | 143 | **证明**： 144 | $$ 145 | \begin{aligned} 146 | \lang {\bf c}_2, {\bf s}_2 \rang &= {\sf BitDecomp}({\bf c}_1)^T \cdot \textbf{B} \cdot {\bf s}_2 \\ 147 | &= {\sf BitDecomp}({\bf c}_1)^T \cdot (2 {\bf e}_2 + {\sf Powersof2}({\bf s}_1)) \\ 148 | &= 2 \lang {\sf BitDecomp}({\bf c}_1), {\bf e}_2 \rang + \lang {\sf BitDecomp}({\bf c}_1), {\sf Powersof2}({\bf s}_1)) \rang \\ 149 | &= 2 \lang {\sf BitDecomp}({\bf c}_1), {\bf e}_2 \rang + \lang {\bf c}_1, {\bf s}_1 \rang \\ 150 | \end{aligned} 151 | $$ 152 | 在此基础上，由于 ${\sf BitDecomp}({\bf c}_1)$ 和 ${\bf e}_2$ 都很小，所以新增噪音 $2 \lang {\sf BitDecomp}({\bf c}_1), {\bf e}_2 \rang$ 的数量级别较小，并且保持2的倍数这一性质，不影响解密。可以认为 $c_2$ 是 $m$ 在密钥 $s_2$ 下的密钥。 153 | 154 | ## 模数切换 155 | 156 | ### 放缩 157 | 158 | 对整数向量 ${\bf x}$ 和整数 $q > p > m$，定义 ${\bf x}^\prime \leftarrow {\sf Scale}({\bf x}, q, p, r)$ 为满足 ${\bf x}^\prime \equiv {\bf x} \mod r$ 中最接近 $(p/q) \cdot {\bf x}$ 的向量 159 | 160 | ### $\ell_1^{(R)}$ 范数 161 | 162 | 通常在实数域上的范数 $\ell_1({\bf s}) = \sum_i \| s[i] \|$ ，拓展到环上向量 ${\bf s} \in R^n$，定义 $\ell_1^{(R)}({\bf s}) = \sum_i \| s[i] \|$ 163 | 164 | ### 引理4 【噪音变化】 165 | 166 | 可视为引理1的扩展版，对环上向量进行补充 167 | 168 | 令 $d$ 为环的度，$q > p > r > 0$ 满足 $q = p = 1 \mod r$。令 ${\bf c} \in R^n$ 并且 ${\bf c}^\prime \leftarrow {\sf Scale}({\bf c}, q, p, r)$。对任意 ${\bf s} \in R^n$，满足 $\| [\lang {\bf c}, {\bf s} \rang]_q \| = q/2 - (q/p) \cdot (r/2) \cdot \sqrt{d} \cdot \gamma(R) \cdot \ell_1^{(R)}({\bf s})$ ，则有 169 | $$ 170 | [\lang {\bf c}^\prime, {\bf s} \rang]_p \equiv [\lang \textbf{c}, \textbf{s} \rang]_q \mod 2 \ \ \textbf{and} \ \ |[\lang \textbf{c}^\prime, \textbf{s} \rang]_p| < (p/q) \cdot \|[\lang \textbf{c}, \textbf{s} \rang]_q\| + (r/2) \cdot \sqrt{d} \cdot \gamma(R) \cdot \ell_1(\textbf{s}) 171 | $$ 172 | 173 | 174 | ## 具体方案实现 FHE 175 | 176 | ### 参数初始化 177 | 178 | **输入**： $(1^\lambda, 1^L, b)$ 179 | 180 | **输出**：初始化参数集合{$params_j = (q_j, d, n_j, N, \chi)$} 181 | 182 | **过程**：用 $b \in \{0,1\}$ 决定采用基于LWE的方案（$d=1$）还是基于RLWE的方案（$n=1$），令 $\mu = (\lambda, L, b) = \theta(\log \lambda + \log L)$ 。 183 | 184 | 循环 $j \leftarrow L ...0$，生成参数 $params_j \leftarrow {\sf E.Setup}(1^\lambda, 1^{(j+1) \cdot \mu}, b)$ 185 | 186 | 构成一个梯度序列的参数，其中模数从 $q_L$($(L+1)\cdot \mu$ bits) 到 $q_0$($\mu$ bits) 187 | 188 | ### 密钥生成算法 189 | 190 | **输入**：初始化产生的参数 $\{params_j\}$ 191 | 192 | **输出**：私钥 $sk \leftarrow {\bf s}_j$，公钥 $pk \leftarrow (\{{\bf A_j}\}, \{\tau_{{\bf s}_{j+1}^{\prime\prime} \rightarrow {\bf s}_j}\})$ 193 | 194 | **过程**：循环 $j \leftarrow L ...0$ 195 | 196 | 1. 生成私钥和公钥 197 | 198 | 计算私钥 $ {\bf s}_j \leftarrow {\sf E.SecretKeyGen}({\bf s}_j)$ 和公钥 $\textbf{A}_j \leftarrow {\sf E.PublicKeyGen}(params_j, {\bf s}_j)$ 199 | 200 | 2. 计算私钥二次项 201 | 202 | 计算 ${\bf s}_j^\prime \leftarrow {\bf s}_j \otimes {\bf s}_j \in R_{q_j}^{\tbinom{n_j+1}{2}}$ 203 | 204 | 3. 平展私钥 205 | 206 | 计算 ${\bf s}_j^{\prime\prime} \leftarrow {\sf BitDecomp}({\bf s}_j^\prime, q_j)$ 207 | 208 | 4. 生成密钥切换 209 | 210 | 计算 $\tau_{{\bf s}_{j+1}^{\prime\prime} \rightarrow {\bf s}_j} \leftarrow {\sf SwitchKeyGen}({\bf s}_{j+1}^{\prime\prime}, {\bf s}_{j})$，当 $j = L$ 时可以忽略该步骤 211 | 212 | ### 加密算法 213 | 214 | **输入**：参数 $params$，公钥 $pk$，消息 $m \in R_2$ 215 | 216 | **输出**：密文 $\textbf{c} \in R_q^{n+1}$ 217 | 218 | **过程**：计算 ${\sf E.Enc}({\bf A}_L, m)$ 219 | 220 | 对明文加密得到第一层的密钥 221 | 222 | ### 解密算法 223 | 224 | **输入**：参数 $params$，私钥 $sk$，密文 ${\bf c}$ 225 | 226 | **输出**：明文 $m$ 227 | 228 | **过程**：假设密文由密钥 ${\bf s}_j$ 加密，计算 ${\sf E.Dec}({\bf s}_j, {\bf c})$ 229 | 230 | 可以参考BV11增加层级标签 231 | 232 | ### 同态加法 233 | 234 | **输入**：公钥 $pk$，加数密文 ${\bf c}_1, {\bf c}_2$ 235 | 236 | **输出**：和密文 ${\bf c}_{add}$ 237 | 238 | **过程**： 239 | 240 | 1. 直接求和 241 | 242 | ${\bf c}_3 \leftarrow {\bf c}_1 + {\bf c}_2 \mod q_j$ 243 | 244 | 2. 重制密文 245 | 246 | ${\bf c}_{add} \leftarrow {\sf FHE.Refresh}({\bf c}_3, \tau_{{\bf s}_{j}^{\prime\prime} \rightarrow {\bf s}_{j-1}}, q_j, q_{j-1})$ 247 | 248 | ### 同态乘法 249 | 250 | **输入**：公钥 $pk$，加数密文 ${\bf c}_1, {\bf c}_2$ 251 | 252 | **输出**：乘积密文 ${\bf c}_{mult}$ 253 | 254 | **过程**： 255 | 256 | 1. 做乘法 257 | 258 | ${\bf c}_3 \leftarrow {\bf c}_1 \otimes {\bf c}_2 \mod q_j$，匹配密钥 ${\bf s}_j^\prime \leftarrow {\bf s}_j \otimes {\bf s}_j$。 259 | 260 | 2. 重制密文 261 | 262 | ${\bf c}_{add} \leftarrow {\sf FHE.Refresh}({\bf c}_3, \tau_{{\bf s}_{j}^{\prime\prime} \rightarrow {\bf s}_{j-1}}, q_j, q_{j-1})$ 263 | 264 | ### Refresh 265 | 266 | **输入**：密文 ${\bf c}$，公钥$\tau_{{\bf s}_{j}^{\prime\prime} \rightarrow {\bf s}_{j-1}}$，两层的模数$q_j$，$q_{j-1}$ 267 | 268 | **输出**：切换后的密文 ${\bf c}^\prime$ 269 | 270 | **过程**： 271 | 272 | 1. 平展：计算 ${\bf c}_1 \leftarrow {\sf Powerof2}({\bf c}, q_j)$，由引理2可知 $\lang {\bf c}_1, {\bf s}_j^{\prime\prime} \rang = \lang {\bf c}, {\bf s}_j^{\prime} \rang \mod q_j$ 273 | 2. 模数切换：计算 ${\bf c}_2 \leftarrow {\sf Scale}({\bf c}_1, q_j, q_{j-1}, 2)$，该密文匹配私钥 $s_j^{\prime\prime}$，适用模数 $q_{j-1}$ 274 | 3. 密钥切换：计算 ${\bf c}_3 \leftarrow {\sf SwitchKey}(\tau_{{\bf s}_{j}^{\prime\prime} \rightarrow {\bf s}_{j-1}}, {\bf c}_2, q_{j-1})$，输出密文 ${\bf c}^\prime = {\bf c}_3$ 275 | 276 | -------------------------------------------------------------------------------- /FHE/BV11_cn.md: -------------------------------------------------------------------------------- 1 | # [BV11]方案 - Efficient Fully Homomorphic Encryption from (Standard) LWE 2 | 3 | ## 安全基础 4 | 5 | 和此前基于格困难问题的方案不同，BV11方案基于LWE（Learning With Error）困难问题。 6 | 7 | 简单来说，该问题中，存在一个密钥向量$\textbf{s}$，和噪音$e$。选择一组随机向量$\{\textbf{a}_i\}$，计算 $\textbf{b}_i=\textbf{a}_i \cdot \textbf{s} + e$。那么在不知道密钥的情况下，难以区分 $\textbf{b}_i$ 和另一组随机值。 8 | 9 | ## 简单介绍 10 | 11 | BV11方案基于LWE问题构造同态加密，传统加密技术的明密文一一对应，即同样的明文在同样密钥的情况下，加密得到的密文是相同的。而这一加密引入了随机采样和随机噪音，使得密文结果并不唯一，并且在解密后得到的信息带有噪音，需要去除噪音后才能得到明文。 12 | 13 | 同态加法的实现是非常平凡的，而同态乘法的结果会产生二次项，导致多项式系数翻倍的问题。为了保证同态乘法后的系数保持线性个数，BV11采用了重线性技术，即采用新的密钥对原密钥进行重制，假设整个计算中需要 $L$ 层乘法，那么需要预先计算出 $L$ 层的密钥。 14 | 15 | 此外，为了防止噪音的规模乘上系数后变得过大，BV11采用了分解技术，将系数分解到二进制形式，每位只能为0或1，这样维持了LWE问题的假设 16 | 17 | ### 加密解密 18 | 19 | 基于LWE困难问题，BV11方案构造了一种加密方式，对消息的1bit进行加密，即$m \in \{0,1\}$。 20 | 21 | 加密计算： $b=\langle\textbf{a},\textbf{s}\rangle+2e+m \in \mathbb{Z}_q$，密文 $c =(\textbf{a},b) \in \mathbb{Z}_q^n \times \mathbb{Z}_q$ 22 | 23 | 如果知道密钥 $\textbf{s}$，则可以完成解密计算：$m + 2e = b - \langle \textbf{a}, \textbf{s} \rangle$，对2取模得到 $m$，$m = [b - \langle \textbf{a}, \textbf{s} \rangle]_2$ 24 | 25 | 解密函数可以写为：$f_{\textbf{a},b}(\textbf{x})=b-\langle{\textbf{a},\textbf{x}}\rangle (mod\ q)=b-\sum_{i=1}^n{\textbf{a}[i]\cdot\textbf{x}[i]} \in \mathbb{Z}_q$，当代入密钥 $\textbf{s}$，即可算出 $f_{\textbf{a},b}(\textbf{s}) = m + 2e$ 26 | 27 | 从这里我们可以看出，因为模数 $q$ 为奇数，为了保障 $2e+m$ 的奇偶性不变，那么噪音 $e$ 的规模不能超过 $q$。 28 | 29 | ### 同态运算 30 | 31 | 假设现有密文$c = (\textbf{a}, b)$，$c^{'} = (\textbf{a}^{'}, b^{'})$，考虑对这两个密文进行同态运算，得到满足同态性质的新密文。 32 | 33 | **同态加法**：$c_{add} = (\textbf{a}_{add}, b_{add}) = (\textbf{a} + \textbf{a}^{'}, b + b^{'})$ 34 | 35 | 那么同样的解密函数得到：$f_{\textbf{a}+\textbf{a}^{'},b+b^{'}}(\textbf{x})=b+b^{'}-\langle{\textbf{a}+\textbf{a}^{'},\textbf{x}}\rangle (mod\ q)=f_{\textbf{a},b}(\textbf{x})+f_{\textbf{a}^{'},b^{'}}(\textbf{x}) = m + m^{'} + 2e + 2e^{'}$ 36 | 37 | 对2取模后与 $m + m^{'}$ 同余，保证计算正确性 38 | 39 | **同态乘法**：$c_{mult} = (\textbf{a}_{mult}, b_{mult})$ 40 | 41 | 先看解密函数的乘法结果：$f_{\textbf{a},b}(\textbf{x})\cdot f_{\textbf{a}^{'},b^{'}}(\textbf{x}) = \left(m+2e\right)\cdot \left(m^{'} + 2e^{'}\right) = m \cdot m^{'} + 2m\cdot e^{'} + 2m^{'} \cdot e + 4e \cdot e^{'}$ 42 | 43 | 对2取模后与 $ m \cdot m^{'} $ 同余，保证计算正确性 44 | 45 | 从具体表达式角度来看：$f_{\textbf{a},b}(\textbf{x})\cdot f_{\textbf{a}^{'},b^{'}}(\textbf{x}) = \left(b-\sum{\textbf{a}[i]\textbf{x}[i]}\right)\cdot \left(b^{'}-\sum{\textbf{a}^{'}[i]\textbf{x}[i]}\right) = h_0+\sum{h_i\cdot \textbf{x}[i]}+\sum{h_{i,j}\cdot \textbf{x}[i]\textbf{x}[j]}$ 46 | 47 | 其中$h_0, h_i, h_{i,j}$都是合并同类项后得到的参数，这里存在一个问题，由于二次项的存在，参数的个数从$n+1$增加到$n^2$左右，以及不能保持同样的线性解密函数。 48 | 49 | 为了解决系数爆炸问题，需要采用**重线性（re-linearization）**技术，通过**转换密钥**保持解密函数的系数个数为线性个数 50 | 51 | **重线性（re-linearization）**：解决系数爆炸问题，核心在于处理二次项 $\textbf{s}[i]\textbf{s}[j]$ ，采用生成新密钥来实现 52 | 53 | 密钥 $\textbf{s}$ 的持有者可以生成另一个密钥 $t$，把 $\textbf{s}[i]\textbf{s}[j]$ 当作消息进行加密隐藏，计算 $b_{i,j} = \textbf{a}_{i,j} \cdot \textbf{t} + e_{i,j} + \textbf{s}[i]\textbf{s}[j]$ 54 | 55 | 令 $\textbf{s}[i] = 1$，那么原来的乘法结果可以写为 $\sum_{i,j}{h_{i,j}\cdot (b_{i,j}-\langle{\textbf{a}_{i,j},\textbf{t}}\rangle)}$ 56 | 57 | 这里又面临一个新的问题，$b_{i,j}-\langle{\textbf{a}_{i,j},\textbf{t}}\rangle \approx \textbf{s}[i]\textbf{s}[j]$，但不代表 $h_{i,j}\cdot (b_{i,j}-\langle{\textbf{a}_{i,j},\textbf{t}}\rangle) \approx h_{i,j} \cdot \textbf{s}[i]\textbf{s}[j]$，因为系数 $h_{i,j}$ 可能很大 58 | 59 | **分解（BitDecomp）**：解决系数绝对值大的问题，需要对 $h_{i,j}$ 展开到二进制形式 60 | 61 | $h_{i,j} = \sum_{\tau=0}^{\lfloor{log\ q}\rfloor}h_{i,j,\tau}2^{\tau}$（因为所有系数都在有限域 $Z_q$ 上，$h_{i,j}$ 最多有 $\lfloor{log\ q}\rfloor$ 位） 62 | 63 | 乘法结果变为$\sum_{i,j,\tau}{h_{i,j,\tau}2^{\tau} \textbf{s}[i]\textbf{s}[j]}$ 64 | 65 | 因为这一改变，对新密钥 $\textbf{t}$ 的使用也需要一点儿变化，令 $b_{i,j,\tau} = \langle \textbf{a}_{i,j,\tau}, \textbf{t}\rangle + e_{i,j,\tau} + 2^{\tau}\textbf{s}[i]\textbf{s}[j]$，有 $2^{\tau}\textbf{s}[i]\textbf{s}[j] \approx b_{i,j,\tau} - \langle \textbf{a}_{i,j,\tau}, \textbf{t} \rangle$ 66 | 67 | 乘法结果变为$\sum_{i,j,\tau}{h_{i,j,\tau} \cdot (b_{i,j,\tau} - \langle \textbf{a}_{i,j,\tau}, \textbf{t} \rangle)}$ 68 | 69 | **密钥链（key chain）**：新设计单个密钥 $\textbf{t}$ 能完成一次乘法，为了完成 $L$ 层乘法，可以用同样的方法构造 $L$ 个依次构造的密钥，称之为密钥链。 70 | 71 | ### Bootstrapping及模数切换 72 | 73 | 进行了一系列同态运算后，密文中的噪音会逐步累积到临界值，为了保障进一步运算后的结果能够还原到正确的明文，需要采用Bootstrapping技术，简单来说，该技术首先对噪音较大的密文进行解密，然后重新用初始噪音进行加密，保证噪音的规模恢复到较小的状态，这样又可以用该密文进一步计算了。但是LWE的解密的复杂度过大，经过解密计算后的明文已经可能出现错误，为了解决这一问题，BV11提出了模数切换技术，这一技术可以保障Bootstrapping的正确执行，进而保障整个加密方案可以无限步骤地计算下去，为全同态加密方案。 74 | 75 | **Bootstrapping**：该技术本身并不复杂，通过对噪音较大的密文进行解密，然后重新用初始噪音进行加密的方法来控制噪音的规模，保障后续计算的正确性。假设噪音规模为 $E$，那么经过一次乘法运算后噪音规模为 $E^2$， $L$ 层乘法之后，噪音规模变为 $E^{2^L}$。而前面我们讲到噪音不能超过模数 $q$，而LWE电路解密的深度是$max(n, \log q)$，不能直接解密。 76 | 77 | **维数-模数压缩（Dimension-Modulus reduction）**：如上文所说，解密电路深度是由维数 $n$ 和模数 $q$ 决定的，因此如果能减小这两个数值，即可得到可计算的解密电路。我们希望将$(n, \log q)$减小到$(k, \log p)$，使得该方案能执行Bootstrapping。 78 | 79 | 技术的主要思路为：假设当前已经完成 $L$ 层计算，采用的密钥为 $S_L$。重新采样维数和维数较小的短密钥 $\widehat{\textbf{s}}$，对当前的长密钥 $S_L$ 进行重制，利用 $\widehat{\textbf{s}}$ 完成密文的维数-模数压缩，最后对短密文解密得到明文。 80 | 81 | ## 具体方案实现 82 | 83 | 普通加密方案分为**密钥生成算法**，**加密算法**，**解密算法**三个部分，全同态加密方案在这三部分之外还增加了**同态计算算法**。 84 | 85 | ### 密钥生成算法 86 | 87 | **输入**：密码学安全参数 $(1^\kappa)$ 88 | 89 | **输出**：私钥 $sk = \widehat{\textbf{s}}$，公钥 $pk = (\textbf{A}, \textbf{b})$，计算密钥 $evk = (\Psi, \widehat{\Psi})$ 90 | 91 | **过程**： 92 | 93 | 1. 生成计算密钥 94 | 95 | 随机采样 $L+1$ 个长密钥 $\textbf{s}_0,...,\textbf{s}_L \stackrel{$}{\leftarrow} \mathbb{Z}^n_q$，对 $\ell \in [L], 0 \le i \le j \le n, \tau \in \{0,...,\lfloor \log q \rfloor \}$，随机采样 $\textbf{a}_{\ell, i, j, \tau} \stackrel{$}{\leftarrow}\mathbb{Z}^n_q$，$e_{\ell, i, j, \tau} \stackrel{$}{\leftarrow} \chi$ 96 | 97 | 计算 $b_{\ell, i, j, \tau} := \langle \textbf{a}_{\ell, i, j, \tau} , \textbf{s}_{\ell} \rangle + 2 \cdot e_{\ell, i, j, \tau} + 2^{\tau} \cdot \textbf{s}_{\ell - 1}[i] \cdot \textbf{s}_{\ell - 1}[j]$，令 $\psi_{\ell, i, j, \tau} := (\textbf{a}_{\ell, i, j, \tau}, b_{\ell, i, j, \tau}) \in \mathbb{Z}_q^n \times \mathbb{Z}_q$，$\Psi \triangleq \{\psi_{\ell, i, j, \tau}\}$ 98 | 99 | 这里可以将 $\psi_{\ell, i, j, \tau}$ 视为对 $2^{\tau} \cdot \textbf{s}_{\ell - 1}[i] \cdot \textbf{s}_{\ell - 1}[j]$ 做基于LWE加密的密文，虽然这里不能解密，因为加密的内容不属于$\{0, 1\}$ 100 | 101 | 2. 生成私钥 102 | 103 | 随机采样 $\widehat{\textbf{s}} \stackrel{$}{\leftarrow}\mathbb{Z}^k_p$，维数为 $k$，模数为 $p$。并对所有 $i \in [n], \tau \in \{0, ..., \lfloor \log q\rfloor \}$，随机采样 $\widehat{\textbf{a}}_{i, \tau} \stackrel{$}{\leftarrow}\mathbb{Z}^k_p$，$\widehat{e}_{i, \tau} \stackrel{$}{\leftarrow}\widehat{\chi}$ 104 | 105 | 计算 $\widehat{b}_{i, \tau} := \langle \widehat{\textbf{a}}_{i, \tau}, \widehat{s} \rangle + \widehat{e}_{i, \tau} + \lfloor \frac{p}{q} \cdot (2^{\tau} \cdot S_L[i]) \rceil \ {(\rm mod}\ p)$，令 $\widehat{\psi}_{i,\tau} := \left( \widehat{\textbf{a}}_{i, \tau} , \widehat{b}_{i, \tau} \right) \in \mathbb{Z}_p^k \times \mathbb{Z}_p$，$\widehat{\Psi} := \{ \widehat{\psi}_{i,\tau} \}_{i \in [n], \tau \in \{0,...,\lfloor \log q \rfloor \}}$ 106 | 107 | 3. 生成公钥 108 | 109 | 选择一致随机矩阵 $\textbf{A} \stackrel{$}{\leftarrow} \mathbb{Z}^{m \times n}_q$ 和噪音向量 $\textbf{e} \stackrel{$}{\leftarrow} \chi^m$，计算 $\textbf{b} := \textbf{A}\textbf{s}_0 + 2\textbf{e}$ 110 | 111 | ## 加密算法 112 | 113 | **输入**：公钥 $pk = (\textbf{A}, \textbf{b})$，待加密消息 $\mu \in \rm{GF(2)}$ 114 | 115 | **输出**：密文 $c = ((\textbf{v}, w), \ell)$ 116 | 117 | **过程**： 118 | 119 | ​ 采样 $\textbf{r} \stackrel{$}{\leftarrow} \{0,1\}^m$，令 $\textbf{v} := \textbf{A}^T \textbf{r}$，$w := \textbf{b}^T\textbf{r} + \mu$ 120 | 121 | ​ $\ell$ 为层级标签，表示该密文经过的乘法次数（电路角度通常称为深度），刚加密后的层级为0 122 | 123 | ​ 因此输出密文为 $c := ((\textbf{v}, w), 0)$ 124 | 125 | ## 同态计算算法 126 | 127 | **输入**：计算函数 $f$，参数密文 $(c_1, ..., c_t)$，计算密钥 $evk = (\Psi, \widehat{\Psi})$ 128 | 129 | **输出**：计算 $L$ 层后输出压缩密文 $\widehat{c} := (\widehat{\textbf{v}}, \widehat{w})$ 130 | 131 | **过程**：函数可以拆分为加法和乘法，因此这里分别讨论加法和乘法两类运算 132 | 133 | 1. 同态加法 134 | 135 | 加法的输入参数可以为任意多个，不妨设为 $c_i = ((\textbf{v}_i, w_i), \ell)$ ，和密文为 $c_{add} = ((\textbf{v}_{add}, w_{add}), \ell)$ 136 | $$ 137 | c_{add} = ((\textbf{v}_{add}, w_{add}), \ell) := \left(\left(\sum_{i}\textbf{v}_i, \sum_{i}w_i \right), \ell\right) 138 | $$ 139 | 可以简单验证和密文满足与明文和 $\sum_i\mu_i$ 模2同余 140 | $$ 141 | w_{add} - \langle \textbf{v}_{add}, \textbf{s}_{\ell} \rangle = \sum_i(w_i - \langle \textbf{v}_{i}, \textbf{s}_{\ell} \rangle) = \sum_i(\mu_i + 2e_i) = \sum_i\mu_i + 2\sum_ie_i 142 | $$ 143 | 144 | 2. 同态乘法 145 | 146 | 乘法的输入参数只能为2个，不妨设为 $c = ((\textbf{v}, w), \ell)$ 和 $c^{'} = ((\textbf{v}^{'}, w^{'}), \ell)$ ，乘积为 $c_{mult} = ((\textbf{v}_{mult}, w_{mult}), \ell + 1)$ 147 | 148 | 先考虑多项式 149 | $$ 150 | \phi(\textbf{x}) = \phi_{(w, \textbf{v}), (w^{'}, \textbf{v}^{'})}(\textbf{x}) \triangleq (w-\langle \textbf{v}, \textbf{x} \rangle) \cdot (w^{'} - \langle \textbf{v}^{'}, \textbf{x} \rangle) 151 | $$ 152 | 打开括号合并同类项后得到新表达式，其中令 $\textbf{x}[0]=1$，因此该表达式可以表示常数项和一次项 153 | $$ 154 | \phi(\textbf{x}) = \sum_{0 \leq i \leq j \leq n} h_{i,j} \cdot \textbf{x}[i] \cdot \textbf{x}[j] 155 | $$ 156 | 拆分 $h_{i,j} = \sum_{\tau=0}^{\lfloor{log\ q}\rfloor} h_{i,j,\tau}2^{\tau}$，保证 $h_{i,j,\tau} \in \{0,1\}$，$\phi(\textbf{x})$ 可以表示为 157 | $$ 158 | \phi(\textbf{x}) = \sum_{\substack{0 \leq i \leq j \leq n \\ \tau \in \{0,...,\lfloor \log q \rfloor\}}}h_{i,j,\tau} \cdot (2^\tau \cdot \textbf{x}[i] \cdot \textbf{x}[j]) 159 | $$ 160 | 利用计算密钥中 $\Psi$ 包含的 $\psi_{\ell, i, j, \tau} := (\textbf{a}_{\ell, i, j, \tau}, b_{\ell, i, j, \tau})$ 满足 $2^{\tau} \cdot \textbf{s}_{\ell}[i] \cdot \textbf{s}_{\ell}[j] \approx b_{\ell, i, j, \tau} - \langle \textbf{a}_{\ell+1, i, j, \tau}, \textbf{s}_{\ell + 1} \rangle$ 161 | 162 | 因此令乘积密文为 $c_{mult} = ((\textbf{v}_{mult}, w_{mult}), \ell + 1)$ 163 | 164 | $$ 165 | \textbf{v}_{mult} := \sum_{\substack{0 \leq i \leq j \leq n \\ \tau \in \{0,...,\lfloor \log q \rfloor\}}} h_{i,j,\tau} \cdot \widehat{\textbf{a}}_{\ell+1,i,j,\tau} 166 | \\ w_{mult} = \sum_{\substack{0 \leq i \leq j \leq n \\ \tau \in \{0,...,\lfloor \log q \rfloor\}}} h_{i,j,\tau} \cdot b_{\ell+1,i,j,\tau} 167 | $$ 168 | 169 | ​ 可以验证乘积密文满足与明文乘积 $\mu\mu^{'}$ 模2同余 170 | $$ 171 | w_{mult} - \langle \textbf{v}_{mult}, \textbf{s}_{\ell + 1} \rangle = \sum_{i,j,\tau} h_{i,j,\tau} (b_{\ell+1, i, j,\tau} - \langle \textbf{a}_{\ell+1,i,j,\tau}, \textbf{s}_{\ell+1} \rangle) = \phi(\textbf{s}_{\ell}) + 2e_1 = \mu\mu^{'} + 2e_2 172 | $$ 173 | 174 | 3. 维数-模数压缩 175 | 176 | 经过 $L$ 层计算后，我们得到密文 $c := ((\textbf{v}, w), L)$，利用压缩密钥 $\widehat{\Psi}$，转换为压缩密文 $\widehat{c}$（从 $(n,q)$ 转为 $(k,p)$ ） 177 | 178 | 首先考虑如下函数，将数值从 $\mathbb{Z}_q$ 转为模 $p$ 的有理数 179 | 180 | $$ 181 | \phi(\textbf{x}) \triangleq \phi_{\textbf{v},w}(\textbf{x}) \triangleq \frac{p}{q} \cdot \left(\frac{q+1}{2} \cdot (w - \langle \textbf{v}, \textbf{x} \rangle) \right) \mod p 182 | $$ 183 | 184 | ​ 除了 $\frac{p}{q}$ 以外的系数仍在 $\mathbb{Z}_q$ 中，可以找到 $h_0, ...,h_n \in \mathbb{Z}_q$ 使得 185 | $$ 186 | \phi(\textbf{x}) = \sum_{i=0}^n h_i \cdot (\frac{p}{q} \cdot \textbf{x}[i]) \mod p 187 | $$ 188 | ​ 同样对 $h_i$ 拆分到二进制，$h_i = \sum_{\tau=0}^{\lfloor \log q \rfloor} h_{i,\tau}$，上式可转换为 189 | $$ 190 | \phi(\textbf{x}) = \sum_{i=0}^n \sum_{\tau=0}^{\lfloor \log q \rfloor} h_{i,\tau} \cdot (\frac{p}{q} \cdot 2^\tau \cdot \textbf{x}[i]) \mod p 191 | $$ 192 | ​ 利用压缩密钥 $\widehat{\Psi}$，计算压缩密文 $\widehat{c} = (\widehat{\textbf{v}}, \widehat{w}) \in \mathbb{Z}_p^k \times \mathbb{Z}_p$ 193 | $$ 194 | \widehat{\textbf{v}} := 2 \cdot \sum_{i=0}^n \sum_{\tau=0}^{\lfloor \log q \rfloor} h_{i,\tau} \cdot \widehat{\textbf{a}}_{i,\tau} \mod p \in \mathbb{Z}_p^k 195 | \\ \widehat{w} := 2 \cdot \sum_{i=0}^n \sum_{\tau=0}^{\lfloor \log q \rfloor} h_{i,\tau} \cdot \widehat{b}_{i,\tau} \mod p \in \mathbb{Z}_p 196 | $$ 197 | ​ 同样可以验证压缩密文和原密文同样满足与明文模2同余 198 | 199 | ## 解密算法 200 | 201 | **输入**：压缩密文 $\widehat{c} := (\widehat{\textbf{v}}, \widehat{w}) \in \mathbb{Z}_p^k \times \mathbb{Z}_p$ 202 | 203 | **输出**：明文 $\mu$ 204 | 205 | **过程**：计算 $\mu^{*} := (\widehat{w} - \lang \widehat{\textbf{v}}, \widehat{\textbf{s}} \rang \mod p) \mod 2$ 206 | 207 | 由前面的计算可以看出 $\widehat{w} - \lang \widehat{\textbf{v}}, \widehat{\textbf{s}} \rang = \mu + 2\widehat{e} \mod p$ 208 | 209 | 只要噪音 $\widehat{e}$ 足够小，解密得到 $\mu^{*} = \mu$，能解密回原消息 210 | 211 | -------------------------------------------------------------------------------- /hello-world-tutorial-contract.md: -------------------------------------------------------------------------------- 1 | # Cortex AI Smart Contract Guide 2 | 3 | Cortex is the first and currently the only public blockchain capable of executing on-chain AI. This tutorial will lead you through the process of deploying a simple “Hello World” AI Dapp. If you prefer to learn by reading the formal documentation, click here (https://github.com/CortexFoundation/tech-doc). 4 | 5 | By the end of this tutorial, you will have learned how to write a simple handwritten digit recognition AI DApp that takes an input image from the user and tells which digit (0-9) is in the image by calling an on-chain AI model on Cortex. 6 | 7 | Prerequsite: 8 | 9 | - Chrome Browser 10 | - Cortex Wallet Chrome Extension (Follow this guide to install it: https://github.com/CortexFoundation/Cortex_Release/tree/master/cortex-wallet if you have not done so) 11 | - Some Cortex TestNet tokens (from faucet or from ERC20 exchanges) 12 | 13 | First make sure from your Cortex wallet that you are on the TestNet. Click on the Cortex wallet extension, enter your password, and at the top of your wallet, you should see: 14 | 15 | ![wallet top](imgs/wallet_top.png) 16 | 17 | If you don't see this yet, click on the dropdown arrow and switch to the Dolores TestNet. (You may need to refresh and then repeat the steps above after this) 18 | 19 | ## Get Started 20 | 21 | Here's the overall code we're going to write - always refer back to it if you get lost. We will go over it step-by-step next. 22 | 23 | ### Overall code 24 | 25 | ```javascript 26 | pragma solidity ^0.4.18; 27 | 28 | contract Infer { 29 | 30 | // initiate the variables 31 | uint256[] public input_data; 32 | // address of the on-chain model 33 | address modelAddr = 0x420bD8d2FE73514E9b77fdBc78c0306Fd8a12866; 34 | // the output will be stored in this variable 35 | uint256 public currentInferResult; 36 | 37 | constructor() public { 38 | // Here we define the dimension of the input_data. To feed an image input of dimension 1 * 1 * 28 * 28 into the model, we need 1 * 1 * 28 * 28 bytes. 39 | // Now, each uint256 in Solidity is 32 bytes. That's why we need divide 1 * 1 * 28 * 28 by 32 and round up. 40 | // To express this division in Solidity, we write (1 * 1 * 28 * 28 + 31) >> 5, where >> 5 means divide by 2^5 and adding the 31 allows us to effectively round up. 41 | input_data = new uint256[]((1 * 1 * 28 * 28 + 31) >> 5); 42 | } 43 | 44 | // you can choose to randomly generate input as opposed to taking the input image from the user 45 | function GenerateRandomInput() public { 46 | input_data[0] = uint(sha256(now)); 47 | for(uint i = 1; i < input_data.length; ++i) { 48 | input_data[i] = uint(sha256(input_data[i - 1])); 49 | } 50 | } 51 | 52 | function SetInput(uint256[] data) public { 53 | for(uint i = 0; i < input_data.length; ++i) { 54 | input_data[i] = data[i]; 55 | } 56 | } 57 | // recognize digit using randomly generated input image 58 | function DigitRecognitionInfer() public { 59 | uint256[] memory output = new uint256[](uint256(1)); 60 | inferArray(modelAddr, input_data, output); 61 | currentInferResult = (output[0] >> (256 - 32)) & ((1 << 32) - 1); 62 | currentInferResult = currentInferResult % 10; 63 | } 64 | } 65 | 66 | ``` 67 | 68 | ### Code Walkthrough 69 | 70 | Now, let's open Remix (link: cerebro.cortexlabs.ai/remix/), the official Cortex IDE, where you can write and later deploy this contract. (You could also write the code in another IDE if you prefer and paste the code into Remix later). 71 | 72 | We will now walk through the code step-by-step. Pay close attention to the comments. 73 | 74 | ### Create a contract 75 | 76 | ```javascript 77 | pragma solidity ^0.4.18; 78 | 79 | contract Infer { 80 | 81 | } 82 | ``` 83 | 84 | ### Initiate basic variables 85 | 86 | ```javascript 87 | // The image input is in the form of a matrix (a uint256 array) of pixels, which has dimension 1 * 3 * 32 * 32. If you're confused why this is the dimension of the input image, see footnotes. 88 | uint256[] public input_data; 89 | // The address of an exisitng on-chain AI model you will call to recognize the digit in the image. If you prefer to use your own digit recognition model, you can upload it to Cortex storage layer first and then substitute this address. 90 | address modelAddr = 0x420bD8d2FE73514E9b77fdBc78c0306Fd8a12866; 91 | // This variable will store the output of the AI inference. 92 | uint256 public currentInferResult; 93 | ``` 94 | 95 | ### Write the function to take user input 96 | 97 | ```javascript 98 | // If we want to input custom user image, we input it as the "data" argument in this function 99 | function SetInput(uint256[] data) public { 100 | for(uint i = 0; i < input_data.length; ++i) { 101 | input_data[i] = data[i]; 102 | } 103 | } 104 | ``` 105 | 106 | ### Write the main function to call the AI model 107 | 108 | Pay attention. This is the most important function in this contract. It takes an input image array of dimension 1 x 3 x 32 x 32 and return a number from 0-9 as the output, telling you which digit is in the image. 109 | 110 | ```javascript 111 | // recognize digit using user input image 112 | function DigitRecognitionInfer() public { 113 | 114 | // initialize variable 115 | uint256[] memory output = new uint256[](1); 116 | // inferArray is a built-in function in CVM, it takes the input "input_data" (which we have set from either the SetInput or GenerateRandomInput function), feed it to this model at "modelAddr" and store the inference result in "output", which you just initialized above and pass in as a placeholder here 117 | inferArray(modelAddr, input_data, output); 118 | 119 | // These two lines below will convert the output into a digit between 0-9 and store it in "currentInferResult". (The conversion has to do with big endian vs. little endian under the hood) 120 | currentInferResult = (output[0] >> (256 - 32)) & ((1 << 32) - 1); 121 | currentInferResult = currentInferResult % 10; 122 | } 123 | ``` 124 | 125 | Note: Notice we did not go over the _GenerateRandomInput()_ and _DigitRecognitionInfer()_ funcitons, because they are not essential. All you need to know is that _GenerateRandomInput()_ will randomly generate an input image and store it in the variable _input_data_ and _DigitRecognitionInfer()_ will call the AI model to recognize which digit is in that randomly generated input image. Not too useful, since most of the time we will be taking custom user input as opposed to randomly generated image, but you can use them to test whether your contract is working properly. 126 | 127 | ## Compile the Contract 128 | 129 | To compile this contract from Remix (link again: cerebro.cortexlabs.ai/remix/), first click on ![the plug](imgs/plug.png) on the left side of the page. 130 | 131 | Then activate the two modules: "Deploy And Run Transactions" and "Solidity Compiler". 132 | 133 | We first need to compile this contract. So click ![the double arrows](imgs/double_arrow.png) on the left. Then click "compile" to compile your contract. Once you have compiled successfully, the icon on the left should look like this: ![compile success](imgs/compiled.png) 134 | 135 | ## Deploy the Contracts 136 | 137 | Now let's deploy this contract to the TestNet. 138 | 139 | Click on the Cortex logo ![cortexlogo](imgs/cortexlogo.png) on the left to go to the Deploy tab. 140 | 141 | Now if a window does not pop up, click on your Cortex wallet on the top right corner to see the transaction window. You may leave everything as default and click on "confirm". (You should see your account filled in automatically; if not, you may need to log into your wallet first and reload) A wallet window should pop up asking you to confirm the transaction like the one below. Review the details and then click "confirm" again. 142 | 143 | ![confirmation](imgs/testNetDeploy.png) 144 | 145 | After a few minutes, your contract should have been successfully deployed and show up under the "Deployed Contracts" section! 146 | 147 | Click on the dropdown menus and you will see all the functions that you can call. Click on the clipboard and you will see the address of your contract. Make sure you save this address somewhere so that you know where your contract was deployed. 148 | 149 | At this stage, you're pretty much done: you can click GenerateRandomInput to generate a random input (remember to always go to your Cortex wallet to confirm function calls) and then click DigitRecognitionInfer to determine what digit the randomly generated input is. If you click on currentInferResult, you will see something like "0: uint256: number", where number is your infer result. 150 | 151 | We have now finished walking through the entire workflow of developing on Cortex. 152 | 153 | However, running the program on randomly generated input images is boring. So for the sake of completeness (and fun), we will go on to show how to take user input, which involves writing a Python script! 154 | 155 | --- 156 | 157 | ## Taking User Input 158 | 159 | Use this Python script to convert your custom image into an array of supported size. 160 | 161 | Open Terminal, first run 162 | 163 | ```Python 164 | pip3 install Pillow 165 | ``` 166 | 167 | Start a Python file, and let's name it "convert.py", which will take your custom image as an input and output an array of pixel values with dimensions compatible with our contract above! Make sure you put your custom image in the same directory as this script. 168 | 169 | ```Python 170 | import sys 171 | from PIL import Image 172 | 173 | img = Image.open(sys.argv[1]) 174 | img = img.resize((28,28)) 175 | img = img.load() 176 | 177 | h = '0123456789abcdef' 178 | s = '' 179 | for i in range(28): 180 | for j in range(28): 181 | t = 0 182 | if type(img[j, i]) is int: 183 | t = img[j, i] // 2 184 | else: 185 | for k in img[j, i]: 186 | t += k 187 | t //= len(img[j, i]) * 2 188 | s += h[t // 16] + h[t % 16] 189 | ret = [] 190 | for i in range(0, len(s), 64): 191 | if i <= len(s): 192 | e = i+64 193 | if e > len(s): 194 | e = len(s) 195 | subs = s[i:e] 196 | if len(subs) < 64: 197 | subs = subs + '0' * (64 - len(subs)) 198 | ret.append('0x' + subs) 199 | else: 200 | ret.append('0x' + '0' * (len(s) - i + 64) + s[i:]) 201 | 202 | print('[' + ','.join(['"' + x + '"' for x in ret]) + ']') 203 | ``` 204 | 205 | Caveat: this python script above only works for MNIST test set - if you want to get the pixel value of your custom images, you may need to modify it accordingly. 206 | 207 | At runtime, run 208 | 209 | ```Python 210 | python3 convert.py your_img_name.jpg 211 | ``` 212 | 213 | Replace your_img_name with your actual image name! 214 | 215 | If everything goes well, Terminal should output an array of pixel values which you can copy. For an image like this （if you need more test images, here's a wonderful image dataset on Kaggle https://www.kaggle.com/scolianni/mnistasjpg): 216 | 217 | drawing 218 | 219 | The Python program should return an array like this: 220 | 221 | ["0x0000000000000000000001000002010005000007000009000000000000000000","0x0000000000060000030000050001010800010100000000000000000000000000","0x030400040000040000090000000a000200000000000000000000000000000000","0x000000000000000500000501000000000000000000000000060007050d110826","0x5a5911020000000000000000000000000000000000004b7f7d7d797e7e7f1603","0x000200040000000000000000000000000324667f77797f7f7d7c64631c000900","0x000000000000000000000000006d7f787f7c7d7d7d7f7f797018000600000000","0x00000001010101000020727f7f7a3b1114377d7f7c3e0a000000000000000000","0x0000000006001f33383a11000000647a7f7f0006000000000100000000000001","0x00000102000500030300257a7f7f020000000000000000000101020200070000","0x000400010000117f7f7e05050000000000000001010100000101000605000000","0x0306317f7f7c0706000000000000000000000000000000000203050000085c7b","0x7f79060000000000010100000000010202000500192f2a02020c7b7b7f680004","0x00000000020000000003070931335b5e767e7e5f5f71797e6909030000000000","0x0002000010365c7b7f797f7a7f7f797d7f787f7f6d3e04000000000001000000","0x3f7f757f7f7b726a797d7f7f7c7f7e7c7f79550600000000000500047e7f7f74","0x652a001a62777f71774736607f787f5a0000000003000b007a797f7f6c75716a","0x7a7d7f772600000a5b7b77790000000000000002527d7f7a7f797e7d7f62351d","0x00090103001b7f4f00000000000c000003115361585b52160105030300000200","0x0000070100000000050007000600020000000300030000000400050002000005","0x000000000007000200000c000004000004000500000000010000030000000000","0x0000000000000000000000000000000000000000000000000000000000000000","0x0000000000000000000000000000000000000000000000000000000000000000","0x0000000000000000000000000000000000000000000000000000000000000000","0x0000000000000000000000000000000000000000000000000000000000000000"] 222 | 223 | ## Running the Contract on Custom Image 224 | 225 | Now let's go back to Remix to infer the contract. 226 | 227 | From ![atAddress](imgs/atAddress.png), type in your contract address and then click the button to load your contract. (If it doesn't load, try compile your contract again) 228 | 229 | Now click on the dropdown arrow and you can see the list of callable functions in the contract like this: 230 | 231 | ![callables](imgs/callable.png) 232 | 233 | Paste your custom image input (that monstrous array above) into SetInput and then click the button, confirm the transaction in the popup window. Wait until the transaction is confirmed. This could take a while! 234 | 235 | Finally, click on `DigitRecognitionInfer`, confirm the transaction. Your final inference result should be stored in `currentInferResult` in a few seconds! 236 | 237 | Expect to see something like this: 238 | ![res](imgs/res.png) 239 | (Don't worry about the 0 in front; your final result is stored is behind the "uint256"). As you can see the model has successfully determined that the image is a 2. 240 | 241 | Congratulations! You are now one of the first people on earth to have successfully written, compiled, deployed and executed an AI DApp. If you have any questions, feel free to reach out to the Cortex team. Join our Telegram group at https://t.me/CortexOfficialENOF 242 | 243 | --- 244 | 245 | ### FAQs 246 | 247 | **Question 1:** Why is the image 1 x 3 x 32 x 32 ? 248 | 249 | **Answer 1:** 250 | 251 | 1: 1 image 252 | 253 | 3: 3 colors in RBG - red, blue, green (each can take on a value from 0 to 255). Varying combinations of RBG can get you different colors. In total, there's 256 x 256 x 256 color combinations. 254 | 255 | 32: 32 rows of pixels 256 | 257 | 32: 32 columns of pixels 258 | 259 | Visualize one such array as 260 | 261 | [ 262 | 263 | red [ [ row1 ],[ row2 ],[ row3 ],[ row4 ]... [ row32 ] ] 264 | 265 | blue [ [ row1 ],[ row2 ],[ row3 ],[ row4 ]... [ row32 ] ] 266 | 267 | green [ [ row1 ],[ row2 ],[ row3 ],[ row4 ]... [ row32 ] ] 268 | 269 | ] 270 | 271 | where [ row1 ] looks like [ [col1],[col2],[col3],[col4]...[col32]] and within each of [col1] lives a value from 0-255, indicating how much red/green/blue it has in that pixel. 272 | 273 | **Question 2:** How do I deploy my contract to MainNet? 274 | 275 | **Answer 2:** Simply switch to MainNet in your Cortex wallet and repeat all steps from above! 276 | -------------------------------------------------------------------------------- /model-writing-tutorial.md: -------------------------------------------------------------------------------- 1 | # Train & Upload AI Models to Cortex 2 | 3 | # Introduction 4 | 5 | Cortex is currently the _only_ public blockchain that allows on-chain execution of machine learning models. Every step of the inference is transparent to the public, verified by blockchain consensus. 6 | 7 | **Why should I upload my model to Cortex?** 8 | 9 | You will get paid a small amount in CTXC each time your model is called in a smart contract. This award comes from part of the miner's reward and its aim is to incentivize machine learning developers to upload their models to the blockchain for the world to use. 10 | 11 | A few essential vocabulary for the rest of the tutorial: 12 | 13 | **CVM**: 14 | 15 | CVM, short for Cortex Virtual Machine, is the core of the Cortex blockchain. It is responsible for executing code on-chain. It is backward compatible with Ethereum's [EVM](https://ethdocs.org/en/latest/introduction/what-is-ethereum.html) with added support for on-chain machine learning inference. While it is virtually impossible to execute nontrivial ML models on the EVM, CVM is specially designed for on-chain ML inference with innovations such as utilization of node machines' GPUs and deterministic quantization of ML models. 16 | 17 | **CVM-Runtime**: 18 | 19 | An open-source deterministic machine learning framework written in C++. During runtime, the CVM (Cortex Virtual Machine) executes your models via CVM-Runtime. It is comparable to other frameworks such as MXNet, PyTorch, TensorFlow etc. with the important distinction that it is deterministic across different computing environments. 20 | 21 | **MRT**: 22 | 23 | MRT (written in Python), short for Model Representation Tool, is one of the most important parts of CVM-Runtime. It quantizes and prepares your model for on-chain execution by CVM-Runtime. At the risk of oversimplification, MRT basically compresses and converts your model from floating point to integer-only so that its execution is efficient and deterministic across different devices. The original research was endorsed by the official team of Amazon MXNet and the details can be read [here](https://medium.com/apache-mxnet/quantizing-neural-network-models-in-mxnet-for-strict-consistency-on-blockchain-b5c950674866). 24 | 25 | In this tutorial, we will write a simple handwritten digit recognition model, convert it via MRT, and upload it to the Cortex blockchain so that it can be executed on-chain. 26 | 27 | We will walk through 4 main stages: 28 | 29 | (1) Install CVM-Runtime and other dependencies. 30 | 31 | (2) Train a model using the MXNet framework. 32 | 33 | (3) Quantize the model trained in stage 2 using MRT. 34 | 35 | (4) Upload the model 36 | 37 | **Prerequisites:** 38 | 39 | - A machine with the Linux Operating System [1]. The official implementation is currently only in Linux, but if you're passionate about implementing CVM-Runtime in a different operating system, pull requests are more than welcomed! 40 | 41 | - A machine with GPU and CUDA installed properly for your Linux version. 42 | 43 | - Working knowledge of Linux programming. 44 | 45 | - Willingness to debug for yourself. While we seek to be as comprehensive as possible in this tutorial, we cannot cover all the idiosyncrasies of different machines and environments. If you notice an issue in the documentation, feel free to open Github issues or pull requests. 46 | 47 | If you encounter any problems during the course, feel free to reach out to our core dev team via our [Telegram](https://t.me/CortexOfficialEN) or [Twitter](https://twitter.com/CTXCBlockchain/). We seek to constantly improve our documentation. 48 | 49 | Let's get started! 50 | 51 | # Tutorial 52 | 53 | ## Stage I: Install CVM-runtime and Other Dependencies 54 | 55 | ### **Install CVM-Runtime** 56 | 57 | ### 1. Git clone the CVM-runtime repository 58 | 59 | ```bash 60 | git clone -b wlt https://github.com/CortexFoundation/cvm-runtime.git 61 | 62 | cd cvm-runtime 63 | ``` 64 | 65 | ### 2. Configure for compilation 66 | 67 | In `config.cmake`, set the `ENABLE_CUDA` variable to `ON` on line 6. This allows us to utilize the GPU. 68 | 69 | ![config](imgs/config.png) 70 | 71 | ### 3. Compile 72 | 73 | You may need to install `make` if you have not already. 74 | 75 | ```bash 76 | make -j8 lib 77 | ``` 78 | 79 | If you encounter this error below 80 | 81 | ![cmake](imgs/cmake.png) 82 | 83 | you should install `cmake`. Try the following commands: 84 | 85 | ```bash 86 | sudo apt-get update 87 | sudo apt-get install cmake 88 | ``` 89 | 90 | Now type `g++` in your shell to see if you have `g++` installed. If not, run `sudo apt-get install g++` 91 | 92 | You might need to switch to a machine with GPU and CUDA installed if you don't have one. 93 | 94 | ### **Install MXNET & MRT dependencies** 95 | 96 | Now go to [MXNET's website](https://mxnet.apache.org/get_started/?platform=linux&language=python&processor=gpu&environ=pip&) to 97 | install the GPU version of MXNET suited for your CUDA. The install command should look something like 98 | 99 | ```bash 100 | pip3 install mxnet-cu102 101 | ``` 102 | 103 | Run `$nvcc --version` to find out your CUDA version; make sure that it matches the number after "cu", in this case it is general version 10. 104 | 105 | Now run the commands below to install other dependencies needed for our model training and quantization later. 106 | 107 | ```bash 108 | pip3 install gluoncv 109 | 110 | make dep 111 | ``` 112 | 113 | # Stage II: Train Your Model 114 | 115 | Execute the following command: 116 | 117 | ```bash 118 | python3 tests/mrt/train_mnist.py 119 | ``` 120 | 121 | This Python program will train a handwritten digit model using MXNet. It will take a few minutes to run. Upon completion, the trained model is stored under `~/mrt_model` (you can alter this path in the `python/mrt/conf.py` file). 122 | 123 | If you don't care about the training process, proceed to the next stage. Otherwise, let's dig a bit into the model training process here. We only sketch the main ideas here - always refer back to the source code `tests/mrt/train_mnist.py` when you're confused. 124 | 125 | Now go ahead and open the `train_mnist.py` file. There are mostly 5 steps: (1) load the data (2) define the model architecture (3) randomly initialize the parameters & set the hyperparameters (4) start training (5) export the model 126 | 127 | ### Step 1: Load the Data 128 | 129 | ```python 130 | # load training and validation data 131 | train_data = mx.gluon.data.vision.MNIST( 132 | train=True).transform_first(data_xform) 133 | val_data = mx.gluon.data.vision.MNIST( 134 | train=False).transform_first(data_xform) 135 | 136 | batch_size = 4 137 | train_loader = mx.gluon.data.DataLoader(train_data, shuffle=True, batch_size=batch_size) 138 | val_loader = mx.gluon.data.DataLoader(val_data, shuffle=False, batch_size=batch_size) 139 | ``` 140 | 141 | ### Step 2: Define the Model 142 | 143 | Our main interest here is the `train_mnist()` function. Notice by default, our code provides 3 architectures that you can choose from (`dapp`, `lenet`, `mlp`). If you do not specify the architecture, the default is `dapp`. 144 | 145 | ```python 146 | if version == 'dapp': 147 | net = nn.HybridSequential(prefix='DApp_') 148 | with net.name_scope(): 149 | net.add( 150 | nn.Conv2D(channels=16, kernel_size=(3, 3), activation='relu'), 151 | nn.MaxPool2D(pool_size=(2, 2), strides=(1, 1)), 152 | nn.Conv2D(channels=32, kernel_size=(3, 3), activation='relu'), 153 | nn.MaxPool2D(pool_size=(2, 2), strides=(1, 1)), 154 | nn.Conv2D(channels=64, kernel_size=(3, 3), activation='relu'), 155 | nn.MaxPool2D(pool_size=(2, 2), strides=(2, 2)), 156 | nn.Conv2D(channels=128, kernel_size=(1, 1), activation='relu'), 157 | nn.MaxPool2D(pool_size=(2, 2), strides=(2, 2)), 158 | nn.Flatten(), 159 | nn.Dense(10, activation=None), 160 | ) 161 | ``` 162 | 163 | ### Step 3: Randomly Initialize the Parameters & Set the Hyperparameters 164 | 165 | ```python 166 | # Random initialize all the mnist model parameters 167 | net.initialize(mx.init.Xavier(), ctx=ctx) 168 | net.summary(nd.zeros((1, 1, 28, 28), ctx=ctx)) 169 | # Set hyperparameters 170 | trainer = gluon.Trainer( 171 | params=net.collect_params(), 172 | optimizer='adam', 173 | optimizer_params={'learning_rate': 1e-3}, 174 | ) 175 | metric = mx.metric.Accuracy() 176 | loss_function = gluon.loss.SoftmaxCrossEntropyLoss() 177 | num_epochs = 5 178 | ``` 179 | 180 | ### Step 4: Start Training 181 | 182 | ```python 183 | for epoch in range(num_epochs): 184 | for inputs, labels in train_loader: 185 | inputs = inputs.as_in_context(ctx) 186 | labels = labels.as_in_context(ctx) 187 | 188 | with autograd.record(): 189 | outputs = net(inputs) 190 | loss = loss_function(outputs, labels) 191 | 192 | loss.backward() 193 | metric.update(labels, outputs) 194 | trainer.step(batch_size=inputs.shape[0]) 195 | 196 | name, acc = metric.get() 197 | print('After epoch {}: {} = {:5.2%}'.format(epoch + 1, name, acc)) 198 | metric.reset() 199 | 200 | for inputs, labels in val_loader: 201 | inputs = inputs.as_in_context(ctx) 202 | labels = labels.as_in_context(ctx) 203 | metric.update(labels, net(inputs)) 204 | print('Validaton: {} = {}'.format(*metric.get())) 205 | assert metric.get()[1] > 0.96 206 | ``` 207 | 208 | ### Step 5: Export the Model 209 | 210 | ```python 211 | sym = net(mx.sym.var('data')) 212 | sym_file, param_file = load_fname(version) 213 | print ("Dump model into ", sym_file, " & ", param_file) 214 | 215 | # dump the mxnet model 216 | with open(sym_file, "w") as fout: 217 | fout.write(sym.tojson()) 218 | net.collect_params().save(param_file) 219 | ``` 220 | 221 | Again, here we only show the general structure of writing a program that trains AI models - refer to the original file under `tests/mrt/train_mnist.py` for the complete code and modify as you see fit! 222 | 223 | # Stage III: Quantize Your Model 224 | 225 | To prepare the model for the Cortex blockchain, we need to quantize it with MRT. Recall from **Introduction** that MRT is a tool originating from Cortex's research in on-chain inference of ML models - it helps us quantize ML models for deterministic inference on the blockchain. 226 | 227 | Execute the following command: 228 | 229 | ```bash 230 | python3 python/mrt/main2.py python/mrt/model_zoo/mnist.ini 231 | ``` 232 | 233 | All the pre-quantized model configuration files are stored in `python/mrt/model_zoo`, and the file `config.example.ini` contains the model configuration for our case. 234 | 235 | **Very Important Note:** 236 | 237 | If you're training a custom model with your custom dataset, keep in mind that you must alter the source code to allow MRT to read your dataset properly. This is because MRT needs to see your dataset (during calibration) in order to get an idea of the range of activation values for each neural layer - only then can it know how to quantize your model for optimal performance. We're working on a more friendly interface regarding this. Meanwhile, feel free to directly alter the source code yourself for workarounds. 238 | 239 | # Stage IV: Upload Your Model 240 | 241 | Now the model is fully quantized. Specifically, `mnist_.json` and `mnist_.params` are your quantized models, stored under `~/mrt_model` (assuming you have not altered the path in the `python/mrt/conf.py` file). Create a separate folder named `data` and rename your `mnist_.json` to `symbol` and your `mnist_.params` to `params`. Run `ls -l` to check your operating system is not hiding the file name extension, because the full file names need to be correct for successful upload. 242 | 243 | We're now ready to upload them! Like most PoW blockchains, transaction data are broadcast out, received by different full nodes and relayed to propagate the entire network; miners then include these transactions in a block later mined into the blockchain. In our case, model file (which is mostly matrices of weights) is the transaction data; to upload this model file, we broadcast it out by seeding it with a torrent file (which is a file including metadata about the model file that enables its reception and relay by other full nodes). 244 | 245 | In this final stage, we will have three main steps: 246 | (1) Notify the Cortex network about the model and generate a torrent file on Cerebro. 247 | (2) Seed/broadcast this torrent 248 | (3) Push the upload progress on Cerebro. 249 | 250 | ### Step 1: Notify the Network & Generate Torrent 251 | 252 | Let's first go the [TestNet Cerebro Explorer](https://cerebro.test.cortexlabs.ai/) to generate a torrent file for our model file, which we will later use to broadcast (upload) the model file to the network. (When you deploy to MainNet, you need to go to the MainNet [Cerebro Explorer](https://cerebro.cortexlabs.ai/)) 253 | 254 | In the menu bar at the top, find "upload" under "AI Contract" 255 | 256 | ![cerebroMenu](imgs/cerebroMenu.png) 257 | 258 | Then choose `Fixed Point Model Data` - recall that our MRT has converted our ML model from float-point to (integer-only) fixed-point for deterministic on-chain inference. 259 | 260 | ![fixedPoint](imgs/fixedPoint.png) 261 | 262 | Now select the two model files simultaneously. 263 | 264 | ![select](imgs/select.png) 265 | 266 | ![upload](imgs/upload.png) 267 | 268 | Change the input shape in the grey box `Shape`. (Note: even if your input is of shape 1,28,28 you need to re-type it anyway, and that's what you should do here if you've been following this tutorial.) 269 | 270 | Now make sure that you have signed into your TestNet wallet and don't have any other transaction confirmation windows open (otherwise the new transaction confirmation window will not pop out). 271 | 272 | Type in the model description (any texts you wish), click `save` and then hit `upload`. A new transaction confirmation window will pop out. Click `confirm`. 273 | 274 | We're not done yet - recall that we're only generating a torrent file here, which includes metadata about the model files that enable their broadcast to other full nodes. 275 | 276 | ### Step 2: Seed/Broadcast This Torrent 277 | 278 | To finish the upload, we now proceed to seed (upload/broadcast) the model files along with the torrent file (the model files' metadata). In this example, we will use [torrentf](https://github.com/CortexFoundation/torrentfs), a client based on the Anacrolix BitTorrent client. 279 | 280 | After confirming the upload transaction on Cerebro above, a torrent file is generated and downloaded. The file name should be something like `3145ad19228c1cd2d051314e72f26c1ce77b7f02.torrent`. The alphanumeric string is called `infohash` - make sure you save this `infohash` and then rename the file to just `torrent` (make sure your operating system is not hiding the file name extension; i.e. make sure the file name is not `torrent.torrent`) 281 | 282 | Next, we create a new folder named `3145ad19228c1cd2d051314e72f26c1ce77b7f02` (same as the infohash) in which we put the `torrent` file alongside the `data` folder containing the two model files. Once that is done, create a new directory called `test` and put everything into it. 283 | 284 | The resulting directory should have this structure: 285 | 286 | ![seed](imgs/seed.png) 287 | 288 | Now, if you don't have Go installed already, install Go first. We recommend this [guide](https://www.linode.com/docs/development/go/install-go-on-ubuntu/), but make sure to install the latest stable version. 289 | 290 | Now run 291 | 292 | ```bash 293 | git clone https://github.com/CortexFoundation/torrentfs.git 294 | cd torrentfs 295 | make 296 | ``` 297 | 298 | Once you finish running the commands above, torrentfs should be compiled. Now we will go back to our root directory and run 299 | 300 | ```bash 301 | torrentfs/build/bin/seeding -dataDir=test 302 | ``` 303 | 304 | Note that this command above is assuming that you have both `torrentfs` and `test` folders in the root directory. If you have a different directory structure, make sure to adjust the command accordingly. 305 | 306 | Once you run this command, the seeding should start and there will be a message telling you that it is seeding. 307 | 308 | Let it seed/broadcast for ~6 hours to ensure the file propagate to the entire network. 309 | 310 | ### Step 3: Push the Upload Progress on Cerebro. 311 | 312 | To prevent spams, the Cortex protocol requires uploaders of large model files to manually push the upload progress by consuming Endorphin (similar to Gas on Ethereum). We need to go to the transaction page to manually send a few more transactions - otherwise, the full nodes will ignore our seed as spam instead of receiving or relaying it. 313 | 314 | To do this, let's go to the transaction page of the uploaded model. Open your wallet and find the transaction in which you uploaded the model. 315 | 316 | ![wallet-transaction](imgs/wallet-transaction.png) 317 | 318 | Click on it to go to the transaction page. 319 | 320 | ![model-transaction](imgs/model-transaction.png) 321 | 322 | On the row "To", you will see "Model XXXXXXXX Created". Click on the XXXXXXXX, which will take you to a new page where you can see: 323 | 324 | ![push-progress](imgs/push-progress.png) 325 | 326 | Click on `send` to push the progress. Keep sending transactions until the upload is complete, at which point you should see: 327 | 328 | ![push-progress-success](imgs/push-progress-success.png) 329 | 330 | Only then, will the full nodes stop ignoring your model files as spam and start receiving or relaying your files. 331 | 332 | Now we're done! It's strongly recommended that you keep seeding for ~6 hours for the model to successfully propagate throughout the entire network. 333 | 334 | # Conclusion 335 | 336 | Voila! You have successfully trained a handwritten digit recognition model and uploaded it to the Cortex blockchain. Now anyone, anywhere in the world can see and use your model. Each time your model is called, you will get a small reward in CTXC (this amount comes from a share of the miner's award). Note that this amount will be added to the balance of the address from which you deployed the model, but usually the reward for one single call to your model is small, so you will need to adjust the unit to Turing to be able to see it. 337 | 338 | If you want to learn how to call your model from a smart contract, we have a [tutorial](https://github.com/CortexFoundation/tech-doc/blob/master/hello-world-tutorial-contract.md) for that too. 339 | 340 | Happy building! 341 |
342 |
343 | 344 | # FAQ 345 | 346 | 1. Relationship between CVM-Runtime, CVM and MRT? 347 | 348 | CVM is the virtual machine that executes code on the Cortex blockchain. CVM-Runtime, if you recall from earlier, is a machine learning framework responsible for on-chain execution of _ML models_. You can consider CVM-Runtime part of the CVM; in fact, it is perhaps the biggest innovation of CVM - before CVM, there was no known way to execute ML models on-chain. MRT is the part of CVM-Runtime that prepares regular float-point ML models for on-chain execution via quantization. You can learn more details from this tree graph [here](https://github.com/CortexFoundation/tech-doc/blob/master/cortex-guide.md). 349 | 350 | # Footnotes 351 | 352 | [1] If you don't have access to a Linux system locally, you can open a Linux EC2 Instance on AWS and connect it to your editor via ssh. There will be a more detailed bonus tutorial on how to set this up - for now, here are the general steps to setting up a Linux system on AWS. Play around with it and use Google if you get stuck. You're always welcomed to ask questions in our Telegram group too! 353 | 354 | > (1) Go to AWS console, set up an EC20 Ubuntu (Ubuntu is one of the most user-friendly Linux systems) Instance. 355 | 356 | > (2) Start a folder named `.ssh`, put in your key pair and start a text file named `config` 357 | 358 | > (3) Open the command palette in Visual Studio code (`command + P` in Mac), type in 359 | > `> Remote-SSH: Connect to Host`. 360 | > Then choose `Add New SSH Host`. 361 | 362 | > Type in your address `ubuntu@your-aws-instance-public-ip` 363 | 364 | > Substitute in your own public ip here. Note that each time you restart your instance, the public ip changes, so you need to reconfigure upon each restart. 365 | 366 | > Type in the absolute path to your configuration file. Your path should end with something like `/.ssh/config` 367 | -------------------------------------------------------------------------------- /cvm/executor/CVM Formalization.md: -------------------------------------------------------------------------------- 1 | # CVM Formalization 2 | 3 | ### Predefined Structure 4 | 5 | #### TShape 6 | 7 | struct `TShape`, a array of `uint32_t`, which is used to standing for data shape. 8 | 9 | #### Optional\ 10 | 11 | the value can be `None`, or type of T. 12 | 13 | #### DLTensor 14 | 15 | Data format，a wrapper of data content pointer. 16 | 17 | *Attributes* 18 | 19 | - dtype must be INT8 or INT32. 20 | 21 | - shape is `TShape` type, storing data shape, for example (1, 2, 3). 22 | 23 | - precision is the max bit of each element take up, -1 stands for non-defined; normal value betweens range (0, 32]; others is non-available, which means a error occurs, either logic error or runtime error. 24 | 25 | *Notes*: we define a `p` precision data is between range $[-\alpha, \alpha], \alpha=2^{p-1}-1$ for all elements, for example: data that precision is 8 means all value is larger than -128 and less than 128, this is strong constraint. 26 | 27 | *Public Interface* 28 | 29 | - ndim is the dimensions of data, for example 3. 30 | 31 | #### Attribute Constant 32 | 33 | *max_attr* = 4096 34 | 35 | *min_attr* = 0 36 | 37 | ## Executor 38 | 39 | ## Ops 40 | 41 | 本节主要介绍 *cvm executor* 的算子形式化描述，包含但不限于 42 | 43 | 1. 输入，输出和参数限制 (restriction of inputs and outputs) 44 | 2. 源代码引用 (source code reference) 45 | 3. 数学形式化描述 (math formalization) 46 | 47 | ##### Inputs & Outputs 48 | 49 | 算子输入输出的数据格式为`DLTensor`，precision属性在(0, 32]范围内。 50 | 51 | ##### Description Format 52 | 53 | 所有的数学描述采用规范： 54 | $$ 55 | Y[y_\text{indexes}] = X[x_\text{indexes}], \\ 56 | \forall \text{given range}, \\ 57 | \text{where } \text{condition}_1 \text{ and } \text{condition}_2 \text{ and } 58 | \cdots \text{ condition}_n 59 | $$ 60 | 描述：对于所有给定取值范围，在指定坐标下，都有$Y=X$，其中定义的变量或条件用$\text{and}$相连接。 61 | 62 | ### Reduce Operator 63 | 64 | Reduce operator perform the reduction function to input data based on the parameters, and the process logic over all the type-based operators is consistent. We abstract the formalization here and introduce the details as belows: 65 | 66 | *Math Formalization* 67 | 68 | Suppose Input `X`, Output `Y`, attributes `axes`, `keepdims`, `exclude`, where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, `axes` is `TShape`, it's dimension is M, $M \in [0, N+1) \and card\{i \mid i \in \text{axes}\}=M$, `keepdims` is `boolean`, `exclude` is `boolean`. 69 | $$ 70 | T = \left\{i \mid \text{axis} \in \text{axes} \and 71 | i = \begin{cases} 72 | \text{axis}, & \text{if axis } \geqslant 0 \\ 73 | \text{axis} + N, & \text{otherwise} 74 | \end{cases} \right\}, \\ 75 | \text{where } card \; \text{T} = M \text{ and } 76 | j \in [0, N), \forall j \in \text{T} 77 | $$ 78 | 79 | $$ 80 | \text{real_axes} = \begin{cases} 81 | \{i \mid i \in [0, N) \and i \notin T\} , & \text{if exclude is true} \\ 82 | T, & \text{otherwise} 83 | \end{cases}, \\ 84 | R = card\{\text{real_axes}\} 85 | $$ 86 | 87 | 88 | 89 | 1. Case `exclude` = true and $M=N$ 90 | 91 | $$ 92 | Y = X 93 | $$ 94 | 95 | 2. Case `exclude` = false and $M = 0$ 96 | 97 | $$ 98 | Y[\underbrace{0, 0, \cdots, 0}_{K}] = \begin{cases} 99 | \sum_{x \in X} x, & \text{if op is sum} \\[1ex] 100 | max\{x \mid x \in X \}, & \text{if op is max} 101 | \end{cases}, \\ 102 | \text{where } K = \begin{cases} 103 | 1, & \text{if keepdims is false} \\ 104 | N, & \text{otherwise} 105 | \end{cases} 106 | $$ 107 | 108 | 3. Case `keepdims` is false 109 | 110 | $$ 111 | Y[d_{I(0)}, d_{I(1)}, \cdots, d_{I(K-1)}] = \\ 112 | \begin{cases} 113 | \sum_{d_{J(0)}=0}^{n_{J(0)}} \cdots \sum_{d_{J(R-1)}=0}^{n_{J(R-1)}} 114 | X[d_0, d_1, \cdots, d_{N-1}], & \text{if op is sum} \\[1ex] 115 | \max \{ X[d_0, d_1, \cdots, d_{N-1}] \mid d_{J(0)} \in [0, n_{J(0)}) \and \cdots \and 116 | d_{J(R-1)} \in [0, n_{J(R-1)}) \}, & \text{if op is max} 117 | \end{cases}, \\ 118 | \forall d_{I(0)} \in [0, n_{I(0)}) \and \cdots \and 119 | d_{I(K-1)} \in [0, n_{I(K-1)}), \\ 120 | \text{where } K = N - R \text{ and } \\ 121 | A = \{ i \mid i \in [0, N) \and i \notin \text{real_axes} \} \text{ and } \\ 122 | B = \{ i \mid i \in [0, N) \and i \in \text{real_axes} \} \text{ and } \\ 123 | I: \{ i \mid i \in [0, K) \} \to A, 124 | \text{ satisfy } I(i) < I(j), \forall 0 \leqslant i < j < K \text{ and } \\ 125 | J : \{ j \mid j \in [0, R) \} \to B, 126 | \text{ satisfy } J(i) < J(j), \forall 0 \leqslant i < j < R 127 | $$ 128 | 129 | 4. Otherwise 130 | 131 | $$ 132 | Y[d_0, d_1, \cdots, d_{N-1}] = M[d_{I(0)}, d_{I(1)}, \cdots, d_{I(K-1)}], \\ 133 | \forall d_{I(0)} \in [0, n_{I(0)}) \and \cdots \and 134 | d_{I(K-1)} \in [0, n_{I(K-1)}) \and \\ 135 | d_{J(0)} = 0 \and \cdots \and d_{J(R-1)} = 0, \\ 136 | \text{where } K = N - R \text{ and } \\ 137 | A = \{ i \mid i \in [0, N) \and i \notin \text{real_axes} \} \text{ and } \\ 138 | B = \{ i \mid i \in [0, N) \and i \in \text{real_axes} \} \text{ and } \\ 139 | I: \{ i \mid i \in [0, K) \} \to A, 140 | \text{ satisfy } I(i) < I(j), \forall 0 \leqslant i < j < K \text{ and } \\ 141 | J : \{ j \mid j \in [0, R) \} \to B, 142 | \text{ satisfy } J(i) < J(j), \forall 0 \leqslant i < j < R \text{ and } \\ 143 | M = \text{reduce_op}(X, \text{axes=axes, keepdims=false, exclude=exclude}) 144 | $$ 145 | 146 | *Example* 147 | 148 | ```python 149 | data = [[[1, 2], [2, 3], [1, 3]], 150 | [[1, 4], [4, 3], [5, 2]], 151 | [[7, 1], [7, 2], [7, 3]]] 152 | 153 | sum(data, axis=(1)) 154 | [[ 4. 8.] 155 | [ 10. 9.] 156 | [ 21. 6.]] 157 | 158 | sum(data, axis=[1,2]) 159 | [ 12. 19. 27.] 160 | ``` 161 | 162 | #### sum 163 | 164 | Test Parameter (sum): 165 | 166 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 34, 67\}, l=\{1, 58\}, r=\{1, 64\}$ 167 | 168 | ​ axis: $(1, )$ 169 | 170 | #### max 171 | 172 | Test Parameter: 173 | 174 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 34, 67\}, l=\{1, 58\}, r=\{1, 64\}$ 175 | 176 | 177 | 178 | ### Broadcast Operator 179 | 180 | Broadcast operator perform the broadcast function to input datas, and the process logic over all the type-based operators is consistent. We abstract the formalization here and introduce the details as belows: 181 | 182 | *Math Formalization* 183 | 184 | Suppose Input `A`, `B`, Output `Y` and broadcast function `BROADCAST_OP`. `A`'s shape is $M$ dimension, exactly $(m_0, m_1, \cdots, m_{M-1})$, `B`'s shape is $N$ dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$. 185 | 186 | 1. Extends `A`'s shape and `B`'s shape into $K = max(M, N)$ dimension by prefixing their shapes with $1$, denoted by $SA$ and $SB$, respectively, where $$SA_i = \begin{cases} 187 | m_{i-K+M}, & i \geqslant K - M \\ 188 | 1, & i < K - M 189 | \end{cases} \text{ and } 190 | SB_i = \begin{cases} 191 | n_{i-K+N}, & i \geqslant K-N \\ 192 | 1, & i < K - N 193 | \end{cases}$$ 194 | 195 | 2. For $\forall i \in [0, K)$, assert $SA_i=SB_i$ or $SA_i=1$ or $SB_i=1$. `Y`'s shape is $K$ dimension, exactly $(k_0, k_1, \cdots k_{K-1}), k_i = \max( SA_i, SB_i )$. 196 | 3. For $\forall i \in [0, K)$, $Y[d_0, d_1, \cdots, d_{K-1}] = 197 | \text{BROADCAST_OP}(A[a_0, a_1, \cdots, a_{K-1}], B[b_0, b_1, \cdots, b_{K-1}])$, where $d_{i} \in [0, k_{i}), a_i = \min(d_{i}, SA_i-1)$ and $b_i = \min(d_{i}, SB_i-1)$. 198 | 199 | Reference: https://github.com/CortexFoundation/CortexTheseus/blob/76320455f0769dbf22115d82181b7ba876c5f942/infernet/src/cvm/ops/cpu/ops.cc#L575 200 | 201 | #### broadcast_add 202 | 203 | set `BROADCAST_OP` to `add`. 204 | 205 | Example: 206 | 207 | ```python 208 | x = [[ 1., 1., 1.], 209 | [ 1., 1., 1.]] 210 | 211 | y = [[ 0.], 212 | [ 1.]] 213 | 214 | broadcast_add(x, y) = [[ 1., 1., 1.], 215 | [ 2., 2., 2.]] 216 | ``` 217 | 218 | Test Parameter: 219 | 220 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 221 | 222 | ​ Y.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 223 | 224 | 225 | 226 | #### broadcast_sub 227 | 228 | set `BROADCAST_OP` to `sub`. 229 | 230 | Test Parameter: 231 | 232 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 233 | 234 | Y.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 235 | 236 | 237 | 238 | #### broadcast_mul 239 | 240 | set `BROADCAST_OP` to `multiply`. 241 | 242 | Test Parameter: 243 | 244 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 245 | 246 | Y.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 247 | 248 | 249 | 250 | #### broadcast_div 251 | 252 | set `BROADCAST_OP` to `divide`. 253 | 254 | Test Parameter: 255 | 256 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 257 | 258 | ​ Y.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 259 | 260 | 261 | 262 | #### broadcast_max 263 | 264 | set `BROADCAST_OP` to `max`. 265 | 266 | Test Parameter: 267 | 268 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 269 | 270 | ​ Y.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 271 | 272 | 273 | 274 | #### 275 | 276 | ### NN Operator 277 | 278 | #### Convolution 279 | 280 | We only supported 2-D convolution operator. 281 | 282 | *Math Formalization* 283 | 284 | Suppose Input `X`, `W`, `B`, and output `Y`, attributes `padding`, `stride`, `dilation`, `groups`, where `X`'s shape is $(N, C, H, W)$, `W`' shape is $(OC, IC, KH, KW)$, and $IC = C \div \text{groups}$, `B` is `Optional`, if `B` is not None, it's shape is $(\text{OC},)$. `padding` is 2-D `TShape`, exactly $(PH, PW), PH,PW \in [min\_attr, max\_attr)$, `stride` is 2-D `TShape`, exactly $(SH, SW) \in [1, max\_attr)$, `dilation` is 2-D `TShape`, exactly $(DH, DW) \in [1, max\_attr)$, `grous` is `int`, the value is in $\{1, C\}$. 285 | 286 | 1. Case `groups` = 1 287 | 288 | $$ 289 | Y[n,i,p,q]=\sum_{j=0}^{C} 290 | \text{kernel}(n, j, p, q, n, i) 291 | + \begin{cases} 292 | 0, & \text{if B is None}\\ 293 | B[i], & \text{otherwise} 294 | \end{cases}, \\ 295 | \forall n \in [0, N) \and i \in [0, OC) \and \\ 296 | p \in \left[0, \left\lfloor{H+2 \cdot \text{PH}-\text{DH} \cdot (\text{KH}-1)-1\over\text{SH}}\right\rfloor+1 \right) \and \\ 297 | q \in \left[0, \left\lfloor{W+2 \cdot \text{PW}-\text{DW} \cdot (\text{KW}-1)-1 \over \text{SW}}\right\rfloor+1 \right),\\ 298 | $$ 299 | where $\text{kernel}$ function is 300 | $$ 301 | \text{kernel}(n, j, p, q, o, i) = \sum_{k_i=0}^{\text{KH}} \sum_{k_j = 0}^{\text{KW}} \text{pad}(p'+k_i*\text{DH},q'+k_j*\text{DW}) \cdot W[o, i, k_i, k_j], \\ 302 | \text{where } p' = p \cdot \text{SH} -\text{PH} \text{ and } 303 | q' = q \cdot \text{SW}-\text{PW} \text{ and } \\ 304 | \text{pad}(p, q) = \begin{cases} 305 | X[n, j, p, q], & \text{ if } p \in [0, H) \and q \in [0, W) \\ 306 | 0, & \text{otherwise} 307 | \end{cases} 308 | $$ 309 | Reference: https://github.com/CortexFoundation/CortexTheseus/blob/76320455f0769dbf22115d82181b7ba876c5f942/infernet/src/cvm/ops/cpu/ops.cc#L475 310 | 311 | 2. Case `groups ` = $C$ 312 | 313 | This case is named *Depth-Wise Convolution*. 314 | $$ 315 | IC = 1 \text{ and }OC = C 316 | $$ 317 | 318 | $$ 319 | Y[n,i,p,q]= \text{kernel}(n,i, p, q, i,0) + \begin{cases} 320 | 0, & \text{if B is None}\\ 321 | B[i], & \text{otherwise} 322 | \end{cases}, \\ 323 | \forall n \in [0, N) \and i \in [0, OC) \and\\ 324 | p \in \left[0, \left\lfloor{H+2 \cdot \text{PH}-\text{DH} \cdot (\text{KH}-1)-1\over\text{SH}}\right\rfloor+1 \right) \and \\ 325 | q \in \left[0, \left\lfloor{W+2 \cdot \text{PW}-\text{DW} \cdot (\text{KW}-1)-1 \over \text{SW}}\right\rfloor+1 \right) 326 | $$ 327 | Reference: https://github.com/CortexFoundation/CortexTheseus/blob/76320455f0769dbf22115d82181b7ba876c5f942/infernet/src/cvm/ops/cpu/ops.cc#L390 328 | 329 | 330 | 331 | #### Dense 332 | 333 | *Math Formalization* 334 | 335 | Suppose Input `X`, `W`, `B`, where `X` shape is $(M * K)$, `W` shape is $(N * K)$, `B` is `Optional`, if `B` is not `NONE`, it's shape is $(N,)$. 336 | 337 | Math: 338 | $$ 339 | Y=X W^T + \begin{cases} 340 | 0, & \text{if B is None} \\ 341 | B, & \text{otherwise} 342 | \end{cases} 343 | $$ 344 | Reference: https://github.com/CortexFoundation/CortexTheseus/blob/76320455f0769dbf22115d82181b7ba876c5f942/infernet/src/cvm/ops/cpu/ops.cc#L86 345 | 346 | Test Parameter: 347 | 348 | ​ X.shape: $(i, j), i = \{1, 14, 27\}, j = \{1, 12, 23\}$ 349 | 350 | ​ W.shape: $(k, j), k = \{1, 18\}, j = \{1, 12, 23\}$ 351 | 352 | ​ units: $k$ 353 | 354 | ​ use_bias:false 355 | 356 | #### Relu 357 | 358 | Test Parameter: 359 | 360 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 361 | 362 | ​ 363 | 364 | #### MaxPooling 365 | 366 | *Math Formalization* 367 | 368 | Suppose Input `X`, Output `Y` and attributes `pool_size`, ` padding`, `strides`, `ceil_mode`, where `X`'s shape is $(N, C, H, W)$, `pool_size` is 2-D `TShape`, exactly $(PSH, PSW)$, `padding` is 2-D `TShape`, exactly $(PH, PW) \in [min\_attr, max\_attr)$, if `padding` is 1-D, which means $PH = PW$, `strides` is 2-D `TShape`, exactly $(SH, SW)$, `ceil_mode` is `boolean`. 369 | $$ 370 | PSH \in [0, H + 2PH + 1), \\ 371 | PSW \in [0, W + 2PW + 1), \\ 372 | PSH > PH \and PSW > PW 373 | $$ 374 | 375 | $$ 376 | Y[n,i,p,q] = \max\{\text{pad}(n, i, p', q') \\ 377 | \mid p' \in [p \cdot \text{SH} -\text{PH}, p \cdot \text{SH} -\text{PH}+\text{PSH}), 378 | q' \in [q \cdot \text{SW}-\text{PW}, q \cdot \text{SW}-\text{PW}+\text{PSW})\}, \\ 379 | \forall n \in [0, N) \and i \in [0, C) \and \\ 380 | p \in \left[0, \text{ceil_func}\left({H+2 \cdot \text{PH}- \text{PSH}\over\text{SH}}\right)+1 \right) \and \\ 381 | q \in \left[0, \text{ceil_func}\left({W+2 \cdot \text{PW}- \text{PSW} \over \text{SW}}\right)+1 \right), \\ 382 | \text{where } \text{ceil_func(val)} = \begin{cases} 383 | \lceil \text{val} \rceil, & \text{if ceil_mode is true} \\ 384 | \lfloor \text{val} \rfloor, & \text{otherwise} 385 | \end{cases} \text{ and } \\ 386 | \text{pad}(n, i, p, q) = \begin{cases} 387 | X[n, i, p, q], & \text{ if } p \in [0, H) \and q \in [0, W) \\ 388 | 0, & \text{otherwise} 389 | \end{cases} 390 | $$ 391 | Reference: https://github.com/CortexFoundation/CortexTheseus/blob/76320455f0769dbf22115d82181b7ba876c5f942/infernet/src/cvm/ops/cpu/ops.cc#L692 392 | 393 | Test Parameter: 394 | 395 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 396 | 397 | ​ pool_size: $(1, 2)$ 398 | 399 | 400 | 401 | #### Upsampling 402 | 403 | *Math Formalization* 404 | 405 | Suppose Input `X`, Output `Y`, attributes `scale`, where `X`'s shape is $(N, C, H, W)$, `scale` is in range $[1, max\_attr)$. 406 | $$ 407 | Y[n, i, h, w] = X[n, i, \left\lfloor {h \over \text{scale}}\right\rfloor, \left\lfloor {w \over \text{scale}}\right\rfloor], \\ 408 | \forall n \in [0, N) \and i \in [0, C) \and 409 | h \in [0, H \cdot \text{scale}) \and w \in [0, W \cdot \text{scale}) 410 | $$ 411 | 412 | Test Parameter: 413 | 414 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 415 | 416 | ​ scale: 2 417 | 418 | ### Elemwise Operator 419 | 420 | #### abs 421 | 422 | *Math Formalization* 423 | 424 | Suppose Input `X`, Output `Y`. 425 | 426 | Math: 427 | $$ 428 | y = \begin{cases} 429 | x, & x \geqslant 0 \\ 430 | -x, & x < 0 431 | \end{cases}, 432 | \forall x \in X 433 | $$ 434 | 435 | Test Parameter: 436 | 437 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 438 | 439 | 440 | 441 | #### cvm_precision 442 | 443 | *Math Formalization* 444 | 445 | Suppose Input `X`, Output `Y`. 446 | 447 | Math: 448 | $$ 449 | y = \begin{cases} 450 | \lceil log_2(abs(x+1)) \rceil, & x \neq 0\\ 451 | 1, & x = 0 452 | \end{cases}, 453 | \forall x \in X 454 | $$ 455 | 456 | #### elemwise_add 457 | 458 | *Math Formalization* 459 | 460 | Suppose Input `A`, `B`, Output `Y`. Where `A` and `B` have the same shape, $(n_0, n_1, \cdots, n_{N-1})$. 461 | $$ 462 | Y = A + B 463 | $$ 464 | 465 | Test Parameter: 466 | 467 | A.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 468 | 469 | B.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 470 | 471 | 472 | 473 | #### elemwise_sub 474 | 475 | *Math Formalization* 476 | 477 | Suppose Input `A`, `B`, Output `Y`. Where `A` and `B` have the same shape, $(n_0, n_1, \cdots, n_{N-1})$. 478 | $$ 479 | Y = A - B 480 | $$ 481 | 482 | Test Parameter: 483 | 484 | A.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 485 | 486 | B.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 487 | 488 | 489 | 490 | #### negative 491 | 492 | *Math Formalization* 493 | 494 | Suppose Input `X`, Output `Y`. 495 | $$ 496 | Y = -X 497 | $$ 498 | 499 | #### clip 500 | 501 | *Math Formalization* 502 | 503 | Suppose Input `X`, Output `Y`, attribute `a_min`, `a_max`. 504 | $$ 505 | y = \begin{cases} 506 | \text{a_max}, & x \geqslant \text{a_max} \\ 507 | x, & x \in (\text{a_min}, \text{a_max}) \\ 508 | \text{a_min}, & x \leqslant \text{a_min} 509 | \end{cases}, 510 | \forall x \in X 511 | $$ 512 | 513 | Test Parameter: 514 | 515 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 516 | 517 | ​ a_max: $10$ 518 | 519 | ​ a_min: $-19$ 520 | 521 | 522 | 523 | #### cvm_clip 524 | 525 | *Math Formalization* 526 | 527 | Suppose Input `X`, Output `Y`, attribute `precision`, where `precision` is in range $[1, 33)$. 528 | $$ 529 | Y = clip(X, \text{a_min}=-\alpha, \text{a_max}=\alpha), \\ 530 | \text{where } \alpha = 2^\text{precision-1}-1 531 | $$ 532 | 533 | Test Parameter: 534 | 535 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 536 | 537 | ​ a_max: 10 538 | 539 | ​ a_min: -19 540 | 541 | ​ precision: 2 542 | 543 | 544 | 545 | #### cvm_right_shift 546 | 547 | *Math Formalization* 548 | 549 | Suppose Input `X`, Output `Y`, attribute `precision`, `shift_bit`, where `precision` is in range $[1, 33)$, and `shift_bit` is in range $[1, 33)$. 550 | $$ 551 | Y = clip(T, \text{a_min} = -\alpha, \text{a_max}=\alpha), \\ 552 | \text{where } T = {\left\lfloor 553 | \left(\left\lfloor \frac{X}{2^{\text{shift_bit} - 1}} \right\rfloor + 1 \right) 554 | \div 2 \right\rfloor} \text{ and } \alpha = 2 ^ {\text{precision} - 1} - 1 555 | $$ 556 | 557 | Test Parameter: 558 | 559 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 560 | 561 | ​ precision: 2 562 | 563 | ​ shift_bit: 2 564 | 565 | #### cvm_left_shift 566 | 567 | *Math Formalization* 568 | 569 | Suppose Input `X`, Output `Y`, attribute `precision`, `shift_bit`, where `precision` is in range $[1, 33)$, and `shift_bit` is in range $[1, 33)$. 570 | $$ 571 | Y = clip(T, \text{a_min} = -\alpha, \text{a_max}=\alpha), \\ 572 | \text{where } T = X * 2^\text{shift_bit} \text{ and } \alpha = 2 ^ {\text{precision} - 1} - 1 573 | $$ 574 | 575 | Test Parameter: 576 | 577 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 578 | 579 | precision: 2 580 | 581 | shift_bit: 2 582 | 583 | #### 584 | 585 | ### Transform Operator 586 | 587 | #### repeat 588 | 589 | *Math Formalization* 590 | 591 | Suppose Input `X`, Output `Y`, attribute `axis`, `repeats`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{\text{axis}}, \cdots, n_{N-1})$, `Y`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{\text{axis}} \cdot repeats, \cdots, n_{N-1})$. Obviously, axis is in range $[0, N)$, repeats is in range $[1, +\infty)$. 592 | $$ 593 | Y[d_0, d_1, \cdots, d_\text{axis}, \cdots, d_{N-1}] = 594 | X[d_0, d_1, \cdots, \left\lfloor{d_\text{axis} \over \text{repeats}}\right\rfloor, \cdots, d_{N-1}], \\ 595 | \forall d_0 \in [0, n_0) \and \cdots \and d_{axis-1} \in [0, n_{axis-1}) \and 596 | d_{axis} \in [0, n_{axis} \cdot \text{repeats}) \and \\ 597 | d_{axis+1} \in [0, n_{axis+1}) \and \cdots \and d_{N-1} \in [0, n_{N-1}) 598 | $$ 599 | 600 | Test Parameter: 601 | 602 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 603 | 604 | ​ repeats: 2 605 | 606 | #### tile 607 | 608 | *Math Formalization* 609 | 610 | Suppose Input `X`, Output `Y`, attribute `reps`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, `reps` is M dimension, exactly $(m_0, m_1, \cdots, m_{M-1})$. 611 | $$ 612 | r \in [1, max\_attr), \forall r \in \text{reps} 613 | $$ 614 | 615 | $$ 616 | Y[k_0, \cdots, k_{K-N-1}, k_{K-N}, k_{K-N+1}, \cdots, k_{K-1}] = \\ 617 | X[k_{K-N+0} \text{ mod } n_0, k_{K-N+1} \text{ mod } n_1, \cdots, k_{K-N+N-1} \text{ mod } n_{N-1}], \\ 618 | \forall k_0 \in [0, S_0) \and \cdots \and k_{K-1} \in [0, S_{K-1}), \\ 619 | \text{where } K = \max\{M, N\} \text{ and } S_i = SX_i \cdot SR_i \text{ and } \\ 620 | SX_p = \begin{cases} 621 | n_{p-K+N}, & p \in [K-N, K-1) \\ 622 | 1, & p \in [0, K-N) 623 | \end{cases} \text{ and } \\ 624 | SR_q = \begin{cases} 625 | m_{q-K+M}, & q \in [K-M, K-1) \\ 626 | 1, & q \in [0, K-M) 627 | \end{cases} 628 | $$ 629 | 630 | Test Parameter: 631 | 632 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 633 | 634 | ​ reps: $(2, 2, 3)$ 635 | 636 | #### flatten 637 | 638 | *Math Formalization* 639 | 640 | Suppose Input `X`, Output `Y`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$. 641 | $$ 642 | Y[\text{flatten_index}(d_0, d_1, \cdots, d_{N-1}, n_0, n_1, \cdots, n_{N-1})] = \\ 643 | X[d_0, d_1, \cdots, d_{N-1}], \\ 644 | \forall d_0 \in [0, n_0) \and d_1 \in [0, n_1) \and \cdots \and 645 | d_{N-1} \in [0, n_{N-1}) 646 | $$ 647 | where $\text{flatten_index}$ is 648 | $$ 649 | \text{flatten_index}(d_0, d_1, \cdots, d_{N-1}, n_0, n_1, \cdots, n_{N-1}) = \\ 650 | d_0 \cdot \prod_{i = 1}^{N-1} n_i + 651 | d_1 \cdot \prod_{i = 2}^{N-1} n_i + 652 | \cdots + d_{N-2} * n_{N-1} + d_{N-1} 653 | $$ 654 | 655 | Test Parameter: 656 | 657 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 658 | 659 | ​ 660 | 661 | #### concatenate 662 | 663 | *Math Formalization* 664 | 665 | Suppose `M` Inputs $I^0, I^1, \cdots, I^{M-1}$, Output `Y`, attribute `axis`. Where all inputs' shape is N dimension, exactly $I^i$'s shape is $(n^i_0, n^i_1, \cdots, n^i_{N-1})$, and `axis` is in range $[0, N)$. 666 | $$ 667 | n^i_j = n^0_j, \forall i \in [1, M) \and j \in [0, N) \and j \neq \text{axis} 668 | $$ 669 | 670 | $$ 671 | Y[d_0, d_1, \cdots, d_\text{axis-1}, \text{new_idx}, d_\text{axis+1}, \cdots, d_{N-1}] = I^i[d_0, d_1, \cdots, d_{N-1}], \\ 672 | \forall d_0 \in [0, n^i_0) \and \cdots \and d_{N-1} \in [0, n^i_{N-1}) 673 | \and i \in [0, M), \\ 674 | \text{where new_idx} = \sum_{j=0}^{i-1} n^j_\text{axis} + d_\text{axis} 675 | $$ 676 | 677 | Test Parameter: 678 | 679 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\} 680 | 681 | ​ Y.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 682 | 683 | ​ 684 | 685 | #### expand_dims 686 | 687 | *Math Formalization* 688 | 689 | Suppose Input `X`, Output `Y`, attributes `axis`, `num_newaxis`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, `axis` is in range $[-N-1, N+1)$, and `num_newaxis` is in range $[min\_attr, max\_attr)$. 690 | $$ 691 | Y[d_0,d_1, \cdots, d_{\text{real_axis}-1}, 692 | \underbrace{0, 0, \cdots, 0}_{\text{num_newaxis}}, 693 | d_\text{real_axis}, \cdots, d_{N-1}] = X[d_0, d_1, \cdots, d_{N-1}], \\ 694 | \forall d_0 \in [0, n_0) \and \cdots \and d_{N-1} \in [0, n_{N-1}), \\ 695 | \text{where real_axis} = 696 | \begin{cases} 697 | \text{axis},& \text{axis} \geqslant 0 \\ 698 | \text{axis} + N,& \text{axis} < 0 699 | \end{cases} 700 | $$ 701 | 702 | Test Parameter: 703 | 704 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 705 | 706 | ​ axis: 2 707 | 708 | #### reshape 709 | 710 | *Math Formalization* 711 | 712 | Suppose Input `X`, Output `Y`, attributes `target_shape`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, and `target_shape` is M dimension, exactly $(m_0, m_1, \cdots, m_{M-1})$ , and satisfy constraint : $m_0 * m_1 * \cdots * m_{M-1} = n_0 * n_1 * \cdots * n_{N-1}$. 713 | $$ 714 | Y[d_0, d_1, \cdots, d_{M-1}] = T[\text{flatten_index}(d_0, d_1, \cdots, d_{M-1}, m_0, m_1, \cdots, m_{N-1})], \\ 715 | \forall d_0 \in [0, m_0) \and \cdots \and d_{N-1} \in [0, m_{N-1}), \\ 716 | \text{where } T = \text{flatten}(X) 717 | $$ 718 | 719 | Test Parameter: 720 | 721 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 722 | 723 | shape: $(r, l, j, i), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 724 | 725 | 726 | 727 | ​ 728 | 729 | #### squeeze 730 | 731 | *Math Formalization* 732 | 733 | Suppose Input `X`, Output `Y`, attributes `axes`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, and `axes` is TShape and dimension is M. 734 | $$ 735 | \text{axis} \in [-N, N), \forall \text{axis} \in \text{axes} 736 | $$ 737 | 738 | $$ 739 | \text{real_axes} = 740 | \begin{cases} 741 | \{\text{axis} \mid \text{axis} \geqslant 0 \and \text{axis} \in \text{axes} \} \bigcup 742 | \{\text{axis} + N \mid \text{axis} < 0 \and \text{axis} \in \text{axis}\}, 743 | & M > 0 \\ 744 | \{\text{axis} \mid n_\text{axis} = 1 \and \text{axis} \in [0, N) \}, & M = 0 745 | \end{cases} \\ 746 | $$ 747 | 748 | $$ 749 | n_\text{axis} = 1, \forall \text{axis} \in \text{real_axis} 750 | $$ 751 | 752 | $$ 753 | Y[d_{I(0)}, d_{I(1)}, \cdots, d_{I(N-K-1)}] = X[d_0, d_1, \cdots, d_{N-1}], \\ 754 | \forall d_0 \in [0, n_0) \and d_1 \in [0, n_1) 755 | \and \cdots \and d_{N-1} \in [0, n_{N-1}), \\ 756 | \text{where } K = card \; \text{real_axes} \text{ and } \\ 757 | I: \{i \mid i \in [0, N-K) \} \to 758 | \{i \mid i \in [0, N) \and i \notin \text{real_axes} \}, \\ 759 | \text{satisfy } I(i) < I(j), \forall 0 \leqslant i < j < N-K 760 | $$ 761 | 762 | Test Parameter: 763 | 764 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 765 | 766 | ​ 767 | 768 | #### transpose 769 | 770 | *Math Formalization* 771 | 772 | Suppose Input `X`, Output `Y`, attributes `axes`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, and `axes` is TShape and dimension is M, where M is in $\{0, N\}$. 773 | $$ 774 | \text{axis} \in [-N, N), \forall \text{axis} \in \text{axes} 775 | $$ 776 | 777 | $$ 778 | Y[d_{\text{real_axes}_0}, d_{\text{real_axes}_1}, \cdots, d_{\text{real_axes}_{N-1}}] = 779 | X[d_0, d_1, \cdots, d_{N-1}], \\ 780 | \forall d_0 \in [0, n_0) \and \cdots \and d_{N-1} \in [0, n_{N-1}), \\ 781 | \text{where real_axes}_i = \begin{cases} 782 | \text{axes}_i, & M = N \and \text{axes}_i \geqslant 0 \\ 783 | \text{axes}_i + N, & M = N \and \text{axes}_i < 0 \\ 784 | N-1-i, & M = 0 785 | \end{cases} \text{ and } \\ 786 | card \; \{\text{real_axes}_i \mid i \in [0, N) \} = N 787 | $$ 788 | 789 | Test Parameter: 790 | 791 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 792 | 793 | ​ 794 | 795 | #### slice | strided_slice 796 | 797 | *Math Formalization* 798 | 799 | Suppose Input `X`, Output `Y`, attributes `begin`, `end`, `strides`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, and `begin`'s shape is B dimension, `end`'s shape is E dimension, `stride`'s shape is S dimension. 800 | $$ 801 | \text{b_arr}[b] = \begin{cases} 802 | \text{begin}[b] + n_i, & b \in [0, N) \and b < B \and begin[b] < 0 \\ 803 | \text{begin}[b], & b \in [0, N) \and b < B \and begin[b] \geqslant 0 \\ 804 | 0, & b \in [0, N) \and b \geqslant B 805 | \end{cases}, b \in [0, N) \\ 806 | \text{e_arr}[e] = \begin{cases} 807 | \text{end}[e] + n_i, & e \in [0, N) \and e < E \and end[e] < 0\\ 808 | \text{end}[e], & e \in [0, N) \and e < E \and end[e] \geqslant 0\\ 809 | n_{e}, & e \in [0, N) \and e \geqslant E 810 | \end{cases}, e \in [0, N) \\ 811 | \text{s_arr}[s] = \begin{cases} 812 | \text{stride}[s], & s \in [0, N) \and s < S \\ 813 | 1, & s \in [0, N) \and s \geqslant S 814 | \end{cases}, s \in [0, N) \\ 815 | \forall \{i \mid i \in [0, N)\}: \text{s_arr}[i] \ne 0 \\ 816 | \text{b_range}(i) = \begin{cases} 817 | -1, & \text{s_arr}[i] < 0 \\ 818 | 0, & \text{s_arr}[i] \geqslant 0 819 | \end{cases} \\ 820 | \text{e_range}(i) = \begin{cases} 821 | n_i - 1, & \text{s_arr}[i] < 0 \\ 822 | n_i, & \text{s_arr}[i] \geqslant 0 823 | \end{cases} \\ 824 | \text{b_vec}[b] = 825 | clip(\text{b_arr}[b], \text{a_min}=\text{b_range}(b), \text{a_max}=\text{e_range}(b)-1), b \in [0, N) \\ 826 | \text{e_vec}[e] = 827 | clip(\text{e_arr}[e], \text{a_min}=\text{b_range}(e), \text{a_max}=\text{e_range}(e)-1), e \in [0, N) \\ 828 | \forall \{i \mid i \in [0, N) \}: 829 | \begin{cases} 830 | \text{b_vec}[i] < \text{e_vec}[i], & \text{s_arr}[i] > 0 \\ 831 | \text{e_vec}[i] < \text{b_vec}[i], & \text{s_arr}[i] < 0 832 | \end{cases} \\ 833 | 834 | Y[d_0, d_1, \cdots, d_{N-1}] = \\ 835 | X[\text{b_vec}[0] + \text{s_arr}[0] * d_0, 836 | \text{b_vec}[1] + \text{s_arr}[1] * d_1, 837 | \cdots, \text{b_vec}[N-1] + \text{s_arr}[N-1] * d_{N-1}]] \\ 838 | \forall (d_0, d_1, \cdots, d_{N-1}), 839 | \text{where } d_j \in [0, 840 | \left\lceil{\text{e_vec}[j] - \text{b_vec}[j] \over \text{s_arr}[j]}\right\rceil) 841 | \and j \in [0, N) 842 | $$ 843 | 844 | Test Parameter: 845 | 846 | X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 847 | 848 | ​ 849 | 850 | #### take 851 | 852 | *Math Formalization* 853 | 854 | Suppose Input `X`, `indices`, Output `Y`, attributes `axis`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, `indices`'s shape is M dimension, exactly $(m_0, m_1, \cdots, m_{M- 1})$, and `axis` is `Optional` . 855 | 856 | 1. Case axis is `None` : 857 | 858 | $$ 859 | T = flatten(X) \\ 860 | Y[d_0, d_1, \cdots, d_{M-1}] = T[clip(\text{xidx}, \text{a_min}=0, \text{a_max}=|T|-1],\\ 861 | \forall (d_0, d_1, \cdots, d_{M-1}), \text{where } d_j \in [0, m_j) \and j \in [0, M) \text{ and }\\ 862 | \text{xidx} = \text{indices}[d_0, d_1, \cdots, d_{M-1}] 863 | $$ 864 | 865 | 2. Case axis is `int`: 866 | 867 | $$ 868 | \text{axis} \in [-N, N) \\ 869 | \text{real_axis} = \begin{cases} 870 | \text{axis}, & \text{axis} \geqslant 0 \\ 871 | \text{axis} + N, & \text{axis} < 0 872 | \end{cases} \\ 873 | Y[d_0, d_1, \cdots, d_{M+N-1}] = X[d_0, \cdots, d_{\text{real_axis}-1}, \text{xdix}, d_{\text{real_axis}+M}, \cdots, d_{M+N-1}], \\ 874 | \forall (d_0, d_1, \cdots, d_{M+N-1}), \text{where } d_j \in \begin{cases} 875 | [0, n_j), & j < \text{real_axis} \\ 876 | [0, m_{j-\text{real_axis}}), & j \in [\text{real_axis}, \text{real_axis}+M) \\ 877 | [0, n_{j-M+1}), & j \in [\text{real_axis} + M, M+N) 878 | \end{cases} \and j \in [0, M+N) : \\ 879 | 880 | \text{where } \text{xidx}{} = clip(\text{indices}[d_{\text{real_axis}}, d_{\text{real_axis}+1}, \cdots, d_{\text{real_axis}+M-1}], \text{a_min}=0, \text{a_max}=n_{\text{real_axis}}-1) 881 | $$ 882 | 883 | 884 | 885 | #### cvm_lut 886 | 887 | *Math Formalization* 888 | 889 | Suppose Input `indices`,`X`, Output `Y`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, `indices`'s shape is M dimension, exactly $(m_0, m_1, \cdots, m_{M- 1})$ . 890 | $$ 891 | Y=take(X, \text{indices}, \text{axis}=\text{None}) 892 | $$ 893 | 894 | 895 | #### slice_like 896 | 897 | *Math Formalization* 898 | 899 | Suppose Input `X`, `shape_like`, Output `Y`, attributes `axes`. Where `X`'s shape is N dimension, exactly $(n_0, n_1, \cdots, n_{N-1})$, `shape_like`'s shape is M dimension, exactly $(m_0, m_1, \cdots, m_{M- 1 })$, and `axes` is `TShape`, `axes`'s shape is K dimension. 900 | $$ 901 | \forall axis \in \text{axes}: axis \in [-N, M)\\ 902 | \text{real_axes} = \begin{cases} 903 | \{j \mid j \in axes \and j \geqslant 0\} \bigcup 904 | \{j + N \mid j \in axes \and j < 0\}, & K > 0\\ 905 | \{0, 1, \cdots, M-1\}, & K = 0 906 | \end{cases} \\ 907 | \forall j \in \text{real_axes}: m_j \leqslant n_j\\ 908 | Y[d_0, d_1, \cdots, d_{N-1}] = X[d_0, d_1, \cdots, d_{N-1}], \\ 909 | \text{where } j \in [0, N) \and d_j \in \begin{cases} 910 | [0, m_j), & j \in \text{real_axes} \\ 911 | [0, n_j), & j \notin \text{real_axes} 912 | \end{cases} 913 | $$ 914 | 915 | Test Parameter: 916 | 917 | ​ X.shape: $(i, j, l, r), \\ i=\{1\}, j=\{1, 14, 27, 40, 53, 66, 79, 92\}, l=\{1, 18, 35, 52, 69, 86\}, r=\{1, 24, 47, 70, 93\}$ 918 | 919 | ​ axis: $(0, 1)$ 920 | 921 | 922 | 923 | ### NMS Operator 924 | 925 | #### get_valid_count 926 | 927 | *Math Formalization* 928 | 929 | Suppose Input `X`, Output `valid_count`, `Y`, attributes `score_threshold`, where `X`'s shape is $(B, N, K), K \geqslant 2$, `score_threshold` is `int`. 930 | $$ 931 | \text{valid_count}[b] = card\{ q \mid q \in [0, N) \and 932 | X[b, q, 1] > \text{score_threshold} \}, \\ 933 | \quad \forall b \in [0, B) 934 | $$ 935 | 936 | $$ 937 | Y[b, \text{idx}, k] = X[b, n, k], \\ 938 | \quad \forall b \in [0, B) \and n \in [0, N) \and 939 | k \in [0, K) \and X[b, n, 1] > \text{score_threshold}, \\ 940 | \quad \text{where idx = } 941 | card \{q \mid q \in [0, n) \and 942 | X[b, q, 1] > \text{score_threshold} \} 943 | $$ 944 | 945 | $$ 946 | Y[b,n, k] = -1, \forall b \in [0, B) \and 947 | n \in [\text{valid_count}[b], N) \and k \in [0, K) 948 | $$ 949 | 950 | 951 | 952 | #### non_max_suppression 953 | 954 | *Math Formalization* 955 | 956 | Suppose Input `X`, `valid_count`, Output `Y`, attributes `iou_threshold`, `max_output_size`, `force_suppress`, `top_k`, where `X`'s shape is $(B, N, K), K = 6$, `iou_threshold` is `int`, the value is in range $(0, +\infty)$, 101 stands for bounding box full-overlap specifically, and larger integer is equivalent to that. `max_output_size` is `int`, `force_suppress` is `boolean`, `top_k` is `int`. 957 | $$ 958 | R[b, i, k] = X[b, I(i), k], \\ 959 | \forall b \in [0, B) \and i \in [0, T) \and k \in [0, K), \\ 960 | \text{where } T = \text{max}\{ 961 | \text{min}(N, \text{valid_count}[b]), 0\} \text{ and } \\ 962 | I: \{ i \mid i \in [0, T) \} \to \{ i \mid i \in [0, T) \}, \\ 963 | \text {satisfy } X[b, I(i), 1] > X[b, I(j), 1] \or \\ 964 | (X[b, I(i), 1] = X[b, I(j), 1] \and I(i) < I(j)), 965 | \forall 0 \leqslant i < j < T 966 | $$ 967 | 968 | $$ 969 | Y[b, n, k] = R[b, \text{IDX}(n), k], \\ 970 | \forall b \in [0, B) \and n \in [0, \min\{T, \text{MOS}, card\{U\}\}) \and 971 | k \in [0, K), \\ 972 | \text{where } \text{TK} = 973 | \begin{cases} 974 | +\infty, & \text{if top_k < 0} \\[1ex] 975 | \text{top_k}, & \text{otherwise} 976 | \end{cases} \text{ and } \\ 977 | \text{MOS} = 978 | \begin{cases} 979 | +\infty, & \text{if max_output_size < 0} \\[1ex] 980 | \text{max_output_size}, & \text{otherwise} 981 | \end{cases} \text{ and } \\ 982 | \text{iou}(p, q) = \begin{cases} 983 | \text{overlap_ratio}(R[b, p, :], R[b, q, :]), & 984 | \begin{array}{} 985 | \text{force_suppress is true} \or R[b, p, 0] = R[b, q, 0] 986 | \end{array} \\[1ex] 987 | 0, & \text{otherwise} 988 | \end{cases} \text{ and } \\ 989 | \text{U} = \{p \mid p \in [0, min\{TK, T\}) \and 990 | R[b,p,0] >= 0 \and \text{iou}(p, q) < \text{iou_threshold}, 991 | \forall q \in U \and q < p\} 992 | \text{ and } \\ 993 | \text{IDX}: \{i \mid i \in [0, card\{U\})\} \to U, \text{satisfy } 994 | \text{IDX}(i) < \text{IDX}(j), \forall 0 \leqslant i < j < card\{U\} 995 | $$ 996 | 997 | $$ 998 | Y[b, n, k] = -1, \\ 999 | \forall b \in [0, B) \and k \in [0, K) \and 1000 | n \in [min\{T, \text{MOS},card\{U\}\}, N) 1001 | $$ 1002 | --------------------------------------------------------------------------------