├── .github └── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature_request.md ├── LICENSE.txt ├── README.md ├── README.txt ├── apps ├── activation │ └── activationTableGenerator.cpp └── include │ └── nnproperties.hpp ├── cpp ├── activation.hpp ├── activationFunctions.hpp ├── constants.hpp ├── dynaq.hpp ├── error.hpp ├── exp.hpp ├── expValues16Bit.hpp ├── expValues32Bit.hpp ├── expValues64Bit.hpp ├── expValues8Bit.hpp ├── fixedPointTransferFunctions.hpp ├── interpolate.hpp ├── limits.hpp ├── log.hpp ├── logValues16Bit.hpp ├── logValues32Bit.hpp ├── logValues64Bit.hpp ├── logValues8Bit.hpp ├── lookupTable.hpp ├── lookupTables.cpp ├── neuralnet.hpp ├── nninit.hpp ├── qformat.hpp ├── qlearn.hpp ├── random.hpp ├── range.hpp ├── sigmoid.hpp ├── sigmoidValues16Bit.hpp ├── sigmoidValues32Bit.hpp ├── sigmoidValues64Bit.hpp ├── sigmoidValues8Bit.hpp ├── signed.hpp ├── tanh.hpp ├── tanhValues16Bit.hpp ├── tanhValues32Bit.hpp ├── tanhValues64Bit.hpp ├── tanhValues8Bit.hpp ├── typeChooser.hpp └── zeroTolerance.hpp ├── docs ├── _config.yml └── index.md ├── examples ├── dqn_maze │ ├── Makefile │ ├── dqn_maze.cpp │ ├── dqn_mazelearner.cpp │ ├── dqn_mazelearner.h │ └── dqn_mazeplot.py ├── maze │ ├── Makefile │ ├── maze.cpp │ ├── mazelearner.cpp │ ├── mazelearner.h │ └── mazeplot.py └── xor │ ├── Makefile │ ├── xor.cpp │ ├── xornet.cpp │ └── xornet.h ├── uml ├── nn │ ├── nn_class.puml │ ├── nn_ssd_init.puml │ └── nn_ssd_train_pt1.puml ├── qformat │ └── qformat_class.puml └── qlearn │ ├── qlearn_class.puml │ └── qlearn_seq.puml └── unit_test ├── nn ├── Makefile ├── nn_plot.py └── nn_unit_test.cpp ├── qformat ├── Makefile └── qformat_unit_test.cpp └── qlearn ├── Makefile └── qlearn_unit_test.cpp /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. 12 | 13 | **To Reproduce** 14 | Steps to reproduce the behavior: 15 | 1. Go to '...' 16 | 2. Click on '....' 17 | 3. Scroll down to '....' 18 | 4. See error 19 | 20 | **Expected behavior** 21 | A clear and concise description of what you expected to happen. 22 | 23 | **Screenshots** 24 | If applicable, add screenshots to help explain your problem. 25 | 26 | **Desktop (please complete the following information):** 27 | - OS: [e.g. iOS] 28 | - Browser [e.g. chrome, safari] 29 | - Version [e.g. 22] 30 | 31 | **Smartphone (please complete the following information):** 32 | - Device: [e.g. iPhone6] 33 | - OS: [e.g. iOS8.1] 34 | - Browser [e.g. stock browser, safari] 35 | - Version [e.g. 22] 36 | 37 | **Additional context** 38 | Add any other context about the problem here. 39 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Is your feature request related to a problem? Please describe.** 11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 12 | 13 | **Describe the solution you'd like** 14 | A clear and concise description of what you want to happen. 15 | 16 | **Describe alternatives you've considered** 17 | A clear and concise description of any alternative solutions or features you've considered. 18 | 19 | **Additional context** 20 | Add any other context or screenshots about the feature request here. 21 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Intel Corporation 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | DISCONTINUATION OF PROJECT 2 | 3 | This project will no longer be maintained by Intel. 4 | 5 | Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project. 6 | 7 | Intel no longer accepts patches to this project. 8 | 9 | If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project. 10 | 11 | Contact: webadmin@linux.intel.com 12 | -------------------------------------------------------------------------------- /README.txt: -------------------------------------------------------------------------------- 1 | # cppnnml 2 | The C++ Neural Network and Machine Learning project is intended to provide a C++ template library for neural nets and machine learning algorithms as well as a Q format library. 3 | The neural networks and machine learning algorithms combined with Q format can be run on embedded systems with very low compute resources. It does not rely upon FPU, GPU, or vectorized 4 | instructions. 5 | 6 | The project wiki is at https://github.com/intel/cppnnml/wiki. I would recommend reading thru some of the examples which will guide you in compiling the code. In order 7 | to produce code with the smallest possible footprint, the code relies upon preprocessor symbols defined either on the command line or within the build system 8 | to ensure that only needed code is compiled (e.g. LUTs for activation functions). 9 | -------------------------------------------------------------------------------- /cpp/activation.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #define NUMBER_OF_ACTIVATION_TABLE_VALUES 96 26 | #define MIN_X_TABLE_VALUE -6 27 | #define MAX_X_TABLE_VALUE 6 28 | #define ACTIVATION_DELTA_SHIFT 3 29 | 30 | -------------------------------------------------------------------------------- /cpp/activationFunctions.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "lookupTable.hpp" 26 | #include "constants.hpp" 27 | #include "sigmoid.hpp" 28 | #include "tanh.hpp" 29 | #include "exp.hpp" 30 | #include "typeChooser.hpp" 31 | 32 | namespace tinymind { 33 | template 34 | struct NullActivationPolicy 35 | { 36 | static ValueType activationFunction(const ValueType& value) 37 | { 38 | return 0; 39 | } 40 | 41 | static ValueType activationFunctionDerivative(const ValueType& value) 42 | { 43 | return 0; 44 | } 45 | }; 46 | 47 | template 48 | struct ReluActivationPolicy 49 | { 50 | static ValueType activationFunction(const ValueType& value) 51 | { 52 | if(value <= 0) 53 | { 54 | return 0; 55 | } 56 | 57 | return value; 58 | } 59 | 60 | static ValueType activationFunctionDerivative(const ValueType& value) 61 | { 62 | if(value <= 0) 63 | { 64 | return 0; 65 | } 66 | 67 | return Constants::one(); 68 | } 69 | }; 70 | 71 | template::FullWidthFieldType MaxValue> 72 | struct CappedReluActivationPolicy 73 | { 74 | typedef typename FullWidthFieldTypeShim::FullWidthFieldType FullWidthFieldType; 75 | static constexpr FullWidthFieldType MAX_VALUE = MaxValue; 76 | 77 | static ValueType activationFunction(const ValueType& value) 78 | { 79 | if(value <= 0) 80 | { 81 | return 0; 82 | } 83 | 84 | if(value > MAX_VALUE) 85 | { 86 | return MAX_VALUE; 87 | } 88 | else 89 | { 90 | return value; 91 | } 92 | } 93 | 94 | static ValueType activationFunctionDerivative(const ValueType& value) 95 | { 96 | return ReluActivationPolicy::activationFunctionDerivative(value); 97 | } 98 | private: 99 | static_assert(ValueType::IsSigned, "Capped Relu activation policy requires a signed type."); 100 | }; 101 | 102 | template 103 | struct SigmoidActivationPolicy 104 | { 105 | typedef LookupTable LookupTableType; 106 | typedef typename SigmoidValuesTableSelector::SigmoidTableType SigmoidTableType; 107 | 108 | static ValueType activationFunction(const ValueType& value) 109 | { 110 | static constexpr ptrdiff_t MAX_ACTIVATION_INDEX = ((sizeof(sigmoidActivationTable.values) / sizeof(sigmoidActivationTable.values[0])) - 1); 111 | 112 | const ValueType result = LookupTableType::getValue(value, &sigmoidActivationTable.values[0], MAX_ACTIVATION_INDEX); 113 | 114 | return result; 115 | } 116 | 117 | static ValueType activationFunctionDerivative(const ValueType& value) 118 | { 119 | //Approximation for 1st derivative of sigmoid 120 | const ValueType oneMinusValue = (Constants::one() - value); 121 | 122 | return (value * oneMinusValue); 123 | } 124 | private: 125 | static const SigmoidTableType sigmoidActivationTable; 126 | static_assert(ValueType::IsSigned, "Sigmoid activation tables require a signed type."); 127 | }; 128 | 129 | template 130 | const typename SigmoidValuesTableSelector::SigmoidTableType SigmoidActivationPolicy::sigmoidActivationTable; 131 | 132 | template 133 | struct TanhActivationPolicy 134 | { 135 | typedef typename ValueType::FullWidthFieldType FullWidthFieldType; 136 | typedef LookupTable LookupTableType; 137 | typedef typename TanhValuesTableSelector::TanhTableType TanhTableType; 138 | 139 | static ValueType activationFunction(const ValueType& value) 140 | { 141 | static constexpr ptrdiff_t MAX_ACTIVATION_INDEX = (((sizeof(FullWidthFieldType) * NUMBER_OF_ACTIVATION_TABLE_VALUES) / sizeof(tanhActivationTable.values[0])) - 1); 142 | 143 | const ValueType result = LookupTableType::getValue(value, &tanhActivationTable.values[0], MAX_ACTIVATION_INDEX); 144 | 145 | return result; 146 | } 147 | 148 | static ValueType activationFunctionDerivative(const ValueType& value) 149 | { 150 | //Approximation for 1st derivative of tanh 151 | const ValueType valueSquared = (value * value); 152 | 153 | return (Constants::one() - valueSquared); 154 | } 155 | private: 156 | static const TanhTableType tanhActivationTable; 157 | static_assert(ValueType::IsSigned, "Tanh activation tables require a signed type."); 158 | }; 159 | 160 | template 161 | const typename TanhValuesTableSelector::TanhTableType TanhActivationPolicy::tanhActivationTable; 162 | 163 | template 164 | struct SoftmaxActivationPolicy 165 | { 166 | typedef typename ValueType::FullWidthFieldType FullWidthFieldType; 167 | typedef LookupTable LookupTableType; 168 | typedef typename ExpValuesTableSelector::ExpTableType ExpTableType; 169 | 170 | static void activationFunction(ValueType const* const values, ValueType* results, const size_t numberOfNerons) 171 | { 172 | static constexpr ptrdiff_t MAX_ACTIVATION_INDEX = (((sizeof(FullWidthFieldType) * NUMBER_OF_ACTIVATION_TABLE_VALUES) / sizeof(expActivationTable.values[0])) - 1); 173 | ValueType result; 174 | ValueType sum(0); 175 | 176 | for(size_t neuron = 0;neuron < numberOfNerons;++neuron) 177 | { 178 | result = LookupTableType::getValue(values[neuron], &expActivationTable.values[0], MAX_ACTIVATION_INDEX); 179 | results[neuron] = result; 180 | sum += result; 181 | } 182 | 183 | for(size_t neuron = 0;neuron < numberOfNerons;++neuron) 184 | { 185 | results[neuron] /= sum; 186 | } 187 | } 188 | 189 | static void activationFunctionDerivative(ValueType const* const values, ValueType const* const targetValues, ValueType* results, const size_t numberOfNerons) 190 | { 191 | for(size_t neuron = 0;neuron < numberOfNerons;++neuron) 192 | { 193 | if(targetValues[neuron] == Constants::one()) 194 | { 195 | results[neuron] = (values[neuron] - Constants::one()); 196 | } 197 | else 198 | { 199 | results[neuron] = values[neuron]; 200 | } 201 | } 202 | } 203 | private: 204 | static const ExpTableType expActivationTable; 205 | static_assert(ValueType::IsSigned, "Exp activation tables require a signed type."); 206 | }; 207 | 208 | template 209 | const typename ExpValuesTableSelector::ExpTableType SoftmaxActivationPolicy::expActivationTable; 210 | } -------------------------------------------------------------------------------- /cpp/constants.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | namespace tinymind { 26 | template 27 | struct Constants 28 | { 29 | static ValueType one() 30 | { 31 | static const ValueType result(1,0); 32 | 33 | return result; 34 | } 35 | 36 | static ValueType negativeOne() 37 | { 38 | static const ValueType result(-1, 0); 39 | 40 | return result; 41 | } 42 | 43 | static ValueType zero() 44 | { 45 | static const ValueType result(0); 46 | 47 | return result; 48 | } 49 | }; 50 | } 51 | -------------------------------------------------------------------------------- /cpp/dynaq.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | namespace tinymind { 26 | template 27 | struct DefaultSimulatedExperiencePolicy 28 | { 29 | typedef typename NeuralNetworkType::NeuralNetworkValueType ValueType; 30 | 31 | DefaultSimulatedExperiencePolicy() : mAverageError(0), mNumberOfSamplesPerAverage(10), mAverageErrorCalculated(false) {} 32 | DefaultSimulatedExperiencePolicy(const int numberOfSamplesPerAverage) : mAverageError(0), mNumberOfSamplesPerAverage(numberOfSamplesPerAverage), mAverageErrorCalculated(false) {} 33 | 34 | bool readyToSimulateExperience() const 35 | { 36 | if(this->mAverageErrorCalculated && NeuralNetworkType::NeuralNetworkTransferFunctionsPolicy::isWithinZeroTolerance(this->mAverageError)) 37 | { 38 | return true; 39 | } 40 | 41 | return false; 42 | } 43 | 44 | void updateModel() 45 | { 46 | static ValueType accumulatedError(0); 47 | static int sampleNumber = 0; 48 | const ValueType error = this->mNeuralNetwork.getError(); 49 | 50 | ++sampleNumber; 51 | accumulatedError += error; 52 | if(sampleNumber == this->mNumberOfSamplesPerAverage) 53 | { 54 | this->mAverageErrorCalculated = true; 55 | accumulatedError /= this->mNumberOfSamplesPerAverage; 56 | this->mAverageError = accumulatedError; 57 | sampleNumber = 0; 58 | accumulatedError = 0; 59 | } 60 | } 61 | private: 62 | ValueType mAverageError; 63 | int mNumberOfSamplesPerAverage; 64 | bool mAverageErrorCalculated; 65 | NeuralNetworkType mNeuralNetwork; 66 | }; 67 | 68 | template 69 | class DynaQLearner 70 | { 71 | public: 72 | typedef typename QLearnerType::ValueType ValueType; 73 | 74 | DynaQLearner(const ValueType& learningRate, const ValueType& discountFactor, const size_t randomActionDecisionPoint) : 75 | mQLearner(learningRate, discountFactor, randomActionDecisionPoint) 76 | { 77 | } 78 | private: 79 | QLearnerType mQLearner; 80 | SimulatedExperiencePolicy mSimulatedExperiencePolicy; 81 | }; 82 | } -------------------------------------------------------------------------------- /cpp/error.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "lookupTable.hpp" 26 | #include "log.hpp" 27 | #include "constants.hpp" 28 | 29 | namespace tinymind { 30 | template 31 | struct MeanSquaredErrorCalculator 32 | { 33 | /** 34 | * Mean-squared error 35 | */ 36 | static ValueType calculateError(ValueType const* const targetValues, ValueType const* const outputValues) 37 | { 38 | ValueType error(0); 39 | 40 | for (uint32_t neuron = 0; neuron < NumberOfOutputNeurons; neuron++) 41 | { 42 | const ValueType delta = (targetValues[neuron] - outputValues[neuron]); 43 | const ValueType deltaSquared = (delta * delta); 44 | error += deltaSquared; 45 | } 46 | 47 | if (NumberOfOutputNeurons > 1) 48 | { 49 | error /= NumberOfOutputNeurons; 50 | } 51 | 52 | return error; 53 | } 54 | }; 55 | 56 | template 57 | struct CrossEntropyLossCalculator 58 | { 59 | typedef typename ValueType::FullWidthFieldType FullWidthFieldType; 60 | typedef LookupTable LookupTableType; 61 | typedef typename LogValuesTableSelector::LogTableType LogTableType; 62 | 63 | /** 64 | * Cross-entropy loss 65 | */ 66 | static ValueType calculateError(ValueType const* const targetValues, ValueType const* const outputValues) 67 | { 68 | static const ptrdiff_t MAX_ACTIVATION_INDEX = (((sizeof(FullWidthFieldType) * NUMBER_OF_ACTIVATION_TABLE_VALUES) / sizeof(logActivationTable.values[0])) - 1); 69 | ValueType error(0); 70 | 71 | for (uint32_t neuron = 0; neuron < NumberOfOutputNeurons; neuron++) 72 | { 73 | error += (targetValues[neuron] * LookupTableType::getValue(outputValues[neuron], &logActivationTable.values[0], MAX_ACTIVATION_INDEX)); 74 | } 75 | 76 | error *= Constants::negativeOne(); 77 | 78 | return error; 79 | } 80 | private: 81 | static const LogTableType logActivationTable; 82 | }; 83 | 84 | template 85 | const typename LogValuesTableSelector::LogTableType CrossEntropyLossCalculator::logActivationTable; 86 | } 87 | -------------------------------------------------------------------------------- /cpp/expValues16Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_EXP_1_15)) 29 | struct ExpValuesTableQ1_15 30 | { 31 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_EXP_1_15)) 34 | #if (defined(TINYMIND_USE_EXP_2_14)) 35 | struct ExpValuesTableQ2_14 36 | { 37 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_EXP_2_14)) 40 | #if (defined(TINYMIND_USE_EXP_3_13)) 41 | struct ExpValuesTableQ3_13 42 | { 43 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_EXP_3_13)) 46 | #if (defined(TINYMIND_USE_EXP_4_12)) 47 | struct ExpValuesTableQ4_12 48 | { 49 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_EXP_4_12)) 52 | #if (defined(TINYMIND_USE_EXP_5_11)) 53 | struct ExpValuesTableQ5_11 54 | { 55 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_EXP_5_11)) 58 | #if (defined(TINYMIND_USE_EXP_6_10)) 59 | struct ExpValuesTableQ6_10 60 | { 61 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_EXP_6_10)) 64 | #if (defined(TINYMIND_USE_EXP_7_9)) 65 | struct ExpValuesTableQ7_9 66 | { 67 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_EXP_7_9)) 70 | #if (defined(TINYMIND_USE_EXP_8_8)) 71 | struct ExpValuesTableQ8_8 72 | { 73 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_EXP_8_8)) 76 | #if (defined(TINYMIND_USE_EXP_9_7)) 77 | struct ExpValuesTableQ9_7 78 | { 79 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_EXP_9_7)) 82 | #if (defined(TINYMIND_USE_EXP_10_6)) 83 | struct ExpValuesTableQ10_6 84 | { 85 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_EXP_10_6)) 88 | #if (defined(TINYMIND_USE_EXP_11_5)) 89 | struct ExpValuesTableQ11_5 90 | { 91 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_EXP_11_5)) 94 | #if (defined(TINYMIND_USE_EXP_12_4)) 95 | struct ExpValuesTableQ12_4 96 | { 97 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_EXP_12_4)) 100 | #if (defined(TINYMIND_USE_EXP_13_3)) 101 | struct ExpValuesTableQ13_3 102 | { 103 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_EXP_13_3)) 106 | #if (defined(TINYMIND_USE_EXP_14_2)) 107 | struct ExpValuesTableQ14_2 108 | { 109 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_EXP_14_2)) 112 | #if (defined(TINYMIND_USE_EXP_15_1)) 113 | struct ExpValuesTableQ15_1 114 | { 115 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_EXP_15_1)) 118 | } 119 | -------------------------------------------------------------------------------- /cpp/expValues32Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_EXP_1_31)) 29 | struct ExpValuesTableQ1_31 30 | { 31 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_EXP_1_31)) 34 | #if (defined(TINYMIND_USE_EXP_2_30)) 35 | struct ExpValuesTableQ2_30 36 | { 37 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_EXP_2_30)) 40 | #if (defined(TINYMIND_USE_EXP_3_29)) 41 | struct ExpValuesTableQ3_29 42 | { 43 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_EXP_3_29)) 46 | #if (defined(TINYMIND_USE_EXP_4_28)) 47 | struct ExpValuesTableQ4_28 48 | { 49 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_EXP_4_28)) 52 | #if (defined(TINYMIND_USE_EXP_5_27)) 53 | struct ExpValuesTableQ5_27 54 | { 55 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_EXP_5_27)) 58 | #if (defined(TINYMIND_USE_EXP_6_26)) 59 | struct ExpValuesTableQ6_26 60 | { 61 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_EXP_6_26)) 64 | #if (defined(TINYMIND_USE_EXP_7_25)) 65 | struct ExpValuesTableQ7_25 66 | { 67 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_EXP_7_25)) 70 | #if (defined(TINYMIND_USE_EXP_8_24)) 71 | struct ExpValuesTableQ8_24 72 | { 73 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_EXP_8_24)) 76 | #if (defined(TINYMIND_USE_EXP_9_23)) 77 | struct ExpValuesTableQ9_23 78 | { 79 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_EXP_9_23)) 82 | #if (defined(TINYMIND_USE_EXP_10_22)) 83 | struct ExpValuesTableQ10_22 84 | { 85 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_EXP_10_22)) 88 | #if (defined(TINYMIND_USE_EXP_11_21)) 89 | struct ExpValuesTableQ11_21 90 | { 91 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_EXP_11_21)) 94 | #if (defined(TINYMIND_USE_EXP_12_20)) 95 | struct ExpValuesTableQ12_20 96 | { 97 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_EXP_12_20)) 100 | #if (defined(TINYMIND_USE_EXP_13_19)) 101 | struct ExpValuesTableQ13_19 102 | { 103 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_EXP_13_19)) 106 | #if (defined(TINYMIND_USE_EXP_14_18)) 107 | struct ExpValuesTableQ14_18 108 | { 109 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_EXP_14_18)) 112 | #if (defined(TINYMIND_USE_EXP_15_17)) 113 | struct ExpValuesTableQ15_17 114 | { 115 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_EXP_15_17)) 118 | #if (defined(TINYMIND_USE_EXP_16_16)) 119 | struct ExpValuesTableQ16_16 120 | { 121 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 122 | }; 123 | #endif // (defined(TINYMIND_USE_EXP_16_16)) 124 | #if (defined(TINYMIND_USE_EXP_17_15)) 125 | struct ExpValuesTableQ17_15 126 | { 127 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 128 | }; 129 | #endif // (defined(TINYMIND_USE_EXP_17_15)) 130 | #if (defined(TINYMIND_USE_EXP_18_14)) 131 | struct ExpValuesTableQ18_14 132 | { 133 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 134 | }; 135 | #endif // (defined(TINYMIND_USE_EXP_18_14)) 136 | #if (defined(TINYMIND_USE_EXP_19_13)) 137 | struct ExpValuesTableQ19_13 138 | { 139 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 140 | }; 141 | #endif // (defined(TINYMIND_USE_EXP_19_13)) 142 | #if (defined(TINYMIND_USE_EXP_20_12)) 143 | struct ExpValuesTableQ20_12 144 | { 145 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 146 | }; 147 | #endif // (defined(TINYMIND_USE_EXP_20_12)) 148 | #if (defined(TINYMIND_USE_EXP_21_11)) 149 | struct ExpValuesTableQ21_11 150 | { 151 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 152 | }; 153 | #endif // (defined(TINYMIND_USE_EXP_21_11)) 154 | #if (defined(TINYMIND_USE_EXP_22_10)) 155 | struct ExpValuesTableQ22_10 156 | { 157 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 158 | }; 159 | #endif // (defined(TINYMIND_USE_EXP_22_10)) 160 | #if (defined(TINYMIND_USE_EXP_23_9)) 161 | struct ExpValuesTableQ23_9 162 | { 163 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 164 | }; 165 | #endif // (defined(TINYMIND_USE_EXP_23_9)) 166 | #if (defined(TINYMIND_USE_EXP_24_8)) 167 | struct ExpValuesTableQ24_8 168 | { 169 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 170 | }; 171 | #endif // (defined(TINYMIND_USE_EXP_24_8)) 172 | #if (defined(TINYMIND_USE_EXP_25_7)) 173 | struct ExpValuesTableQ25_7 174 | { 175 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 176 | }; 177 | #endif // (defined(TINYMIND_USE_EXP_25_7)) 178 | #if (defined(TINYMIND_USE_EXP_26_6)) 179 | struct ExpValuesTableQ26_6 180 | { 181 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 182 | }; 183 | #endif // (defined(TINYMIND_USE_EXP_26_6)) 184 | #if (defined(TINYMIND_USE_EXP_27_5)) 185 | struct ExpValuesTableQ27_5 186 | { 187 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 188 | }; 189 | #endif // (defined(TINYMIND_USE_EXP_27_5)) 190 | #if (defined(TINYMIND_USE_EXP_28_4)) 191 | struct ExpValuesTableQ28_4 192 | { 193 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 194 | }; 195 | #endif // (defined(TINYMIND_USE_EXP_28_4)) 196 | #if (defined(TINYMIND_USE_EXP_29_3)) 197 | struct ExpValuesTableQ29_3 198 | { 199 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 200 | }; 201 | #endif // (defined(TINYMIND_USE_EXP_29_3)) 202 | #if (defined(TINYMIND_USE_EXP_30_2)) 203 | struct ExpValuesTableQ30_2 204 | { 205 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 206 | }; 207 | #endif // (defined(TINYMIND_USE_EXP_30_2)) 208 | #if (defined(TINYMIND_USE_EXP_31_1)) 209 | struct ExpValuesTableQ31_1 210 | { 211 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 212 | }; 213 | #endif // (defined(TINYMIND_USE_EXP_31_1)) 214 | } 215 | -------------------------------------------------------------------------------- /cpp/expValues64Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_EXP_1_63)) 29 | struct ExpValuesTableQ1_63 30 | { 31 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_EXP_1_63)) 34 | #if (defined(TINYMIND_USE_EXP_2_62)) 35 | struct ExpValuesTableQ2_62 36 | { 37 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_EXP_2_62)) 40 | #if (defined(TINYMIND_USE_EXP_3_61)) 41 | struct ExpValuesTableQ3_61 42 | { 43 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_EXP_3_61)) 46 | #if (defined(TINYMIND_USE_EXP_4_60)) 47 | struct ExpValuesTableQ4_60 48 | { 49 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_EXP_4_60)) 52 | #if (defined(TINYMIND_USE_EXP_5_59)) 53 | struct ExpValuesTableQ5_59 54 | { 55 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_EXP_5_59)) 58 | #if (defined(TINYMIND_USE_EXP_6_58)) 59 | struct ExpValuesTableQ6_58 60 | { 61 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_EXP_6_58)) 64 | #if (defined(TINYMIND_USE_EXP_7_57)) 65 | struct ExpValuesTableQ7_57 66 | { 67 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_EXP_7_57)) 70 | #if (defined(TINYMIND_USE_EXP_8_56)) 71 | struct ExpValuesTableQ8_56 72 | { 73 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_EXP_8_56)) 76 | #if (defined(TINYMIND_USE_EXP_9_55)) 77 | struct ExpValuesTableQ9_55 78 | { 79 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_EXP_9_55)) 82 | #if (defined(TINYMIND_USE_EXP_10_54)) 83 | struct ExpValuesTableQ10_54 84 | { 85 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_EXP_10_54)) 88 | #if (defined(TINYMIND_USE_EXP_11_53)) 89 | struct ExpValuesTableQ11_53 90 | { 91 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_EXP_11_53)) 94 | #if (defined(TINYMIND_USE_EXP_12_52)) 95 | struct ExpValuesTableQ12_52 96 | { 97 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_EXP_12_52)) 100 | #if (defined(TINYMIND_USE_EXP_13_51)) 101 | struct ExpValuesTableQ13_51 102 | { 103 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_EXP_13_51)) 106 | #if (defined(TINYMIND_USE_EXP_14_50)) 107 | struct ExpValuesTableQ14_50 108 | { 109 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_EXP_14_50)) 112 | #if (defined(TINYMIND_USE_EXP_15_49)) 113 | struct ExpValuesTableQ15_49 114 | { 115 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_EXP_15_49)) 118 | #if (defined(TINYMIND_USE_EXP_16_48)) 119 | struct ExpValuesTableQ16_48 120 | { 121 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 122 | }; 123 | #endif // (defined(TINYMIND_USE_EXP_16_48)) 124 | #if (defined(TINYMIND_USE_EXP_17_47)) 125 | struct ExpValuesTableQ17_47 126 | { 127 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 128 | }; 129 | #endif // (defined(TINYMIND_USE_EXP_17_47)) 130 | #if (defined(TINYMIND_USE_EXP_18_46)) 131 | struct ExpValuesTableQ18_46 132 | { 133 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 134 | }; 135 | #endif // (defined(TINYMIND_USE_EXP_18_46)) 136 | #if (defined(TINYMIND_USE_EXP_19_45)) 137 | struct ExpValuesTableQ19_45 138 | { 139 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 140 | }; 141 | #endif // (defined(TINYMIND_USE_EXP_19_45)) 142 | #if (defined(TINYMIND_USE_EXP_20_44)) 143 | struct ExpValuesTableQ20_44 144 | { 145 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 146 | }; 147 | #endif // (defined(TINYMIND_USE_EXP_20_44)) 148 | #if (defined(TINYMIND_USE_EXP_21_43)) 149 | struct ExpValuesTableQ21_43 150 | { 151 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 152 | }; 153 | #endif // (defined(TINYMIND_USE_EXP_21_43)) 154 | #if (defined(TINYMIND_USE_EXP_22_42)) 155 | struct ExpValuesTableQ22_42 156 | { 157 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 158 | }; 159 | #endif // (defined(TINYMIND_USE_EXP_22_42)) 160 | #if (defined(TINYMIND_USE_EXP_23_41)) 161 | struct ExpValuesTableQ23_41 162 | { 163 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 164 | }; 165 | #endif // (defined(TINYMIND_USE_EXP_23_41)) 166 | #if (defined(TINYMIND_USE_EXP_24_40)) 167 | struct ExpValuesTableQ24_40 168 | { 169 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 170 | }; 171 | #endif // (defined(TINYMIND_USE_EXP_24_40)) 172 | #if (defined(TINYMIND_USE_EXP_25_39)) 173 | struct ExpValuesTableQ25_39 174 | { 175 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 176 | }; 177 | #endif // (defined(TINYMIND_USE_EXP_25_39)) 178 | #if (defined(TINYMIND_USE_EXP_26_38)) 179 | struct ExpValuesTableQ26_38 180 | { 181 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 182 | }; 183 | #endif // (defined(TINYMIND_USE_EXP_26_38)) 184 | #if (defined(TINYMIND_USE_EXP_27_37)) 185 | struct ExpValuesTableQ27_37 186 | { 187 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 188 | }; 189 | #endif // (defined(TINYMIND_USE_EXP_27_37)) 190 | #if (defined(TINYMIND_USE_EXP_28_36)) 191 | struct ExpValuesTableQ28_36 192 | { 193 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 194 | }; 195 | #endif // (defined(TINYMIND_USE_EXP_28_36)) 196 | #if (defined(TINYMIND_USE_EXP_29_35)) 197 | struct ExpValuesTableQ29_35 198 | { 199 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 200 | }; 201 | #endif // (defined(TINYMIND_USE_EXP_29_35)) 202 | #if (defined(TINYMIND_USE_EXP_30_34)) 203 | struct ExpValuesTableQ30_34 204 | { 205 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 206 | }; 207 | #endif // (defined(TINYMIND_USE_EXP_30_34)) 208 | #if (defined(TINYMIND_USE_EXP_31_33)) 209 | struct ExpValuesTableQ31_33 210 | { 211 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 212 | }; 213 | #endif // (defined(TINYMIND_USE_EXP_31_33)) 214 | #if (defined(TINYMIND_USE_EXP_32_32)) 215 | struct ExpValuesTableQ32_32 216 | { 217 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 218 | }; 219 | #endif // (defined(TINYMIND_USE_EXP_32_32)) 220 | #if (defined(TINYMIND_USE_EXP_33_31)) 221 | struct ExpValuesTableQ33_31 222 | { 223 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 224 | }; 225 | #endif // (defined(TINYMIND_USE_EXP_33_31)) 226 | #if (defined(TINYMIND_USE_EXP_34_30)) 227 | struct ExpValuesTableQ34_30 228 | { 229 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 230 | }; 231 | #endif // (defined(TINYMIND_USE_EXP_34_30)) 232 | #if (defined(TINYMIND_USE_EXP_35_29)) 233 | struct ExpValuesTableQ35_29 234 | { 235 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 236 | }; 237 | #endif // (defined(TINYMIND_USE_EXP_35_29)) 238 | #if (defined(TINYMIND_USE_EXP_36_28)) 239 | struct ExpValuesTableQ36_28 240 | { 241 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 242 | }; 243 | #endif // (defined(TINYMIND_USE_EXP_36_28)) 244 | #if (defined(TINYMIND_USE_EXP_37_27)) 245 | struct ExpValuesTableQ37_27 246 | { 247 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 248 | }; 249 | #endif // (defined(TINYMIND_USE_EXP_37_27)) 250 | #if (defined(TINYMIND_USE_EXP_38_26)) 251 | struct ExpValuesTableQ38_26 252 | { 253 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 254 | }; 255 | #endif // (defined(TINYMIND_USE_EXP_38_26)) 256 | #if (defined(TINYMIND_USE_EXP_39_25)) 257 | struct ExpValuesTableQ39_25 258 | { 259 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 260 | }; 261 | #endif // (defined(TINYMIND_USE_EXP_39_25)) 262 | #if (defined(TINYMIND_USE_EXP_40_24)) 263 | struct ExpValuesTableQ40_24 264 | { 265 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 266 | }; 267 | #endif // (defined(TINYMIND_USE_EXP_40_24)) 268 | #if (defined(TINYMIND_USE_EXP_41_23)) 269 | struct ExpValuesTableQ41_23 270 | { 271 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 272 | }; 273 | #endif // (defined(TINYMIND_USE_EXP_41_23)) 274 | #if (defined(TINYMIND_USE_EXP_42_22)) 275 | struct ExpValuesTableQ42_22 276 | { 277 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 278 | }; 279 | #endif // (defined(TINYMIND_USE_EXP_42_22)) 280 | #if (defined(TINYMIND_USE_EXP_43_21)) 281 | struct ExpValuesTableQ43_21 282 | { 283 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 284 | }; 285 | #endif // (defined(TINYMIND_USE_EXP_43_21)) 286 | #if (defined(TINYMIND_USE_EXP_44_20)) 287 | struct ExpValuesTableQ44_20 288 | { 289 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 290 | }; 291 | #endif // (defined(TINYMIND_USE_EXP_44_20)) 292 | #if (defined(TINYMIND_USE_EXP_45_19)) 293 | struct ExpValuesTableQ45_19 294 | { 295 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 296 | }; 297 | #endif // (defined(TINYMIND_USE_EXP_45_19)) 298 | #if (defined(TINYMIND_USE_EXP_46_18)) 299 | struct ExpValuesTableQ46_18 300 | { 301 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 302 | }; 303 | #endif // (defined(TINYMIND_USE_EXP_46_18)) 304 | #if (defined(TINYMIND_USE_EXP_47_17)) 305 | struct ExpValuesTableQ47_17 306 | { 307 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 308 | }; 309 | #endif // (defined(TINYMIND_USE_EXP_47_17)) 310 | #if (defined(TINYMIND_USE_EXP_48_16)) 311 | struct ExpValuesTableQ48_16 312 | { 313 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 314 | }; 315 | #endif // (defined(TINYMIND_USE_EXP_48_16)) 316 | #if (defined(TINYMIND_USE_EXP_49_15)) 317 | struct ExpValuesTableQ49_15 318 | { 319 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 320 | }; 321 | #endif // (defined(TINYMIND_USE_EXP_49_15)) 322 | #if (defined(TINYMIND_USE_EXP_50_14)) 323 | struct ExpValuesTableQ50_14 324 | { 325 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 326 | }; 327 | #endif // (defined(TINYMIND_USE_EXP_50_14)) 328 | #if (defined(TINYMIND_USE_EXP_51_13)) 329 | struct ExpValuesTableQ51_13 330 | { 331 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 332 | }; 333 | #endif // (defined(TINYMIND_USE_EXP_51_13)) 334 | #if (defined(TINYMIND_USE_EXP_52_12)) 335 | struct ExpValuesTableQ52_12 336 | { 337 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 338 | }; 339 | #endif // (defined(TINYMIND_USE_EXP_52_12)) 340 | #if (defined(TINYMIND_USE_EXP_53_11)) 341 | struct ExpValuesTableQ53_11 342 | { 343 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 344 | }; 345 | #endif // (defined(TINYMIND_USE_EXP_53_11)) 346 | #if (defined(TINYMIND_USE_EXP_54_10)) 347 | struct ExpValuesTableQ54_10 348 | { 349 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 350 | }; 351 | #endif // (defined(TINYMIND_USE_EXP_54_10)) 352 | #if (defined(TINYMIND_USE_EXP_55_9)) 353 | struct ExpValuesTableQ55_9 354 | { 355 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 356 | }; 357 | #endif // (defined(TINYMIND_USE_EXP_55_9)) 358 | #if (defined(TINYMIND_USE_EXP_56_8)) 359 | struct ExpValuesTableQ56_8 360 | { 361 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 362 | }; 363 | #endif // (defined(TINYMIND_USE_EXP_56_8)) 364 | #if (defined(TINYMIND_USE_EXP_57_7)) 365 | struct ExpValuesTableQ57_7 366 | { 367 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 368 | }; 369 | #endif // (defined(TINYMIND_USE_EXP_57_7)) 370 | #if (defined(TINYMIND_USE_EXP_58_6)) 371 | struct ExpValuesTableQ58_6 372 | { 373 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 374 | }; 375 | #endif // (defined(TINYMIND_USE_EXP_58_6)) 376 | #if (defined(TINYMIND_USE_EXP_59_5)) 377 | struct ExpValuesTableQ59_5 378 | { 379 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 380 | }; 381 | #endif // (defined(TINYMIND_USE_EXP_59_5)) 382 | #if (defined(TINYMIND_USE_EXP_60_4)) 383 | struct ExpValuesTableQ60_4 384 | { 385 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 386 | }; 387 | #endif // (defined(TINYMIND_USE_EXP_60_4)) 388 | #if (defined(TINYMIND_USE_EXP_61_3)) 389 | struct ExpValuesTableQ61_3 390 | { 391 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 392 | }; 393 | #endif // (defined(TINYMIND_USE_EXP_61_3)) 394 | #if (defined(TINYMIND_USE_EXP_62_2)) 395 | struct ExpValuesTableQ62_2 396 | { 397 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 398 | }; 399 | #endif // (defined(TINYMIND_USE_EXP_62_2)) 400 | #if (defined(TINYMIND_USE_EXP_63_1)) 401 | struct ExpValuesTableQ63_1 402 | { 403 | static const uint64_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 404 | }; 405 | #endif // (defined(TINYMIND_USE_EXP_63_1)) 406 | } 407 | -------------------------------------------------------------------------------- /cpp/expValues8Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_EXP_1_7)) 29 | struct ExpValuesTableQ1_7 30 | { 31 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_EXP_1_7)) 34 | #if (defined(TINYMIND_USE_EXP_2_6)) 35 | struct ExpValuesTableQ2_6 36 | { 37 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_EXP_2_6)) 40 | #if (defined(TINYMIND_USE_EXP_3_5)) 41 | struct ExpValuesTableQ3_5 42 | { 43 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_EXP_3_5)) 46 | #if (defined(TINYMIND_USE_EXP_4_4)) 47 | struct ExpValuesTableQ4_4 48 | { 49 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_EXP_4_4)) 52 | #if (defined(TINYMIND_USE_EXP_5_3)) 53 | struct ExpValuesTableQ5_3 54 | { 55 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_EXP_5_3)) 58 | #if (defined(TINYMIND_USE_EXP_6_2)) 59 | struct ExpValuesTableQ6_2 60 | { 61 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_EXP_6_2)) 64 | #if (defined(TINYMIND_USE_EXP_7_1)) 65 | struct ExpValuesTableQ7_1 66 | { 67 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_EXP_7_1)) 70 | } 71 | -------------------------------------------------------------------------------- /cpp/fixedPointTransferFunctions.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | #include "activationFunctions.hpp" 27 | #include "sigmoid.hpp" 28 | #include "tanh.hpp" 29 | #include "interpolate.hpp" 30 | #include "error.hpp" 31 | #include "nninit.hpp" 32 | #include "zeroTolerance.hpp" 33 | #include "constants.hpp" 34 | 35 | namespace tinymind { 36 | template< 37 | typename ValueType, 38 | class TransferFunctionRandomNumberGeneratorPolicy, 39 | class TransferFunctionHiddenNeuronActivationPolicy, 40 | class TransferFunctionOutputNeuronActivationPolicy, 41 | unsigned NumberOfOutputNeurons = 1, 42 | class TransferFunctionNetworkInitializationPolicy = tinymind::DefaultNetworkInitializer, 43 | class TransferFunctionErrorCalculatorPolicy = tinymind::MeanSquaredErrorCalculator, 44 | class TransferFunctionZeroTolerancePolicy = tinymind::ZeroToleranceCalculator > 45 | struct FixedPointTransferFunctions 46 | { 47 | typedef ValueType TransferFunctionsValueType; 48 | typedef TransferFunctionRandomNumberGeneratorPolicy RandomNumberGeneratorPolicy; 49 | typedef TransferFunctionHiddenNeuronActivationPolicy HiddenNeuronActivationPolicy; 50 | typedef TransferFunctionOutputNeuronActivationPolicy OutputNeuronActivationPolicy; 51 | typedef TransferFunctionNetworkInitializationPolicy NetworkIntializationPolicy; 52 | typedef TransferFunctionErrorCalculatorPolicy ErrorCalulatorPolicy; 53 | typedef TransferFunctionZeroTolerancePolicy ZeroToleranceCalculatorPolicy; 54 | typedef typename ValueType::FullWidthFieldType FullWidthFieldType; 55 | 56 | static const unsigned NumberOfTransferFunctionsOutputNeurons = NumberOfOutputNeurons; 57 | 58 | static ValueType calculateError(ValueType const* const targetValues, ValueType const* const outputValues) 59 | { 60 | return ErrorCalulatorPolicy::calculateError(targetValues, outputValues); 61 | } 62 | 63 | static ValueType calculateOutputGradient(const ValueType& targetValue, const ValueType& outputValue) 64 | { 65 | const ValueType delta = targetValue - outputValue; 66 | 67 | return (delta * OutputNeuronActivationPolicy::activationFunctionDerivative(outputValue)); 68 | } 69 | 70 | static ValueType generateRandomWeight() 71 | { 72 | return RandomNumberGeneratorPolicy::generateRandomWeight(); 73 | } 74 | 75 | static ValueType hiddenNeuronActivationFunction(const ValueType& value) 76 | { 77 | return HiddenNeuronActivationPolicy::activationFunction(value); 78 | } 79 | 80 | static ValueType hiddenNeuronActivationFunctionDerivative(const ValueType& value) 81 | { 82 | return HiddenNeuronActivationPolicy::activationFunctionDerivative(value); 83 | } 84 | 85 | static ValueType initialAccelerationRate() 86 | { 87 | return NetworkIntializationPolicy::initialAccelerationRate(); 88 | } 89 | 90 | static ValueType initialBiasOutputValue() 91 | { 92 | return NetworkIntializationPolicy::initialBiasOutputValue(); 93 | } 94 | 95 | static ValueType initialDeltaWeight() 96 | { 97 | return NetworkIntializationPolicy::initialDeltaWeight(); 98 | } 99 | 100 | static ValueType initialGradientValue() 101 | { 102 | return NetworkIntializationPolicy::initialGradientValue(); 103 | } 104 | 105 | static ValueType initialLearningRate() 106 | { 107 | return NetworkIntializationPolicy::initialLearningRate(); 108 | } 109 | 110 | static ValueType initialMomentumRate() 111 | { 112 | return NetworkIntializationPolicy::initialMomentumRate(); 113 | } 114 | 115 | static ValueType initialOutputValue() 116 | { 117 | return NetworkIntializationPolicy::initialOutputValue(); 118 | } 119 | 120 | static bool isWithinZeroTolerance(const ValueType& value) 121 | { 122 | return ZeroToleranceCalculatorPolicy::isWithinZeroTolerance(value); 123 | } 124 | 125 | static ValueType negate(const ValueType& value) 126 | { 127 | const ValueType result = value * Constants::negativeOne(); 128 | 129 | return result; 130 | } 131 | 132 | static ValueType noOpDeltaWeight() 133 | { 134 | return NetworkIntializationPolicy::noOpDeltaWeight(); 135 | } 136 | 137 | static ValueType noOpWeight() 138 | { 139 | return NetworkIntializationPolicy::noOpWeight(); 140 | } 141 | 142 | static ValueType outputNeuronActivationFunction(const ValueType& value) 143 | { 144 | return OutputNeuronActivationPolicy::activationFunction(value); 145 | } 146 | 147 | static void outputNeuronActivationFunction(ValueType const* const values, ValueType* results, const size_t numberOfNerons) 148 | { 149 | OutputNeuronActivationPolicy::activationFunction(values, results, numberOfNerons); 150 | } 151 | 152 | static ValueType outputNeuronActivationFunctionDerivative(const ValueType& value) 153 | { 154 | return OutputNeuronActivationPolicy::activationFunctionDerivative(value); 155 | } 156 | 157 | static void outputNeuronActivationFunctionDerivative(ValueType const* const values, ValueType const* const targetValues, ValueType* results, const size_t numberOfNerons) 158 | { 159 | OutputNeuronActivationPolicy::activationFunctionDerivative(values, targetValues, results, numberOfNerons); 160 | } 161 | }; 162 | } 163 | -------------------------------------------------------------------------------- /cpp/interpolate.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | namespace tinymind { 26 | template 27 | T linearInterpolation(const T& x, const T& x0, const T& x1, const T& y0, const T& y1) 28 | { 29 | if(x1 == x0) 30 | { 31 | return y0; 32 | } 33 | 34 | const T result = y0 + ((y1 - y0) * ((x - x0) / (x1 - x0))); 35 | 36 | return result; 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /cpp/limits.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | namespace tinymind { 26 | template 27 | struct limits 28 | { 29 | }; 30 | 31 | template<> 32 | struct limits 33 | { 34 | static constexpr int8_t max = 127; 35 | }; 36 | 37 | template<> 38 | struct limits 39 | { 40 | static constexpr uint8_t max = 255; 41 | }; 42 | 43 | template<> 44 | struct limits 45 | { 46 | static constexpr int16_t max = 32767; 47 | }; 48 | 49 | template<> 50 | struct limits 51 | { 52 | static constexpr uint16_t max = 65535; 53 | }; 54 | 55 | template<> 56 | struct limits 57 | { 58 | static constexpr int32_t max = 2147483647; 59 | }; 60 | 61 | template<> 62 | struct limits 63 | { 64 | static constexpr uint32_t max = 4294967295; 65 | }; 66 | 67 | template<> 68 | struct limits 69 | { 70 | static constexpr int64_t max = 9223372036854775807ULL; 71 | }; 72 | 73 | template<> 74 | struct limits 75 | { 76 | static constexpr uint64_t max = 18446744073709551615ULL; 77 | }; 78 | } -------------------------------------------------------------------------------- /cpp/logValues16Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_LOG_1_15)) 29 | struct LogValuesTableQ1_15 30 | { 31 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_LOG_1_15)) 34 | #if (defined(TINYMIND_USE_LOG_2_14)) 35 | struct LogValuesTableQ2_14 36 | { 37 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_LOG_2_14)) 40 | #if (defined(TINYMIND_USE_LOG_3_13)) 41 | struct LogValuesTableQ3_13 42 | { 43 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_LOG_3_13)) 46 | #if (defined(TINYMIND_USE_LOG_4_12)) 47 | struct LogValuesTableQ4_12 48 | { 49 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_LOG_4_12)) 52 | #if (defined(TINYMIND_USE_LOG_5_11)) 53 | struct LogValuesTableQ5_11 54 | { 55 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_LOG_5_11)) 58 | #if (defined(TINYMIND_USE_LOG_6_10)) 59 | struct LogValuesTableQ6_10 60 | { 61 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_LOG_6_10)) 64 | #if (defined(TINYMIND_USE_LOG_7_9)) 65 | struct LogValuesTableQ7_9 66 | { 67 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_LOG_7_9)) 70 | #if (defined(TINYMIND_USE_LOG_8_8)) 71 | struct LogValuesTableQ8_8 72 | { 73 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_LOG_8_8)) 76 | #if (defined(TINYMIND_USE_LOG_9_7)) 77 | struct LogValuesTableQ9_7 78 | { 79 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_LOG_9_7)) 82 | #if (defined(TINYMIND_USE_LOG_10_6)) 83 | struct LogValuesTableQ10_6 84 | { 85 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_LOG_10_6)) 88 | #if (defined(TINYMIND_USE_LOG_11_5)) 89 | struct LogValuesTableQ11_5 90 | { 91 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_LOG_11_5)) 94 | #if (defined(TINYMIND_USE_LOG_12_4)) 95 | struct LogValuesTableQ12_4 96 | { 97 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_LOG_12_4)) 100 | #if (defined(TINYMIND_USE_LOG_13_3)) 101 | struct LogValuesTableQ13_3 102 | { 103 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_LOG_13_3)) 106 | #if (defined(TINYMIND_USE_LOG_14_2)) 107 | struct LogValuesTableQ14_2 108 | { 109 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_LOG_14_2)) 112 | #if (defined(TINYMIND_USE_LOG_15_1)) 113 | struct LogValuesTableQ15_1 114 | { 115 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_LOG_15_1)) 118 | } 119 | -------------------------------------------------------------------------------- /cpp/logValues32Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_LOG_1_31)) 29 | struct LogValuesTableQ1_31 30 | { 31 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_LOG_1_31)) 34 | #if (defined(TINYMIND_USE_LOG_2_30)) 35 | struct LogValuesTableQ2_30 36 | { 37 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_LOG_2_30)) 40 | #if (defined(TINYMIND_USE_LOG_3_29)) 41 | struct LogValuesTableQ3_29 42 | { 43 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_LOG_3_29)) 46 | #if (defined(TINYMIND_USE_LOG_4_28)) 47 | struct LogValuesTableQ4_28 48 | { 49 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_LOG_4_28)) 52 | #if (defined(TINYMIND_USE_LOG_5_27)) 53 | struct LogValuesTableQ5_27 54 | { 55 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_LOG_5_27)) 58 | #if (defined(TINYMIND_USE_LOG_6_26)) 59 | struct LogValuesTableQ6_26 60 | { 61 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_LOG_6_26)) 64 | #if (defined(TINYMIND_USE_LOG_7_25)) 65 | struct LogValuesTableQ7_25 66 | { 67 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_LOG_7_25)) 70 | #if (defined(TINYMIND_USE_LOG_8_24)) 71 | struct LogValuesTableQ8_24 72 | { 73 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_LOG_8_24)) 76 | #if (defined(TINYMIND_USE_LOG_9_23)) 77 | struct LogValuesTableQ9_23 78 | { 79 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_LOG_9_23)) 82 | #if (defined(TINYMIND_USE_LOG_10_22)) 83 | struct LogValuesTableQ10_22 84 | { 85 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_LOG_10_22)) 88 | #if (defined(TINYMIND_USE_LOG_11_21)) 89 | struct LogValuesTableQ11_21 90 | { 91 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_LOG_11_21)) 94 | #if (defined(TINYMIND_USE_LOG_12_20)) 95 | struct LogValuesTableQ12_20 96 | { 97 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_LOG_12_20)) 100 | #if (defined(TINYMIND_USE_LOG_13_19)) 101 | struct LogValuesTableQ13_19 102 | { 103 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_LOG_13_19)) 106 | #if (defined(TINYMIND_USE_LOG_14_18)) 107 | struct LogValuesTableQ14_18 108 | { 109 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_LOG_14_18)) 112 | #if (defined(TINYMIND_USE_LOG_15_17)) 113 | struct LogValuesTableQ15_17 114 | { 115 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_LOG_15_17)) 118 | #if (defined(TINYMIND_USE_LOG_16_16)) 119 | struct LogValuesTableQ16_16 120 | { 121 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 122 | }; 123 | #endif // (defined(TINYMIND_USE_LOG_16_16)) 124 | #if (defined(TINYMIND_USE_LOG_17_15)) 125 | struct LogValuesTableQ17_15 126 | { 127 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 128 | }; 129 | #endif // (defined(TINYMIND_USE_LOG_17_15)) 130 | #if (defined(TINYMIND_USE_LOG_18_14)) 131 | struct LogValuesTableQ18_14 132 | { 133 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 134 | }; 135 | #endif // (defined(TINYMIND_USE_LOG_18_14)) 136 | #if (defined(TINYMIND_USE_LOG_19_13)) 137 | struct LogValuesTableQ19_13 138 | { 139 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 140 | }; 141 | #endif // (defined(TINYMIND_USE_LOG_19_13)) 142 | #if (defined(TINYMIND_USE_LOG_20_12)) 143 | struct LogValuesTableQ20_12 144 | { 145 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 146 | }; 147 | #endif // (defined(TINYMIND_USE_LOG_20_12)) 148 | #if (defined(TINYMIND_USE_LOG_21_11)) 149 | struct LogValuesTableQ21_11 150 | { 151 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 152 | }; 153 | #endif // (defined(TINYMIND_USE_LOG_21_11)) 154 | #if (defined(TINYMIND_USE_LOG_22_10)) 155 | struct LogValuesTableQ22_10 156 | { 157 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 158 | }; 159 | #endif // (defined(TINYMIND_USE_LOG_22_10)) 160 | #if (defined(TINYMIND_USE_LOG_23_9)) 161 | struct LogValuesTableQ23_9 162 | { 163 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 164 | }; 165 | #endif // (defined(TINYMIND_USE_LOG_23_9)) 166 | #if (defined(TINYMIND_USE_LOG_24_8)) 167 | struct LogValuesTableQ24_8 168 | { 169 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 170 | }; 171 | #endif // (defined(TINYMIND_USE_LOG_24_8)) 172 | #if (defined(TINYMIND_USE_LOG_25_7)) 173 | struct LogValuesTableQ25_7 174 | { 175 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 176 | }; 177 | #endif // (defined(TINYMIND_USE_LOG_25_7)) 178 | #if (defined(TINYMIND_USE_LOG_26_6)) 179 | struct LogValuesTableQ26_6 180 | { 181 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 182 | }; 183 | #endif // (defined(TINYMIND_USE_LOG_26_6)) 184 | #if (defined(TINYMIND_USE_LOG_27_5)) 185 | struct LogValuesTableQ27_5 186 | { 187 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 188 | }; 189 | #endif // (defined(TINYMIND_USE_LOG_27_5)) 190 | #if (defined(TINYMIND_USE_LOG_28_4)) 191 | struct LogValuesTableQ28_4 192 | { 193 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 194 | }; 195 | #endif // (defined(TINYMIND_USE_LOG_28_4)) 196 | #if (defined(TINYMIND_USE_LOG_29_3)) 197 | struct LogValuesTableQ29_3 198 | { 199 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 200 | }; 201 | #endif // (defined(TINYMIND_USE_LOG_29_3)) 202 | #if (defined(TINYMIND_USE_LOG_30_2)) 203 | struct LogValuesTableQ30_2 204 | { 205 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 206 | }; 207 | #endif // (defined(TINYMIND_USE_LOG_30_2)) 208 | #if (defined(TINYMIND_USE_LOG_31_1)) 209 | struct LogValuesTableQ31_1 210 | { 211 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 212 | }; 213 | #endif // (defined(TINYMIND_USE_LOG_31_1)) 214 | } 215 | -------------------------------------------------------------------------------- /cpp/logValues8Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_LOG_1_7)) 29 | struct LogValuesTableQ1_7 30 | { 31 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_LOG_1_7)) 34 | #if (defined(TINYMIND_USE_LOG_2_6)) 35 | struct LogValuesTableQ2_6 36 | { 37 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_LOG_2_6)) 40 | #if (defined(TINYMIND_USE_LOG_3_5)) 41 | struct LogValuesTableQ3_5 42 | { 43 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_LOG_3_5)) 46 | #if (defined(TINYMIND_USE_LOG_4_4)) 47 | struct LogValuesTableQ4_4 48 | { 49 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_LOG_4_4)) 52 | #if (defined(TINYMIND_USE_LOG_5_3)) 53 | struct LogValuesTableQ5_3 54 | { 55 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_LOG_5_3)) 58 | #if (defined(TINYMIND_USE_LOG_6_2)) 59 | struct LogValuesTableQ6_2 60 | { 61 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_LOG_6_2)) 64 | #if (defined(TINYMIND_USE_LOG_7_1)) 65 | struct LogValuesTableQ7_1 66 | { 67 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_LOG_7_1)) 70 | } 71 | -------------------------------------------------------------------------------- /cpp/lookupTable.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | #include "interpolate.hpp" 27 | #include 28 | 29 | namespace tinymind { 30 | template 31 | struct LookupTable 32 | { 33 | typedef typename ValueType::FullWidthFieldType FullWidthFieldType; 34 | typedef typename ValueType::FullWidthValueType FullWidthValueType; 35 | 36 | static ValueType getValue(const ValueType& value, typename ValueType::FullWidthFieldType const* const tableValues, const ptrdiff_t maxActivationIndex) 37 | { 38 | static constexpr FullWidthValueType MIN_X_ACTIVATION_VALUE = static_cast(MIN_X_TABLE_VALUE * (1 << ValueType::NumberOfFractionalBits)); 39 | static constexpr FullWidthValueType MAX_X_ACTIVATION_VALUE = static_cast(MAX_X_TABLE_VALUE * (1 << ValueType::NumberOfFractionalBits)); 40 | static constexpr FullWidthValueType ACTIVATION_DELTA_X = static_cast((1 << (ValueType::NumberOfFractionalBits - ACTIVATION_DELTA_SHIFT))); 41 | 42 | ValueType result; 43 | 44 | if (value.getValue() <= MIN_X_ACTIVATION_VALUE) 45 | { 46 | result = tableValues[0]; 47 | } 48 | else if (value.getValue() >= MAX_X_ACTIVATION_VALUE) 49 | { 50 | result = tableValues[maxActivationIndex]; 51 | } 52 | else 53 | { 54 | const ptrdiff_t lowerIndex = static_cast((value.getValue() - static_cast(MIN_X_ACTIVATION_VALUE)) / static_cast(ACTIVATION_DELTA_X)); 55 | if (lowerIndex >= maxActivationIndex) 56 | { 57 | result = tableValues[maxActivationIndex]; 58 | } 59 | else 60 | { 61 | const ptrdiff_t upperIndex = lowerIndex + 1; 62 | const FullWidthValueType lowerX = static_cast(MIN_X_ACTIVATION_VALUE + (lowerIndex * ACTIVATION_DELTA_X)); 63 | const FullWidthValueType upperX = static_cast(MIN_X_ACTIVATION_VALUE + (upperIndex * ACTIVATION_DELTA_X)); 64 | 65 | if (tableValues[lowerIndex] == tableValues[upperIndex]) 66 | { 67 | result = tableValues[upperIndex]; 68 | } 69 | else if (tableValues[lowerIndex] == static_cast(value.getValue())) 70 | { 71 | result = tableValues[lowerIndex]; 72 | } 73 | else if (tableValues[upperIndex] == static_cast(value.getValue())) 74 | { 75 | result = tableValues[upperIndex]; 76 | } 77 | else 78 | { 79 | result = linearInterpolation( 80 | value, 81 | static_cast(lowerX), 82 | static_cast(upperX), 83 | static_cast(tableValues[lowerIndex]), 84 | static_cast(tableValues[upperIndex])); 85 | } 86 | } 87 | } 88 | 89 | return result; 90 | } 91 | }; 92 | } 93 | -------------------------------------------------------------------------------- /cpp/nninit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "constants.hpp" 26 | 27 | namespace tinymind { 28 | template 29 | struct DefaultNetworkInitializer 30 | { 31 | static ValueType initialAccelerationRate() 32 | { 33 | static const ValueType rate(0, (1 << (ValueType::NumberOfFractionalBits - 4))); 34 | 35 | return rate; 36 | } 37 | 38 | static ValueType initialBiasOutputValue() 39 | { 40 | return Constants::one(); 41 | } 42 | 43 | static ValueType initialDeltaWeight() 44 | { 45 | return Constants::zero(); 46 | } 47 | 48 | static ValueType initialGradientValue() 49 | { 50 | return Constants::zero(); 51 | } 52 | 53 | static ValueType initialLearningRate() 54 | { 55 | static const ValueType rate(0, (1 << (ValueType::NumberOfFractionalBits - 2))); 56 | 57 | return rate; 58 | } 59 | 60 | static ValueType initialMomentumRate() 61 | { 62 | static const ValueType rate(0, (1 << (ValueType::NumberOfFractionalBits - 1))); 63 | 64 | return rate; 65 | } 66 | 67 | static ValueType initialOutputValue() 68 | { 69 | return Constants::zero(); 70 | } 71 | 72 | static ValueType noOpDeltaWeight() 73 | { 74 | return Constants::one(); 75 | } 76 | 77 | static ValueType noOpWeight() 78 | { 79 | return Constants::one(); 80 | } 81 | }; 82 | } 83 | -------------------------------------------------------------------------------- /cpp/random.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | namespace tinymind { 26 | template 27 | struct NullRandomNumberPolicy 28 | { 29 | static ValueType generateRandomWeight() 30 | { 31 | return 0; 32 | } 33 | 34 | size_t getRandomActionDecisionPoint() const 35 | { 36 | return 0; 37 | } 38 | 39 | static ValueType randInt(ValueType const& min, ValueType const& max) 40 | { 41 | return 0; 42 | } 43 | 44 | void setRandomActionDecisionPoint(const size_t randomActionDecisionPoint) 45 | { 46 | } 47 | }; 48 | } 49 | -------------------------------------------------------------------------------- /cpp/range.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activationFunctions.hpp" 26 | #include "constants.hpp" 27 | 28 | namespace tinymind { 29 | template 30 | struct RangeConverter 31 | { 32 | static void convertToFractionOfRange(ValueType& value, const ValueType& minValue, const ValueType& maxValue) 33 | { 34 | ValueType range; 35 | 36 | if(maxValue > minValue) 37 | { 38 | range = (maxValue - minValue); 39 | value -= minValue; 40 | value /= range; 41 | } 42 | } 43 | }; 44 | 45 | template 46 | struct ActivationOutputRangeConverter 47 | { 48 | static void convertToOutputValue(OutputValueType& value, const OutputValueType& minValue, const OutputValueType& maxValue) 49 | { 50 | } 51 | }; 52 | 53 | template 54 | struct ActivationOutputRangeConverter > 55 | { 56 | // The tanh activation function outputs values from -1..1 so we must scale the output such that -1 == min value, 1 == max value. 57 | static void convertToOutputValue(OutputValueType& value, const OutputValueType& minValue, const OutputValueType& maxValue) 58 | { 59 | const OutputValueType range = (maxValue - minValue); 60 | 61 | value += Constants::one(); 62 | value *= range; 63 | value /= 2; 64 | value += minValue; 65 | } 66 | }; 67 | 68 | template 69 | struct ActivationOutputRangeConverter > 70 | { 71 | static void convertToOutputValue(OutputValueType& value, const OutputValueType& minValue, const OutputValueType& maxValue) 72 | { 73 | const OutputValueType range = (maxValue - minValue); 74 | 75 | value *= range; 76 | value += minValue; 77 | } 78 | }; 79 | } -------------------------------------------------------------------------------- /cpp/sigmoidValues16Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_SIGMOID_1_15)) 29 | struct SigmoidValuesTableQ1_15 30 | { 31 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_SIGMOID_1_15)) 34 | #if (defined(TINYMIND_USE_SIGMOID_2_14)) 35 | struct SigmoidValuesTableQ2_14 36 | { 37 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_SIGMOID_2_14)) 40 | #if (defined(TINYMIND_USE_SIGMOID_3_13)) 41 | struct SigmoidValuesTableQ3_13 42 | { 43 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_SIGMOID_3_13)) 46 | #if (defined(TINYMIND_USE_SIGMOID_4_12)) 47 | struct SigmoidValuesTableQ4_12 48 | { 49 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_SIGMOID_4_12)) 52 | #if (defined(TINYMIND_USE_SIGMOID_5_11)) 53 | struct SigmoidValuesTableQ5_11 54 | { 55 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_SIGMOID_5_11)) 58 | #if (defined(TINYMIND_USE_SIGMOID_6_10)) 59 | struct SigmoidValuesTableQ6_10 60 | { 61 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_SIGMOID_6_10)) 64 | #if (defined(TINYMIND_USE_SIGMOID_7_9)) 65 | struct SigmoidValuesTableQ7_9 66 | { 67 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_SIGMOID_7_9)) 70 | #if (defined(TINYMIND_USE_SIGMOID_8_8)) 71 | struct SigmoidValuesTableQ8_8 72 | { 73 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_SIGMOID_8_8)) 76 | #if (defined(TINYMIND_USE_SIGMOID_9_7)) 77 | struct SigmoidValuesTableQ9_7 78 | { 79 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_SIGMOID_9_7)) 82 | #if (defined(TINYMIND_USE_SIGMOID_10_6)) 83 | struct SigmoidValuesTableQ10_6 84 | { 85 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_SIGMOID_10_6)) 88 | #if (defined(TINYMIND_USE_SIGMOID_11_5)) 89 | struct SigmoidValuesTableQ11_5 90 | { 91 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_SIGMOID_11_5)) 94 | #if (defined(TINYMIND_USE_SIGMOID_12_4)) 95 | struct SigmoidValuesTableQ12_4 96 | { 97 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_SIGMOID_12_4)) 100 | #if (defined(TINYMIND_USE_SIGMOID_13_3)) 101 | struct SigmoidValuesTableQ13_3 102 | { 103 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_SIGMOID_13_3)) 106 | #if (defined(TINYMIND_USE_SIGMOID_14_2)) 107 | struct SigmoidValuesTableQ14_2 108 | { 109 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_SIGMOID_14_2)) 112 | #if (defined(TINYMIND_USE_SIGMOID_15_1)) 113 | struct SigmoidValuesTableQ15_1 114 | { 115 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_SIGMOID_15_1)) 118 | } 119 | -------------------------------------------------------------------------------- /cpp/sigmoidValues32Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_SIGMOID_1_31)) 29 | struct SigmoidValuesTableQ1_31 30 | { 31 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_SIGMOID_1_31)) 34 | #if (defined(TINYMIND_USE_SIGMOID_2_30)) 35 | struct SigmoidValuesTableQ2_30 36 | { 37 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_SIGMOID_2_30)) 40 | #if (defined(TINYMIND_USE_SIGMOID_3_29)) 41 | struct SigmoidValuesTableQ3_29 42 | { 43 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_SIGMOID_3_29)) 46 | #if (defined(TINYMIND_USE_SIGMOID_4_28)) 47 | struct SigmoidValuesTableQ4_28 48 | { 49 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_SIGMOID_4_28)) 52 | #if (defined(TINYMIND_USE_SIGMOID_5_27)) 53 | struct SigmoidValuesTableQ5_27 54 | { 55 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_SIGMOID_5_27)) 58 | #if (defined(TINYMIND_USE_SIGMOID_6_26)) 59 | struct SigmoidValuesTableQ6_26 60 | { 61 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_SIGMOID_6_26)) 64 | #if (defined(TINYMIND_USE_SIGMOID_7_25)) 65 | struct SigmoidValuesTableQ7_25 66 | { 67 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_SIGMOID_7_25)) 70 | #if (defined(TINYMIND_USE_SIGMOID_8_24)) 71 | struct SigmoidValuesTableQ8_24 72 | { 73 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_SIGMOID_8_24)) 76 | #if (defined(TINYMIND_USE_SIGMOID_9_23)) 77 | struct SigmoidValuesTableQ9_23 78 | { 79 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_SIGMOID_9_23)) 82 | #if (defined(TINYMIND_USE_SIGMOID_10_22)) 83 | struct SigmoidValuesTableQ10_22 84 | { 85 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_SIGMOID_10_22)) 88 | #if (defined(TINYMIND_USE_SIGMOID_11_21)) 89 | struct SigmoidValuesTableQ11_21 90 | { 91 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_SIGMOID_11_21)) 94 | #if (defined(TINYMIND_USE_SIGMOID_12_20)) 95 | struct SigmoidValuesTableQ12_20 96 | { 97 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_SIGMOID_12_20)) 100 | #if (defined(TINYMIND_USE_SIGMOID_13_19)) 101 | struct SigmoidValuesTableQ13_19 102 | { 103 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_SIGMOID_13_19)) 106 | #if (defined(TINYMIND_USE_SIGMOID_14_18)) 107 | struct SigmoidValuesTableQ14_18 108 | { 109 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_SIGMOID_14_18)) 112 | #if (defined(TINYMIND_USE_SIGMOID_15_17)) 113 | struct SigmoidValuesTableQ15_17 114 | { 115 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_SIGMOID_15_17)) 118 | #if (defined(TINYMIND_USE_SIGMOID_16_16)) 119 | struct SigmoidValuesTableQ16_16 120 | { 121 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 122 | }; 123 | #endif // (defined(TINYMIND_USE_SIGMOID_16_16)) 124 | #if (defined(TINYMIND_USE_SIGMOID_17_15)) 125 | struct SigmoidValuesTableQ17_15 126 | { 127 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 128 | }; 129 | #endif // (defined(TINYMIND_USE_SIGMOID_17_15)) 130 | #if (defined(TINYMIND_USE_SIGMOID_18_14)) 131 | struct SigmoidValuesTableQ18_14 132 | { 133 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 134 | }; 135 | #endif // (defined(TINYMIND_USE_SIGMOID_18_14)) 136 | #if (defined(TINYMIND_USE_SIGMOID_19_13)) 137 | struct SigmoidValuesTableQ19_13 138 | { 139 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 140 | }; 141 | #endif // (defined(TINYMIND_USE_SIGMOID_19_13)) 142 | #if (defined(TINYMIND_USE_SIGMOID_20_12)) 143 | struct SigmoidValuesTableQ20_12 144 | { 145 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 146 | }; 147 | #endif // (defined(TINYMIND_USE_SIGMOID_20_12)) 148 | #if (defined(TINYMIND_USE_SIGMOID_21_11)) 149 | struct SigmoidValuesTableQ21_11 150 | { 151 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 152 | }; 153 | #endif // (defined(TINYMIND_USE_SIGMOID_21_11)) 154 | #if (defined(TINYMIND_USE_SIGMOID_22_10)) 155 | struct SigmoidValuesTableQ22_10 156 | { 157 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 158 | }; 159 | #endif // (defined(TINYMIND_USE_SIGMOID_22_10)) 160 | #if (defined(TINYMIND_USE_SIGMOID_23_9)) 161 | struct SigmoidValuesTableQ23_9 162 | { 163 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 164 | }; 165 | #endif // (defined(TINYMIND_USE_SIGMOID_23_9)) 166 | #if (defined(TINYMIND_USE_SIGMOID_24_8)) 167 | struct SigmoidValuesTableQ24_8 168 | { 169 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 170 | }; 171 | #endif // (defined(TINYMIND_USE_SIGMOID_24_8)) 172 | #if (defined(TINYMIND_USE_SIGMOID_25_7)) 173 | struct SigmoidValuesTableQ25_7 174 | { 175 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 176 | }; 177 | #endif // (defined(TINYMIND_USE_SIGMOID_25_7)) 178 | #if (defined(TINYMIND_USE_SIGMOID_26_6)) 179 | struct SigmoidValuesTableQ26_6 180 | { 181 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 182 | }; 183 | #endif // (defined(TINYMIND_USE_SIGMOID_26_6)) 184 | #if (defined(TINYMIND_USE_SIGMOID_27_5)) 185 | struct SigmoidValuesTableQ27_5 186 | { 187 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 188 | }; 189 | #endif // (defined(TINYMIND_USE_SIGMOID_27_5)) 190 | #if (defined(TINYMIND_USE_SIGMOID_28_4)) 191 | struct SigmoidValuesTableQ28_4 192 | { 193 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 194 | }; 195 | #endif // (defined(TINYMIND_USE_SIGMOID_28_4)) 196 | #if (defined(TINYMIND_USE_SIGMOID_29_3)) 197 | struct SigmoidValuesTableQ29_3 198 | { 199 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 200 | }; 201 | #endif // (defined(TINYMIND_USE_SIGMOID_29_3)) 202 | #if (defined(TINYMIND_USE_SIGMOID_30_2)) 203 | struct SigmoidValuesTableQ30_2 204 | { 205 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 206 | }; 207 | #endif // (defined(TINYMIND_USE_SIGMOID_30_2)) 208 | #if (defined(TINYMIND_USE_SIGMOID_31_1)) 209 | struct SigmoidValuesTableQ31_1 210 | { 211 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 212 | }; 213 | #endif // (defined(TINYMIND_USE_SIGMOID_31_1)) 214 | } 215 | -------------------------------------------------------------------------------- /cpp/sigmoidValues8Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_SIGMOID_1_7)) 29 | struct SigmoidValuesTableQ1_7 30 | { 31 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_SIGMOID_1_7)) 34 | #if (defined(TINYMIND_USE_SIGMOID_2_6)) 35 | struct SigmoidValuesTableQ2_6 36 | { 37 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_SIGMOID_2_6)) 40 | #if (defined(TINYMIND_USE_SIGMOID_3_5)) 41 | struct SigmoidValuesTableQ3_5 42 | { 43 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_SIGMOID_3_5)) 46 | #if (defined(TINYMIND_USE_SIGMOID_4_4)) 47 | struct SigmoidValuesTableQ4_4 48 | { 49 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_SIGMOID_4_4)) 52 | #if (defined(TINYMIND_USE_SIGMOID_5_3)) 53 | struct SigmoidValuesTableQ5_3 54 | { 55 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_SIGMOID_5_3)) 58 | #if (defined(TINYMIND_USE_SIGMOID_6_2)) 59 | struct SigmoidValuesTableQ6_2 60 | { 61 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_SIGMOID_6_2)) 64 | #if (defined(TINYMIND_USE_SIGMOID_7_1)) 65 | struct SigmoidValuesTableQ7_1 66 | { 67 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_SIGMOID_7_1)) 70 | } 71 | -------------------------------------------------------------------------------- /cpp/signed.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | namespace tinymind { 26 | template 27 | struct IsSigned 28 | {}; 29 | 30 | template<> 31 | struct IsSigned 32 | { 33 | static constexpr bool result = true; 34 | }; 35 | 36 | template<> 37 | struct IsSigned 38 | { 39 | static constexpr bool result = true; 40 | }; 41 | 42 | template<> 43 | struct IsSigned 44 | { 45 | static constexpr bool result = true; 46 | }; 47 | 48 | template<> 49 | struct IsSigned 50 | { 51 | static constexpr bool result = true; 52 | }; 53 | 54 | template<> 55 | struct IsSigned 56 | { 57 | static constexpr bool result = false; 58 | }; 59 | 60 | template<> 61 | struct IsSigned 62 | { 63 | static constexpr bool result = false; 64 | }; 65 | 66 | template<> 67 | struct IsSigned 68 | { 69 | static constexpr bool result = false; 70 | }; 71 | 72 | template<> 73 | struct IsSigned 74 | { 75 | static constexpr bool result = false; 76 | }; 77 | } 78 | -------------------------------------------------------------------------------- /cpp/tanhValues16Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_TANH_1_15)) 29 | struct TanhValuesTableQ1_15 30 | { 31 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_TANH_1_15)) 34 | #if (defined(TINYMIND_USE_TANH_2_14)) 35 | struct TanhValuesTableQ2_14 36 | { 37 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_TANH_2_14)) 40 | #if (defined(TINYMIND_USE_TANH_3_13)) 41 | struct TanhValuesTableQ3_13 42 | { 43 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_TANH_3_13)) 46 | #if (defined(TINYMIND_USE_TANH_4_12)) 47 | struct TanhValuesTableQ4_12 48 | { 49 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_TANH_4_12)) 52 | #if (defined(TINYMIND_USE_TANH_5_11)) 53 | struct TanhValuesTableQ5_11 54 | { 55 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_TANH_5_11)) 58 | #if (defined(TINYMIND_USE_TANH_6_10)) 59 | struct TanhValuesTableQ6_10 60 | { 61 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_TANH_6_10)) 64 | #if (defined(TINYMIND_USE_TANH_7_9)) 65 | struct TanhValuesTableQ7_9 66 | { 67 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_TANH_7_9)) 70 | #if (defined(TINYMIND_USE_TANH_8_8)) 71 | struct TanhValuesTableQ8_8 72 | { 73 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_TANH_8_8)) 76 | #if (defined(TINYMIND_USE_TANH_9_7)) 77 | struct TanhValuesTableQ9_7 78 | { 79 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_TANH_9_7)) 82 | #if (defined(TINYMIND_USE_TANH_10_6)) 83 | struct TanhValuesTableQ10_6 84 | { 85 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_TANH_10_6)) 88 | #if (defined(TINYMIND_USE_TANH_11_5)) 89 | struct TanhValuesTableQ11_5 90 | { 91 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_TANH_11_5)) 94 | #if (defined(TINYMIND_USE_TANH_12_4)) 95 | struct TanhValuesTableQ12_4 96 | { 97 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_TANH_12_4)) 100 | #if (defined(TINYMIND_USE_TANH_13_3)) 101 | struct TanhValuesTableQ13_3 102 | { 103 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_TANH_13_3)) 106 | #if (defined(TINYMIND_USE_TANH_14_2)) 107 | struct TanhValuesTableQ14_2 108 | { 109 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_TANH_14_2)) 112 | #if (defined(TINYMIND_USE_TANH_15_1)) 113 | struct TanhValuesTableQ15_1 114 | { 115 | static const uint16_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_TANH_15_1)) 118 | } 119 | -------------------------------------------------------------------------------- /cpp/tanhValues32Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_TANH_1_31)) 29 | struct TanhValuesTableQ1_31 30 | { 31 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_TANH_1_31)) 34 | #if (defined(TINYMIND_USE_TANH_2_30)) 35 | struct TanhValuesTableQ2_30 36 | { 37 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_TANH_2_30)) 40 | #if (defined(TINYMIND_USE_TANH_3_29)) 41 | struct TanhValuesTableQ3_29 42 | { 43 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_TANH_3_29)) 46 | #if (defined(TINYMIND_USE_TANH_4_28)) 47 | struct TanhValuesTableQ4_28 48 | { 49 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_TANH_4_28)) 52 | #if (defined(TINYMIND_USE_TANH_5_27)) 53 | struct TanhValuesTableQ5_27 54 | { 55 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_TANH_5_27)) 58 | #if (defined(TINYMIND_USE_TANH_6_26)) 59 | struct TanhValuesTableQ6_26 60 | { 61 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_TANH_6_26)) 64 | #if (defined(TINYMIND_USE_TANH_7_25)) 65 | struct TanhValuesTableQ7_25 66 | { 67 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_TANH_7_25)) 70 | #if (defined(TINYMIND_USE_TANH_8_24)) 71 | struct TanhValuesTableQ8_24 72 | { 73 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 74 | }; 75 | #endif // (defined(TINYMIND_USE_TANH_8_24)) 76 | #if (defined(TINYMIND_USE_TANH_9_23)) 77 | struct TanhValuesTableQ9_23 78 | { 79 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 80 | }; 81 | #endif // (defined(TINYMIND_USE_TANH_9_23)) 82 | #if (defined(TINYMIND_USE_TANH_10_22)) 83 | struct TanhValuesTableQ10_22 84 | { 85 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 86 | }; 87 | #endif // (defined(TINYMIND_USE_TANH_10_22)) 88 | #if (defined(TINYMIND_USE_TANH_11_21)) 89 | struct TanhValuesTableQ11_21 90 | { 91 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 92 | }; 93 | #endif // (defined(TINYMIND_USE_TANH_11_21)) 94 | #if (defined(TINYMIND_USE_TANH_12_20)) 95 | struct TanhValuesTableQ12_20 96 | { 97 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 98 | }; 99 | #endif // (defined(TINYMIND_USE_TANH_12_20)) 100 | #if (defined(TINYMIND_USE_TANH_13_19)) 101 | struct TanhValuesTableQ13_19 102 | { 103 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 104 | }; 105 | #endif // (defined(TINYMIND_USE_TANH_13_19)) 106 | #if (defined(TINYMIND_USE_TANH_14_18)) 107 | struct TanhValuesTableQ14_18 108 | { 109 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 110 | }; 111 | #endif // (defined(TINYMIND_USE_TANH_14_18)) 112 | #if (defined(TINYMIND_USE_TANH_15_17)) 113 | struct TanhValuesTableQ15_17 114 | { 115 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 116 | }; 117 | #endif // (defined(TINYMIND_USE_TANH_15_17)) 118 | #if (defined(TINYMIND_USE_TANH_16_16)) 119 | struct TanhValuesTableQ16_16 120 | { 121 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 122 | }; 123 | #endif // (defined(TINYMIND_USE_TANH_16_16)) 124 | #if (defined(TINYMIND_USE_TANH_17_15)) 125 | struct TanhValuesTableQ17_15 126 | { 127 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 128 | }; 129 | #endif // (defined(TINYMIND_USE_TANH_17_15)) 130 | #if (defined(TINYMIND_USE_TANH_18_14)) 131 | struct TanhValuesTableQ18_14 132 | { 133 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 134 | }; 135 | #endif // (defined(TINYMIND_USE_TANH_18_14)) 136 | #if (defined(TINYMIND_USE_TANH_19_13)) 137 | struct TanhValuesTableQ19_13 138 | { 139 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 140 | }; 141 | #endif // (defined(TINYMIND_USE_TANH_19_13)) 142 | #if (defined(TINYMIND_USE_TANH_20_12)) 143 | struct TanhValuesTableQ20_12 144 | { 145 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 146 | }; 147 | #endif // (defined(TINYMIND_USE_TANH_20_12)) 148 | #if (defined(TINYMIND_USE_TANH_21_11)) 149 | struct TanhValuesTableQ21_11 150 | { 151 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 152 | }; 153 | #endif // (defined(TINYMIND_USE_TANH_21_11)) 154 | #if (defined(TINYMIND_USE_TANH_22_10)) 155 | struct TanhValuesTableQ22_10 156 | { 157 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 158 | }; 159 | #endif // (defined(TINYMIND_USE_TANH_22_10)) 160 | #if (defined(TINYMIND_USE_TANH_23_9)) 161 | struct TanhValuesTableQ23_9 162 | { 163 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 164 | }; 165 | #endif // (defined(TINYMIND_USE_TANH_23_9)) 166 | #if (defined(TINYMIND_USE_TANH_24_8)) 167 | struct TanhValuesTableQ24_8 168 | { 169 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 170 | }; 171 | #endif // (defined(TINYMIND_USE_TANH_24_8)) 172 | #if (defined(TINYMIND_USE_TANH_25_7)) 173 | struct TanhValuesTableQ25_7 174 | { 175 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 176 | }; 177 | #endif // (defined(TINYMIND_USE_TANH_25_7)) 178 | #if (defined(TINYMIND_USE_TANH_26_6)) 179 | struct TanhValuesTableQ26_6 180 | { 181 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 182 | }; 183 | #endif // (defined(TINYMIND_USE_TANH_26_6)) 184 | #if (defined(TINYMIND_USE_TANH_27_5)) 185 | struct TanhValuesTableQ27_5 186 | { 187 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 188 | }; 189 | #endif // (defined(TINYMIND_USE_TANH_27_5)) 190 | #if (defined(TINYMIND_USE_TANH_28_4)) 191 | struct TanhValuesTableQ28_4 192 | { 193 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 194 | }; 195 | #endif // (defined(TINYMIND_USE_TANH_28_4)) 196 | #if (defined(TINYMIND_USE_TANH_29_3)) 197 | struct TanhValuesTableQ29_3 198 | { 199 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 200 | }; 201 | #endif // (defined(TINYMIND_USE_TANH_29_3)) 202 | #if (defined(TINYMIND_USE_TANH_30_2)) 203 | struct TanhValuesTableQ30_2 204 | { 205 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 206 | }; 207 | #endif // (defined(TINYMIND_USE_TANH_30_2)) 208 | #if (defined(TINYMIND_USE_TANH_31_1)) 209 | struct TanhValuesTableQ31_1 210 | { 211 | static const uint32_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 212 | }; 213 | #endif // (defined(TINYMIND_USE_TANH_31_1)) 214 | } 215 | -------------------------------------------------------------------------------- /cpp/tanhValues8Bit.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "activation.hpp" 26 | 27 | namespace tinymind { 28 | #if (defined(TINYMIND_USE_TANH_1_7)) 29 | struct TanhValuesTableQ1_7 30 | { 31 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 32 | }; 33 | #endif // (defined(TINYMIND_USE_TANH_1_7)) 34 | #if (defined(TINYMIND_USE_TANH_2_6)) 35 | struct TanhValuesTableQ2_6 36 | { 37 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 38 | }; 39 | #endif // (defined(TINYMIND_USE_TANH_2_6)) 40 | #if (defined(TINYMIND_USE_TANH_3_5)) 41 | struct TanhValuesTableQ3_5 42 | { 43 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 44 | }; 45 | #endif // (defined(TINYMIND_USE_TANH_3_5)) 46 | #if (defined(TINYMIND_USE_TANH_4_4)) 47 | struct TanhValuesTableQ4_4 48 | { 49 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 50 | }; 51 | #endif // (defined(TINYMIND_USE_TANH_4_4)) 52 | #if (defined(TINYMIND_USE_TANH_5_3)) 53 | struct TanhValuesTableQ5_3 54 | { 55 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 56 | }; 57 | #endif // (defined(TINYMIND_USE_TANH_5_3)) 58 | #if (defined(TINYMIND_USE_TANH_6_2)) 59 | struct TanhValuesTableQ6_2 60 | { 61 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 62 | }; 63 | #endif // (defined(TINYMIND_USE_TANH_6_2)) 64 | #if (defined(TINYMIND_USE_TANH_7_1)) 65 | struct TanhValuesTableQ7_1 66 | { 67 | static const uint8_t values[NUMBER_OF_ACTIVATION_TABLE_VALUES]; 68 | }; 69 | #endif // (defined(TINYMIND_USE_TANH_7_1)) 70 | } 71 | -------------------------------------------------------------------------------- /cpp/typeChooser.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | namespace tinymind { 26 | template 27 | struct FullWidthType 28 | {}; 29 | 30 | template<> 31 | struct FullWidthType<8, false> 32 | { 33 | typedef uint8_t FractionalPartFieldType; 34 | typedef uint8_t FixedPartFieldType; 35 | typedef uint8_t FullWidthFieldType; 36 | typedef uint8_t FullWidthValueType; 37 | }; 38 | 39 | template<> 40 | struct FullWidthType<8, true> 41 | { 42 | typedef uint8_t FractionalPartFieldType; 43 | typedef int8_t FixedPartFieldType; 44 | typedef uint8_t FullWidthFieldType; 45 | typedef int8_t FullWidthValueType; 46 | }; 47 | 48 | template<> 49 | struct FullWidthType<16, false> 50 | { 51 | typedef uint16_t FractionalPartFieldType; 52 | typedef uint16_t FixedPartFieldType; 53 | typedef uint16_t FullWidthFieldType; 54 | typedef uint16_t FullWidthValueType; 55 | }; 56 | 57 | template<> 58 | struct FullWidthType<16, true> 59 | { 60 | typedef uint16_t FractionalPartFieldType; 61 | typedef int16_t FixedPartFieldType; 62 | typedef uint16_t FullWidthFieldType; 63 | typedef int16_t FullWidthValueType; 64 | }; 65 | 66 | template<> 67 | struct FullWidthType<32, false> 68 | { 69 | typedef uint32_t FractionalPartFieldType; 70 | typedef uint32_t FixedPartFieldType; 71 | typedef uint32_t FullWidthFieldType; 72 | typedef uint32_t FullWidthValueType; 73 | }; 74 | 75 | template<> 76 | struct FullWidthType<32, true> 77 | { 78 | typedef uint32_t FractionalPartFieldType; 79 | typedef int32_t FixedPartFieldType; 80 | typedef uint32_t FullWidthFieldType; 81 | typedef int32_t FullWidthValueType; 82 | }; 83 | 84 | template<> 85 | struct FullWidthType<64, false> 86 | { 87 | typedef uint64_t FractionalPartFieldType; 88 | typedef uint64_t FixedPartFieldType; 89 | typedef uint64_t FullWidthFieldType; 90 | typedef uint64_t FullWidthValueType; 91 | }; 92 | 93 | template<> 94 | struct FullWidthType<64, true> 95 | { 96 | typedef uint64_t FractionalPartFieldType; 97 | typedef int64_t FixedPartFieldType; 98 | typedef uint64_t FullWidthFieldType; 99 | typedef int64_t FullWidthValueType; 100 | }; 101 | 102 | template 103 | struct FullWidthFieldTypeChooser 104 | { 105 | static_assert(NumBits <= 64, "NumBits must be <= 64."); 106 | static constexpr unsigned Result = (NumBits <= 8) ? 8 : (NumBits <= 16) ? 16 : (NumBits <= 32) ? 32 : 64; 107 | typedef typename FullWidthType::FractionalPartFieldType FractionalPartFieldType; 108 | typedef typename FullWidthType::FixedPartFieldType FixedPartFieldType; 109 | typedef typename FullWidthType::FullWidthFieldType FullWidthFieldType; 110 | typedef typename FullWidthType::FullWidthValueType FullWidthValueType; 111 | }; 112 | 113 | // Meant to be overloaded by client code not using tinymind Q format value types 114 | template 115 | struct FullWidthFieldTypeShim 116 | { 117 | typedef typename ValueType::FullWidthFieldType FullWidthFieldType; 118 | }; 119 | } -------------------------------------------------------------------------------- /cpp/zeroTolerance.hpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include "constants.hpp" 26 | 27 | namespace tinymind { 28 | template 29 | struct ZeroToleranceCalculator 30 | { 31 | static bool isWithinZeroTolerance(const ValueType& value) 32 | { 33 | static const ValueType zeroTolerance(1 << (ValueType::NumberOfFractionalBits - 7)); 34 | static const ValueType negativeTolerance = (Constants::negativeOne() * zeroTolerance); 35 | 36 | return ((Constants::zero() == value) || ((value < zeroTolerance) && (value > negativeTolerance))); 37 | } 38 | }; 39 | } 40 | -------------------------------------------------------------------------------- /docs/_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-architect -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | 2 | 3 |

Tinymind

4 |

Years ago, I began writing this code with the goal of instantiating and running neural networks and machine learning algorithms within embedded systems. I did not have an FPU, GPU, or any vectorized instructions at my disposal so I needed to ensure I could run these algorithms with only a very simple CPU. The end result is this library of C++ templates. The code within the templates are contained within the namespace tinymind.

5 |

My inspiration for these libraries is Andrei Alexandrescu's Modern C++ Design. This book was my first exposure to the power of C++ templates and template metaprogramming. In the book he describes how to design your code using policy classes as template parameters to customize behavior. I used this idea to make the code within these libraries as small and efficient as possible.

6 |

Today, tinymind provides: Feed-forward neural networks, recurrent neural networks, Q format, and Q learning.

7 |

See the wiki pages and tutorials which document how the code is designed as well as how to use the code within tinymind to instantiate and run neural networks as well as machine learning algorithms on virtually any device which has C++ compiler support.

8 |

Dan McLeran

9 |

danmcleran@gmail.com

10 |

daniel.mcleran@intel.com

11 | 12 | 13 | -------------------------------------------------------------------------------- /examples/dqn_maze/Makefile: -------------------------------------------------------------------------------- 1 | # Simple Makefile for the DQN maze example 2 | # Tell the code to compile the Q16.16 tanh activation function LUT 3 | default : 4 | # Make an output dir to hold the executable 5 | mkdir -p ~/dqn_maze 6 | # Build the example with default build flags 7 | g++ -O3 -Wall -o ~/dqn_maze/dqn_maze dqn_maze.cpp dqn_mazelearner.cpp ../../cpp/lookupTables.cpp -I../../cpp -I../../apps/include/ -DTINYMIND_USE_TANH_16_16 8 | 9 | debug : 10 | # Make an output dir to hold the executable 11 | mkdir -p ~/dqn_maze 12 | # Build the example with default build flags 13 | g++ -g -Wall -o ~/dqn_maze/dqn_maze dqn_maze.cpp dqn_mazelearner.cpp ../../cpp/lookupTables.cpp -I../../cpp -I../../apps/include/ -DTINYMIND_USE_TANH_16_16 14 | 15 | # Remove all object files 16 | clean: 17 | rm *.o 18 | -------------------------------------------------------------------------------- /examples/dqn_maze/dqn_maze.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | /* 24 | Q-Learning unit test. Learn the best path out of a simple maze. 25 | 26 | 5 == Outside the maze 27 | ________________________________________________ 28 | | | | 29 | | | | 30 | | 0 | 1 / 5 31 | | | | 32 | |____________/ ________|__/ __________________|_______________________ 33 | | | | | 34 | | | / | 35 | | 4 | 3 | 2 | 36 | | / | | 37 | |__/ __________________|_______________________|_______________________| 38 | 5 39 | 40 | The paths out of the maze: 41 | 42 | 0->4->5 43 | 0->4->3->1->5 44 | 1->5 45 | 1->3->4->5 46 | 2->3->1->5 47 | 2->3->4->5 48 | 3->1->5 49 | 3->4->5 50 | 4->5 51 | 4->3->1->5 52 | */ 53 | 54 | #include 55 | #include 56 | #include 57 | 58 | #include "dqn_mazelearner.h" 59 | 60 | extern QLearnerType qLearner; 61 | 62 | int main(const int argc, char *argv[]) 63 | { 64 | using namespace std; 65 | state_t state; 66 | action_t action; 67 | typename MazeEnvironmentType::ParentType::experience_t experience; 68 | string logEntry; 69 | ofstream logFile; 70 | size_t randomActionDecisionPoint; 71 | 72 | logFile.open("dqn_maze_training.txt"); 73 | if(!logFile.is_open()) 74 | { 75 | cerr << "Failed to open output file." << endl; 76 | return -1; 77 | } 78 | 79 | // Init reward table 80 | for(state = 0;state < NUMBER_OF_STATES;++state) 81 | { 82 | for(action = 0;action < NUMBER_OF_ACTIONS;++action) 83 | { 84 | if (qLearner.getEnvironment().isActionValidForState(state, action)) 85 | { 86 | if (5 == action) 87 | { 88 | qLearner.getEnvironment().setRewardForStateAndAction(state, action, MazeEnvironmentType::EnvironmentRewardValue); 89 | } 90 | else 91 | { 92 | qLearner.getEnvironment().setRewardForStateAndAction(state, action, MazeEnvironmentType::EnvironmentNoRewardValue); 93 | } 94 | } 95 | else 96 | { 97 | qLearner.getEnvironment().setRewardForStateAndAction(state, action, MazeEnvironmentType::EnvironmentInvalidActionValue); 98 | } 99 | } 100 | } 101 | 102 | qLearner.getEnvironment().setRandomActionDecisionPoint(100); 103 | qLearner.getEnvironment().setGoalState(5); 104 | 105 | // randomly search the maze for the reward, keep updating the Q table 106 | for (auto i = 0; i < 500; ++i) 107 | { 108 | // after 400 random iterations, scale down the randomness on every iteration 109 | if (i >= 400) 110 | { 111 | randomActionDecisionPoint = qLearner.getEnvironment().getRandomActionDecisionPoint(); 112 | if (randomActionDecisionPoint > 0) 113 | { 114 | --randomActionDecisionPoint; 115 | qLearner.getEnvironment().setRandomActionDecisionPoint(randomActionDecisionPoint); 116 | } 117 | } 118 | 119 | qLearner.startNewEpisode(); 120 | 121 | state = rand() % NUMBER_OF_STATES; 122 | cout << "*** starting in state " << (int)state << " ***" << endl; 123 | logEntry.clear(); 124 | logEntry += to_string(state); 125 | logEntry += ","; 126 | action = qLearner.takeAction(state); 127 | cout << "take action " << (int)action << endl; 128 | logEntry += to_string(action); 129 | logEntry += ","; 130 | experience.state = state; 131 | experience.action = action; 132 | experience.reward = qLearner.getEnvironment().getRewardForStateAndAction(experience.state, experience.action); 133 | experience.newState = static_cast(experience.action); 134 | qLearner.updateFromExperience(experience); 135 | 136 | // look until we find the cheese 137 | while (qLearner.getState() != qLearner.getEnvironment().getGoalState()) 138 | { 139 | action = qLearner.takeAction(qLearner.getState()); 140 | cout << "take action " << (int)action << endl; 141 | logEntry += to_string(action); 142 | logEntry += ","; 143 | experience.state = qLearner.getState(); 144 | experience.action = action; 145 | experience.reward = qLearner.getEnvironment().getRewardForStateAndAction(experience.state, experience.action); 146 | experience.newState = static_cast(experience.action); 147 | qLearner.updateFromExperience(experience); 148 | } 149 | 150 | logFile << logEntry << endl; 151 | } 152 | 153 | logFile.close(); 154 | logFile.open("dqn_maze_test.txt"); 155 | if(!logFile.is_open()) 156 | { 157 | cerr << "Failed to open output file." << endl; 158 | return -1; 159 | } 160 | 161 | // trainging is done, now run some test iterations 162 | for (auto i = 0; i < 100; ++i) 163 | { 164 | qLearner.startNewEpisode(); 165 | 166 | state = rand() % NUMBER_OF_STATES; 167 | cout << "*** starting in state " << (int)state << " ***" << endl; 168 | logEntry.clear(); 169 | logEntry += to_string(state); 170 | logEntry += ","; 171 | action = qLearner.takeAction(state); 172 | cout << "take action " << (int)action << endl; 173 | logEntry += to_string(action); 174 | logEntry += ","; 175 | experience.state = state; 176 | experience.action = action; 177 | experience.reward = qLearner.getEnvironment().getRewardForStateAndAction(experience.state, experience.action); 178 | experience.newState = static_cast(experience.action); 179 | qLearner.updateFromExperience(experience); 180 | 181 | // look until we find the cheese 182 | while (qLearner.getState() != qLearner.getEnvironment().getGoalState()) 183 | { 184 | action = qLearner.takeAction(qLearner.getState()); 185 | cout << "take action " << (int)action << endl; 186 | logEntry += to_string(action); 187 | logEntry += ","; 188 | experience.state = qLearner.getState(); 189 | experience.action = action; 190 | experience.reward = qLearner.getEnvironment().getRewardForStateAndAction(experience.state, experience.action); 191 | experience.newState = static_cast(experience.action); 192 | qLearner.updateFromExperience(experience); 193 | } 194 | 195 | logFile << logEntry << endl; 196 | } 197 | 198 | return 0; 199 | } -------------------------------------------------------------------------------- /examples/dqn_maze/dqn_mazelearner.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #include "dqn_mazelearner.h" 24 | 25 | #define LEARNING_RATE (QValueType(1,0) / QValueType(5,0)) 26 | #define DISCOUNT_FACTOR (QValueType(8,0) / QValueType(10,0)) 27 | QLearnerType qLearner(LEARNING_RATE, DISCOUNT_FACTOR, 100); 28 | -------------------------------------------------------------------------------- /examples/dqn_maze/dqn_mazelearner.h: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | 31 | #include "qformat.hpp" 32 | #include "qlearn.hpp" 33 | #include "constants.hpp" 34 | #include "fixedPointTransferFunctions.hpp" 35 | #include "activationFunctions.hpp" 36 | #include "neuralnet.hpp" 37 | #include "nnproperties.hpp" 38 | 39 | // 6 rooms and 6 actions 40 | #define NUMBER_OF_STATES 6 41 | #define NUMBER_OF_ACTIONS 6 42 | 43 | typedef uint8_t state_t; 44 | typedef uint8_t action_t; 45 | 46 | static std::default_random_engine generator(time(nullptr)); 47 | static std::uniform_real_distribution distribution(-1.0, 1.0); 48 | 49 | template 50 | struct UniformRealRandomNumberGenerator 51 | { 52 | typedef tinymind::ValueConverter WeightConverterPolicy; 53 | 54 | static ValueType generateRandomWeight() 55 | { 56 | const double temp = distribution(generator); 57 | const ValueType weight = WeightConverterPolicy::convertToDestinationType(temp); 58 | 59 | return weight; 60 | } 61 | }; 62 | 63 | template 64 | struct DQNMazeEnvironmentRandomNumberGeneratorPolicy 65 | { 66 | DQNMazeEnvironmentRandomNumberGeneratorPolicy() 67 | { 68 | srand(static_cast(time(0))); // seed random number generator 69 | } 70 | 71 | size_t getRandomActionDecisionPoint() const 72 | { 73 | return this->mRandomActionDecisionPoint; 74 | } 75 | 76 | T randInt(const T min, const T max) 77 | { 78 | // Generate a random number between min..max 79 | const T value = (rand() % 80 | (max + 81 | 1 - 82 | min)) + 83 | min; 84 | return value; 85 | } 86 | 87 | void setRandomActionDecisionPoint(const size_t randomActionDecisionPoint) 88 | { 89 | if(randomActionDecisionPoint <= 100) 90 | { 91 | this->mRandomActionDecisionPoint = randomActionDecisionPoint; 92 | } 93 | } 94 | private: 95 | size_t mRandomActionDecisionPoint; 96 | }; 97 | 98 | template< typename StateType, 99 | typename ActionType, 100 | typename ValueType, 101 | size_t NumberOfStates, 102 | size_t NumberOfActions, 103 | typename RewardPolicyType, 104 | template class QLearningPolicy = tinymind::DefaultLearningPolicy 105 | > 106 | struct DQNMazeEnvironment : public tinymind::QLearningEnvironment 107 | { 108 | typedef tinymind::QLearningEnvironment ParentType; 109 | static constexpr size_t EnvironmentNumberOfStates = ParentType::EnvironmentNumberOfStates; 110 | static constexpr size_t EnvironmentNumberOfActions = ParentType::EnvironmentNumberOfActions; 111 | static constexpr size_t EnvironmentInvalidState = ParentType::EnvironmentInvalidState; 112 | static constexpr size_t EnvironmentInvalidAction = ParentType::EnvironmentInvalidAction; 113 | static const ValueType EnvironmentRewardValue; 114 | static const ValueType EnvironmentNoRewardValue; 115 | static const ValueType EnvironmentInvalidActionValue; 116 | 117 | DQNMazeEnvironment(const ValueType& learningRate, const ValueType& discountFactor, const size_t randomActionDecisionPoint) : 118 | ParentType(learningRate, discountFactor, randomActionDecisionPoint), mGoalState(EnvironmentInvalidState) 119 | { 120 | } 121 | 122 | StateType getGoalState() const 123 | { 124 | return this->mGoalState; 125 | } 126 | 127 | static void getInputValues(const StateType state, ValueType *pInputs) 128 | { 129 | static const ValueType MAX_STATE = ValueType((NumberOfStates - 1),0); 130 | ValueType input = (ValueType(state, 0) / MAX_STATE); 131 | 132 | *pInputs = input; 133 | } 134 | 135 | size_t getNextStateForStateActionPair(const StateType state, const ActionType action) const 136 | { 137 | return action; 138 | } 139 | 140 | ValueType getRewardForStateAndAction(const StateType state, const ActionType action) const 141 | { 142 | return this->mRewardPolicy.getRewardForStateAndAction(state, action); 143 | } 144 | 145 | size_t getValidActionsForState(const StateType state, ActionType* pActions) const 146 | { 147 | size_t numberOfValidActions = 0; 148 | 149 | for(ActionType action = 0;action < NumberOfActions;++action) 150 | { 151 | if(EnvironmentInvalidActionValue != this->getRewardForStateAndAction(state, action)) 152 | { 153 | *pActions = action; 154 | ++pActions; 155 | ++numberOfValidActions; 156 | } 157 | } 158 | 159 | return numberOfValidActions; 160 | } 161 | 162 | bool isActionValidForState(const StateType state, const ActionType action) const 163 | { 164 | bool valid; 165 | switch(state) 166 | { 167 | case 0: 168 | if (4 == action) 169 | { 170 | valid = true; 171 | } 172 | else 173 | { 174 | valid = false; 175 | } 176 | break; 177 | case 1: 178 | if ((3 == action) || (5 == action)) 179 | { 180 | valid = true; 181 | } 182 | else 183 | { 184 | valid = false; 185 | } 186 | break; 187 | case 2: 188 | if (3 == action) 189 | { 190 | valid = true; 191 | } 192 | else 193 | { 194 | valid = false; 195 | } 196 | break; 197 | case 3: 198 | if ((1 == action) || (4 == action) || (2 == action)) 199 | { 200 | valid = true; 201 | } 202 | else 203 | { 204 | valid = false; 205 | } 206 | break; 207 | case 4: 208 | if ((0 == action) || (5 == action) || (3 == action)) 209 | { 210 | valid = true; 211 | } 212 | else 213 | { 214 | valid = false; 215 | } 216 | break; 217 | case 5: 218 | if ((1 == action) || (4 == action) || (5 == action)) 219 | { 220 | valid = true; 221 | } 222 | else 223 | { 224 | valid = false; 225 | } 226 | break; 227 | default: 228 | valid = false; 229 | break; 230 | } 231 | 232 | return valid; 233 | } 234 | 235 | bool isGoalState(const StateType state) const 236 | { 237 | return (state == this->mGoalState); 238 | } 239 | 240 | void setGoalState(const StateType goalState) 241 | { 242 | if (goalState < EnvironmentNumberOfStates) 243 | { 244 | this->mGoalState = goalState; 245 | } 246 | } 247 | 248 | void setRewardForStateAndAction(const StateType state, const ActionType action, const ValueType& reward) 249 | { 250 | this->mRewardPolicy.setRewardForStateAndAction(state, action, reward); 251 | } 252 | 253 | void takeAction(const typename ParentType::experience_t& experience) 254 | { 255 | this->mExperience = experience; 256 | } 257 | 258 | typename ParentType::experience_t mExperience; 259 | StateType mGoalState; 260 | private: 261 | RewardPolicyType mRewardPolicy; 262 | }; 263 | 264 | template class QLearningPolicy> 265 | const ValueType DQNMazeEnvironment::EnvironmentRewardValue = ValueType(1,0); 266 | 267 | template class QLearningPolicy> 268 | const ValueType DQNMazeEnvironment::EnvironmentNoRewardValue = ValueType(0,0); 269 | 270 | template class QLearningPolicy> 271 | const ValueType DQNMazeEnvironment::EnvironmentInvalidActionValue = ValueType(-1, 0); 272 | 273 | // signed Q-format type to use as reward 274 | typedef tinymind::QValue<16, 16, true> QValueType; 275 | 276 | // Define the action->reward policy 277 | typedef tinymind::QTableRewardPolicy QTableRewardPolicyType; 278 | 279 | // define the maze environment type 280 | typedef DQNMazeEnvironment MazeEnvironmentType; 281 | 282 | // define the neural network for DQN 283 | #define NUMBER_OF_INPUT_LAYER_NEURONS 1 284 | #define NUMBER_OF_HIDDEN_LAYERS 1 285 | #define NUMBER_OF_HIDDEN_LAYER_NEURONS (NUMBER_OF_ACTIONS + 2) 286 | #define NUMBER_OF_OUTPUT_LAYER_NEURONS NUMBER_OF_ACTIONS 287 | #define NUMBER_OF_ITERATIONS_FOR_TARGET_NN_UPDATE 10 288 | 289 | typedef tinymind::FixedPointTransferFunctions< QValueType, 290 | UniformRealRandomNumberGenerator, 291 | tinymind::TanhActivationPolicy, 292 | tinymind::TanhActivationPolicy, 293 | NUMBER_OF_OUTPUT_LAYER_NEURONS> TransferFunctionsType; 294 | typedef tinymind::MultilayerPerceptron< QValueType, 295 | NUMBER_OF_INPUT_LAYER_NEURONS, 296 | NUMBER_OF_HIDDEN_LAYERS, 297 | NUMBER_OF_HIDDEN_LAYER_NEURONS, 298 | NUMBER_OF_OUTPUT_LAYER_NEURONS, 299 | TransferFunctionsType> NeuralNetworkType; 300 | 301 | typedef tinymind::QValueNeuralNetworkPolicy QValuePolicyType; 304 | 305 | // define the Q-learner type 306 | typedef tinymind::QLearner QLearnerType; -------------------------------------------------------------------------------- /examples/dqn_maze/dqn_mazeplot.py: -------------------------------------------------------------------------------- 1 | ''' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | ''' 22 | 23 | import os 24 | import sys 25 | 26 | import matplotlib.pyplot as pyplot 27 | 28 | if __name__ == '__main__': 29 | assert(len(sys.argv) == 2) 30 | 31 | dataPath = sys.argv[1] 32 | assert(os.path.exists(dataPath)) 33 | 34 | fileName = dataPath.split(os.path.sep)[-1] 35 | 36 | startStateToDataDict = {} 37 | 38 | with open(dataPath) as f: 39 | line = f.readline() 40 | while(line): 41 | s = line.strip().split(',') 42 | startState = int(s[0], 10) 43 | if not startState in startStateToDataDict.keys(): 44 | startStateToDataDict[startState] = [] 45 | moves = [] 46 | for i in range(1, len(s) - 1): 47 | moves.append(int(s[i], 10)) 48 | 49 | startStateToDataDict[startState].append(moves) 50 | line = f.readline() 51 | 52 | for startState in startStateToDataDict.keys(): 53 | pyplot.figure() 54 | pyplot.title("%s\nState state = %d" % (fileName, startState)) 55 | for line in startStateToDataDict[startState]: 56 | data = [startState] + line 57 | pyplot.plot(range(len(data)), data, '-D') 58 | 59 | pyplot.show() 60 | 61 | -------------------------------------------------------------------------------- /examples/maze/Makefile: -------------------------------------------------------------------------------- 1 | # Simple Makefile for the table-based q-learner example 2 | default : 3 | # Make an output dir to hold the executable 4 | mkdir -p ~/maze 5 | # Build the example with default build flags 6 | g++ -O3 -Wall -o ~/maze/maze maze.cpp mazelearner.cpp -I../../cpp 7 | 8 | debug : 9 | # Make an output dir to hold the executable 10 | mkdir -p ~/maze 11 | # Build the example with default build flags 12 | g++ -g -Wall -o ~/maze/maze maze.cpp mazelearner.cpp -I../../cpp 13 | 14 | # Remove all object files 15 | clean: 16 | rm *.o 17 | -------------------------------------------------------------------------------- /examples/maze/maze.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | /* 24 | Q-Learning unit test. Learn the best path out of a simple maze. 25 | 26 | 5 == Outside the maze 27 | ________________________________________________ 28 | | | | 29 | | | | 30 | | 0 | 1 / 5 31 | | | | 32 | |____________/ ________|__/ __________________|_______________________ 33 | | | | | 34 | | | / | 35 | | 4 | 3 | 2 | 36 | | / | | 37 | |__/ __________________|_______________________|_______________________| 38 | 5 39 | 40 | The paths out of the maze: 41 | 42 | 0->4->5 43 | 0->4->3->1->5 44 | 1->5 45 | 1->3->4->5 46 | 2->3->1->5 47 | 2->3->4->5 48 | 3->1->5 49 | 3->4->5 50 | 4->5 51 | 4->3->1->5 52 | */ 53 | 54 | #include 55 | #include 56 | #include 57 | 58 | #include "mazelearner.h" 59 | 60 | extern QLearnerType qLearner; 61 | 62 | int main(const int argc, char *argv[]) 63 | { 64 | using namespace std; 65 | state_t state; 66 | action_t action; 67 | typename MazeEnvironmentType::ParentType::experience_t experience; 68 | string logEntry; 69 | ofstream logFile; 70 | size_t randomActionDecisionPoint; 71 | 72 | logFile.open("maze_training.txt"); 73 | if(!logFile.is_open()) 74 | { 75 | cerr << "Failed to open output file." << endl; 76 | return -1; 77 | } 78 | 79 | // Init reward table 80 | for(state = 0;state < NUMBER_OF_STATES;++state) 81 | { 82 | for(action = 0;action < NUMBER_OF_ACTIONS;++action) 83 | { 84 | if (qLearner.getEnvironment().isActionValidForState(state, action)) 85 | { 86 | if (5 == action) 87 | { 88 | qLearner.getEnvironment().setRewardForStateAndAction(state, action, MazeEnvironmentType::EnvironmentRewardValue); 89 | } 90 | else 91 | { 92 | qLearner.getEnvironment().setRewardForStateAndAction(state, action, MazeEnvironmentType::EnvironmentNoRewardValue); 93 | } 94 | } 95 | else 96 | { 97 | qLearner.getEnvironment().setRewardForStateAndAction(state, action, MazeEnvironmentType::EnvironmentInvalidActionValue); 98 | } 99 | } 100 | } 101 | 102 | qLearner.getEnvironment().setRandomActionDecisionPoint(100); 103 | qLearner.getEnvironment().setGoalState(5); 104 | 105 | // randomly search the maze for the reward, keep updating the Q table 106 | for (auto i = 0; i < 500; ++i) 107 | { 108 | // after 400 random iterations, scale down the randomness on every iteration 109 | if (i >= 400) 110 | { 111 | randomActionDecisionPoint = qLearner.getEnvironment().getRandomActionDecisionPoint(); 112 | if (randomActionDecisionPoint > 0) 113 | { 114 | --randomActionDecisionPoint; 115 | qLearner.getEnvironment().setRandomActionDecisionPoint(randomActionDecisionPoint); 116 | } 117 | } 118 | 119 | qLearner.startNewEpisode(); 120 | 121 | state = rand() % NUMBER_OF_STATES; 122 | cout << "*** starting in state " << (int)state << " ***" << endl; 123 | logEntry.clear(); 124 | logEntry += to_string(state); 125 | logEntry += ","; 126 | action = qLearner.takeAction(state); 127 | cout << "take action " << (int)action << endl; 128 | logEntry += to_string(action); 129 | logEntry += ","; 130 | experience.state = state; 131 | experience.action = action; 132 | experience.reward = qLearner.getEnvironment().getRewardForStateAndAction(experience.state, experience.action); 133 | experience.newState = static_cast(experience.action); 134 | qLearner.updateFromExperience(experience); 135 | 136 | // look until we find the cheese 137 | while (qLearner.getState() != qLearner.getEnvironment().getGoalState()) 138 | { 139 | action = qLearner.takeAction(qLearner.getState()); 140 | cout << "take action " << (int)action << endl; 141 | logEntry += to_string(action); 142 | logEntry += ","; 143 | experience.state = qLearner.getState(); 144 | experience.action = action; 145 | experience.reward = qLearner.getEnvironment().getRewardForStateAndAction(experience.state, experience.action); 146 | experience.newState = static_cast(experience.action); 147 | qLearner.updateFromExperience(experience); 148 | } 149 | 150 | logFile << logEntry << endl; 151 | } 152 | 153 | logFile.close(); 154 | logFile.open("maze_test.txt"); 155 | if(!logFile.is_open()) 156 | { 157 | cerr << "Failed to open output file." << endl; 158 | return -1; 159 | } 160 | 161 | // trainging is done, now run some test iterations 162 | for (auto i = 0; i < 100; ++i) 163 | { 164 | qLearner.startNewEpisode(); 165 | 166 | state = rand() % NUMBER_OF_STATES; 167 | cout << "*** starting in state " << (int)state << " ***" << endl; 168 | logEntry.clear(); 169 | logEntry += to_string(state); 170 | logEntry += ","; 171 | action = qLearner.takeAction(state); 172 | cout << "take action " << (int)action << endl; 173 | logEntry += to_string(action); 174 | logEntry += ","; 175 | experience.state = state; 176 | experience.action = action; 177 | experience.reward = qLearner.getEnvironment().getRewardForStateAndAction(experience.state, experience.action); 178 | experience.newState = static_cast(experience.action); 179 | qLearner.updateFromExperience(experience); 180 | 181 | // look until we find the cheese 182 | while (qLearner.getState() != qLearner.getEnvironment().getGoalState()) 183 | { 184 | action = qLearner.takeAction(qLearner.getState()); 185 | cout << "take action " << (int)action << endl; 186 | logEntry += to_string(action); 187 | logEntry += ","; 188 | experience.state = qLearner.getState(); 189 | experience.action = action; 190 | experience.reward = qLearner.getEnvironment().getRewardForStateAndAction(experience.state, experience.action); 191 | experience.newState = static_cast(experience.action); 192 | qLearner.updateFromExperience(experience); 193 | } 194 | 195 | logFile << logEntry << endl; 196 | } 197 | 198 | return 0; 199 | } -------------------------------------------------------------------------------- /examples/maze/mazelearner.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #include "mazelearner.h" 24 | 25 | #define LEARNING_RATE (QValueType(1,0) / QValueType(5,0)) 26 | #define DISCOUNT_FACTOR (QValueType(8,0) / QValueType(10,0)) 27 | QLearnerType qLearner(LEARNING_RATE, DISCOUNT_FACTOR, 100); 28 | -------------------------------------------------------------------------------- /examples/maze/mazelearner.h: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | 30 | #include "qformat.hpp" 31 | #include "qlearn.hpp" 32 | #include "constants.hpp" 33 | 34 | // 6 rooms and 6 actions 35 | #define NUMBER_OF_STATES 6 36 | #define NUMBER_OF_ACTIONS 6 37 | 38 | typedef uint8_t state_t; 39 | typedef uint8_t action_t; 40 | 41 | template 42 | struct MazeEnvironmentRandomNumberGeneratorPolicy 43 | { 44 | MazeEnvironmentRandomNumberGeneratorPolicy() 45 | { 46 | srand(static_cast(time(0))); // seed random number generator 47 | } 48 | 49 | size_t getRandomActionDecisionPoint() const 50 | { 51 | return this->mRandomActionDecisionPoint; 52 | } 53 | 54 | T randInt(const T min, const T max) 55 | { 56 | // Generate a random number between min..max 57 | const T value = (rand() % 58 | (max + 59 | 1 - 60 | min)) + 61 | min; 62 | return value; 63 | } 64 | 65 | void setRandomActionDecisionPoint(const size_t randomActionDecisionPoint) 66 | { 67 | if(randomActionDecisionPoint <= 100) 68 | { 69 | this->mRandomActionDecisionPoint = randomActionDecisionPoint; 70 | } 71 | } 72 | private: 73 | size_t mRandomActionDecisionPoint; 74 | }; 75 | 76 | template< typename StateType, 77 | typename ActionType, 78 | typename ValueType, 79 | size_t NumberOfStates, 80 | size_t NumberOfActions, 81 | typename RewardPolicyType, 82 | template class QLearningPolicy = tinymind::DefaultLearningPolicy 83 | > 84 | struct MazeEnvironment : public tinymind::QLearningEnvironment 85 | { 86 | typedef tinymind::QLearningEnvironment ParentType; 87 | static constexpr size_t EnvironmentNumberOfStates = ParentType::EnvironmentNumberOfStates; 88 | static constexpr size_t EnvironmentNumberOfActions = ParentType::EnvironmentNumberOfActions; 89 | static constexpr size_t EnvironmentInvalidState = ParentType::EnvironmentInvalidState; 90 | static constexpr size_t EnvironmentInvalidAction = ParentType::EnvironmentInvalidAction; 91 | static const ValueType EnvironmentRewardValue; 92 | static const ValueType EnvironmentNoRewardValue; 93 | static const ValueType EnvironmentInvalidActionValue; 94 | 95 | MazeEnvironment(const ValueType& learningRate, const ValueType& discountFactor, const size_t randomActionDecisionPoint) : 96 | ParentType(learningRate, discountFactor, randomActionDecisionPoint), mGoalState(EnvironmentInvalidState) 97 | { 98 | } 99 | 100 | StateType getGoalState() const 101 | { 102 | return this->mGoalState; 103 | } 104 | 105 | size_t getNextStateForStateActionPair(const StateType state, const ActionType action) const 106 | { 107 | return action; 108 | } 109 | 110 | ValueType getRewardForStateAndAction(const StateType state, const ActionType action) const 111 | { 112 | return this->mRewardPolicy.getRewardForStateAndAction(state, action); 113 | } 114 | 115 | size_t getValidActionsForState(const StateType state, ActionType* pActions) const 116 | { 117 | size_t numberOfValidActions = 0; 118 | 119 | for(ActionType action = 0;action < NumberOfActions;++action) 120 | { 121 | if(EnvironmentInvalidActionValue != this->getRewardForStateAndAction(state, action)) 122 | { 123 | *pActions = action; 124 | ++pActions; 125 | ++numberOfValidActions; 126 | } 127 | } 128 | 129 | return numberOfValidActions; 130 | } 131 | 132 | bool isActionValidForState(const StateType state, const ActionType action) const 133 | { 134 | bool valid; 135 | switch(state) 136 | { 137 | case 0: 138 | if (4 == action) 139 | { 140 | valid = true; 141 | } 142 | else 143 | { 144 | valid = false; 145 | } 146 | break; 147 | case 1: 148 | if ((3 == action) || (5 == action)) 149 | { 150 | valid = true; 151 | } 152 | else 153 | { 154 | valid = false; 155 | } 156 | break; 157 | case 2: 158 | if (3 == action) 159 | { 160 | valid = true; 161 | } 162 | else 163 | { 164 | valid = false; 165 | } 166 | break; 167 | case 3: 168 | if ((1 == action) || (4 == action) || (2 == action)) 169 | { 170 | valid = true; 171 | } 172 | else 173 | { 174 | valid = false; 175 | } 176 | break; 177 | case 4: 178 | if ((0 == action) || (5 == action) || (3 == action)) 179 | { 180 | valid = true; 181 | } 182 | else 183 | { 184 | valid = false; 185 | } 186 | break; 187 | case 5: 188 | if ((1 == action) || (4 == action) || (5 == action)) 189 | { 190 | valid = true; 191 | } 192 | else 193 | { 194 | valid = false; 195 | } 196 | break; 197 | default: 198 | valid = false; 199 | break; 200 | } 201 | 202 | return valid; 203 | } 204 | 205 | bool isGoalState(const StateType state) const 206 | { 207 | return (state == this->mGoalState); 208 | } 209 | 210 | void setGoalState(const StateType goalState) 211 | { 212 | if (goalState < EnvironmentNumberOfStates) 213 | { 214 | this->mGoalState = goalState; 215 | } 216 | } 217 | 218 | void setRewardForStateAndAction(const StateType state, const ActionType action, const ValueType& reward) 219 | { 220 | this->mRewardPolicy.setRewardForStateAndAction(state, action, reward); 221 | } 222 | 223 | void takeAction(const typename ParentType::experience_t& experience) 224 | { 225 | this->mExperience = experience; 226 | } 227 | 228 | typename ParentType::experience_t mExperience; 229 | StateType mGoalState; 230 | private: 231 | RewardPolicyType mRewardPolicy; 232 | }; 233 | 234 | template class QLearningPolicy> 235 | const ValueType MazeEnvironment::EnvironmentRewardValue = ValueType(1,0); 236 | 237 | template class QLearningPolicy> 238 | const ValueType MazeEnvironment::EnvironmentNoRewardValue = ValueType(0,0); 239 | 240 | template class QLearningPolicy> 241 | const ValueType MazeEnvironment::EnvironmentInvalidActionValue = ValueType(-1, 0); 242 | 243 | // unsigned Q-format type to use as reward 244 | typedef tinymind::QValue<16,16,false> QValueType; 245 | 246 | // Define the action->reward policy 247 | typedef tinymind::QTableRewardPolicy QTableRewardPolicyType; 248 | 249 | // define the maze environment type 250 | typedef MazeEnvironment MazeEnvironmentType; 251 | 252 | // define the Q-learner type 253 | typedef tinymind::QLearner QLearnerType; 254 | -------------------------------------------------------------------------------- /examples/maze/mazeplot.py: -------------------------------------------------------------------------------- 1 | ''' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | ''' 22 | 23 | import os 24 | import sys 25 | 26 | import matplotlib.pyplot as pyplot 27 | 28 | if __name__ == '__main__': 29 | assert(len(sys.argv) == 2) 30 | 31 | dataPath = sys.argv[1] 32 | assert(os.path.exists(dataPath)) 33 | 34 | fileName = dataPath.split(os.path.sep)[-1] 35 | 36 | startStateToDataDict = {} 37 | 38 | with open(dataPath) as f: 39 | line = f.readline() 40 | while(line): 41 | s = line.strip().split(',') 42 | startState = int(s[0], 10) 43 | if not startState in startStateToDataDict.keys(): 44 | startStateToDataDict[startState] = [] 45 | moves = [] 46 | for i in range(1, len(s) - 1): 47 | moves.append(int(s[i], 10)) 48 | 49 | startStateToDataDict[startState].append(moves) 50 | line = f.readline() 51 | 52 | for startState in startStateToDataDict.keys(): 53 | pyplot.figure() 54 | pyplot.title("%s\nState state = %d" % (fileName, startState)) 55 | for line in startStateToDataDict[startState]: 56 | data = [startState] + line 57 | pyplot.plot(range(len(data)), data, '-D') 58 | 59 | pyplot.show() 60 | 61 | -------------------------------------------------------------------------------- /examples/xor/Makefile: -------------------------------------------------------------------------------- 1 | # Simple Makefile for the XOR neural network example 2 | # Enable C++ iostreams 3 | # Tell the code to compile the Q8.8 tanh activation function LUT 4 | default : 5 | # Make an output dir to hold the executable 6 | mkdir -p ~/xor 7 | # Build the example with default build flags 8 | g++ -O3 -Wall -o ~/xor/xor xor.cpp xornet.cpp ../../cpp/lookupTables.cpp -DENABLE_OSTREAMS -DTINYMIND_USE_TANH_8_8 -I../../cpp/ -I../../apps/include 9 | 10 | debug : 11 | # Make an output dir to hold the executable 12 | mkdir -p ~/xor 13 | # Build the example with default build flags 14 | g++ -g -Wall -o ~/xor/xor xor.cpp xornet.cpp ../../cpp/lookupTables.cpp -DENABLE_OSTREAMS -DTINYMIND_USE_TANH_8_8 -I../../cpp/ -I../../apps/include 15 | 16 | # Remove all object files 17 | clean: 18 | rm *.o 19 | -------------------------------------------------------------------------------- /examples/xor/xor.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #include 24 | #include 25 | #include 26 | #include 27 | 28 | #include "constants.hpp" 29 | #include "nnproperties.hpp" 30 | 31 | #include "xornet.h" 32 | 33 | #define TRAINING_ITERATIONS 2000 34 | #define NUM_SAMPLES_AVG_ERROR 20 35 | 36 | static void generateXorTrainingValue(ValueType& x, ValueType& y, ValueType& z) 37 | { 38 | const int randX = rand() & 0x1; 39 | const int randY = rand() & 0x1; 40 | const int result = (randX ^ randY); 41 | 42 | x = ValueType(randX, 0); 43 | y = ValueType(randY, 0); 44 | z = ValueType(result, 0); 45 | } 46 | 47 | extern NeuralNetworkType testNeuralNet; 48 | 49 | int main(const int argc, char *argv[]) 50 | { 51 | using namespace std; 52 | 53 | srand(static_cast(time(0))); // seed random number generator 54 | 55 | char const* const path = "nn_fixed_xor.txt"; 56 | ofstream results(path); 57 | ValueType values[NeuralNetworkType::NumberOfInputLayerNeurons]; 58 | ValueType output[NeuralNetworkType::NumberOfOutputLayerNeurons]; 59 | ValueType learnedValues[NeuralNetworkType::NumberOfInputLayerNeurons]; 60 | ValueType error; 61 | 62 | tinymind::NetworkPropertiesFileManager::writeHeader(testNeuralNet, results); 63 | 64 | for (auto i = 0; i < TRAINING_ITERATIONS; ++i) 65 | { 66 | generateXorTrainingValue(values[0], values[1], output[0]); 67 | 68 | testNeuralNet.feedForward(&values[0]); 69 | error = testNeuralNet.calculateError(&output[0]); 70 | if (!NeuralNetworkType::NeuralNetworkTransferFunctionsPolicy::isWithinZeroTolerance(error)) 71 | { 72 | testNeuralNet.trainNetwork(&output[0]); 73 | } 74 | testNeuralNet.getLearnedValues(&learnedValues[0]); 75 | 76 | tinymind::NetworkPropertiesFileManager::storeNetworkProperties(testNeuralNet, results, &output[0], &learnedValues[0]); 77 | results << error << std::endl; 78 | } 79 | 80 | return 0; 81 | } 82 | -------------------------------------------------------------------------------- /examples/xor/xornet.cpp: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | 24 | #include 25 | #include "xornet.h" 26 | 27 | NeuralNetworkType testNeuralNet; -------------------------------------------------------------------------------- /examples/xor/xornet.h: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | */ 22 | 23 | #pragma once 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | 30 | #include "qformat.hpp" 31 | #include "activationFunctions.hpp" 32 | #include "fixedPointTransferFunctions.hpp" 33 | #include "neuralnet.hpp" 34 | 35 | // Q-Format value type 36 | static constexpr size_t NUMBER_OF_FIXED_BITS = 8; 37 | static constexpr size_t NUMBER_OF_FRACTIONAL_BITS = 8; 38 | typedef tinymind::QValue ValueType; 39 | // typedef the underlying full-width representation type 40 | typedef typename ValueType::FullWidthValueType FullWidthValueType; 41 | // Neural network architecture 42 | static constexpr size_t NUMBER_OF_INPUTS = 2; 43 | static constexpr size_t NUMBER_OF_HIDDEN_LAYERS = 1; 44 | static constexpr size_t NUMBER_OF_NEURONS_PER_HIDDEN_LAYER = 3; 45 | static constexpr size_t NUMBER_OF_OUTPUTS = 1; 46 | 47 | // Random number generator 48 | struct RandomNumberGenerator 49 | { 50 | static ValueType generateRandomWeight() 51 | { 52 | // Generate a random number between -1..1 in the Q Format full width type 53 | const FullWidthValueType weight = (rand() % 54 | (tinymind::Constants::one().getValue() + 55 | tinymind::Constants::one().getValue() - 56 | tinymind::Constants::negativeOne().getValue())) + 57 | tinymind::Constants::negativeOne().getValue(); 58 | 59 | return weight; 60 | } 61 | }; 62 | 63 | // Typedef of transfer functions for the fixed-point neural network 64 | typedef tinymind::FixedPointTransferFunctions< ValueType, 65 | RandomNumberGenerator, 66 | tinymind::TanhActivationPolicy, 67 | tinymind::TanhActivationPolicy> TransferFunctionsType; 68 | 69 | // typedef the neural network itself 70 | typedef tinymind::MultilayerPerceptron< ValueType, 71 | NUMBER_OF_INPUTS, 72 | NUMBER_OF_HIDDEN_LAYERS, 73 | NUMBER_OF_NEURONS_PER_HIDDEN_LAYER, 74 | NUMBER_OF_OUTPUTS, 75 | TransferFunctionsType> NeuralNetworkType; 76 | -------------------------------------------------------------------------------- /uml/nn/nn_class.puml: -------------------------------------------------------------------------------- 1 | /' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | *'/ 22 | 23 | @startuml 24 | class InputLayerType { 25 | } 26 | 27 | class InnerHiddenLayerManager { 28 | } 29 | 30 | class LastHiddenLayerType { 31 | } 32 | 33 | class OutputLayerType { 34 | } 35 | 36 | class TrainingPolicyType { 37 | } 38 | 39 | class GradientsManagerType { 40 | } 41 | 42 | class TransferFunctionsPolicy { 43 | } 44 | 45 | class MultilayerPerceptron { 46 | +{static} size_t NeuralNetworkNumberOfHiddenLayers 47 | +{static} size_t NumberOfInnerHiddenLayers 48 | +{static} size_t NumberOfInputLayerNeurons 49 | +{static} size_t NumberOfHiddenLayerNeurons 50 | +{static} size_t NumberOfOutputLayerNeurons 51 | +{static} size_t NeuralNetworkRecurrentConnectionDepth 52 | +{static} size_t NeuralNetworkBatchSize 53 | #ValueType mError 54 | } 55 | 56 | class RecurrentMultilayerPerceptron { 57 | } 58 | 59 | MultilayerPerceptron *-- InputLayerType 60 | MultilayerPerceptron *-- InnerHiddenLayerManager 61 | MultilayerPerceptron *-- LastHiddenLayerType 62 | MultilayerPerceptron *-- OutputLayerType 63 | MultilayerPerceptron *-- RecurrentLayerType 64 | MultilayerPerceptron *-- TrainingPolicyType 65 | MultilayerPerceptron *-- GradientsManagerType 66 | MultilayerPerceptron - TransferFunctionsPolicy 67 | RecurrentMultilayerPerceptron -|> MultilayerPerceptron 68 | ElmanNetwork -|> RecurrentMultilayerPerceptron 69 | 70 | @enduml -------------------------------------------------------------------------------- /uml/nn/nn_ssd_init.puml: -------------------------------------------------------------------------------- 1 | /' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | *'/ 22 | 23 | @startuml 24 | title Neural Network Initialization 25 | NeuralNet -> InputLayerType : initializeNeurons 26 | NeuralNet -> InnerHiddenLayerManager : initializeInnerHiddenLayerNeurons 27 | loop NumberOfInnerHiddenLayers 28 | InnerHiddenLayerManager -> InnerHiddenLayerType : initializeNeurons 29 | end 30 | NeuralNet -> LastHiddenLayerType : initializeNeurons 31 | NeuralNet -> OutputLayerType : initializeNeurons 32 | NeuralNet -> RecurrentLayerType : initializeNeurons 33 | NeuralNet -> TrainingPolicyType : initialize 34 | TrainingPolicyType -> TransferFunctionsPolicy : initialLearningRate 35 | TrainingPolicyType -> TransferFunctionsPolicy : initialMomentumRate 36 | TrainingPolicyType -> TransferFunctionsPolicy : initialAccelerationRate 37 | NeuralNet -> InputLayerType : initializeWeights 38 | loop NumberOfNeurons 39 | InputLayerType -> Neuron : initializeWeights 40 | loop NumberOfOutgoingConnections 41 | Neuron -> TransferFunctionsPolicy : generateRandomWeight 42 | end 43 | end 44 | NeuralNet -> InnerHiddenLayerManager : initializeInnerHiddenLayerWeights 45 | loop NumberOfInnerHiddenLayers 46 | InnerHiddenLayerManager -> InnerHiddenLayerType : initializeWeights 47 | loop NumberOfNeurons 48 | InnerHiddenLayerType -> Neuron : initializeWeights 49 | loop NumberOfOutgoingConnections 50 | Neuron -> TransferFunctionsPolicy : generateRandomWeight 51 | end 52 | end 53 | end 54 | NeuralNet -> LastHiddenLayerType : initializeWeights 55 | loop NumberOfNeurons 56 | LastHiddenLayerType -> Neuron : initializeWeights 57 | loop NumberOfOutgoingConnections 58 | Neuron -> TransferFunctionsPolicy : generateRandomWeight 59 | end 60 | end 61 | NeuralNet -> OutputLayerType : initializeWeights 62 | loop NumberOfNeurons 63 | OutputLayerType -> Neuron : initializeWeights 64 | loop NumberOfOutgoingConnections 65 | Neuron -> TransferFunctionsPolicy : generateRandomWeight 66 | end 67 | end 68 | NeuralNet -> RecurrentLayerType : initializeWeights 69 | loop NumberOfNeurons 70 | RecurrentLayerType -> Neuron : initializeWeights 71 | loop NumberOfOutgoingConnections 72 | Neuron -> TransferFunctionsPolicy : generateRandomWeight 73 | end 74 | end 75 | @enduml -------------------------------------------------------------------------------- /uml/nn/nn_ssd_train_pt1.puml: -------------------------------------------------------------------------------- 1 | /' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | *'/ 22 | 23 | @startuml 24 | title Simple Feed Forward Neural Network Training 25 | Application -> NeuralNet : feedForward 26 | NeuralNet -> InputLayerType : feedForward 27 | NeuralNet -> HiddenLayerFeedForwardManagerType : feedForward 28 | note over InnerHiddenLayerManager : InnerHiddenLayers are all hidden layers before\nthe last one, which feeds the OutputLayer 29 | loop 1 to Number Of Inner Hidden Layers 30 | HiddenLayerFeedForwardManagerType -> InnerHiddenLayerType : feedForward 31 | end 32 | note over InnerHiddenLayerManager : InnerHiddenLayers may not exists\nbut the last one always does 33 | HiddenLayerFeedForwardManagerType -> LastHiddenLayerType : feedForward 34 | NeuralNet -> OutputLayerType : feedForward 35 | Application -> NeuralNet : error = calculateError 36 | loop Number Of Output Neurons 37 | NeuralNet -> OutputLayer : getLearnedValue 38 | end 39 | NeuralNet -> TransferFunctionsPolicy : error = calculateError(learned values) 40 | Application -> TransferFunctionsPolicy : train = !isWithinZeroTolerance(error) 41 | alt train 42 | Application -> NeuralNet : trainNetwork 43 | end 44 | @enduml -------------------------------------------------------------------------------- /uml/qformat/qformat_class.puml: -------------------------------------------------------------------------------- 1 | /' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | *'/ 22 | 23 | @startuml qformat_class 24 | 25 | title QValue 26 | 27 | class QTypeChooser { 28 | } 29 | 30 | class FullWidthFieldTypeChooser { 31 | } 32 | 33 | class MultiplicationResultFullWidthFieldTypeChooser { 34 | } 35 | 36 | class DivisionResultFullWidthFieldTypeChooser { 37 | } 38 | 39 | class QValueRoundingPolicy { 40 | {static} T round(const T& value) 41 | } 42 | 43 | class SignExtender { 44 | {static} void signExtend(const T& value) 45 | } 46 | 47 | class ShiftPolicy { 48 | {static} ResultType shfit(const SourceType& value) 49 | } 50 | 51 | class NoShiftPolicy { 52 | {static} ResultType shfit(const SourceType& value) 53 | } 54 | 55 | class ShiftLeftPolicy { 56 | {static} ResultType shfit(const SourceType& value) 57 | } 58 | 59 | class ShiftRightPolicy { 60 | {static} ResultType shfit(const SourceType& value) 61 | } 62 | 63 | class QValue { 64 | } 65 | 66 | QValue -- QTypeChooser 67 | QTypeChooser -- FullWidthFieldTypeChooser 68 | QTypeChooser -- MultiplicationResultFullWidthFieldTypeChooser 69 | QTypeChooser -- DivisionResultFullWidthFieldTypeChooser 70 | MultiplicationResultFullWidthFieldTypeChooser -- FullWidthFieldTypeChooser 71 | DivisionResultFullWidthFieldTypeChooser -- FullWidthFieldTypeChooser 72 | QValue -- QValueRoundingPolicy 73 | QValue -- SignExtender 74 | QValue -- ShiftPolicy 75 | ShiftPolicy -- NoShiftPolicy 76 | ShiftPolicy -- ShiftLeftPolicy 77 | ShiftPolicy -- ShiftRightPolicy 78 | 79 | @enduml -------------------------------------------------------------------------------- /uml/qlearn/qlearn_class.puml: -------------------------------------------------------------------------------- 1 | /' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | *'/ 22 | 23 | @startuml 24 | class DefaultRewardPolicy { 25 | +ValueType getRewardForStateAndAction(const size_t state, const size_t action) const 26 | +void setRewardForStateAndAction(const size_t state, const size_t action, const ValueType& reward) 27 | -ValueType mRewardTable[NumberOfStates][NumberOfActions] 28 | } 29 | RewardPolicy <|-- DefaultRewardPolicy 30 | 31 | class QLearningEnvironment::experience_t { 32 | +size_t state 33 | +size_t action 34 | +ValueType reward 35 | +size_t newState 36 | } 37 | 38 | class QLearningEnvironment { 39 | #ValueType mLearningRate; 40 | #ValueType mDiscountFactor; 41 | #size_t mRandomActionDecisionPoint; 42 | #RandomNumberGeneratorPolicy mRandomNumberGenerator; 43 | #RewardPolicy mRewardPolicy; 44 | } 45 | QLearningEnvironment <|-- EnvironmentType 46 | QLearningEnvironment -- RewardPolicy 47 | 48 | class QLearner { 49 | +void updateFromExperience(const QLearningEnvironment::experience_t& experience) 50 | -EnvironmentType mEnvironment 51 | -StateType mState 52 | -ValueType mAccumulatedReward 53 | -QValuePolicy mQValuePolicy 54 | } 55 | 56 | QLearner -- StateToActionPolicy 57 | QLearner *-- EnvironmentType 58 | QLearner *-- QValuePolicy 59 | QLearner -- QLearningEnvironment::experience_t 60 | 61 | class QTablePolicy { 62 | +ValueType getQValue(const size_t state, const size_t action) const 63 | +void setQValue(const size_t state, const size_t action, const ValueType& value) 64 | -ValueType mQTable[EnvironmentType::EnvironmentNumberOfStates][EnvironmentType::EnvironmentNumberOfActions] 65 | } 66 | QValuePolicy <|-- QTablePolicy 67 | 68 | class StateToActionPolicy { 69 | +{static} size_t selectActionForState() 70 | } 71 | 72 | StateToActionPolicy <|-- ArgMaxPolicy 73 | 74 | StateToActionPolicy -- QValuePolicy 75 | 76 | @enduml -------------------------------------------------------------------------------- /uml/qlearn/qlearn_seq.puml: -------------------------------------------------------------------------------- 1 | /' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | *'/ 22 | 23 | @startuml 24 | autonumber 25 | BandSelectionProgram -> QLearner : init 26 | QLearner -> EnvironmentType : init 27 | QLearner -> EnvironmentType : getInitialQTableValue 28 | QLearner -> QValuePolicy : setQValue 29 | BandSelectionProgram -> QLearner : setState 30 | BandSelectionProgram -> QLearner : iterate 31 | activate QLearner 32 | QLearner -> QLearner : chooseAction 33 | activate QLearner 34 | QLearner -> EnvironmentType : getValidActionsForState 35 | QLearner -> QLearnerStateToActionPolicy : selectActionForState 36 | deactivate QLearner 37 | QLearner -> EnvironmentType : takeAction 38 | deactivate QLearner 39 | BandSelectionProgram -> QLearner : getEnvironment 40 | BandSelectionProgram -> BandSelectionEnvironment : getRewardForStateAndAction 41 | BandSelectionEnvironment -> BandSelectionRewardPolicy : getRewardForStateAndAction 42 | BandSelectionProgram -> QLearner : updateFromExperience 43 | @enduml -------------------------------------------------------------------------------- /unit_test/nn/Makefile: -------------------------------------------------------------------------------- 1 | # Simple Makefile for the neural network unit tests 2 | # Enable C++ iostreams 3 | # Tell the code to compile the activation function LUTs used by the unit tests 4 | nn_unit_test: nn_unit_test.cpp ../../cpp/lookupTables.cpp 5 | # Make an output dir to hold the executable 6 | mkdir -p ~/nn 7 | # Build the unit tests 8 | g++ -g -Wall -o ~/nn/nn_unit_test nn_unit_test.cpp ../../cpp/lookupTables.cpp -DENABLE_OSTREAMS -DTINYMIND_USE_SIGMOID_8_8 -DTINYMIND_USE_SIGMOID_16_16 -DTINYMIND_USE_LOG_16_16 -DTINYMIND_USE_TANH_8_8 -DTINYMIND_USE_TANH_8_24 -DTINYMIND_USE_TANH_16_16 -DTINYMIND_USE_EXP_16_16 -I../../cpp -I../../apps/include -I${BOOST_HOME} 9 | 10 | nn_unit_test_cov: nn_unit_test.cpp ../../cpp/lookupTables.cpp 11 | # Build the unit tests with coverage info 12 | g++ --coverage -o nn_unit_test nn_unit_test.cpp ../../cpp/lookupTables.cpp -DENABLE_OSTREAMS -DTINYMIND_USE_SIGMOID_8_8 -DTINYMIND_USE_SIGMOID_16_16 -DTINYMIND_USE_LOG_16_16 -DTINYMIND_USE_TANH_8_8 -DTINYMIND_USE_TANH_8_24 -DTINYMIND_USE_TANH_16_16 -DTINYMIND_USE_EXP_16_16 -I../../cpp -I../../apps/include -I${BOOST_HOME} 13 | # Run the executable program 14 | ./nn_unit_test 15 | # Run lcov 16 | lcov -t "nn_unit_test" -c -d . -o nn_unit_test_cov.info 17 | # Make the output dir for HTML reports 18 | mkdir -p nn_unit_test_cov_html 19 | # Generate the HTML report 20 | genhtml -o ./nn_unit_test_cov_html nn_unit_test_cov.info 21 | 22 | clean: 23 | rm -f nn_unit_test 24 | rm -f *.o 25 | rm -f *.gcda 26 | rm -f *.gcno 27 | rm -f *.txt 28 | rm -f *.bin 29 | rm -drf nn_unit_test_cov_html 30 | rm -f nn_unit_test_cov.info 31 | -------------------------------------------------------------------------------- /unit_test/nn/nn_plot.py: -------------------------------------------------------------------------------- 1 | ''' 2 | * Copyright (c) 2020 Intel Corporation 3 | * 4 | * Permission is hereby granted, free of charge, to any person obtaining a copy 5 | * of this software and associated documentation files (the "Software"), to deal 6 | * in the Software without restriction, including without limitation the rights 7 | * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | * copies of the Software, and to permit persons to whom the Software is 9 | * furnished to do so, subject to the following conditions: 10 | * 11 | * The above copyright notice and this permission notice shall be included in all 12 | * copies or substantial portions of the Software. 13 | * 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | * SOFTWARE. 21 | ''' 22 | 23 | import os 24 | import sys 25 | 26 | import matplotlib.pyplot as pyplot 27 | 28 | if __name__ == '__main__': 29 | assert(len(sys.argv) == 2) 30 | 31 | dataPath = sys.argv[1] 32 | assert(os.path.exists(dataPath)) 33 | 34 | fileName = dataPath.split(os.path.sep)[-1] 35 | 36 | names = [] 37 | data = [] 38 | with open(dataPath) as f: 39 | line = f.readline() 40 | names = line.strip(' \r\n').split(',') 41 | line = f.readline() 42 | while(line): 43 | s = line.strip(' \r\n').split(',') 44 | data.extend([float(value) for value in s]) 45 | line = f.readline() 46 | 47 | pyplot.figure() 48 | pyplot.suptitle("%s\n%s" % ("NN Plot", fileName)) 49 | index = 0 50 | rows = int(len(names) / 4) 51 | cols = int((len(names) / rows) + (len(names) % rows)) 52 | xax = None 53 | for row in range(rows): 54 | for col in range(cols): 55 | if(index < len(names)): 56 | values = data[index::len(names)] 57 | if xax: 58 | ax = pyplot.subplot2grid((rows,cols), (row,col), sharex=xax) 59 | else: 60 | ax = pyplot.subplot2grid((rows,cols), (row,col)) 61 | xax = ax 62 | ax.plot(range(len(values)), values, label = names[index]) 63 | ax.grid(True) 64 | ax.set_title(names[index], fontsize=7) 65 | ax.tick_params(labelsize=6) 66 | index += 1 67 | pyplot.show() 68 | -------------------------------------------------------------------------------- /unit_test/qformat/Makefile: -------------------------------------------------------------------------------- 1 | # Simple Makefile for the q-format unit tests 2 | # Enable C++ iostreams 3 | qformat_unit_test: qformat_unit_test.cpp 4 | # Make an output dir to hold the executable 5 | mkdir -p ~/qformat 6 | # Build the unit tests 7 | g++ -g -Wall -o ~/qformat/qformat_unit_test qformat_unit_test.cpp -DENABLE_OSTREAMS -I../../cpp -I${BOOST_HOME} 8 | 9 | # Remove all object files 10 | clean: 11 | rm *.o -------------------------------------------------------------------------------- /unit_test/qlearn/Makefile: -------------------------------------------------------------------------------- 1 | # Simple Makefile for the q-learn unit tests 2 | # Enable C++ iostreams 3 | # Tell the code to compile the Q16.16 tanh activation function LUT 4 | qlearn_unit_test: qlearn_unit_test.cpp ../../cpp/lookupTables.cpp 5 | # Make an output dir to hold the executable 6 | mkdir -p ~/qlearn 7 | # Build the unit tests 8 | g++ -g -Wall -o ~/qlearn/qlearn_unit_test qlearn_unit_test.cpp ../../cpp/lookupTables.cpp -DENABLE_OSTREAMS -I../../cpp -I../../apps/include -DTINYMIND_USE_TANH_16_16 -I${BOOST_HOME} 9 | 10 | # Remove all object files 11 | clean: 12 | rm *.o --------------------------------------------------------------------------------