├── .gitattributes ├── .gitignore ├── LICENSE.md ├── README.md ├── SharpLearning.Examples.Integration.sln ├── SharpLearning.Examples.sln ├── azure-pipelines.yaml └── src ├── SharpLearning.Examples.Integration ├── CntkIntegrations.cs ├── DataSetUtilities.cs ├── Properties │ └── AssemblyInfo.cs ├── SharpLearning.Examples.Integration.csproj ├── app.config └── packages.config └── SharpLearning.Examples ├── Containers └── MatrixExamples.cs ├── CrossValidation ├── CrossValidationExamples.cs └── TrainTestSplitExamples.cs ├── FeatureTransformations └── FeatureNormalizationExample.cs ├── Guides ├── EnsembleLearningGuide.cs ├── HyperparameterTuningGuide.cs └── IntroductionGuide.cs ├── InputOutput └── CsvParserExamples.cs ├── Learners ├── ClassificationLearnerExamples.cs └── RegressionLearnerExamples.cs ├── LearningCurves └── LearningCurvesExamples.cs ├── Metrics ├── ClassificationMetricExamples.cs └── RegressionMetricExamples.cs ├── ModelSelection └── ClassificationFindBestDefaultModelExample.cs ├── NeuralNets ├── BatchNormalizationExamples.cs ├── ClassificationNeuralNetExamples.cs ├── OptimizerExamples.cs ├── RegressionNeuralNetExamples.cs └── ValidationSetForSelectingBestIterationExamples.cs ├── Optimization └── HyperparameterTuningExamples.cs ├── Properties ├── AssemblyInfo.cs ├── Resources.Designer.cs └── Resources.resx ├── Resources ├── cifar10_test_small.csv ├── cifar10_train_small.csv ├── mnist_small_test.csv ├── mnist_small_train.csv └── winequality-white.csv ├── SharpLearning.Examples.csproj └── packages.config /.gitattributes: -------------------------------------------------------------------------------- 1 | * text=auto 2 | 3 | *.cs text diff=csharp 4 | *.msbuild text 5 | *.config text 6 | *.conf text 7 | *.cscfg text 8 | *.csdef text 9 | *.xml text 10 | *.js text 11 | *.ps1 text 12 | 13 | *.DotSettings text eol=lf 14 | *.sh text eol=lf 15 | 16 | *.csproj text merge=union 17 | *.fsproj text merge=union 18 | *.sln text eol=crlf merge=union 19 | 20 | *.jpg binary 21 | *.png binary 22 | *.gif binary 23 | 24 | *.doc diff=astextplain 25 | *.DOC diff=astextplain 26 | *.docx diff=astextplain 27 | *.DOCX diff=astextplain 28 | *.pdf diff=astextplain 29 | *.PDF diff=astextplain 30 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Build Directories 2 | Build/ 3 | BuildTest/ 4 | packages/ 5 | nupkgs/ 6 | bin/ 7 | obj/ 8 | out/ 9 | 10 | # User-specific Files 11 | *.user 12 | *.suo 13 | *.sln.docstates 14 | *.userprefs 15 | .vagrant 16 | .vs 17 | 18 | # Test & Analysis Results 19 | [Tt]est[Rr]esult*/ 20 | [Bb]uild[Ll]og.* 21 | *.VisualState.xml 22 | TestResult.xml 23 | TestResults.xml 24 | coverage.* 25 | *.dotCover 26 | *.psess 27 | *.vsp 28 | *.vspx 29 | *.lnt 30 | *.svclog 31 | *.log 32 | 33 | # Caches 34 | _ReSharper* 35 | _TeamCity* 36 | *.[Rr]e[Ss]harper 37 | *.ide/ 38 | *.[Cc]ache 39 | *.ncb 40 | *.sdf 41 | *.aps 42 | *.opensdf 43 | *.opendb 44 | *.cachefile 45 | *.swp 46 | 47 | # Upgrades 48 | _UpgradeReport_Files/ 49 | Backup*/ 50 | UpgradeLog*.XML 51 | UpgradeLog*.htm 52 | *.bak 53 | *.orig 54 | 55 | # OS 56 | $RECYCLE.BIN/ 57 | .DS_Store 58 | Thumbs.db 59 | ehthumbs.db 60 | Desktop.ini 61 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | COPYRIGHT 2 | 3 | All contributions by Mads Dabros: 4 | Copyright (c) 2014, Mads Dabros. 5 | All rights reserved. 6 | 7 | All other contributions: 8 | Copyright (c) 2017, the respective contributors. 9 | All rights reserved. 10 | 11 | Each contributor holds copyright over their respective contributions. 12 | The project versioning (Git) records all such contribution source information. 13 | 14 | LICENSE 15 | 16 | The MIT License (MIT) 17 | 18 | Permission is hereby granted, free of charge, to any person obtaining a copy 19 | of this software and associated documentation files (the "Software"), to deal 20 | in the Software without restriction, including without limitation the rights 21 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 22 | copies of the Software, and to permit persons to whom the Software is 23 | furnished to do so, subject to the following conditions: 24 | 25 | The above copyright notice and this permission notice shall be included in all 26 | copies or substantial portions of the Software. 27 | 28 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 29 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 30 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 31 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 32 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 33 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 34 | SOFTWARE. 35 | 36 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | SharpLearning.Examples 2 | ====================== 3 | [![Build Status](https://machinelearning.visualstudio.com/sharplearning-github-build/_apis/build/status/SharpLearning.Examples-CI?branchName=master)](https://machinelearning.visualstudio.com/sharplearning-github-build/_build/latest?definitionId=30&branchName=master) 4 | 5 | SharpLearning.Examples contains code examples to show how to use the machine learning library [SharpLearning](https://github.com/mdabros/SharpLearning) 6 | 7 | The examples are structured in a unittest project, where each test is a separate example. 8 | There are no assertions and the unit test framework is only used as an easy way of running the examples. 9 | Information about how to get started can be found in the SharpLearning [wiki](https://github.com/mdabros/SharpLearning/wiki) 10 | 11 | An example showing how to read data, create a RegressionDecisionTreeLearner and learn a 12 | RegressionDecisionTreeModel can be seen below: 13 | 14 | ```c# 15 | [TestMethod] 16 | public void RegressionLearner_Learn_And_Predict() 17 | { 18 | // Use StreamReader(filepath) when running from filesystem 19 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 20 | var targetName = "quality"; 21 | 22 | // read feature matrix (all columns except quality) 23 | var observations = parser.EnumerateRows(c => c != targetName) 24 | .ToF64Matrix(); 25 | 26 | // read regression targets 27 | var targets = parser.EnumerateRows(targetName) 28 | .ToF64Vector(); 29 | 30 | // create learner 31 | var learner = new RegressionDecisionTreeLearner(maximumTreeDepth: 5); 32 | 33 | // learns a RegressionDecisionTreeModel 34 | var model = learner.Learn(observations, targets); 35 | 36 | // use the model to predict the training data 37 | var predictions = model.Predict(observations); 38 | } 39 | ``` 40 | 41 | Currently there are only examples showing general usage of the learners and models. 42 | Specific examples for the individual learner/model types is planned. However, except from different hyperparameters, 43 | the the individual learner types are used in exactly the same way. Same example as above but with a RandomForestLearner instead of a DecisionTreeLearner: 44 | 45 | 46 | ```c# 47 | [TestMethod] 48 | public void RegressionLearner_Learn_And_Predict() 49 | { 50 | // Use StreamReader(filepath) when running from filesystem 51 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 52 | var targetName = "quality"; 53 | 54 | // read feature matrix (all columns except quality) 55 | var observations = parser.EnumerateRows(c => c != targetName) 56 | .ToF64Matrix(); 57 | 58 | // read regression targets 59 | var targets = parser.EnumerateRows(targetName) 60 | .ToF64Vector(); 61 | 62 | // create learner (RandomForest) 63 | var learner = new RegressionRandomForestLearner(trees: 100); 64 | 65 | // learns a RegressionRandomForestTreeModel 66 | var model = learner.Learn(observations, targets); 67 | 68 | // use the model to predict the training data 69 | var predictions = model.Predict(observations); 70 | } 71 | ``` 72 | 73 | Datasets 74 | --------- 75 | SharpLearning.Examples contains 3 datasets in csv format for use with the examples: 76 | 77 | **MNIST_SMALL** 78 | - *mnist_small_train* - 1000 observations from the original MNIST training data set 79 | - *mnist_small_test* - 1000 observations from the original MNIST test data set 80 | 81 | The original and complete dataset can be found here: 82 | http://yann.lecun.com/exdb/mnist/ 83 | 84 | **CIFAR10_SMALL** 85 | - *cifar10_train_small* - 1000 observations from the original CIFAR10 training data set 86 | - *cifar10_test_small* - 1000 observations from the original CIFAR10 test data set 87 | 88 | The original and complete dataset can be found here: 89 | https://www.cs.toronto.edu/~kriz/cifar.html 90 | 91 | **Wine Quality Data Set** 92 | - *winequality-white* - the complete white wine quality dataset 93 | 94 | The original dataset can be found here: 95 | http://archive.ics.uci.edu/ml/datasets/Wine+Quality 96 | 97 | Installation 98 | ------------ 99 | 100 | Installation instructions for SharpLearning are availible in the main SharpLearning repository: 101 | [SharpLearning](https://github.com/mdabros/SharpLearning) 102 | 103 | -------------------------------------------------------------------------------- /SharpLearning.Examples.Integration.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio 15 4 | VisualStudioVersion = 15.0.28307.705 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SharpLearning.Examples.Integration", "src\SharpLearning.Examples.Integration\SharpLearning.Examples.Integration.csproj", "{1F54BD04-7F1B-4577-BEBF-E6CB123CD483}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|x64 = Debug|x64 11 | Release|x64 = Release|x64 12 | EndGlobalSection 13 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 14 | {1F54BD04-7F1B-4577-BEBF-E6CB123CD483}.Debug|x64.ActiveCfg = Debug|x64 15 | {1F54BD04-7F1B-4577-BEBF-E6CB123CD483}.Debug|x64.Build.0 = Debug|x64 16 | {1F54BD04-7F1B-4577-BEBF-E6CB123CD483}.Release|x64.ActiveCfg = Release|x64 17 | {1F54BD04-7F1B-4577-BEBF-E6CB123CD483}.Release|x64.Build.0 = Release|x64 18 | EndGlobalSection 19 | GlobalSection(SolutionProperties) = preSolution 20 | HideSolutionNode = FALSE 21 | EndGlobalSection 22 | GlobalSection(ExtensibilityGlobals) = postSolution 23 | SolutionGuid = {F5959C23-A726-4316-A910-CDE015550AFB} 24 | EndGlobalSection 25 | EndGlobal 26 | -------------------------------------------------------------------------------- /SharpLearning.Examples.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio 15 4 | VisualStudioVersion = 15.0.28307.705 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{DDD13518-91F1-4239-977E-133B073D5CC0}" 7 | ProjectSection(SolutionItems) = preProject 8 | azure-pipelines.yaml = azure-pipelines.yaml 9 | README.md = README.md 10 | EndProjectSection 11 | EndProject 12 | Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SharpLearning.Examples", "src\SharpLearning.Examples\SharpLearning.Examples.csproj", "{D730F7CD-AAA3-4702-9A23-16435A7944D3}" 13 | EndProject 14 | Global 15 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 16 | Debug|Any CPU = Debug|Any CPU 17 | Release|Any CPU = Release|Any CPU 18 | EndGlobalSection 19 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 20 | {D730F7CD-AAA3-4702-9A23-16435A7944D3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 21 | {D730F7CD-AAA3-4702-9A23-16435A7944D3}.Debug|Any CPU.Build.0 = Debug|Any CPU 22 | {D730F7CD-AAA3-4702-9A23-16435A7944D3}.Release|Any CPU.ActiveCfg = Release|Any CPU 23 | {D730F7CD-AAA3-4702-9A23-16435A7944D3}.Release|Any CPU.Build.0 = Release|Any CPU 24 | EndGlobalSection 25 | GlobalSection(SolutionProperties) = preSolution 26 | HideSolutionNode = FALSE 27 | EndGlobalSection 28 | GlobalSection(ExtensibilityGlobals) = postSolution 29 | SolutionGuid = {E539FD1A-CF46-492E-ADBD-0984C400A4CE} 30 | EndGlobalSection 31 | EndGlobal 32 | -------------------------------------------------------------------------------- /azure-pipelines.yaml: -------------------------------------------------------------------------------- 1 | trigger: 2 | - master 3 | 4 | # Pull request validation against master branch only. 5 | pr: 6 | - master 7 | 8 | variables: 9 | buildPlatform: # Use default. 10 | buildConfiguration: 'Release' 11 | testAssembliesSearchPattern: 'Build\**\SharpLearning.Examples.dll' 12 | 13 | jobs: 14 | - job: 'CI' 15 | displayName: 'CI' 16 | pool: 17 | name: Hosted VS2017 18 | 19 | steps: 20 | - task: NuGetCommand@2 21 | displayName: 'NuGet Restore' 22 | inputs: 23 | restoreSolution: '**\*.sln' 24 | nugetConfigPath: nuget.config 25 | 26 | - task: VSBuild@1 27 | displayName: 'Build solution **\*.sln' 28 | inputs: 29 | vsVersion: 15.0 30 | platform: '$(buildPlatform)' 31 | configuration: '$(buildConfiguration)' 32 | 33 | - task: VisualStudioTestPlatformInstaller@1 34 | displayName: 'Visual Studio Test Platform Installer' 35 | 36 | - task: VSTest@2 37 | displayName: 'Test Assemblies' 38 | inputs: 39 | testAssemblyVer2: | 40 | $(testAssembliesSearchPattern) 41 | !**\*TestAdapter*.dll 42 | !**\*.Testing.dll 43 | !**\*IntegrationTests*.dll 44 | !**\*TestFramework*.dll 45 | !**\obj\** 46 | vsTestVersion: toolsInstaller 47 | runInParallel: false 48 | runTestsInIsolation: true 49 | codeCoverageEnabled: true 50 | configuration: '$(buildConfiguration)' -------------------------------------------------------------------------------- /src/SharpLearning.Examples.Integration/CntkIntegrations.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Collections.Generic; 3 | using System.Diagnostics; 4 | using System.Linq; 5 | using CNTK; 6 | using CntkCatalyst; 7 | using CntkCatalyst.LayerFunctions; 8 | using Microsoft.VisualStudio.TestTools.UnitTesting; 9 | using SharpLearning.CrossValidation; 10 | using SharpLearning.CrossValidation.CrossValidators; 11 | using SharpLearning.CrossValidation.Samplers; 12 | using SharpLearning.Examples.Integration; 13 | using SharpLearning.FeatureTransformations.MatrixTransforms; 14 | using SharpLearning.Metrics.Regression; 15 | using SharpLearning.Neural; 16 | using SharpLearning.Neural.Activations; 17 | using SharpLearning.Neural.Layers; 18 | using SharpLearning.Neural.Learners; 19 | using SharpLearning.Neural.Loss; 20 | using SharpLearning.Neural.Optimizers; 21 | 22 | namespace SharpLearning.Examples.IntegrationWithOtherMLPackages 23 | { 24 | [TestClass] 25 | public class CntkIntegrations 26 | { 27 | /// 28 | /// Pure SharpLearning Example for comparison. 29 | /// 30 | [TestMethod] 31 | public void SharpLearning_Example() 32 | { 33 | // Load data 34 | var (observations, targets) = DataSetUtilities.LoadWinequalityWhite(); 35 | 36 | // transform data for neural net 37 | var transform = new MinMaxTransformer(0.0, 1.0); 38 | transform.Transform(observations, observations); 39 | 40 | var featureCount = observations.ColumnCount; 41 | 42 | // define the neural net. 43 | var net = new NeuralNet(); 44 | net.Add(new InputLayer(inputUnits: featureCount)); 45 | net.Add(new DenseLayer(32, Activation.Relu)); 46 | net.Add(new DenseLayer(32, Activation.Relu)); 47 | net.Add(new SquaredErrorRegressionLayer()); 48 | 49 | // using only 10 iteration to make the example run faster. 50 | // using square error as error metric. This is only used for reporting progress. 51 | var learner = new RegressionNeuralNetLearner(net, iterations: 10, loss: new SquareLoss(), 52 | optimizerMethod: OptimizerMethod.Adam); 53 | 54 | var cv = new RandomCrossValidation(10, seed: 232); 55 | var predictions = cv.CrossValidate(learner, observations, targets); 56 | 57 | Trace.WriteLine(FormatErrorString(targets, predictions)); 58 | } 59 | 60 | /// 61 | /// Note that this uses the CntkCatalyst package, which adds layers, and model API to CNTK C#. 62 | /// Ongoing effort to refactor/add SharpLearning utilities to make integration with CNTK, and other ML frameworks, easier. 63 | /// This also serve to improve the CntkCatalyst extensions. 64 | /// 65 | [TestMethod] 66 | public void SharpLearning_With_Cntk_Example() 67 | { 68 | // Load data 69 | var (observations, targets) = DataSetUtilities.LoadWinequalityWhite(); 70 | 71 | // transform data for neural net 72 | var transform = new MinMaxTransformer(0.0, 1.0); 73 | transform.Transform(observations, observations); 74 | 75 | var featureCount = observations.ColumnCount; 76 | var observationCount = observations.RowCount; 77 | var targetCount = 1; 78 | 79 | var inputShape = new int[] { featureCount, 1 }; 80 | var outputShape = new int[] { targetCount }; 81 | 82 | // Convert data to float, and wrap as minibatch data. 83 | var observationsFloat = observations.Data().Select(v => (float)v).ToArray(); 84 | var observationsData = new MemoryMinibatchData(observationsFloat, inputShape, observationCount); 85 | var targetsFloat = targets.Select(v => (float)v).ToArray(); 86 | var targetsData = new MemoryMinibatchData(targetsFloat, outputShape, observationCount); 87 | 88 | var dataType = DataType.Float; 89 | var device = DeviceDescriptor.CPUDevice; 90 | 91 | // setup input and target variables. 92 | var inputVariable = Layers.Input(inputShape, dataType); 93 | var targetVariable = Variable.InputVariable(outputShape, dataType); 94 | 95 | // setup name to variable 96 | var nameToVariable = new Dictionary 97 | { 98 | { "observations", inputVariable }, 99 | { "targets", targetVariable }, 100 | }; 101 | 102 | // Get cross validation folds. 103 | var sampler = new RandomIndexSampler(seed: 24); 104 | var crossValidationIndexSets = CrossValidationUtilities 105 | .GetKFoldCrossValidationIndexSets(sampler, foldCount: 10, targets: targets); 106 | var predictions = new double[observationCount]; 107 | 108 | // Run cross validation loop. 109 | foreach (var set in crossValidationIndexSets) 110 | { 111 | // setup data. 112 | var trainingNameToData = new Dictionary 113 | { 114 | { "observations", observationsData.GetSamples(set.trainingIndices) }, 115 | { "targets", targetsData.GetSamples(set.trainingIndices) } 116 | }; 117 | 118 | var validationNameToData = new Dictionary 119 | { 120 | { "observations", observationsData.GetSamples(set.validationIndices) }, 121 | { "targets", targetsData.GetSamples(set.validationIndices) } 122 | }; 123 | 124 | var trainSource = new MemoryMinibatchSource(nameToVariable, trainingNameToData, seed: 232, randomize: true); 125 | var validationSource = new MemoryMinibatchSource(nameToVariable, validationNameToData, seed: 232, randomize: false); 126 | 127 | // Create model and fit. 128 | var model = CreateModel(inputVariable, targetVariable, targetCount, dataType, device); 129 | model.Fit(trainSource, batchSize: 128, epochs: 10); 130 | 131 | // Predict. 132 | var predictionsRaw = model.Predict(validationSource); 133 | var currentPredictions = predictionsRaw.Select(v => (double)v.Single()).ToArray(); 134 | 135 | // set cross-validation predictions 136 | var validationIndices = set.validationIndices; 137 | for (int i = 0; i < validationIndices.Length; i++) 138 | { 139 | predictions[validationIndices[i]] = currentPredictions[i]; 140 | } 141 | } 142 | 143 | Trace.WriteLine(FormatErrorString(targets, predictions)); 144 | } 145 | 146 | static Model CreateModel(Function inputVariable, Variable targetVariable, int targetCount, 147 | DataType dataType, DeviceDescriptor device) 148 | { 149 | var random = new Random(232); 150 | Func weightInit = () => Initializers.GlorotNormal(random.Next()); 151 | var biasInit = Initializers.Zero(); 152 | 153 | // Create the architecture. 154 | var network = inputVariable 155 | 156 | .Dense(32, weightInit(), biasInit, device, dataType) 157 | .ReLU() 158 | .Dense(32, weightInit(), biasInit, device, dataType) 159 | .ReLU() 160 | .Dense(targetCount, weightInit(), biasInit, device, dataType); 161 | 162 | // loss 163 | var lossFunc = Losses.MeanSquaredError(network.Output, targetVariable); 164 | var metricFunc = Losses.MeanAbsoluteError(network.Output, targetVariable); 165 | 166 | // setup trainer. 167 | var learner = CntkCatalyst.Learners.Adam(network.Parameters()); 168 | var trainer = CNTKLib.CreateTrainer(network, lossFunc, metricFunc, new LearnerVector { learner }); 169 | 170 | var model = new Model(trainer, network, dataType, device); 171 | 172 | Trace.WriteLine(model.Summary()); 173 | return model; 174 | } 175 | 176 | string FormatErrorString(double[] targets, double[] predictions) 177 | { 178 | var metric = new MeanSquaredErrorRegressionMetric(); 179 | var error = metric.Error(targets, predictions); 180 | return $"MSE: {error}"; 181 | } 182 | } 183 | } 184 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples.Integration/Properties/AssemblyInfo.cs: -------------------------------------------------------------------------------- 1 | using System.Reflection; 2 | using System.Runtime.CompilerServices; 3 | using System.Runtime.InteropServices; 4 | 5 | [assembly: AssemblyTitle("SharpLearning.Examples.Integration")] 6 | [assembly: AssemblyDescription("")] 7 | [assembly: AssemblyConfiguration("")] 8 | [assembly: AssemblyCompany("")] 9 | [assembly: AssemblyProduct("SharpLearning.Examples.Integration")] 10 | [assembly: AssemblyCopyright("Copyright © 2019")] 11 | [assembly: AssemblyTrademark("")] 12 | [assembly: AssemblyCulture("")] 13 | 14 | [assembly: ComVisible(false)] 15 | 16 | [assembly: Guid("1f54bd04-7f1b-4577-bebf-e6cb123cd483")] 17 | 18 | // [assembly: AssemblyVersion("1.0.*")] 19 | [assembly: AssemblyVersion("0.2.0.0")] 20 | [assembly: AssemblyFileVersion("0.2.0.0")] 21 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples.Integration/SharpLearning.Examples.Integration.csproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | Debug 7 | AnyCPU 8 | {1F54BD04-7F1B-4577-BEBF-E6CB123CD483} 9 | Library 10 | Properties 11 | SharpLearning.Examples.Integration 12 | SharpLearning.Examples.Integration 13 | v4.6.1 14 | 512 15 | {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} 16 | 15.0 17 | $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) 18 | $(ProgramFiles)\Common Files\microsoft shared\VSTT\$(VisualStudioVersion)\UITestExtensionPackages 19 | False 20 | UnitTest 21 | 22 | 23 | 24 | 25 | true 26 | ..\..\Build\Debug\ 27 | DEBUG;TRACE 28 | full 29 | x64 30 | prompt 31 | MinimumRecommendedRules.ruleset 32 | 33 | 34 | ..\..\Build\Release\ 35 | TRACE 36 | true 37 | pdbonly 38 | x64 39 | prompt 40 | MinimumRecommendedRules.ruleset 41 | 42 | 43 | 44 | ..\..\packages\CNTK.GPU.2.6.0\lib\netstandard2.0\Cntk.Core.Managed-2.6.dll 45 | 46 | 47 | ..\..\packages\CntkCatalyst.0.1.0-alpha.11\lib\net461\CntkCatalyst.dll 48 | 49 | 50 | ..\..\packages\MathNet.Numerics.4.8.1\lib\net461\MathNet.Numerics.dll 51 | 52 | 53 | ..\..\packages\MSTest.TestFramework.1.3.2\lib\net45\Microsoft.VisualStudio.TestPlatform.TestFramework.dll 54 | 55 | 56 | ..\..\packages\MSTest.TestFramework.1.3.2\lib\net45\Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions.dll 57 | 58 | 59 | ..\..\packages\SharpLearning.Common.Interfaces.0.31.4\lib\net461\SharpLearning.Common.Interfaces.dll 60 | 61 | 62 | ..\..\packages\SharpLearning.Containers.0.31.4\lib\net461\SharpLearning.Containers.dll 63 | 64 | 65 | ..\..\packages\SharpLearning.CrossValidation.0.31.4\lib\net461\SharpLearning.CrossValidation.dll 66 | 67 | 68 | ..\..\packages\SharpLearning.FeatureTransformations.0.31.4\lib\net461\SharpLearning.FeatureTransformations.dll 69 | 70 | 71 | ..\..\packages\SharpLearning.InputOutput.0.31.4\lib\net461\SharpLearning.InputOutput.dll 72 | 73 | 74 | ..\..\packages\SharpLearning.Metrics.0.31.4\lib\net461\SharpLearning.Metrics.dll 75 | 76 | 77 | ..\..\packages\SharpLearning.Neural.0.31.4\lib\net461\SharpLearning.Neural.dll 78 | 79 | 80 | 81 | 82 | 83 | 84 | ..\..\packages\System.ValueTuple.4.5.0\lib\net461\System.ValueTuple.dll 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples.Integration/app.config: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples.Integration/packages.config: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Containers/MatrixExamples.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using Microsoft.VisualStudio.TestTools.UnitTesting; 3 | using SharpLearning.Containers.Matrices; 4 | 5 | namespace SharpLearning.Examples.Containers 6 | { 7 | [TestClass] 8 | public class MatrixExamples 9 | { 10 | [TestMethod] 11 | public void F64Matrix_Create() 12 | { 13 | // matrix is created row wise 14 | var matrix = new F64Matrix(new double[] { 1, 2, 3, 15 | 4, 5, 6 }, 2, 3); 16 | 17 | // dimensions 18 | Assert.AreEqual(2, matrix.RowCount); 19 | Assert.AreEqual(3, matrix.ColumnCount); 20 | } 21 | 22 | [TestMethod] 23 | public void F64Matrix_GetRow() 24 | { 25 | // matrix is created row wise 26 | var matrix = new F64Matrix(new double[] { 1, 2, 3, 27 | 4, 5, 6 }, 2, 3); 28 | // returns the row as an array 29 | var row = matrix.Row(1); // [4, 5, 6] 30 | } 31 | 32 | [TestMethod] 33 | public void F64Matrix_GetColumn() 34 | { 35 | // matrix is created row wise 36 | var matrix = new F64Matrix(new double[] { 1, 2, 3, 37 | 4, 5, 6 }, 2, 3); 38 | // returns the column as an array 39 | var column = matrix.Column(1); // [2, 5] 40 | } 41 | 42 | [TestMethod] 43 | public void F64Matrix_GetRows() 44 | { 45 | // matrix is created row wise 46 | var matrix = new F64Matrix(new double[] { 1, 2, 3, 47 | 4, 5, 6, 48 | 7, 8, 9}, 3, 3); 49 | 50 | // returns selected rows a a new matrix 51 | var rows = matrix.Rows(new int[] { 0, 2 }); // [1, 2, 3, 52 | // 7, 8, 9] 53 | } 54 | 55 | [TestMethod] 56 | public void F64Matrix_GetColumns() 57 | { 58 | // matrix is created row wise 59 | var matrix = new F64Matrix(new double[] { 1, 2, 3, 60 | 4, 5, 6, 61 | 7, 8, 9}, 3, 3); 62 | 63 | // returns selected columns a a new matrix 64 | var cols = matrix.Columns(new int[] { 0, 2 }); // [1, 3, 65 | // 4, 6 66 | // 7, 9] 67 | } 68 | } 69 | } 70 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/CrossValidation/CrossValidationExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using Microsoft.VisualStudio.TestTools.UnitTesting; 4 | using SharpLearning.Containers; 5 | using SharpLearning.CrossValidation.CrossValidators; 6 | using SharpLearning.DecisionTrees.Learners; 7 | using SharpLearning.Examples.Properties; 8 | using SharpLearning.InputOutput.Csv; 9 | using SharpLearning.Metrics.Classification; 10 | using SharpLearning.Metrics.Regression; 11 | 12 | namespace SharpLearning.Examples.CrossValidation 13 | { 14 | [TestClass] 15 | public class CrossValidationExamples 16 | { 17 | [TestMethod] 18 | public void CrossValidation_CrossValidate() 19 | { 20 | #region Read data 21 | 22 | // Use StreamReader(filepath) when running from filesystem 23 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 24 | var targetName = "quality"; 25 | 26 | // read feature matrix (all columns different from the targetName) 27 | var observations = parser.EnumerateRows(c => c != targetName) 28 | .ToF64Matrix(); 29 | 30 | // read targets 31 | var targets = parser.EnumerateRows(targetName) 32 | .ToF64Vector(); 33 | 34 | #endregion 35 | 36 | // creates cross validator, observations are shuffled randomly 37 | var cv = new RandomCrossValidation(crossValidationFolds: 5, seed: 42); 38 | 39 | // create learner 40 | var learner = new RegressionDecisionTreeLearner(); 41 | 42 | // cross-validated predictions 43 | var cvPredictions = cv.CrossValidate(learner, observations, targets); 44 | 45 | // metric for measuring model error 46 | var metric = new MeanSquaredErrorRegressionMetric(); 47 | 48 | // cross-validation provides an estimate on how the model will perform on unseen data 49 | Trace.WriteLine("Cross-validation error: " + metric.Error(targets, cvPredictions)); 50 | 51 | // train and predict training set for comparison. 52 | var predictions = learner.Learn(observations, targets).Predict(observations); 53 | 54 | // The training set is NOT a good estimate of how well the model will perfrom on unseen data. 55 | Trace.WriteLine("Training error: " + metric.Error(targets, predictions)); 56 | } 57 | 58 | [TestMethod] 59 | public void CrossValidation_CrossValidate_ProbabilityPredictions() 60 | { 61 | #region Read data 62 | 63 | // Use StreamReader(filepath) when running from filesystem 64 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 65 | var targetName = "quality"; 66 | 67 | // read feature matrix (all columns different from the targetName) 68 | var observations = parser.EnumerateRows(c => c != targetName) 69 | .ToF64Matrix(); 70 | 71 | // read targets 72 | var targets = parser.EnumerateRows(targetName) 73 | .ToF64Vector(); 74 | 75 | #endregion 76 | 77 | // creates cross validator, observations are shuffled randomly 78 | var cv = new RandomCrossValidation(crossValidationFolds: 5, seed: 42); 79 | 80 | // create learner 81 | var learner = new ClassificationDecisionTreeLearner(); 82 | 83 | // cross-validated predictions 84 | var cvPredictions = cv.CrossValidate(learner, observations, targets); 85 | 86 | // metric for measuring model error 87 | var metric = new LogLossClassificationProbabilityMetric(); 88 | 89 | // cross-validation provides an estimate on how the model will perform on unseen data 90 | Trace.WriteLine("Cross-validation error: " + metric.Error(targets, cvPredictions)); 91 | 92 | // train and predict training set for comparison 93 | var predictions = learner.Learn(observations, targets).PredictProbability(observations); 94 | 95 | // The training set is NOT a good estimate of how well the model will perfrom on unseen data. 96 | Trace.WriteLine("Training error: " + metric.Error(targets, predictions)); 97 | } 98 | } 99 | } 100 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/CrossValidation/TrainTestSplitExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using Microsoft.VisualStudio.TestTools.UnitTesting; 4 | using SharpLearning.CrossValidation.TrainingTestSplitters; 5 | using SharpLearning.DecisionTrees.Learners; 6 | using SharpLearning.Examples.Properties; 7 | using SharpLearning.InputOutput.Csv; 8 | using SharpLearning.Metrics.Regression; 9 | 10 | namespace SharpLearning.Examples.CrossValidation 11 | { 12 | [TestClass] 13 | public class TrainTestSplitExamples 14 | { 15 | [TestMethod] 16 | public void TrainingTestSplitter_SplitSet() 17 | { 18 | #region Read data 19 | 20 | // Use StreamReader(filepath) when running from filesystem 21 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 22 | var targetName = "quality"; 23 | 24 | // read feature matrix (all columns different from the targetName) 25 | var observations = parser.EnumerateRows(c => c != targetName) 26 | .ToF64Matrix(); 27 | 28 | // read targets 29 | var targets = parser.EnumerateRows(targetName) 30 | .ToF64Vector(); 31 | 32 | #endregion 33 | 34 | // creates training test splitter, observations are shuffled randomly 35 | var splitter = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 36 | 37 | var trainingTestSplit = splitter.SplitSet(observations, targets); 38 | var trainingSet = trainingTestSplit.TrainingSet; 39 | var testSet = trainingTestSplit.TestSet; 40 | 41 | var learner = new RegressionDecisionTreeLearner(); 42 | var model = learner.Learn(trainingSet.Observations, trainingSet.Targets); 43 | 44 | // predict test set 45 | var testPredictions = model.Predict(testSet.Observations); 46 | 47 | // metric for measuring model error 48 | var metric = new MeanSquaredErrorRegressionMetric(); 49 | 50 | // The test set provides an estimate on how the model will perform on unseen data 51 | Trace.WriteLine("Test error: " + metric.Error(testSet.Targets, testPredictions)); 52 | 53 | // predict training set for comparison 54 | var trainingPredictions = model.Predict(trainingSet.Observations); 55 | 56 | // The training set is NOT a good estimate of how well the model will perfrom on unseen data. 57 | Trace.WriteLine("Training error: " + metric.Error(trainingSet.Targets, trainingPredictions)); 58 | } 59 | } 60 | } 61 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/FeatureTransformations/FeatureNormalizationExample.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using System.Linq; 4 | using System.Text; 5 | using Microsoft.VisualStudio.TestTools.UnitTesting; 6 | using SharpLearning.Examples.Properties; 7 | using SharpLearning.FeatureTransformations.MatrixTransforms; 8 | using SharpLearning.InputOutput.Csv; 9 | using SharpLearning.InputOutput.Serialization; 10 | using SharpLearning.Neural; 11 | using SharpLearning.Neural.Layers; 12 | using SharpLearning.Neural.Learners; 13 | using SharpLearning.Neural.Loss; 14 | 15 | namespace SharpLearning.Examples.FeatureTransformations 16 | { 17 | [TestClass] 18 | public class FeatureNormalizationExample 19 | { 20 | [TestMethod] 21 | public void FeatureNormalization_Normalize() 22 | { 23 | // Use StreamReader(filepath) when running from filesystem 24 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 25 | var targetName = "quality"; 26 | 27 | // read feature matrix (all columns different from the targetName) 28 | var observations = parser.EnumerateRows(c => c != targetName) 29 | .ToF64Matrix(); 30 | 31 | // create minmax normalizer (normalizes each feature from 0.0 to 1.0) 32 | var minMaxTransformer = new MinMaxTransformer(0.0, 1.0); 33 | 34 | // transforms features using the feature normalization transform 35 | minMaxTransformer.Transform(observations, observations); 36 | 37 | // read targets 38 | var targets = parser.EnumerateRows(targetName) 39 | .ToF64Vector(); 40 | 41 | // Create neural net. 42 | var net = new NeuralNet(); 43 | net.Add(new InputLayer(observations.ColumnCount)); 44 | net.Add(new SquaredErrorRegressionLayer()); 45 | 46 | // Create regression learner. 47 | var learner = new RegressionNeuralNetLearner(net, new SquareLoss()); 48 | 49 | // learns a neural net regression model. 50 | var model = learner.Learn(observations, targets); 51 | 52 | // serializer for saving the MinMaxTransformer 53 | var serializer = new GenericXmlDataContractSerializer(); 54 | 55 | // Serialize transform for use with the model. 56 | // Replace this with StreamWriter for use with file system. 57 | var data = new StringBuilder(); 58 | var writer = new StringWriter(data); 59 | serializer.Serialize(minMaxTransformer, () => writer); 60 | 61 | // Deserialize transform for use with the model. 62 | // Replace this with StreamReader for use with file system. 63 | var reader = new StringReader(data.ToString()); 64 | var deserializedMinMaxTransform = serializer.Deserialize(() => reader); 65 | 66 | // Normalize observation and predict using the model. 67 | var normalizedObservation = deserializedMinMaxTransform.Transform(observations.Row(0)); 68 | var prediction = model.Predict(normalizedObservation); 69 | 70 | Trace.WriteLine($"Prediction: {prediction}"); 71 | } 72 | } 73 | } 74 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Guides/EnsembleLearningGuide.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using Microsoft.VisualStudio.TestTools.UnitTesting; 4 | using SharpLearning.AdaBoost.Learners; 5 | using SharpLearning.Common.Interfaces; 6 | using SharpLearning.Containers.Extensions; 7 | using SharpLearning.Containers.Matrices; 8 | using SharpLearning.CrossValidation.TrainingTestSplitters; 9 | using SharpLearning.Ensemble.Learners; 10 | using SharpLearning.Examples.Properties; 11 | using SharpLearning.GradientBoost.Learners; 12 | using SharpLearning.InputOutput.Csv; 13 | using SharpLearning.Metrics.Regression; 14 | using SharpLearning.RandomForest.Learners; 15 | 16 | namespace SharpLearning.Examples.Guides 17 | { 18 | [TestClass] 19 | public class EnsembleLearningGuide 20 | { 21 | [TestMethod] 22 | public void RegressionEnsembleLearner() 23 | { 24 | #region read and split data 25 | // Use StreamReader(filepath) when running from filesystem 26 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 27 | var targetName = "quality"; 28 | 29 | // read feature matrix 30 | var observations = parser.EnumerateRows(c => c != targetName) 31 | .ToF64Matrix(); 32 | 33 | // read regression targets 34 | var targets = parser.EnumerateRows(targetName) 35 | .ToF64Vector(); 36 | 37 | // creates training test splitter, 38 | // Since this is a regression problem, we use the random training/test set splitter. 39 | // 30 % of the data is used for the test set. 40 | var splitter = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 41 | 42 | var trainingTestSplit = splitter.SplitSet(observations, targets); 43 | var trainSet = trainingTestSplit.TrainingSet; 44 | var testSet = trainingTestSplit.TestSet; 45 | #endregion 46 | 47 | // create the list of learners to include in the ensemble 48 | var ensembleLearners = new IIndexedLearner[] 49 | { 50 | new RegressionAdaBoostLearner(maximumTreeDepth: 15), 51 | new RegressionRandomForestLearner(runParallel: false), 52 | new RegressionSquareLossGradientBoostLearner(iterations: 198, learningRate: 0.028, maximumTreeDepth: 12, 53 | subSampleRatio: 0.559, featuresPrSplit: 10, runParallel: false) 54 | 55 | }; 56 | 57 | // create the ensemble learner 58 | var learner = new RegressionEnsembleLearner(learners: ensembleLearners); 59 | 60 | // the ensemble learnr combines all the provided learners 61 | // into a single ensemble model. 62 | var model = learner.Learn(trainSet.Observations, trainSet.Targets); 63 | 64 | // predict the training and test set. 65 | var trainPredictions = model.Predict(trainSet.Observations); 66 | var testPredictions = model.Predict(testSet.Observations); 67 | 68 | // since this is a regression problem we are using square error as metric 69 | // for evaluating how well the model performs. 70 | var metric = new MeanSquaredErrorRegressionMetric(); 71 | 72 | // measure the error on training and test set. 73 | var trainError = metric.Error(trainSet.Targets, trainPredictions); 74 | var testError = metric.Error(testSet.Targets, testPredictions); 75 | 76 | // The ensemble model achieves a lower test error 77 | // then any of the individual models: 78 | 79 | // RegressionAdaBoostLearner: 0.4005 80 | // RegressionRandomForestLearner: 0.4037 81 | // RegressionSquareLossGradientBoostLearner: 0.3936 82 | TraceTrainingAndTestError(trainError, testError); 83 | } 84 | 85 | 86 | static void TraceTrainingAndTestError(double trainError, double testError) 87 | { 88 | Trace.WriteLine(string.Format("Train error: {0:0.0000} - Test error: {1:0.0000}", 89 | trainError, testError)); 90 | } 91 | 92 | } 93 | } 94 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Guides/HyperparameterTuningGuide.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Diagnostics; 3 | using System.IO; 4 | using Microsoft.VisualStudio.TestTools.UnitTesting; 5 | using SharpLearning.CrossValidation.TrainingTestSplitters; 6 | using SharpLearning.Examples.Properties; 7 | using SharpLearning.GradientBoost.Learners; 8 | using SharpLearning.InputOutput.Csv; 9 | using SharpLearning.Metrics.Regression; 10 | using SharpLearning.Optimization; 11 | 12 | namespace SharpLearning.Examples.Guides 13 | { 14 | [TestClass] 15 | public class HyperparameterTuningGuide 16 | { 17 | [TestMethod] 18 | public void GradientBoost_Default_Parameters() 19 | { 20 | #region read and split data 21 | // Use StreamReader(filepath) when running from filesystem 22 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 23 | var targetName = "quality"; 24 | 25 | // read feature matrix 26 | var observations = parser.EnumerateRows(c => c != targetName) 27 | .ToF64Matrix(); 28 | 29 | // read regression targets 30 | var targets = parser.EnumerateRows(targetName) 31 | .ToF64Vector(); 32 | 33 | // creates training test splitter, 34 | // Since this is a regression problem, we use the random training/test set splitter. 35 | // 30 % of the data is used for the test set. 36 | var splitter = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 37 | 38 | var trainingTestSplit = splitter.SplitSet(observations, targets); 39 | var trainSet = trainingTestSplit.TrainingSet; 40 | var testSet = trainingTestSplit.TestSet; 41 | #endregion 42 | 43 | // create learner with default parameters 44 | var learner = new RegressionSquareLossGradientBoostLearner(runParallel: false); 45 | 46 | // learn model with found parameters 47 | var model = learner.Learn(trainSet.Observations, trainSet.Targets); 48 | 49 | // predict the training and test set. 50 | var trainPredictions = model.Predict(trainSet.Observations); 51 | var testPredictions = model.Predict(testSet.Observations); 52 | 53 | // since this is a regression problem we are using square error as metric 54 | // for evaluating how well the model performs. 55 | var metric = new MeanSquaredErrorRegressionMetric(); 56 | 57 | // measure the error on training and test set. 58 | var trainError = metric.Error(trainSet.Targets, trainPredictions); 59 | var testError = metric.Error(testSet.Targets, testPredictions); 60 | 61 | TraceTrainingAndTestError(trainError, testError); 62 | } 63 | 64 | [TestMethod] 65 | public void GradientBoost_Optimize_Hyperparameters() 66 | { 67 | #region read and split data 68 | // Use StreamReader(filepath) when running from filesystem 69 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 70 | var targetName = "quality"; 71 | 72 | // read feature matrix 73 | var observations = parser.EnumerateRows(c => c != targetName) 74 | .ToF64Matrix(); 75 | 76 | // read regression targets 77 | var targets = parser.EnumerateRows(targetName) 78 | .ToF64Vector(); 79 | 80 | // creates training test splitter, 81 | // Since this is a regression problem, we use the random training/test set splitter. 82 | // 30 % of the data is used for the test set. 83 | var splitter = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 84 | 85 | var trainingTestSplit = splitter.SplitSet(observations, targets); 86 | var trainSet = trainingTestSplit.TrainingSet; 87 | var testSet = trainingTestSplit.TestSet; 88 | #endregion 89 | 90 | // since this is a regression problem we are using square error as metric 91 | // for evaluating how well the model performs. 92 | var metric = new MeanSquaredErrorRegressionMetric(); 93 | 94 | // Usually better results can be achieved by tuning a gradient boost learner 95 | 96 | var numberOfFeatures = trainSet.Observations.ColumnCount; 97 | 98 | // Parameter specs for the optimizer 99 | // best parameter to tune on random forest is featuresPrSplit. 100 | var parameters = new IParameterSpec[] 101 | { 102 | new MinMaxParameterSpec(min: 80, max: 300, 103 | transform: Transform.Linear, parameterType: ParameterType.Discrete), // iterations 104 | 105 | new MinMaxParameterSpec(min: 0.02, max: 0.2, 106 | transform: Transform.Log10, parameterType: ParameterType.Continuous), // learning rate 107 | 108 | new MinMaxParameterSpec(min: 8, max: 15, 109 | transform: Transform.Linear, parameterType: ParameterType.Discrete), // maximumTreeDepth 110 | 111 | new MinMaxParameterSpec(min: 0.5, max: 0.9, 112 | transform: Transform.Linear, parameterType: ParameterType.Continuous), // subSampleRatio 113 | 114 | new MinMaxParameterSpec(min: 1, max: numberOfFeatures, 115 | transform: Transform.Linear, parameterType: ParameterType.Discrete), // featuresPrSplit 116 | }; 117 | 118 | // Further split the training data to have a validation set to measure 119 | // how well the model generalizes to unseen data during the optimization. 120 | var validationSplit = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24) 121 | .SplitSet(trainSet.Observations, trainSet.Targets); 122 | 123 | 124 | // Define optimizer objective (function to minimize) 125 | Func minimize = p => 126 | { 127 | // create the candidate learner using the current optimization parameters. 128 | var candidateLearner = new RegressionSquareLossGradientBoostLearner( 129 | iterations: (int)p[0], 130 | learningRate: p[1], 131 | maximumTreeDepth: (int)p[2], 132 | subSampleRatio: p[3], 133 | featuresPrSplit: (int)p[4], 134 | runParallel: false); 135 | 136 | var candidateModel = candidateLearner.Learn(validationSplit.TrainingSet.Observations, 137 | validationSplit.TrainingSet.Targets); 138 | 139 | var validationPredictions = candidateModel.Predict(validationSplit.TestSet.Observations); 140 | var candidateError = metric.Error(validationSplit.TestSet.Targets, validationPredictions); 141 | 142 | // trace current error 143 | Trace.WriteLine(string.Format("Candidate Error: {0:0.0000}, Candidate Parameters: {1}", 144 | candidateError, string.Join(", ", p))); 145 | 146 | return new OptimizerResult(p, candidateError); 147 | }; 148 | 149 | // create random search optimizer 150 | var optimizer = new RandomSearchOptimizer(parameters, iterations: 30, runParallel: true); 151 | 152 | // find best hyperparameters 153 | var result = optimizer.OptimizeBest(minimize); 154 | var best = result.ParameterSet; 155 | 156 | // create the final learner using the best hyperparameters. 157 | var learner = new RegressionSquareLossGradientBoostLearner( 158 | iterations: (int)best[0], 159 | learningRate: best[1], 160 | maximumTreeDepth: (int)best[2], 161 | subSampleRatio: best[3], 162 | featuresPrSplit: (int)best[4], 163 | runParallel: false); 164 | 165 | // learn model with found parameters 166 | var model = learner.Learn(trainSet.Observations, trainSet.Targets); 167 | 168 | // predict the training and test set. 169 | var trainPredictions = model.Predict(trainSet.Observations); 170 | var testPredictions = model.Predict(testSet.Observations); 171 | 172 | // measure the error on training and test set. 173 | var trainError = metric.Error(trainSet.Targets, trainPredictions); 174 | var testError = metric.Error(testSet.Targets, testPredictions); 175 | 176 | // Optimizer found hyperparameters. 177 | Trace.WriteLine(string.Format("Found parameters, iterations: {0}, learning rate {1:0.000}: maximumTreeDepth: {2}, subSampleRatio {3:0.000}, featuresPrSplit: {4} ", 178 | (int)best[0], best[1], (int)best[2], best[3], (int)best[4])); 179 | TraceTrainingAndTestError(trainError, testError); 180 | } 181 | 182 | static void TraceTrainingAndTestError(double trainError, double testError) 183 | { 184 | Trace.WriteLine(string.Format("Train error: {0:0.0000} - Test error: {1:0.0000}", 185 | trainError, testError)); 186 | } 187 | } 188 | } 189 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Guides/IntroductionGuide.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using System.Linq; 4 | using System.Text; 5 | using Microsoft.VisualStudio.TestTools.UnitTesting; 6 | using SharpLearning.Common.Interfaces; 7 | using SharpLearning.CrossValidation.TrainingTestSplitters; 8 | using SharpLearning.Examples.Properties; 9 | using SharpLearning.InputOutput.Csv; 10 | using SharpLearning.InputOutput.Serialization; 11 | using SharpLearning.Metrics.Regression; 12 | using SharpLearning.RandomForest.Learners; 13 | using SharpLearning.RandomForest.Models; 14 | 15 | namespace SharpLearning.Examples.Guides 16 | { 17 | [TestClass] 18 | public class IntroductionGuide 19 | { 20 | 21 | [TestMethod] 22 | public void RandomForest_Default_Parameters() 23 | { 24 | // Use StreamReader(filepath) when running from filesystem 25 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 26 | var targetName = "quality"; 27 | 28 | // read feature matrix 29 | var observations = parser.EnumerateRows(c => c != targetName) 30 | .ToF64Matrix(); 31 | 32 | // read regression targets 33 | var targets = parser.EnumerateRows(targetName) 34 | .ToF64Vector(); 35 | 36 | // creates training test splitter, 37 | // Since this is a regression problem, we use the random training/test set splitter. 38 | // 30 % of the data is used for the test set. 39 | var splitter = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 40 | 41 | var trainingTestSplit = splitter.SplitSet(observations, targets); 42 | var trainSet = trainingTestSplit.TrainingSet; 43 | var testSet = trainingTestSplit.TestSet; 44 | 45 | // Create the learner and learn the model. 46 | var learner = new RegressionRandomForestLearner(trees: 100); 47 | var model = learner.Learn(trainSet.Observations, trainSet.Targets); 48 | 49 | // predict the training and test set. 50 | var trainPredictions = model.Predict(trainSet.Observations); 51 | var testPredictions = model.Predict(testSet.Observations); 52 | 53 | // since this is a regression problem we are using square error as metric 54 | // for evaluating how well the model performs. 55 | var metric = new MeanSquaredErrorRegressionMetric(); 56 | 57 | // measure the error on training and test set. 58 | var trainError = metric.Error(trainSet.Targets, trainPredictions); 59 | var testError = metric.Error(testSet.Targets, testPredictions); 60 | 61 | TraceTrainingAndTestError(trainError, testError); 62 | } 63 | 64 | 65 | [TestMethod] 66 | public void RandomForest_Default_Parameters_Save_Load_Model_Using_Static_Methods() 67 | { 68 | #region read and split data 69 | // Use StreamReader(filepath) when running from filesystem 70 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 71 | var targetName = "quality"; 72 | 73 | // read feature matrix 74 | var observations = parser.EnumerateRows(c => c != targetName) 75 | .ToF64Matrix(); 76 | 77 | // read regression targets 78 | var targets = parser.EnumerateRows(targetName) 79 | .ToF64Vector(); 80 | 81 | // creates training test splitter, 82 | // Since this is a regression problem, we use the random training/test set splitter. 83 | // 30 % of the data is used for the test set. 84 | var splitter = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 85 | 86 | var trainingTestSplit = splitter.SplitSet(observations, targets); 87 | var trainSet = trainingTestSplit.TrainingSet; 88 | var testSet = trainingTestSplit.TestSet; 89 | #endregion 90 | 91 | // create learner with default parameters 92 | var learner = new RegressionRandomForestLearner(trees: 100); 93 | 94 | // learn model with found parameters 95 | var model = learner.Learn(trainSet.Observations, trainSet.Targets); 96 | 97 | // predict the training and test set. 98 | var trainPredictions = model.Predict(trainSet.Observations); 99 | var testPredictions = model.Predict(testSet.Observations); 100 | 101 | // since this is a regression problem we are using square error as metric 102 | // for evaluating how well the model performs. 103 | var metric = new MeanSquaredErrorRegressionMetric(); 104 | 105 | // measure the error on training and test set. 106 | var trainError = metric.Error(trainSet.Targets, trainPredictions); 107 | var testError = metric.Error(testSet.Targets, testPredictions); 108 | 109 | TraceTrainingAndTestError(trainError, testError); 110 | 111 | //Save model, in the file system use new StreamWriter(filePath); 112 | // default format is xml. 113 | var savedModel = new StringWriter(); 114 | model.Save(() => savedModel); 115 | 116 | // load model, in the file system use new StreamReader(filePath); 117 | // default format is xml. 118 | var loadedModel = RegressionForestModel.Load(() => new StringReader(savedModel.ToString())); 119 | } 120 | 121 | [TestMethod] 122 | public void RandomForest_Default_Parameters_Save_Load_Model_Using_Serializer() 123 | { 124 | #region read and split data 125 | // Use StreamReader(filepath) when running from filesystem 126 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 127 | var targetName = "quality"; 128 | 129 | // read feature matrix 130 | var observations = parser.EnumerateRows(c => c != targetName) 131 | .ToF64Matrix(); 132 | 133 | // read regression targets 134 | var targets = parser.EnumerateRows(targetName) 135 | .ToF64Vector(); 136 | 137 | // creates training test splitter, 138 | // Since this is a regression problem, we use the random training/test set splitter. 139 | // 30 % of the data is used for the test set. 140 | var splitter = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 141 | 142 | var trainingTestSplit = splitter.SplitSet(observations, targets); 143 | var trainSet = trainingTestSplit.TrainingSet; 144 | var testSet = trainingTestSplit.TestSet; 145 | #endregion 146 | 147 | // create learner with default parameters 148 | var learner = new RegressionRandomForestLearner(trees: 100); 149 | 150 | // learn model with found parameters 151 | var model = learner.Learn(trainSet.Observations, trainSet.Targets); 152 | 153 | // predict the training and test set. 154 | var trainPredictions = model.Predict(trainSet.Observations); 155 | var testPredictions = model.Predict(testSet.Observations); 156 | 157 | // since this is a regression problem we are using square error as metric 158 | // for evaluating how well the model performs. 159 | var metric = new MeanSquaredErrorRegressionMetric(); 160 | 161 | // measure the error on training and test set. 162 | var trainError = metric.Error(trainSet.Targets, trainPredictions); 163 | var testError = metric.Error(testSet.Targets, testPredictions); 164 | 165 | TraceTrainingAndTestError(trainError, testError); 166 | 167 | //Save/load model as xml, in the file system use new StreamWriter(filePath); 168 | var xmlSerializer = new GenericXmlDataContractSerializer(); 169 | var savedModelXml = new StringWriter(); 170 | 171 | xmlSerializer 172 | .Serialize>(model, () => savedModelXml); 173 | 174 | var loadedModelXml = xmlSerializer 175 | .Deserialize>(() => new StringReader(savedModelXml.ToString())); 176 | 177 | //Save/load model as binary, in the file system use new StreamWriter(filePath); 178 | var binarySerializer = new GenericBinarySerializer(); 179 | var savedModelBinary = new StringWriter(); 180 | 181 | binarySerializer 182 | .Serialize>(model, () => savedModelBinary); 183 | 184 | var loadedModelBinary = binarySerializer 185 | .Deserialize>(() => new StringReader(savedModelBinary.ToString())); 186 | 187 | } 188 | 189 | [TestMethod] 190 | public void RandomForest_Default_Parameters_Variable_Importance() 191 | { 192 | #region read and split data 193 | // Use StreamReader(filepath) when running from filesystem 194 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 195 | var targetName = "quality"; 196 | 197 | // read feature matrix 198 | var observations = parser.EnumerateRows(c => c != targetName) 199 | .ToF64Matrix(); 200 | 201 | // read regression targets 202 | var targets = parser.EnumerateRows(targetName) 203 | .ToF64Vector(); 204 | 205 | // creates training test splitter, 206 | // Since this is a regression problem, we use the random training/test set splitter. 207 | // 30 % of the data is used for the test set. 208 | var splitter = new RandomTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 209 | 210 | var trainingTestSplit = splitter.SplitSet(observations, targets); 211 | var trainSet = trainingTestSplit.TrainingSet; 212 | var testSet = trainingTestSplit.TestSet; 213 | #endregion 214 | 215 | // create learner with default parameters 216 | var learner = new RegressionRandomForestLearner(trees: 100); 217 | 218 | // learn model with found parameters 219 | var model = learner.Learn(trainSet.Observations, trainSet.Targets); 220 | 221 | // predict the training and test set. 222 | var trainPredictions = model.Predict(trainSet.Observations); 223 | var testPredictions = model.Predict(testSet.Observations); 224 | 225 | // since this is a regression problem we are using square error as metric 226 | // for evaluating how well the model performs. 227 | var metric = new MeanSquaredErrorRegressionMetric(); 228 | 229 | // measure the error on training and test set. 230 | var trainError = metric.Error(trainSet.Targets, trainPredictions); 231 | var testError = metric.Error(testSet.Targets, testPredictions); 232 | 233 | TraceTrainingAndTestError(trainError, testError); 234 | 235 | // the variable importance requires the featureNameToIndex 236 | // from the data set. This mapping describes the relation 237 | // from column name to index in the feature matrix. 238 | var featureNameToIndex = parser.EnumerateRows(c => c != targetName) 239 | .First().ColumnNameToIndex; 240 | 241 | // Get the variable importance from the model. 242 | // Variable importance is a measure made by to model 243 | // of how important each feature is. 244 | var importances = model.GetVariableImportance(featureNameToIndex); 245 | 246 | // trace normalized importances as csv. 247 | var importanceCsv = new StringBuilder(); 248 | importanceCsv.Append("FeatureName;Importance"); 249 | foreach (var feature in importances) 250 | { 251 | importanceCsv.AppendLine(); 252 | importanceCsv.Append(string.Format("{0};{1:0.00}", 253 | feature.Key, feature.Value)); 254 | } 255 | 256 | Trace.WriteLine(importanceCsv); 257 | } 258 | 259 | 260 | static void TraceTrainingAndTestError(double trainError, double testError) 261 | { 262 | Trace.WriteLine(string.Format("Train error: {0:0.0000} - Test error: {1:0.0000}", 263 | trainError, testError)); 264 | } 265 | } 266 | } 267 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/InputOutput/CsvParserExamples.cs: -------------------------------------------------------------------------------- 1 | using System.IO; 2 | using Microsoft.VisualStudio.TestTools.UnitTesting; 3 | using SharpLearning.Examples.Properties; 4 | using SharpLearning.InputOutput.Csv; 5 | 6 | namespace SharpLearning.Examples.InputOutput 7 | { 8 | [TestClass] 9 | public class CsvParserExamples 10 | { 11 | [TestMethod] 12 | public void CsvParser_Read_F64Matrix() 13 | { 14 | // Use StreamReader(filepath) when running from filesystem 15 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 16 | 17 | // Read numeric feature matrix using conditional expression (exclude colum with header value quality) 18 | var obserationsConditionals = parser.EnumerateRows(c => c != "quality") 19 | .ToF64Matrix(); 20 | 21 | // Read numeric feature matrix using specified header values 22 | var obserationsSpecified = parser.EnumerateRows("volatile acidity", "citric acid", "residual sugar") 23 | .ToF64Matrix(); 24 | } 25 | 26 | [TestMethod] 27 | public void CsvParser_Read_F64Vector() 28 | { 29 | // Use StreamReader(filepath) when running from filesystem 30 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 31 | 32 | // Read double array using specified header values 33 | var targets = parser.EnumerateRows("quality") 34 | .ToF64Vector(); 35 | } 36 | 37 | [TestMethod] 38 | public void CsvParser_Read_StringMatrix() 39 | { 40 | // Use StreamReader(filepath) when running from filesystem 41 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 42 | 43 | // Read feature matrix as strings using conditional expression (exclude colum with header value quality) 44 | var obserationsConditionals = parser.EnumerateRows(c => c != "quality") 45 | .ToStringMatrix(); 46 | 47 | // Read feature matrix as strings using specified header values 48 | var obserationsSpecified = parser.EnumerateRows("volatile acidity", "citric acid", "residual sugar") 49 | .ToStringMatrix(); 50 | } 51 | 52 | [TestMethod] 53 | public void CsvParser_Read_StringVector() 54 | { 55 | // Use StreamReader(filepath) when running from filesystem 56 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 57 | 58 | // Read string array using specified header values 59 | var targets = parser.EnumerateRows("quality") 60 | .ToStringVector(); 61 | } 62 | } 63 | } 64 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Learners/ClassificationLearnerExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Collections.Generic; 2 | using System.Diagnostics; 3 | using System.IO; 4 | using System.Linq; 5 | using System.Text; 6 | using Microsoft.VisualStudio.TestTools.UnitTesting; 7 | using SharpLearning.DecisionTrees.Learners; 8 | using SharpLearning.DecisionTrees.Models; 9 | using SharpLearning.Examples.Properties; 10 | using SharpLearning.InputOutput.Csv; 11 | using SharpLearning.Metrics.Classification; 12 | 13 | namespace SharpLearning.Examples.Learners 14 | { 15 | [TestClass] 16 | public class ClassificationLearnerExamples 17 | { 18 | [TestMethod] 19 | public void ClassificationLearner_Learn() 20 | { 21 | // Use StreamReader(filepath) when running from filesystem 22 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 23 | var targetName = "quality"; 24 | 25 | // read feature matrix 26 | var observations = parser.EnumerateRows(c => c != targetName) 27 | .ToF64Matrix(); 28 | 29 | // read classification targets 30 | var targets = parser.EnumerateRows(targetName) 31 | .ToF64Vector(); 32 | 33 | // create learner 34 | var learner = new ClassificationDecisionTreeLearner(); 35 | 36 | // learns a ClassificationDecisionTreeModel 37 | var model = learner.Learn(observations, targets); 38 | } 39 | 40 | [TestMethod] 41 | public void ClassificationModel_Predict() 42 | { 43 | #region learner creation 44 | 45 | // Use StreamReader(filepath) when running from filesystem 46 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 47 | var targetName = "quality"; 48 | 49 | // read feature matrix 50 | var observations = parser.EnumerateRows(c => c != targetName) 51 | .ToF64Matrix(); 52 | 53 | // read classification targets 54 | var targets = parser.EnumerateRows(targetName) 55 | .ToF64Vector(); 56 | 57 | // create learner 58 | var learner = new ClassificationDecisionTreeLearner(); 59 | #endregion 60 | 61 | // learns a ClassificationDecisionTreeModel 62 | var model = learner.Learn(observations, targets); 63 | 64 | // predict all observations 65 | var predictions = model.Predict(observations); 66 | 67 | // predict single observation 68 | var prediction = model.Predict(observations.Row(0)); 69 | } 70 | 71 | [TestMethod] 72 | public void ClassificationModel_PredictProbability() 73 | { 74 | #region learner creation 75 | 76 | // Use StreamReader(filepath) when running from filesystem 77 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 78 | var targetName = "quality"; 79 | 80 | // read feature matrix 81 | var observations = parser.EnumerateRows(c => c != targetName) 82 | .ToF64Matrix(); 83 | 84 | // read classification targets 85 | var targets = parser.EnumerateRows(targetName) 86 | .ToF64Vector(); 87 | 88 | // create learner 89 | var learner = new ClassificationDecisionTreeLearner(maximumTreeDepth: 5); 90 | #endregion 91 | 92 | // learns a ClassificationDecisionTreeModel 93 | var model = learner.Learn(observations, targets); 94 | 95 | // predict probabilities for all observations 96 | var probabilityPredictions = model.PredictProbability(observations); 97 | 98 | // predict probabilities for single observation 99 | var probabilityPrediction = model.PredictProbability(observations.Row(0)); 100 | 101 | // the predicted class 102 | var predictedClass = probabilityPrediction.Prediction; 103 | 104 | // trace class probabilities 105 | probabilityPrediction.Probabilities.ToList() 106 | .ForEach(p => Trace.WriteLine(p.Key + ": " + p.Value)); 107 | } 108 | 109 | [TestMethod] 110 | public void ClassificationModel_PredictProbability_Threshold_On_Probability() 111 | { 112 | #region learner creation 113 | 114 | // Use StreamReader(filepath) when running from filesystem 115 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 116 | var targetName = "quality"; 117 | 118 | // read feature matrix 119 | var observations = parser.EnumerateRows(c => c != targetName) 120 | .ToF64Matrix(); 121 | 122 | // read classification targets and convert to binary problem (low quality/high quality). 123 | var targets = parser.EnumerateRows(targetName) 124 | .ToF64Vector().Select(t => t < 5 ? 0.0 : 1.0).ToArray(); 125 | 126 | var translation = new Dictionary { { 0.0, "Low quality" }, { 1.0, "High quality" } }; 127 | 128 | // create learner 129 | var learner = new ClassificationDecisionTreeLearner(maximumTreeDepth: 5); 130 | #endregion 131 | 132 | // learns a ClassificationDecisionTreeModel 133 | var model = learner.Learn(observations, targets); 134 | 135 | // predict probabilities for all observations 136 | var probabilityPredictions = model.PredictProbability(observations); 137 | 138 | // zip target and probabilities to keep order 139 | var zip = targets.Zip(probabilityPredictions, (t, p) => new { Target = t, Prediction = p }); 140 | 141 | // threhold on the probabilty of the predicted class. 142 | // This will remove the obserations that the model is uncertain about. 143 | var probabilityThreshold = 0.90; 144 | var thresholdedResult = zip.Where(kvp => kvp.Prediction.Probabilities[kvp.Prediction.Prediction] > probabilityThreshold); 145 | 146 | // evaluate the resulting observations 147 | var thresholdedPredictions = thresholdedResult.Select(p => p.Prediction).ToArray(); 148 | var thresholdedTargets = thresholdedResult.Select(p => p.Target).ToArray(); 149 | 150 | // evaluate only on probability thresholded data 151 | var metric = new LogLossClassificationProbabilityMetric(); 152 | Trace.WriteLine("ProbabilityThresholded Result:"); 153 | Trace.WriteLine(metric.ErrorString(thresholdedTargets, thresholdedPredictions, translation)); 154 | Trace.WriteLine(""); 155 | 156 | // evaluate on all data for comparison 157 | Trace.WriteLine("All data result:"); 158 | Trace.WriteLine(metric.ErrorString(targets, probabilityPredictions, translation)); 159 | } 160 | 161 | [TestMethod] 162 | public void ClassificationModel_FeatureImportance() 163 | { 164 | #region learner creation 165 | 166 | // Use StreamReader(filepath) when running from filesystem 167 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 168 | var targetName = "quality"; 169 | 170 | // read feature matrix 171 | var observations = parser.EnumerateRows(c => c != targetName) 172 | .ToF64Matrix(); 173 | 174 | // read classification targets 175 | var targets = parser.EnumerateRows(targetName) 176 | .ToF64Vector(); 177 | 178 | // create learner 179 | var learner = new ClassificationDecisionTreeLearner(); 180 | 181 | #endregion 182 | 183 | // learns a ClassificationDecisionTreeModel 184 | var model = learner.Learn(observations, targets); 185 | 186 | // raw feature importance 187 | var rawImportance = model.GetRawVariableImportance(); 188 | 189 | // Normalized and named feature importance. 190 | // This gives information about which features/variables the learner found important (higher is more important). 191 | var featureNameToIndex = parser.EnumerateRows(c => c != targetName).First().ColumnNameToIndex; 192 | var importance = model.GetVariableImportance(featureNameToIndex); 193 | 194 | // trace normalized importances 195 | var importanceCsv = new StringBuilder(); 196 | importanceCsv.Append("FeatureName;Importance"); 197 | foreach (var feature in importance) 198 | { 199 | importanceCsv.AppendLine(); 200 | importanceCsv.Append(feature.Key + ";" + feature.Value); 201 | } 202 | 203 | Trace.WriteLine(importanceCsv); 204 | } 205 | 206 | [TestMethod] 207 | public void ClassificationModel_Save_Load() 208 | { 209 | #region learner creation 210 | 211 | // Use StreamReader(filepath) when running from filesystem 212 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 213 | var targetName = "quality"; 214 | 215 | // read feature matrix 216 | var observations = parser.EnumerateRows(c => c != targetName) 217 | .ToF64Matrix(); 218 | 219 | // read classification targets 220 | var targets = parser.EnumerateRows(targetName) 221 | .ToF64Vector(); 222 | 223 | // create learner 224 | var learner = new ClassificationDecisionTreeLearner(); 225 | 226 | #endregion 227 | 228 | // learns a ClassificationDecisionTreeModel 229 | var model = learner.Learn(observations, targets); 230 | 231 | var writer = new StringWriter(); 232 | model.Save(() => writer); 233 | 234 | // save to file 235 | //model.Save(() => new StreamWriter(filePath)); 236 | 237 | var text = writer.ToString(); 238 | var loadedModel = ClassificationDecisionTreeModel.Load(() => new StringReader(text)); 239 | 240 | // load from file 241 | //ClassificationDecisionTreeModel.Load(() => new StreamReader(filePath)); 242 | } 243 | } 244 | } 245 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Learners/RegressionLearnerExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using System.Linq; 4 | using System.Text; 5 | using Microsoft.VisualStudio.TestTools.UnitTesting; 6 | using SharpLearning.DecisionTrees.Learners; 7 | using SharpLearning.DecisionTrees.Models; 8 | using SharpLearning.Examples.Properties; 9 | using SharpLearning.InputOutput.Csv; 10 | 11 | namespace SharpLearning.Examples.Learners 12 | { 13 | [TestClass] 14 | public class RegressionLearnerExamples 15 | { 16 | [TestMethod] 17 | public void RegressionLearner_Learn() 18 | { 19 | // Use StreamReader(filepath) when running from filesystem 20 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 21 | var targetName = "quality"; 22 | 23 | // read feature matrix 24 | var observations = parser.EnumerateRows(c => c != targetName) 25 | .ToF64Matrix(); 26 | 27 | // read regression targets 28 | var targets = parser.EnumerateRows(targetName) 29 | .ToF64Vector(); 30 | 31 | // create learner 32 | var learner = new RegressionDecisionTreeLearner(); 33 | 34 | // learns a RegressionDecisionTreeModel 35 | var model = learner.Learn(observations, targets); 36 | } 37 | 38 | [TestMethod] 39 | public void RegressionModel_Predict() 40 | { 41 | #region learner creation 42 | 43 | // Use StreamReader(filepath) when running from filesystem 44 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 45 | var targetName = "quality"; 46 | 47 | // read feature matrix 48 | var observations = parser.EnumerateRows(c => c != targetName) 49 | .ToF64Matrix(); 50 | 51 | // read regression targets 52 | var targets = parser.EnumerateRows(targetName) 53 | .ToF64Vector(); 54 | 55 | // create learner 56 | var learner = new RegressionDecisionTreeLearner(); 57 | #endregion 58 | 59 | // learns a RegressionDecisionTreeModel 60 | var model = learner.Learn(observations, targets); 61 | 62 | // predict all observations 63 | var predictions = model.Predict(observations); 64 | 65 | // predict single observation 66 | var prediction = model.Predict(observations.Row(0)); 67 | } 68 | 69 | [TestMethod] 70 | public void RegressionModel_FeatureImportance() 71 | { 72 | #region learner creation 73 | 74 | // Use StreamReader(filepath) when running from filesystem 75 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 76 | var targetName = "quality"; 77 | 78 | // read feature matrix 79 | var observations = parser.EnumerateRows(c => c != targetName) 80 | .ToF64Matrix(); 81 | 82 | // read regression targets 83 | var targets = parser.EnumerateRows(targetName) 84 | .ToF64Vector(); 85 | 86 | // create learner 87 | var learner = new RegressionDecisionTreeLearner(); 88 | #endregion 89 | 90 | // learns a RegressionDecisionTreeModel 91 | var model = learner.Learn(observations, targets); 92 | 93 | // raw feature importance 94 | var rawImportance = model.GetRawVariableImportance(); 95 | 96 | // normalized and named feature importance 97 | var featureNameToIndex = parser.EnumerateRows(c => c != targetName).First().ColumnNameToIndex; 98 | var importance = model.GetVariableImportance(featureNameToIndex); 99 | 100 | // trace normalized importances 101 | var importanceCsv = new StringBuilder(); 102 | importanceCsv.Append("FeatureName;Importance"); 103 | foreach (var feature in importance) 104 | { 105 | importanceCsv.AppendLine(); 106 | importanceCsv.Append(feature.Key + ";" + feature.Value); 107 | } 108 | 109 | Trace.WriteLine(importanceCsv); 110 | } 111 | 112 | [TestMethod] 113 | public void RegressionModel_Save_Load() 114 | { 115 | #region learner creation 116 | 117 | // Use StreamReader(filepath) when running from filesystem 118 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 119 | var targetName = "quality"; 120 | 121 | // read feature matrix 122 | var observations = parser.EnumerateRows(c => c != targetName) 123 | .ToF64Matrix(); 124 | 125 | // read regression targets 126 | var targets = parser.EnumerateRows(targetName) 127 | .ToF64Vector(); 128 | 129 | // create learner 130 | var learner = new RegressionDecisionTreeLearner(); 131 | 132 | #endregion 133 | 134 | // learns a ClassificationDecisionTreeModel 135 | var model = learner.Learn(observations, targets); 136 | 137 | var writer = new StringWriter(); 138 | model.Save(() => writer); 139 | 140 | // save to file 141 | //model.Save(() => new StreamWriter(filePath)); 142 | 143 | var text = writer.ToString(); 144 | var loadedModel = RegressionDecisionTreeModel.Load(() => new StringReader(text)); 145 | 146 | // load from file 147 | //RegressionDecisionTreeModel.Load(() => new StreamReader(filePath)); 148 | } 149 | } 150 | } 151 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/LearningCurves/LearningCurvesExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using System.Linq; 4 | using Microsoft.VisualStudio.TestTools.UnitTesting; 5 | using SharpLearning.Containers; 6 | using SharpLearning.CrossValidation.LearningCurves; 7 | using SharpLearning.DecisionTrees.Learners; 8 | using SharpLearning.Examples.Properties; 9 | using SharpLearning.InputOutput.Csv; 10 | using SharpLearning.Metrics.Classification; 11 | using SharpLearning.Metrics.Regression; 12 | 13 | namespace SharpLearning.Examples.LearningCurves 14 | { 15 | [TestClass] 16 | public class LearningCurvesExamples 17 | { 18 | [TestMethod] 19 | public void LearningCurves_Calculate() 20 | { 21 | #region Read data 22 | 23 | // Use StreamReader(filepath) when running from filesystem 24 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 25 | var targetName = "quality"; 26 | 27 | // read feature matrix 28 | var observations = parser.EnumerateRows(c => c != targetName) 29 | .ToF64Matrix(); 30 | 31 | // read regression targets 32 | var targets = parser.EnumerateRows(targetName) 33 | .ToF64Vector(); 34 | 35 | #endregion 36 | 37 | // metric for measuring model error 38 | var metric = new MeanSquaredErrorRegressionMetric(); 39 | 40 | // creates cross validator, observations are shuffled randomly 41 | var learningCurveCalculator = new RandomShuffleLearningCurvesCalculator(metric, 42 | samplePercentages: new double[] { 0.05, 0.1, 0.2, 0.4, 0.8, 1.0 }, 43 | trainingPercentage: 0.7, numberOfShufflesPrSample: 5); 44 | 45 | // create learner 46 | var learner = new RegressionDecisionTreeLearner(maximumTreeDepth: 5); 47 | 48 | // calculate learning curve 49 | var learningCurve = learningCurveCalculator.Calculate(learner, observations, targets); 50 | 51 | // write to csv 52 | var writer = new StringWriter(); 53 | learningCurve.Write(() => writer); 54 | 55 | // trace result 56 | // Plotting the learning curves will help determine if the model has high bias or high variance. 57 | // This information can be used to determine what to try next in order to improve the model. 58 | Trace.WriteLine(writer.ToString()); 59 | 60 | // alternatively, write to file 61 | //learningCurve.Write(() => new StreamWriter(filePath)); 62 | } 63 | 64 | [TestMethod] 65 | public void LearningCurves_Calculate_ProbabilityPrediction() 66 | { 67 | #region Read data 68 | 69 | // Use StreamReader(filepath) when running from filesystem 70 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 71 | var targetName = "quality"; 72 | 73 | // read feature matrix 74 | var observations = parser.EnumerateRows(c => c != targetName) 75 | .ToF64Matrix(); 76 | 77 | // read classification targets and convert to binary problem (low quality/high quality). 78 | var targets = parser.EnumerateRows(targetName) 79 | .ToF64Vector().Select(t => t < 5 ? 0.0 : 1.0).ToArray(); 80 | 81 | #endregion 82 | 83 | // metric for measuring model error 84 | var metric = new LogLossClassificationProbabilityMetric(); 85 | 86 | // creates cross validator, observations are shuffled randomly 87 | var learningCurveCalculator = new RandomShuffleLearningCurvesCalculator(metric, 88 | samplePercentages: new double[] { 0.05, 0.1, 0.2, 0.4, 0.8, 1.0 }, 89 | trainingPercentage: 0.7, numberOfShufflesPrSample: 5); 90 | 91 | // create learner 92 | var learner = new ClassificationDecisionTreeLearner(maximumTreeDepth: 5); 93 | 94 | // calculate learning curve 95 | var learningCurve = learningCurveCalculator.Calculate(learner, observations, targets); 96 | 97 | // write to csv 98 | var writer = new StringWriter(); 99 | learningCurve.Write(() => writer); 100 | 101 | // trace result 102 | // Plotting the learning curves will help determine if the model has high bias or high variance. 103 | // This information can be used to determine what to try next in order to improve the model. 104 | Trace.WriteLine(writer.ToString()); 105 | 106 | // alternatively, write to file 107 | //learningCurve.Write(() => new StreamWriter(filePath)); 108 | } 109 | } 110 | } 111 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Metrics/ClassificationMetricExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Collections.Generic; 2 | using System.Diagnostics; 3 | using Microsoft.VisualStudio.TestTools.UnitTesting; 4 | using SharpLearning.Metrics.Classification; 5 | 6 | namespace SharpLearning.Examples.Metrics 7 | { 8 | [TestClass] 9 | public class ClassificationMetricExamples 10 | { 11 | [TestMethod] 12 | public void ClassificationMetric_Error() 13 | { 14 | var targets = new double[] { 1, 2, 2, 2, 3, 1, 1, 2, 3 }; 15 | var predictions = new double[] { 1, 2, 2, 2, 1, 2, 2, 1, 3 }; 16 | 17 | var metric = new TotalErrorClassificationMetric(); 18 | 19 | Trace.WriteLine("Error: " + metric.Error(targets, predictions)); 20 | } 21 | 22 | [TestMethod] 23 | public void ClassificationMetric_ErrorString() 24 | { 25 | var targets = new double[] { 1, 2, 2, 2, 3, 1, 1, 2, 3 }; 26 | var predictions = new double[] { 1, 2, 2, 2, 1, 2, 2, 1, 3 }; 27 | 28 | var metric = new TotalErrorClassificationMetric(); 29 | 30 | Trace.WriteLine(metric.ErrorString(targets, predictions)); 31 | } 32 | 33 | [TestMethod] 34 | public void ClassificationMetric_ErrorString_Translate_Target_Values_To_Names() 35 | { 36 | var targets = new double[] { 1, 2, 2, 2, 3, 1, 1, 2, 3 }; 37 | var predictions = new double[] { 1, 2, 2, 2, 1, 2, 2, 1, 3 }; 38 | 39 | var translation = new Dictionary { { 1.0, "Quality1" }, { 2.0, "Quality2" }, { 3.0, "Quality3" } }; 40 | var metric = new TotalErrorClassificationMetric(); 41 | 42 | Trace.WriteLine(metric.ErrorString(targets, predictions, translation)); 43 | } 44 | 45 | [TestMethod] 46 | public void ClassificationMetric_On_Strings() 47 | { 48 | var targets = new string[] { "Quality1", "Quality2", "Quality2", "Quality2", "Quality3", "Quality1", "Quality1", "Quality2", "Quality3" }; 49 | var predictions = new string[] { "Quality1", "Quality2", "Quality2", "Quality2", "Quality1", "Quality2", "Quality2", "Quality1", "Quality3" }; 50 | 51 | var metric = new TotalErrorClassificationMetric(); 52 | Trace.WriteLine(metric.ErrorString(targets, predictions)); 53 | } 54 | } 55 | } 56 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Metrics/RegressionMetricExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using Microsoft.VisualStudio.TestTools.UnitTesting; 3 | using SharpLearning.Metrics.Regression; 4 | 5 | namespace SharpLearning.Examples.Metrics 6 | { 7 | [TestClass] 8 | public class RegressionMetricExamples 9 | { 10 | [TestMethod] 11 | public void MeanSquaredErrorRegressionMetric_Error() 12 | { 13 | var targets = new double[] { 1, 2, 2, 2, 3, 1, 1, 2, 3 }; 14 | var predictions = new double[] { 1, 2, 2, 2, 1, 2, 2, 1, 3 }; 15 | 16 | var metric = new MeanSquaredErrorRegressionMetric(); 17 | Trace.WriteLine("Error: " + metric.Error(targets, predictions)); 18 | } 19 | } 20 | } 21 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/ModelSelection/ClassificationFindBestDefaultModelExample.cs: -------------------------------------------------------------------------------- 1 | using System.Collections.Generic; 2 | using System.Diagnostics; 3 | using System.IO; 4 | using Microsoft.VisualStudio.TestTools.UnitTesting; 5 | using SharpLearning.AdaBoost.Learners; 6 | using SharpLearning.Common.Interfaces; 7 | using SharpLearning.CrossValidation.TrainingTestSplitters; 8 | using SharpLearning.DecisionTrees.Learners; 9 | using SharpLearning.Examples.Properties; 10 | using SharpLearning.FeatureTransformations.MatrixTransforms; 11 | using SharpLearning.GradientBoost.Learners; 12 | using SharpLearning.InputOutput.Csv; 13 | using SharpLearning.Metrics.Classification; 14 | using SharpLearning.RandomForest.Learners; 15 | 16 | namespace SharpLearning.Examples.ModelSelection 17 | { 18 | [TestClass] 19 | public class ClassificationFindBestDefaultModelExample 20 | { 21 | [TestMethod] 22 | public void Classification_Find_Best_Model_With_Default_Parameters() 23 | { 24 | #region Read and Transform Data 25 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 26 | var targetName = "quality"; 27 | 28 | // read feature matrix (all columns different from the targetName) 29 | var observations = parser.EnumerateRows(c => c != targetName) 30 | .ToF64Matrix(); 31 | 32 | // create minmax normalizer (normalizes each feature from 0.0 to 1.0) 33 | var minMaxTransformer = new MinMaxTransformer(0.0, 1.0); 34 | 35 | // transforms features using the feature normalization transform 36 | minMaxTransformer.Transform(observations, observations); 37 | 38 | // read targets 39 | var targets = parser.EnumerateRows(targetName) 40 | .ToF64Vector(); 41 | #endregion 42 | 43 | // split data 44 | // creates training test splitter, training and test set are splittet 45 | // to have equal distribution of classes in both set. 46 | var splitter = new StratifiedTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 47 | 48 | var trainingTestSplit = splitter.SplitSet(observations, targets); 49 | var trainingSet = trainingTestSplit.TrainingSet; 50 | var testSet = trainingTestSplit.TestSet; 51 | 52 | // Create list of all classification learners (with default parameters) 53 | var learners = new List> 54 | { 55 | new ClassificationDecisionTreeLearner(), 56 | new ClassificationRandomForestLearner(), 57 | new ClassificationExtremelyRandomizedTreesLearner(), 58 | new ClassificationAdaBoostLearner(), 59 | new ClassificationBinomialGradientBoostLearner(), 60 | }; 61 | 62 | // metric for measuring the error 63 | var metric = new TotalErrorClassificationMetric(); 64 | 65 | // try all learners 66 | var testPredictions = new double[testSet.Targets.Length]; 67 | var testObservation = new double[trainingSet.Observations.ColumnCount]; 68 | foreach (var learner in learners) 69 | { 70 | // train model 71 | var model = learner.Learn(trainingSet.Observations, trainingSet.Targets); 72 | 73 | // iterate over test set and predict each observation 74 | for (int i = 0; i < testSet.Targets.Length; i++) 75 | { 76 | testSet.Observations.Row(i, testObservation); 77 | testPredictions[i] = model.Predict(testObservation); 78 | } 79 | 80 | // measure error on test set 81 | var error = metric.Error(testSet.Targets, testPredictions); 82 | 83 | // Trace learner type and error to output window 84 | Trace.WriteLine(string.Format("{0}: {1:0.0000}", learner.GetType().Name, error)); 85 | } 86 | } 87 | } 88 | } 89 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/NeuralNets/BatchNormalizationExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using System.Linq; 4 | using Microsoft.VisualStudio.TestTools.UnitTesting; 5 | using SharpLearning.Containers.Matrices; 6 | using SharpLearning.Examples.Properties; 7 | using SharpLearning.InputOutput.Csv; 8 | using SharpLearning.Metrics.Classification; 9 | using SharpLearning.Neural; 10 | using SharpLearning.Neural.Layers; 11 | using SharpLearning.Neural.Learners; 12 | using SharpLearning.Neural.Loss; 13 | 14 | namespace SharpLearning.Examples.NeuralNets 15 | { 16 | [TestClass] 17 | 18 | public class BatchNormalizationExamples 19 | { 20 | [TestMethod] 21 | public void Classification_Neural_Net_Using_BatchNormalization() 22 | { 23 | #region Read Data 24 | // Use StreamReader(filepath) when running from filesystem 25 | var trainingParser = new CsvParser(() => new StringReader(Resources.cifar10_train_small)); 26 | var testParser = new CsvParser(() => new StringReader(Resources.cifar10_test_small)); 27 | 28 | var targetName = "label"; 29 | var id = "id"; 30 | 31 | var featureNames = trainingParser.EnumerateRows(v => v != targetName && v != id).First().ColumnNameToIndex.Keys.ToArray(); 32 | 33 | var index = 0.0; 34 | var targetNameToTargetValue = trainingParser.EnumerateRows(targetName) 35 | .ToStringVector().Distinct().ToDictionary(v => v, v => index++); 36 | 37 | // read feature matrix (training) 38 | var trainingObservations = trainingParser 39 | .EnumerateRows(featureNames) 40 | .ToF64Matrix(); 41 | // read classification targets (training) 42 | var trainingTargets = trainingParser.EnumerateRows(targetName) 43 | .ToStringVector().Select(v => targetNameToTargetValue[v]).ToArray(); 44 | 45 | // read feature matrix (test) 46 | var testObservations = testParser 47 | .EnumerateRows(featureNames) 48 | .ToF64Matrix(); 49 | // read classification targets (test) 50 | var testTargets = testParser.EnumerateRows(targetName) 51 | .ToStringVector().Select(v => targetNameToTargetValue[v]).ToArray(); 52 | #endregion 53 | 54 | // transform pixel values to be between 0 and 1. 55 | trainingObservations.Map(p => p / 255); 56 | testObservations.Map(p => p / 255); 57 | 58 | // the output layer must know the number of classes. 59 | var numberOfClasses = trainingTargets.Distinct().Count(); 60 | 61 | // batch normalization can be added to all layers with weights + biases. 62 | // Batch normalization will increase the error reduction pr. iteration 63 | // but will also make each iteration more slow due to the extra work. 64 | // Batch normalization usually has the best effect on deeper networks. 65 | var useBatchNorm = true; 66 | var net = new NeuralNet(); 67 | net.Add(new InputLayer(width: 32, height: 32, depth: 3)); // CIFAR data is 32x32x3. 68 | net.Add(new Conv2DLayer(3, 3, 32) { BatchNormalization = useBatchNorm }); // activate batch normalization. 69 | net.Add(new MaxPool2DLayer(2, 2)); 70 | net.Add(new DropoutLayer(0.25)); 71 | net.Add(new Conv2DLayer(3, 3, 64) { BatchNormalization = useBatchNorm }); // activate batch normalization. 72 | net.Add(new Conv2DLayer(3, 3, 64) { BatchNormalization = useBatchNorm }); // activate batch normalization. 73 | net.Add(new MaxPool2DLayer(2, 2)); 74 | net.Add(new DropoutLayer(0.25)); 75 | net.Add(new DenseLayer(512) { BatchNormalization = useBatchNorm }); // activate batch normalization. 76 | net.Add(new DropoutLayer(0.5)); 77 | net.Add(new SoftMaxLayer(numberOfClasses)); 78 | 79 | // using classification accuracy as error metric. 80 | // When using a validation set, the error metric 81 | // is used for selecting the best iteration based on models error on the validation set. 82 | var learner = new ClassificationNeuralNetLearner(net, iterations: 5, loss: new AccuracyLoss()); 83 | 84 | var model = learner.Learn(trainingObservations, trainingTargets); 85 | 86 | var metric = new TotalErrorClassificationMetric(); 87 | var predictions = model.Predict(testObservations); 88 | 89 | Trace.WriteLine("Test Error: " + metric.Error(testTargets, predictions)); 90 | } 91 | } 92 | } 93 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/NeuralNets/ClassificationNeuralNetExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using System.Linq; 4 | using Microsoft.VisualStudio.TestTools.UnitTesting; 5 | using SharpLearning.Containers.Matrices; 6 | using SharpLearning.Examples.Properties; 7 | using SharpLearning.InputOutput.Csv; 8 | using SharpLearning.Metrics.Classification; 9 | using SharpLearning.Neural; 10 | using SharpLearning.Neural.Activations; 11 | using SharpLearning.Neural.Layers; 12 | using SharpLearning.Neural.Learners; 13 | using SharpLearning.Neural.Loss; 14 | 15 | namespace SharpLearning.Examples.NeuralNets 16 | { 17 | [TestClass] 18 | public class ClassificationNeuralNetExamples 19 | { 20 | [TestMethod] 21 | public void Classification_Convolutional_Neural_Net() 22 | { 23 | #region Read Data 24 | 25 | // Use StreamReader(filepath) when running from filesystem 26 | var trainingParser = new CsvParser(() => new StringReader(Resources.mnist_small_train)); 27 | var testParser = new CsvParser(() => new StringReader(Resources.mnist_small_test)); 28 | 29 | var targetName = "Class"; 30 | 31 | var featureNames = trainingParser.EnumerateRows(c => c != targetName).First().ColumnNameToIndex.Keys.ToArray(); 32 | 33 | // read feature matrix (training) 34 | var trainingObservations = trainingParser 35 | .EnumerateRows(featureNames) 36 | .ToF64Matrix(); 37 | // read classification targets (training) 38 | var trainingTargets = trainingParser.EnumerateRows(targetName) 39 | .ToF64Vector(); 40 | 41 | // read feature matrix (test) 42 | var testObservations = testParser 43 | .EnumerateRows(featureNames) 44 | .ToF64Matrix(); 45 | // read classification targets (test) 46 | var testTargets = testParser.EnumerateRows(targetName) 47 | .ToF64Vector(); 48 | #endregion 49 | 50 | // transform pixel values to be between 0 and 1. 51 | trainingObservations.Map(p => p / 255); 52 | testObservations.Map(p => p / 255); 53 | 54 | // the output layer must know the number of classes. 55 | var numberOfClasses = trainingTargets.Distinct().Count(); 56 | 57 | // define the neural net. 58 | var net = new NeuralNet(); 59 | net.Add(new InputLayer(width: 28, height: 28, depth: 1)); // MNIST data is 28x28x1. 60 | net.Add(new Conv2DLayer(filterWidth: 5, filterHeight: 5, filterCount: 32)); 61 | net.Add(new MaxPool2DLayer(poolWidth: 2, poolHeight: 2)); 62 | net.Add(new DropoutLayer(0.5)); 63 | net.Add(new DenseLayer(256, Activation.Relu)); 64 | net.Add(new DropoutLayer(0.5)); 65 | net.Add(new SoftMaxLayer(numberOfClasses)); 66 | 67 | // using only 10 iteration to make the example run faster. 68 | // using classification accuracy as error metric. This is only used for reporting progress. 69 | var learner = new ClassificationNeuralNetLearner(net, iterations: 10, loss: new AccuracyLoss()); 70 | var model = learner.Learn(trainingObservations, trainingTargets); 71 | 72 | var metric = new TotalErrorClassificationMetric(); 73 | var predictions = model.Predict(testObservations); 74 | 75 | Trace.WriteLine("Test Error: " + metric.Error(testTargets, predictions)); 76 | } 77 | 78 | [TestMethod] 79 | public void Classification_Standard_Neural_Net() 80 | { 81 | #region Read Data 82 | // Use StreamReader(filepath) when running from filesystem 83 | var trainingParser = new CsvParser(() => new StringReader(Resources.mnist_small_train)); 84 | var testParser = new CsvParser(() => new StringReader(Resources.mnist_small_test)); 85 | 86 | var targetName = "Class"; 87 | 88 | var featureNames = trainingParser.EnumerateRows(c => c != targetName).First().ColumnNameToIndex.Keys.ToArray(); 89 | 90 | // read feature matrix (training) 91 | var trainingObservations = trainingParser 92 | .EnumerateRows(featureNames) 93 | .ToF64Matrix(); 94 | // read classification targets (training) 95 | var trainingTargets = trainingParser.EnumerateRows(targetName) 96 | .ToF64Vector(); 97 | 98 | // read feature matrix (test) 99 | var testObservations = testParser 100 | .EnumerateRows(featureNames) 101 | .ToF64Matrix(); 102 | // read classification targets (test) 103 | var testTargets = testParser.EnumerateRows(targetName) 104 | .ToF64Vector(); 105 | #endregion 106 | 107 | // transform pixel values to be between 0 and 1. 108 | trainingObservations.Map(p => p / 255); 109 | testObservations.Map(p => p / 255); 110 | 111 | // the output layer must know the number of classes. 112 | var numberOfClasses = trainingTargets.Distinct().Count(); 113 | 114 | var net = new NeuralNet(); 115 | net.Add(new InputLayer(width: 28, height: 28, depth: 1)); // MNIST data is 28x28x1. 116 | net.Add(new DropoutLayer(0.2)); 117 | net.Add(new DenseLayer(800, Activation.Relu)); 118 | net.Add(new DropoutLayer(0.5)); 119 | net.Add(new DenseLayer(800, Activation.Relu)); 120 | net.Add(new DropoutLayer(0.5)); 121 | net.Add(new SoftMaxLayer(numberOfClasses)); 122 | 123 | // using only 10 iteration to make the example run faster. 124 | // using classification accuracy as error metric. This is only used for reporting progress. 125 | var learner = new ClassificationNeuralNetLearner(net, iterations: 10, loss: new AccuracyLoss()); 126 | var model = learner.Learn(trainingObservations, trainingTargets); 127 | 128 | var metric = new TotalErrorClassificationMetric(); 129 | var predictions = model.Predict(testObservations); 130 | 131 | Trace.WriteLine("Test Error: " + metric.Error(testTargets, predictions)); 132 | } 133 | 134 | [TestMethod] 135 | public void Classification_LogisticRegression_Neural_Net() 136 | { 137 | #region Read Data 138 | // Use StreamReader(filepath) when running from filesystem 139 | var trainingParser = new CsvParser(() => new StringReader(Resources.mnist_small_train)); 140 | var testParser = new CsvParser(() => new StringReader(Resources.mnist_small_test)); 141 | 142 | var targetName = "Class"; 143 | 144 | var featureNames = trainingParser.EnumerateRows(c => c != targetName).First().ColumnNameToIndex.Keys.ToArray(); 145 | 146 | // read feature matrix (training) 147 | var trainingObservations = trainingParser 148 | .EnumerateRows(featureNames) 149 | .ToF64Matrix(); 150 | // read classification targets (training) 151 | var trainingTargets = trainingParser.EnumerateRows(targetName) 152 | .ToF64Vector(); 153 | 154 | // read feature matrix (test) 155 | var testObservations = testParser 156 | .EnumerateRows(featureNames) 157 | .ToF64Matrix(); 158 | // read classification targets (test) 159 | var testTargets = testParser.EnumerateRows(targetName) 160 | .ToF64Vector(); 161 | #endregion 162 | 163 | // transform pixel values to be between 0 and 1. 164 | trainingObservations.Map(p => p / 255); 165 | testObservations.Map(p => p / 255); 166 | 167 | // the output layer must know the number of classes. 168 | var numberOfClasses = trainingTargets.Distinct().Count(); 169 | 170 | var net = new NeuralNet(); 171 | net.Add(new InputLayer(width: 28, height: 28, depth: 1)); // MNIST data is 28x28x1. 172 | net.Add(new SoftMaxLayer(numberOfClasses)); // No hidden layers and SoftMax output layer corresponds to logistic regression classifer. 173 | 174 | // using only 10 iteration to make the example run faster. 175 | // using classification accuracy as error metric. This is only used for reporting progress. 176 | var learner = new ClassificationNeuralNetLearner(net, iterations: 10, loss: new AccuracyLoss()); 177 | var model = learner.Learn(trainingObservations, trainingTargets); 178 | 179 | var metric = new TotalErrorClassificationMetric(); 180 | var predictions = model.Predict(testObservations); 181 | 182 | Trace.WriteLine("Test Error: " + metric.Error(testTargets, predictions)); 183 | } 184 | 185 | [TestMethod] 186 | public void Classification_LinearSvm_Neural_Net() 187 | { 188 | #region Read Data 189 | // Use StreamReader(filepath) when running from filesystem 190 | var trainingParser = new CsvParser(() => new StringReader(Resources.mnist_small_train)); 191 | var testParser = new CsvParser(() => new StringReader(Resources.mnist_small_test)); 192 | 193 | var targetName = "Class"; 194 | 195 | var featureNames = trainingParser.EnumerateRows(c => c != targetName).First().ColumnNameToIndex.Keys.ToArray(); 196 | 197 | // read feature matrix (training) 198 | var trainingObservations = trainingParser 199 | .EnumerateRows(featureNames) 200 | .ToF64Matrix(); 201 | // read classification targets (training) 202 | var trainingTargets = trainingParser.EnumerateRows(targetName) 203 | .ToF64Vector(); 204 | 205 | // read feature matrix (test) 206 | var testObservations = testParser 207 | .EnumerateRows(featureNames) 208 | .ToF64Matrix(); 209 | // read classification targets (test) 210 | var testTargets = testParser.EnumerateRows(targetName) 211 | .ToF64Vector(); 212 | #endregion 213 | 214 | // transform pixel values to be between 0 and 1. 215 | trainingObservations.Map(p => p / 255); 216 | testObservations.Map(p => p / 255); 217 | 218 | // the output layer must know the number of classes. 219 | var numberOfClasses = trainingTargets.Distinct().Count(); 220 | 221 | var net = new NeuralNet(); 222 | net.Add(new InputLayer(width: 28, height: 28, depth: 1)); // MNIST data is 28x28x1. 223 | net.Add(new SvmLayer(numberOfClasses)); // No hidden layers and Svm output layer corresponds to linear Svm. 224 | 225 | // using only 10 iteration to make the example run faster. 226 | // using classification accuracy as error metric. This is only used for reporting progress. 227 | var learner = new ClassificationNeuralNetLearner(net, iterations: 10, loss: new AccuracyLoss()); 228 | var model = learner.Learn(trainingObservations, trainingTargets); 229 | 230 | var metric = new TotalErrorClassificationMetric(); 231 | var predictions = model.Predict(testObservations); 232 | 233 | Trace.WriteLine("Test Error: " + metric.Error(testTargets, predictions)); 234 | } 235 | } 236 | } 237 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/NeuralNets/OptimizerExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using System.Linq; 4 | using Microsoft.VisualStudio.TestTools.UnitTesting; 5 | using SharpLearning.Containers.Matrices; 6 | using SharpLearning.Examples.Properties; 7 | using SharpLearning.InputOutput.Csv; 8 | using SharpLearning.Metrics.Classification; 9 | using SharpLearning.Neural; 10 | using SharpLearning.Neural.Activations; 11 | using SharpLearning.Neural.Layers; 12 | using SharpLearning.Neural.Learners; 13 | using SharpLearning.Neural.Loss; 14 | using SharpLearning.Neural.Optimizers; 15 | 16 | namespace SharpLearning.Examples.NeuralNets 17 | { 18 | [TestClass] 19 | 20 | public class OptimizerExamples 21 | { 22 | [TestMethod] 23 | public void Classification_Convolutional_Neural_Net_Select_Optimizer_Method() 24 | { 25 | #region Read Data 26 | 27 | // Use StreamReader(filepath) when running from filesystem 28 | var trainingParser = new CsvParser(() => new StringReader(Resources.mnist_small_train)); 29 | var testParser = new CsvParser(() => new StringReader(Resources.mnist_small_test)); 30 | 31 | var targetName = "Class"; 32 | 33 | var featureNames = trainingParser.EnumerateRows(c => c != targetName).First().ColumnNameToIndex.Keys.ToArray(); 34 | 35 | // read feature matrix (training) 36 | var trainingObservations = trainingParser 37 | .EnumerateRows(featureNames) 38 | .ToF64Matrix(); 39 | // read classification targets (training) 40 | var trainingTargets = trainingParser.EnumerateRows(targetName) 41 | .ToF64Vector(); 42 | 43 | // read feature matrix (test) 44 | var testObservations = testParser 45 | .EnumerateRows(featureNames) 46 | .ToF64Matrix(); 47 | // read classification targets (test) 48 | var testTargets = testParser.EnumerateRows(targetName) 49 | .ToF64Vector(); 50 | #endregion 51 | 52 | // transform pixel values to be between 0 and 1. 53 | trainingObservations.Map(p => p / 255); 54 | testObservations.Map(p => p / 255); 55 | 56 | // the output layer must know the number of classes. 57 | var numberOfClasses = trainingTargets.Distinct().Count(); 58 | 59 | // define the neural net. 60 | var net = new NeuralNet(); 61 | net.Add(new InputLayer(width: 28, height: 28, depth: 1)); // MNIST data is 28x28x1. 62 | net.Add(new Conv2DLayer(filterWidth: 5, filterHeight: 5, filterCount: 32)); 63 | net.Add(new MaxPool2DLayer(poolWidth: 2, poolHeight: 2)); 64 | net.Add(new DropoutLayer(0.5)); 65 | net.Add(new DenseLayer(256, Activation.Relu)); 66 | net.Add(new DropoutLayer(0.5)); 67 | net.Add(new SoftMaxLayer(numberOfClasses)); 68 | 69 | // Different optimizer methods can be selected. 70 | // SharpLearning default is RMSProp. 71 | // Recommendations would be: RMSProp, Adam, Nadam or Adadelta. 72 | var optimizerMethod = OptimizerMethod.Nadam; 73 | 74 | // using only 10 iteration to make the example run faster. 75 | var learner = new ClassificationNeuralNetLearner(net, iterations: 10, loss: new AccuracyLoss(), 76 | optimizerMethod: optimizerMethod); 77 | 78 | var model = learner.Learn(trainingObservations, trainingTargets); 79 | var metric = new TotalErrorClassificationMetric(); 80 | var predictions = model.Predict(testObservations); 81 | 82 | Trace.WriteLine("Test Error: " + metric.Error(testTargets, predictions)); 83 | } 84 | } 85 | } -------------------------------------------------------------------------------- /src/SharpLearning.Examples/NeuralNets/RegressionNeuralNetExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using Microsoft.VisualStudio.TestTools.UnitTesting; 4 | using SharpLearning.Containers.Matrices; 5 | using SharpLearning.Examples.Properties; 6 | using SharpLearning.FeatureTransformations.MatrixTransforms; 7 | using SharpLearning.InputOutput.Csv; 8 | using SharpLearning.Metrics.Regression; 9 | using SharpLearning.Neural; 10 | using SharpLearning.Neural.Activations; 11 | using SharpLearning.Neural.Layers; 12 | using SharpLearning.Neural.Learners; 13 | using SharpLearning.Neural.Loss; 14 | 15 | namespace SharpLearning.Examples.NeuralNets 16 | { 17 | [TestClass] 18 | public class NeuralNetExamples 19 | { 20 | [TestMethod] 21 | public void Regression_Standard_Neural_Net() 22 | { 23 | #region Read Data 24 | 25 | // Use StreamReader(filepath) when running from filesystem 26 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 27 | var targetName = "quality"; 28 | 29 | // read feature matrix 30 | var observations = parser.EnumerateRows(c => c != targetName) 31 | .ToF64Matrix(); 32 | 33 | // read classification targets 34 | var targets = parser.EnumerateRows(targetName) 35 | .ToF64Vector(); 36 | 37 | #endregion 38 | 39 | // transform pixel values to be between 0 and 1. 40 | var minMaxTransformer = new MinMaxTransformer(0.0, 1.0); 41 | minMaxTransformer.Transform(observations, observations); 42 | 43 | var numberOfFeatures = observations.ColumnCount; 44 | 45 | // define the neural net. 46 | var net = new NeuralNet(); 47 | net.Add(new InputLayer(inputUnits: numberOfFeatures)); 48 | net.Add(new DropoutLayer(0.2)); 49 | net.Add(new DenseLayer(800, Activation.Relu)); 50 | net.Add(new DropoutLayer(0.5)); 51 | net.Add(new DenseLayer(800, Activation.Relu)); 52 | net.Add(new DropoutLayer(0.5)); 53 | net.Add(new SquaredErrorRegressionLayer()); 54 | 55 | // using only 10 iteration to make the example run faster. 56 | // using square error as error metric. This is only used for reporting progress. 57 | var learner = new RegressionNeuralNetLearner(net, iterations: 10, loss: new SquareLoss()); 58 | var model = learner.Learn(observations, targets); 59 | 60 | var metric = new MeanSquaredErrorRegressionMetric(); 61 | var predictions = model.Predict(observations); 62 | 63 | Trace.WriteLine("Training Error: " + metric.Error(targets, predictions)); 64 | } 65 | 66 | [TestMethod] 67 | public void Regression_Standard_Neural_Net_FeatureTransform_Normalization() 68 | { 69 | #region Read Data 70 | 71 | // Use StreamReader(filepath) when running from filesystem 72 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 73 | var targetName = "quality"; 74 | 75 | // read feature matrix 76 | var observations = parser.EnumerateRows(c => c != targetName) 77 | .ToF64Matrix(); 78 | 79 | // read classification targets 80 | var targets = parser.EnumerateRows(targetName) 81 | .ToF64Vector(); 82 | 83 | #endregion 84 | 85 | // transform pixel values to be between 0 and 1 86 | // and shift each feature to have a mean value of zero. 87 | var minMaxTransformer = new MinMaxTransformer(0.0, 1.0); 88 | var meanZeroTransformer = new MeanZeroFeatureTransformer(); 89 | 90 | minMaxTransformer.Transform(observations, observations); 91 | meanZeroTransformer.Transform(observations, observations); 92 | 93 | var numberOfFeatures = observations.ColumnCount; 94 | 95 | // define the neural net. 96 | var net = new NeuralNet(); 97 | net.Add(new InputLayer(inputUnits: numberOfFeatures)); 98 | net.Add(new DropoutLayer(0.2)); 99 | net.Add(new DenseLayer(800, Activation.Relu)); 100 | net.Add(new DropoutLayer(0.5)); 101 | net.Add(new DenseLayer(800, Activation.Relu)); 102 | net.Add(new DropoutLayer(0.5)); 103 | net.Add(new SquaredErrorRegressionLayer()); 104 | 105 | // using only 10 iteration to make the example run faster. 106 | // using square error as error metric. This is only used for reporting progress. 107 | var learner = new RegressionNeuralNetLearner(net, iterations: 10, loss: new SquareLoss()); 108 | var model = learner.Learn(observations, targets); 109 | 110 | var metric = new MeanSquaredErrorRegressionMetric(); 111 | var predictions = model.Predict(observations); 112 | 113 | Trace.WriteLine("Training Error: " + metric.Error(targets, predictions)); 114 | } 115 | } 116 | } 117 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/NeuralNets/ValidationSetForSelectingBestIterationExamples.cs: -------------------------------------------------------------------------------- 1 | using System.Diagnostics; 2 | using System.IO; 3 | using System.Linq; 4 | using Microsoft.VisualStudio.TestTools.UnitTesting; 5 | using SharpLearning.Containers.Matrices; 6 | using SharpLearning.CrossValidation.TrainingTestSplitters; 7 | using SharpLearning.Examples.Properties; 8 | using SharpLearning.InputOutput.Csv; 9 | using SharpLearning.Metrics.Classification; 10 | using SharpLearning.Neural; 11 | using SharpLearning.Neural.Activations; 12 | using SharpLearning.Neural.Layers; 13 | using SharpLearning.Neural.Learners; 14 | using SharpLearning.Neural.Loss; 15 | 16 | namespace SharpLearning.Examples.NeuralNets 17 | { 18 | [TestClass] 19 | 20 | public class ValidationSetForSelectingBestIterationExamples 21 | { 22 | [TestMethod] 23 | public void Classification_Neural_Net_Using_ValidtionSet_For_Selecting_The_best_Model() 24 | { 25 | #region Read Data 26 | // Use StreamReader(filepath) when running from filesystem 27 | var trainingParser = new CsvParser(() => new StringReader(Resources.mnist_small_train)); 28 | var testParser = new CsvParser(() => new StringReader(Resources.mnist_small_test)); 29 | 30 | var targetName = "Class"; 31 | 32 | var featureNames = trainingParser.EnumerateRows(c => c != targetName).First().ColumnNameToIndex.Keys.ToArray(); 33 | 34 | // read feature matrix (training) 35 | var trainingObservations = trainingParser 36 | .EnumerateRows(featureNames) 37 | .ToF64Matrix(); 38 | // read classification targets (training) 39 | var trainingTargets = trainingParser.EnumerateRows(targetName) 40 | .ToF64Vector(); 41 | 42 | // read feature matrix (test) 43 | var testObservations = testParser 44 | .EnumerateRows(featureNames) 45 | .ToF64Matrix(); 46 | // read classification targets (test) 47 | var testTargets = testParser.EnumerateRows(targetName) 48 | .ToF64Vector(); 49 | #endregion 50 | 51 | // transform pixel values to be between 0 and 1. 52 | trainingObservations.Map(p => p / 255); 53 | testObservations.Map(p => p / 255); 54 | 55 | // create training validation split 56 | var splitter = new StratifiedTrainingTestIndexSplitter(trainingPercentage: 0.7, seed: 24); 57 | var split = splitter.SplitSet(trainingObservations, trainingTargets); 58 | 59 | // the output layer must know the number of classes. 60 | var numberOfClasses = trainingTargets.Distinct().Count(); 61 | 62 | var net = new NeuralNet(); 63 | net.Add(new InputLayer(width: 28, height: 28, depth: 1)); // MNIST data is 28x28x1. 64 | net.Add(new DenseLayer(800, Activation.Relu)); 65 | net.Add(new SoftMaxLayer(numberOfClasses)); 66 | 67 | // using classification accuracy as error metric. 68 | // When using a validation set, the error metric 69 | // is used for selecting the best iteration based on models error on the validation set. 70 | var learner = new ClassificationNeuralNetLearner(net, iterations: 10, loss: new AccuracyLoss()); 71 | 72 | var model = learner.Learn(split.TrainingSet.Observations, split.TrainingSet.Targets,//); 73 | split.TestSet.Observations, split.TestSet.Targets); // the validation set for estimating how well the network generalises to new data. 74 | 75 | var metric = new TotalErrorClassificationMetric(); 76 | var predictions = model.Predict(testObservations); 77 | 78 | Trace.WriteLine("Test Error: " + metric.Error(testTargets, predictions)); 79 | } 80 | } 81 | } 82 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Optimization/HyperparameterTuningExamples.cs: -------------------------------------------------------------------------------- 1 | using System; 2 | using System.Diagnostics; 3 | using System.IO; 4 | using Microsoft.VisualStudio.TestTools.UnitTesting; 5 | using SharpLearning.CrossValidation.CrossValidators; 6 | using SharpLearning.DecisionTrees.Learners; 7 | using SharpLearning.Examples.Properties; 8 | using SharpLearning.InputOutput.Csv; 9 | using SharpLearning.Metrics.Regression; 10 | using SharpLearning.Optimization; 11 | 12 | namespace SharpLearning.Examples.Optimization 13 | { 14 | [TestClass] 15 | public class HyperparameterTuningExamples 16 | { 17 | [TestMethod] 18 | public void Hyper_Parameter_Tuning() 19 | { 20 | #region Read data 21 | 22 | // Use StreamReader(filepath) when running from filesystem 23 | var parser = new CsvParser(() => new StringReader(Resources.winequality_white)); 24 | var targetName = "quality"; 25 | 26 | // read feature matrix 27 | var observations = parser.EnumerateRows(c => c != targetName) 28 | .ToF64Matrix(); 29 | 30 | // read classification targets 31 | var targets = parser.EnumerateRows(targetName) 32 | .ToF64Vector(); 33 | 34 | #endregion 35 | 36 | // metric to minimize 37 | var metric = new MeanSquaredErrorRegressionMetric(); 38 | 39 | // Parameter ranges for the optimizer 40 | var paramers = new IParameterSpec[] 41 | { 42 | new MinMaxParameterSpec(min: 1, max: 100, 43 | transform: Transform.Linear, parameterType: ParameterType.Discrete), // maximumTreeDepth 44 | new MinMaxParameterSpec(min: 1, max: 16, 45 | transform: Transform.Linear, parameterType: ParameterType.Discrete), // minimumSplitSize 46 | }; 47 | 48 | // create random search optimizer 49 | var optimizer = new RandomSearchOptimizer(paramers, iterations: 30, runParallel: true); 50 | 51 | // other availible optimizers 52 | // GridSearchOptimizer 53 | // GlobalizedBoundedNelderMeadOptimizer 54 | // ParticleSwarmOptimizer 55 | // BayesianOptimizer 56 | 57 | // function to minimize 58 | Func minimize = p => 59 | { 60 | var cv = new RandomCrossValidation(crossValidationFolds: 5, seed: 42); 61 | var optlearner = new RegressionDecisionTreeLearner(maximumTreeDepth: (int)p[0], minimumSplitSize: (int)p[1]); 62 | var predictions = cv.CrossValidate(optlearner, observations, targets); 63 | var error = metric.Error(targets, predictions); 64 | 65 | Trace.WriteLine(string.Format("Candidate Error: {0:0.0000}, Candidate Parameters: {1}", 66 | error, string.Join(", ", p))); 67 | 68 | return new OptimizerResult(p, error); 69 | }; 70 | 71 | // run optimizer 72 | var result = optimizer.OptimizeBest(minimize); 73 | var bestParameters = result.ParameterSet; 74 | 75 | Trace.WriteLine("Result: " + result.Error); 76 | 77 | // create learner with found parameters 78 | var learner = new RegressionDecisionTreeLearner(maximumTreeDepth: (int)bestParameters[0], minimumSplitSize: (int)bestParameters[1]); 79 | 80 | // learn model with found parameters 81 | var model = learner.Learn(observations, targets); 82 | } 83 | } 84 | } 85 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Properties/AssemblyInfo.cs: -------------------------------------------------------------------------------- 1 | using System.Reflection; 2 | using System.Runtime.CompilerServices; 3 | using System.Runtime.InteropServices; 4 | 5 | // General Information about an assembly is controlled through the following 6 | // set of attributes. Change these attribute values to modify the information 7 | // associated with an assembly. 8 | [assembly: AssemblyTitle("SharpLearning.Examples")] 9 | [assembly: AssemblyDescription("")] 10 | [assembly: AssemblyConfiguration("")] 11 | [assembly: AssemblyCompany("")] 12 | [assembly: AssemblyProduct("SharpLearning.Examples")] 13 | [assembly: AssemblyCopyright("Copyright © 2015")] 14 | [assembly: AssemblyTrademark("")] 15 | [assembly: AssemblyCulture("")] 16 | 17 | // Setting ComVisible to false makes the types in this assembly not visible 18 | // to COM components. If you need to access a type in this assembly from 19 | // COM, set the ComVisible attribute to true on that type. 20 | [assembly: ComVisible(false)] 21 | 22 | // The following GUID is for the ID of the typelib if this project is exposed to COM 23 | [assembly: Guid("b61d4af1-5a9c-438c-bab8-2c8b3c736056")] 24 | 25 | // Version information for an assembly consists of the following four values: 26 | // 27 | // Major Version 28 | // Minor Version 29 | // Build Number 30 | // Revision 31 | // 32 | // You can specify all the values or you can default the Build and Revision Numbers 33 | // by using the '*' as shown below: 34 | // [assembly: AssemblyVersion("1.0.*")] 35 | [assembly: AssemblyVersion("0.2.0.0")] 36 | [assembly: AssemblyFileVersion("0.2.0.0")] 37 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Properties/Resources.Designer.cs: -------------------------------------------------------------------------------- 1 | //------------------------------------------------------------------------------ 2 | // 3 | // This code was generated by a tool. 4 | // Runtime Version:4.0.30319.42000 5 | // 6 | // Changes to this file may cause incorrect behavior and will be lost if 7 | // the code is regenerated. 8 | // 9 | //------------------------------------------------------------------------------ 10 | 11 | namespace SharpLearning.Examples.Properties { 12 | using System; 13 | 14 | 15 | /// 16 | /// A strongly-typed resource class, for looking up localized strings, etc. 17 | /// 18 | // This class was auto-generated by the StronglyTypedResourceBuilder 19 | // class via a tool like ResGen or Visual Studio. 20 | // To add or remove a member, edit your .ResX file then rerun ResGen 21 | // with the /str option, or rebuild your VS project. 22 | [global::System.CodeDom.Compiler.GeneratedCodeAttribute("System.Resources.Tools.StronglyTypedResourceBuilder", "15.0.0.0")] 23 | [global::System.Diagnostics.DebuggerNonUserCodeAttribute()] 24 | [global::System.Runtime.CompilerServices.CompilerGeneratedAttribute()] 25 | internal class Resources { 26 | 27 | private static global::System.Resources.ResourceManager resourceMan; 28 | 29 | private static global::System.Globalization.CultureInfo resourceCulture; 30 | 31 | [global::System.Diagnostics.CodeAnalysis.SuppressMessageAttribute("Microsoft.Performance", "CA1811:AvoidUncalledPrivateCode")] 32 | internal Resources() { 33 | } 34 | 35 | /// 36 | /// Returns the cached ResourceManager instance used by this class. 37 | /// 38 | [global::System.ComponentModel.EditorBrowsableAttribute(global::System.ComponentModel.EditorBrowsableState.Advanced)] 39 | internal static global::System.Resources.ResourceManager ResourceManager { 40 | get { 41 | if (object.ReferenceEquals(resourceMan, null)) { 42 | global::System.Resources.ResourceManager temp = new global::System.Resources.ResourceManager("SharpLearning.Examples.Properties.Resources", typeof(Resources).Assembly); 43 | resourceMan = temp; 44 | } 45 | return resourceMan; 46 | } 47 | } 48 | 49 | /// 50 | /// Overrides the current thread's CurrentUICulture property for all 51 | /// resource lookups using this strongly typed resource class. 52 | /// 53 | [global::System.ComponentModel.EditorBrowsableAttribute(global::System.ComponentModel.EditorBrowsableState.Advanced)] 54 | internal static global::System.Globalization.CultureInfo Culture { 55 | get { 56 | return resourceCulture; 57 | } 58 | set { 59 | resourceCulture = value; 60 | } 61 | } 62 | 63 | /// 64 | /// Looks up a localized string similar to id;Pixel0_Channel0;Pixel1_Channel0;Pixel2_Channel0;Pixel3_Channel0;Pixel4_Channel0;Pixel5_Channel0;Pixel6_Channel0;Pixel7_Channel0;Pixel8_Channel0;Pixel9_Channel0;Pixel10_Channel0;Pixel11_Channel0;Pixel12_Channel0;Pixel13_Channel0;Pixel14_Channel0;Pixel15_Channel0;Pixel16_Channel0;Pixel17_Channel0;Pixel18_Channel0;Pixel19_Channel0;Pixel20_Channel0;Pixel21_Channel0;Pixel22_Channel0;Pixel23_Channel0;Pixel24_Channel0;Pixel25_Channel0;Pixel26_Channel0;Pixel27_Channel0;Pixel28_Channel0;Pixel29_Channel0;Pixel30_C [rest of string was truncated]";. 65 | /// 66 | internal static string cifar10_test_small { 67 | get { 68 | return ResourceManager.GetString("cifar10_test_small", resourceCulture); 69 | } 70 | } 71 | 72 | /// 73 | /// Looks up a localized string similar to id;Pixel0_Channel0;Pixel1_Channel0;Pixel2_Channel0;Pixel3_Channel0;Pixel4_Channel0;Pixel5_Channel0;Pixel6_Channel0;Pixel7_Channel0;Pixel8_Channel0;Pixel9_Channel0;Pixel10_Channel0;Pixel11_Channel0;Pixel12_Channel0;Pixel13_Channel0;Pixel14_Channel0;Pixel15_Channel0;Pixel16_Channel0;Pixel17_Channel0;Pixel18_Channel0;Pixel19_Channel0;Pixel20_Channel0;Pixel21_Channel0;Pixel22_Channel0;Pixel23_Channel0;Pixel24_Channel0;Pixel25_Channel0;Pixel26_Channel0;Pixel27_Channel0;Pixel28_Channel0;Pixel29_Channel0;Pixel30_C [rest of string was truncated]";. 74 | /// 75 | internal static string cifar10_train_small { 76 | get { 77 | return ResourceManager.GetString("cifar10_train_small", resourceCulture); 78 | } 79 | } 80 | 81 | /// 82 | /// Looks up a localized string similar to Class;Pixel1;Pixel2;Pixel3;Pixel4;Pixel5;Pixel6;Pixel7;Pixel8;Pixel9;Pixel10;Pixel11;Pixel12;Pixel13;Pixel14;Pixel15;Pixel16;Pixel17;Pixel18;Pixel19;Pixel20;Pixel21;Pixel22;Pixel23;Pixel24;Pixel25;Pixel26;Pixel27;Pixel28;Pixel29;Pixel30;Pixel31;Pixel32;Pixel33;Pixel34;Pixel35;Pixel36;Pixel37;Pixel38;Pixel39;Pixel40;Pixel41;Pixel42;Pixel43;Pixel44;Pixel45;Pixel46;Pixel47;Pixel48;Pixel49;Pixel50;Pixel51;Pixel52;Pixel53;Pixel54;Pixel55;Pixel56;Pixel57;Pixel58;Pixel59;Pixel60;Pixel61;Pixel62;Pixel63;Pixel64;Pix [rest of string was truncated]";. 83 | /// 84 | internal static string mnist_small_test { 85 | get { 86 | return ResourceManager.GetString("mnist_small_test", resourceCulture); 87 | } 88 | } 89 | 90 | /// 91 | /// Looks up a localized string similar to Class;Pixel1;Pixel2;Pixel3;Pixel4;Pixel5;Pixel6;Pixel7;Pixel8;Pixel9;Pixel10;Pixel11;Pixel12;Pixel13;Pixel14;Pixel15;Pixel16;Pixel17;Pixel18;Pixel19;Pixel20;Pixel21;Pixel22;Pixel23;Pixel24;Pixel25;Pixel26;Pixel27;Pixel28;Pixel29;Pixel30;Pixel31;Pixel32;Pixel33;Pixel34;Pixel35;Pixel36;Pixel37;Pixel38;Pixel39;Pixel40;Pixel41;Pixel42;Pixel43;Pixel44;Pixel45;Pixel46;Pixel47;Pixel48;Pixel49;Pixel50;Pixel51;Pixel52;Pixel53;Pixel54;Pixel55;Pixel56;Pixel57;Pixel58;Pixel59;Pixel60;Pixel61;Pixel62;Pixel63;Pixel64;Pix [rest of string was truncated]";. 92 | /// 93 | internal static string mnist_small_train { 94 | get { 95 | return ResourceManager.GetString("mnist_small_train", resourceCulture); 96 | } 97 | } 98 | 99 | /// 100 | /// Looks up a localized string similar to "fixed acidity";"volatile acidity";"citric acid";"residual sugar";"chlorides";"free sulfur dioxide";"total sulfur dioxide";"density";"pH";"sulphates";"alcohol";"quality" 101 | ///7;0.27;0.36;20.7;0.045;45;170;1.001;3;0.45;8.8;6 102 | ///6.3;0.3;0.34;1.6;0.049;14;132;0.994;3.3;0.49;9.5;6 103 | ///8.1;0.28;0.4;6.9;0.05;30;97;0.9951;3.26;0.44;10.1;6 104 | ///7.2;0.23;0.32;8.5;0.058;47;186;0.9956;3.19;0.4;9.9;6 105 | ///7.2;0.23;0.32;8.5;0.058;47;186;0.9956;3.19;0.4;9.9;6 106 | ///8.1;0.28;0.4;6.9;0.05;30;97;0.9951;3.26;0.44;10.1;6 107 | ///6.2;0.32;0.16;7;0.045;30;136;0.9 [rest of string was truncated]";. 108 | /// 109 | internal static string winequality_white { 110 | get { 111 | return ResourceManager.GetString("winequality_white", resourceCulture); 112 | } 113 | } 114 | } 115 | } 116 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/Properties/Resources.resx: -------------------------------------------------------------------------------- 1 |  2 | 3 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | text/microsoft-resx 110 | 111 | 112 | 2.0 113 | 114 | 115 | System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 116 | 117 | 118 | System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 119 | 120 | 121 | 122 | ..\Resources\mnist_small_test.csv;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;Windows-1252 123 | 124 | 125 | ..\Resources\mnist_small_train.csv;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;Windows-1252 126 | 127 | 128 | ..\Resources\cifar10_test_small.csv;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;Windows-1252 129 | 130 | 131 | ..\Resources\cifar10_train_small.csv;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;Windows-1252 132 | 133 | 134 | ..\Resources\winequality-white.csv;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;Windows-1252 135 | 136 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/SharpLearning.Examples.csproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | Debug 5 | AnyCPU 6 | {D730F7CD-AAA3-4702-9A23-16435A7944D3} 7 | Library 8 | Properties 9 | SharpLearning.Examples 10 | SharpLearning.Examples 11 | v4.6.2 12 | 512 13 | {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} 14 | 10.0 15 | $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) 16 | $(ProgramFiles)\Common Files\microsoft shared\VSTT\$(VisualStudioVersion)\UITestExtensionPackages 17 | False 18 | UnitTest 19 | 20 | 21 | 22 | 23 | 24 | true 25 | full 26 | false 27 | ..\..\Build\Debug\ 28 | DEBUG;TRACE 29 | prompt 30 | 4 31 | 32 | 33 | pdbonly 34 | true 35 | ..\..\Build\Release\ 36 | TRACE 37 | prompt 38 | 4 39 | 40 | 41 | 42 | ..\..\packages\MathNet.Numerics.4.8.1\lib\net461\MathNet.Numerics.dll 43 | 44 | 45 | ..\..\packages\SharpLearning.AdaBoost.0.31.7\lib\net461\SharpLearning.AdaBoost.dll 46 | 47 | 48 | ..\..\packages\SharpLearning.Common.Interfaces.0.31.7\lib\net461\SharpLearning.Common.Interfaces.dll 49 | 50 | 51 | ..\..\packages\SharpLearning.Containers.0.31.7\lib\net461\SharpLearning.Containers.dll 52 | 53 | 54 | ..\..\packages\SharpLearning.CrossValidation.0.31.7\lib\net461\SharpLearning.CrossValidation.dll 55 | 56 | 57 | ..\..\packages\SharpLearning.DecisionTrees.0.31.7\lib\net461\SharpLearning.DecisionTrees.dll 58 | 59 | 60 | ..\..\packages\SharpLearning.Ensemble.0.31.7\lib\net461\SharpLearning.Ensemble.dll 61 | 62 | 63 | ..\..\packages\SharpLearning.FeatureTransformations.0.31.7\lib\net461\SharpLearning.FeatureTransformations.dll 64 | 65 | 66 | ..\..\packages\SharpLearning.GradientBoost.0.31.7\lib\net461\SharpLearning.GradientBoost.dll 67 | 68 | 69 | ..\..\packages\SharpLearning.InputOutput.0.31.7\lib\net461\SharpLearning.InputOutput.dll 70 | 71 | 72 | ..\..\packages\SharpLearning.Metrics.0.31.7\lib\net461\SharpLearning.Metrics.dll 73 | 74 | 75 | ..\..\packages\SharpLearning.Neural.0.31.7\lib\net461\SharpLearning.Neural.dll 76 | 77 | 78 | ..\..\packages\SharpLearning.Optimization.0.31.7\lib\net461\SharpLearning.Optimization.dll 79 | 80 | 81 | ..\..\packages\SharpLearning.RandomForest.0.31.7\lib\net461\SharpLearning.RandomForest.dll 82 | 83 | 84 | 85 | 86 | 87 | ..\..\packages\System.ValueTuple.4.5.0\lib\net461\System.ValueTuple.dll 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | True 126 | True 127 | Resources.resx 128 | 129 | 130 | 131 | 132 | 133 | ResXFileCodeGenerator 134 | Resources.Designer.cs 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | False 150 | 151 | 152 | False 153 | 154 | 155 | False 156 | 157 | 158 | False 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. 169 | 170 | 171 | 172 | 179 | -------------------------------------------------------------------------------- /src/SharpLearning.Examples/packages.config: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | --------------------------------------------------------------------------------