├── LICENSE-TECH-PREVIEW.txt ├── LICENSE.txt ├── README.adoc └── legal ├── licenses └── Pay-As-You-Go.txt └── privacy └── cloud.txt /LICENSE-TECH-PREVIEW.txt: -------------------------------------------------------------------------------- 1 | Lost Tech LLC permits use of this technical preview of Gradient for the duration of the trial free of charge for non-commercial use, or in small businesses (to qualify as a small business, the company must have both under $1M/year profit and under $5M/year revenue). This product can be used in any other commercial environment, as long as it, any its derivatives or outputs are not used to obtain commerial gains. 2 | 3 | Reverse engineering is prohibited to the extent, permitted by law. 4 | 5 | The product is distributed as is. Lost Tech LLC can not be held liable for any amount, surpassing the cost of the product. 6 | 7 | 8 | THE OPEN SOURCE PROMISE 9 | 10 | In the unlikely event this project is no longer maintained (new versions are not released) for at least 18 consecutive months, its source code should be released under Apache License 2.0. If the code is not released under these circumstances, Lost Tech LLC permits binary decompilation to obtain sources. 11 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2020 Lost Tech LLC 2 | 3 | 1. For non-commercial use and small businesses (must have 4 | under $5M/year turnover and under $1M/year profit): 5 | 6 | Permission is hereby granted, free of charge, to use and copy the Software 7 | for own purposes. Redistribution is permitted only to non-commercial entities, 8 | who must agree to the same terms. Decompilation and other forms of reverse 9 | engineering of the Software are prohibited. 10 | 11 | 2. For commercial use a separate license must be acquired 12 | via https://losttech.software/buy_gradient.html 13 | 14 | 3. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 15 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES 16 | OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 17 | IN NO EVENT SHALL THE LOST TECH LLC BE LIABLE FOR ANY CLAIM, DAMAGES 18 | OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 19 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 20 | DEALINGS IN THE SOFTWARE. -------------------------------------------------------------------------------- /README.adoc: -------------------------------------------------------------------------------- 1 | :toc: macro 2 | :toc-title: 3 | :toclevels: 3 4 | :language: csharp 5 | 6 | # LostTech.TensorFlow (formerly Gradient) 7 | This repository serves as a public issue tracker and documentation for LostTech.TensorFlow, full TensorFlow binding for .NET 8 | 9 | [link=https://www.nuget.org/packages/LostTech.TensorFlow/] 10 | image::https://img.shields.io/nuget/v/LostTech.TensorFlow.svg[NuGet] 11 | 12 | LostTech.TensorFlow enables .NET developers to use the complete set of TensorFlow APIs from any .NET language. 13 | 14 | You can find samples at https://github.com/losttech/Gradient-Samples 15 | 16 | Full API documentation is available at https://gradient.docs.losttech.software/ 17 | 18 | [quote, Char-RNN trained on Shakespeare] 19 | He shall speak not reverbering injurance. 20 | 21 | # Contents 22 | toc::[] 23 | 24 | # Getting started 25 | ## Supported platforms 26 | LostTech.TensorFlow is fully supported on Windows, MacOS, and Linux on AMD64/x64 27 | architecture. It might work on other OS/CPU architecture combinations, but will 28 | only be supported on the best effort basis. 29 | 30 | ## Installation 31 | ### Install Python + Tensorflow 32 | Before installing LostTech.TensorFlow, you should ensure, that you have 33 | TensorFlow installed and working (or use <> from NuGet): 34 | 35 | . Install Python 3.7 64-bit. If you have Visual Studio 2017+, it is possible to install it as a component. Otherwise, get one from https://www.python.org/downloads/ or your system's package manager 36 | . Install TensorFlow 1.15.0 (1.15.1+ release has a https://github.com/tensorflow/tensorflow/issues/36417[critical bug]) 37 | using pip (https://www.tensorflow.org/install/[official instructions]): 38 | .. Find `python` executable in the installation directory (VS installs to: C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\python.exe) 39 | .. Open command line to the directory, containing `python` 40 | .. Execute `.\python -m pip install "tensorflow-gpu==1.15.0"` or `.\python -m pip install "tensorflow-cpu==1.15.0"`. 41 | GPU acceleration requires matching CuDNN and CUDA 10 installed, 42 | see https://www.tensorflow.org/install/gpu#older_versions_of_tensorflow[instructions]. 43 | . Check the installation by launching `python`, and running [source,python]`import tensorflow`. It should succeed. 44 | 45 | ### Add Nuget package to your project 46 | 47 | LostTech.TensorFlow packages are published on https://www.nuget.org/packages/LostTech.TensorFlow/[Nuget]. 48 | Nuget page lists the commands, necessary to install the package into your 49 | project. For dotnet-based projects CLI command is 50 | 51 | [source,powershell] 52 | ---- 53 | dotnet add package LostTech.TensorFlow --pre 54 | ---- 55 | 56 | If using the new package management features of `.csproj`, this could also be achieved by adding the following line to it: 57 | 58 | [source,xml] 59 | ---- 60 | 61 | ---- 62 | 63 | See the example project file https://github.com/losttech/Gradient-Samples/blob/master/BasicMath/BasicMath.csproj[here]. 64 | 65 | ## First steps 66 | 67 | ### Packages and Namespaces 68 | In most cases, you will need to add `using tensorflow;` at the beginning of your 69 | file. In many cases you will also need `using LostTech.TensorFlow;` and 70 | `using numpy;`. 71 | 72 | [source,csharp] 73 | ---- 74 | using numpy; 75 | using tensorflow; 76 | using LostTech.TensorFlow; 77 | ---- 78 | 79 | Commercial users will also need to add a https://www.nuget.org/packages/LostTech.Gradient.License.Azure/[licensing package for Pay-As-You-Go]: 80 | [link=https://www.nuget.org/packages/LostTech.Gradient.License.Azure/] 81 | image::https://img.shields.io/nuget/v/LostTech.Gradient.License.Azure.svg[NuGet] 82 | 83 | And configure the key from the https://lt-tf-lic-portal.azurewebsites.net/Home/Subscriptions[Subscriptions Page]. 84 | 85 | [source,csharp] 86 | ---- 87 | using LostTech.TensorFlow; 88 | using LostTech.TensorFlow.Licensing; 89 | TensorFlowSetup.Instance.UseLicense(new AzureLicense("...Key Goes Here...")); 90 | TensorFlowSetup.Instance.EnsureInitialized(); 91 | ---- 92 | 93 | ### Logging 94 | https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/compat/v1/logging[TensorFlow logging] 95 | is separate from LostTech.TensorFlow logging. This section discusses the later. 96 | 97 | LostTech.TensorFlow core runtime library is https://www.nuget.org/packages/Gradient.Runtime/[Gradient.Runtime]. 98 | To configure runtime logging, set appropriate properties of https://gradient.docs.losttech.software/Runtime/v0.4.2/LostTech.Gradient/GradientLog.htm[`LostTech.Gradient.GradientLog`] static class, e.g.: 99 | [source,csharp] 100 | GradientLog.OutputWriter = Console.Out; 101 | 102 | ### Selecting Python environment 103 | If you want to use TensorFlow with non-default configuration (e.g. different versions instead of Python 3.7 + TensorFlow 1.15), 104 | use one of `LostTech.Gradient.GradientEngine.UseEnvironment*` methods before accessing any TensorFlow methods to select the desired TensorFlow installation. 105 | 106 | We also recommend to explicitly call `TensorFlowSetup.Instance.EnsureInitialized()` 107 | to be able to catch any problems with TensorFlow installation. This is especially important 108 | in production systems. 109 | 110 | ### Using packaged environment 111 | [link=https://www.nuget.org/packages/LostTech.TensorFlow.Python/] 112 | image::https://img.shields.io/nuget/v/LostTech.TensorFlow.Python.svg[NuGet] 113 | 1. Install NuGet package https://www.nuget.org/packages/LostTech.TensorFlow.Python[LostTech.TensorFlow.Python] 114 | 2. Deploy TensorFlow from the package and configure the engine to use it: 115 | 116 | [source,csharp] 117 | ---- 118 | var pyEnv = LostTech.TensorFlow.PackagedTensorFlow.EnsureDeployed(DIRECTORY); 119 | GradientEngine.UseEnvironment(pyEnv); 120 | ---- 121 | 122 | ### Old style TF 123 | Prior to the recent changes, the main way to use TensorFlow was to contstruct a computation graph, and then run it in a session. Most of the existing examples will use this mode. 124 | 125 | #### Constructing compute graph 126 | `Graph` creation methods are located in the `tf` class from `tensorflow` namespace. For example: 127 | 128 | [source,csharp] 129 | ---- 130 | var a = tf.constant(5.0, name: "a"); 131 | var b = tf.constant(10.0, name: "b"); 132 | 133 | var sum = tf.add(a, b, name: "sum"); 134 | var div = tf.div(a, b, name: "div"); 135 | ---- 136 | 137 | #### Running computation 138 | Next, you need to create a `Session` to run your graph one or multiple times. Sessions allocate CPU, GPU and memory resources, and hold the states of variables. 139 | 140 | NOTE: In GPU mode, TensorFlow will attempt to allocate all the GPU memory to itself at that stage, 141 | so ensure you don't have any other programs extensively using it, or https://stackoverflow.com/questions/34199233/how-to-prevent-tensorflow-from-allocating-the-totality-of-a-gpu-memory[turn down TensorFlow memory allocation] 142 | 143 | Since TensorFlow sessions hold unmanaged resources, they have to be used with `IDisposable` pattern: 144 | [source,csharp] 145 | ---- 146 | var session = new Session(); 147 | using(session.StartUsing()) { 148 | ...do something with the session... 149 | }); 150 | ---- 151 | 152 | Now that you have a `Session` to work with, you can actually compute the values in the graph: 153 | 154 | [source,csharp] 155 | ---- 156 | var session = new Session(); 157 | using(session.StartUsing()) { 158 | Console.WriteLine($"a = {session.run(a)}"); 159 | Console.WriteLine($"b = {session.run(b)}"); 160 | Console.WriteLine($"a + b = {session.run(sum)}"); 161 | Console.WriteLine($"a / b = {session.run(div)}"); 162 | }; 163 | ---- 164 | 165 | The full code for this example is available at our https://github.com/losttech/Gradient-Samples/tree/master/BasicMath[samples repository] 166 | 167 | # Porting Python code to LostTech.TensorFlow + C# 168 | In most cases converting Python code, that uses TensorFlow, should be as easy as using C# syntax instead of Python one: 169 | 170 | * add `new` to class constructor calls: `Class()` -> `new Class()`. 171 | 172 | __Its easy to spot class construction vs simple function calls in Python: 173 | by convention function names there start with a lower case letter like `min`, 174 | while in class names the first letter is capitalized: `Session` __ 175 | 176 | * to pass named paramters, use `:` instead of `=`: `make_layer(kernel_bias=2.0)` -> `make_layer(kernel_bias: 2.0)` 177 | * to get a subrange of a `Tensor` , use <> syntax (if available): `tensor[1..-2]` -> `tensor[1..^3]` (when using C# 8 ranges, note, that the right side in C# is *INCLUSIVE*, while in Python it is *EXCLUSIVE*). A single element can be addressed as usual: `tensor[1]` 178 | 179 | ## Names of classes and functions 180 | Generally, LostTech.TensorFlow follows TensorFlow https://www.tensorflow.org/versions/r1.15/api_docs/python/tf[Python API] naming. 181 | There are, though, language-based differences: 182 | 183 | * in Python modules (roughly equivalent to namespaces) can directly contain functions. In .NET every function must be a part of some type. For that reason LostTech.TensorFlow exposes static classes, named after the innermost module name to contain module functions and properties (but not classes). For example, Python's `tensorflow.contrib.data` module has a correspoding C# class `tensorflow.contrib.data.data`. So an equivalent of Python's `tensorflow.contrib.data.group_by_window` would be `tensorflow.contrib.data.data.group_by_window`. This mostly applies to the unofficial APIs. 184 | * most of the official API's functions and properties (but *not* classes) are exposed via a special class `tensorflow.tf`. Combined with `using tensorflow;` this enables invoking TensorFlow functions as neatly as: `tf.placeholder(...)`, `tf.keras.activations.relu(...)`, etc 185 | 186 | __there is also a similar class__ `numpy.np` __for NumPy functions__ 187 | 188 | * class names and namespaces are mostly the same as in Python API. 189 | E.g. https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/Session[`tf.Session`] is in `tensorflow` namespace, 190 | and can be instantiated via `new tensorflow.Session(...)` or simply `new Session(...)` with `using tensorflow;` 191 | 192 | * some APIs have multiple aliases, like https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/add[tf.add]. 193 | Only one of the aliases is exposed by LostTech.TensorFlow. Usually the shortest one. 194 | 195 | * in case of name conflicts (e.g. C# does not allow both `shape` property and `set_shape` method in the same class), 196 | one of the conflicting names is exposed with suffix `$$_$$`. For example: `set_shape$$_$$`, which should be easy to find in IDE autocomplete list. 197 | 198 | * (very rare) due to the way LostTech.TensorFlow works, non-official classes, functions and properties might be exposed via unexpected namespaces. 199 | IDE should be able to help find classes (by suggesting to add an appropriate `using namespace;`). For functions and properties, one might try to find the class, corresponding to their containing module (see the example with `tensorflow.contrib.data` above, you could search for the `data` class). Another less convenient alternative is to use Visual Studio's Object Explorer. 200 | 201 | * (rare) some classes and functions, exposed by TensorFlow might only be exposed as function-typed properties. 202 | For example, https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/ConfigProto[`ConfigProto`], 203 | that is used to configure `tf.Session` does not have a correspoing class in LostTech.TensorFlow. 204 | To create an instance of `ConfigProto`, you must call its constructor via `ConfigProto` property in [title="tensorflow.core.protobuf.config_pb2"]`config_pb2` class: `config_pb2.ConfigProto.CreateInstance()`. 205 | 206 | ## Parameter and return types 207 | LostTech.TensorFlow tries hard to expose statically-typed API, but the underlying TensorFlow code is inherently dynamic. 208 | In many cases LostTech.TensorFlow over-generalizes or under-generalizes underlying parameter and return types. 209 | 210 | When the parameter type is over-generalized, it simply means you loose a hint as to what can actually be passed. 211 | LostTech.TensorFlow's parameter may be `IEnumerable`, but the function can reject anything except a `PythonSet`. 212 | In these cases you can either refer to the https://www.tensorflow.org/versions/r1.15/api_docs/python/tf[official documentation], 213 | or quickly try it, and see if the error you get explains what the function actually expects. 214 | 215 | For convenience, any 1D .NET arrays are passed as instances of `PythonList` 216 | by default. This also applies to enumerables produced by `System.Linq`. 217 | This behavior can be turned off using `IsEnabled` properties in 218 | https://gradient.docs.losttech.software/Runtime/v0.4.2/LostTech.Gradient.Codecs/[LostTech.Gradient.Codecs]. 219 | 220 | ### Dynamic overloads 221 | TL;DR; when you can't pass something or get `InvalidCastException`, replace `tf.func_name(...)` -> `tf.func_name_dyn(...)`, 222 | and `new Class(...)` -> `Class.NewDyn(...)`. 223 | 224 | When the parameter or return type is under-generalized, you will not be able to use LostTech.TensorFlow's statically-typed API. 225 | A function parameter may say, that it only accepts `int` and `bool`, but you know from documentation/sample, 226 | that you have to pass a `Tensor`. Another common example is when LostTech.TensorFlow thinks the parameter must be 227 | of a derived class, when a base class would actually also be ok. For example, parameter `cell` might be of type `LSTMCell`, 228 | but actually you should be able to pass any `RNNCell`, where `class LSTMCell: RNNCell`. 229 | Do not try converting the value you want to pass to the expected type. It will not work. 230 | For these cases LostTech.TensorFlow provides dynamic API alongside statically-typed one. 231 | 232 | Every function from original API will have an untyped overload, whose name ends with `_dyn`. 233 | All its parameters intentionally allow anything to be passed (type `object`). 234 | It also returns a `dynamic` type. 235 | 236 | Same applies to properties. For each `SomeType property{get;set;}` there's a `dynamic property_dyn{get;set;}`. 237 | 238 | Every class with constructors will have an untyped static factory method, named `NewDyn`, 239 | which allows you to call class constructor similar to untyped function overloads in the previos paragraph. 240 | 241 | Please, report to this issue tracker, if you have to call dynamic overloads a lot to get your model running. 242 | We will try to fix that in the next version. 243 | 244 | In some cases even that is not enough. If you need to call a method or access a property of an instance of some class, 245 | and that method/property is not exposed by LostTech.TensorFlow, convert the instance to `dynamic`, and try to call it that way. 246 | See https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/types/using-type-dynamic 247 | 248 | ## Passing functions 249 | Many TensorFlow APIs accept functions as parameters. 250 | If the parameter type is known to be a function, LostTech.TensorFlow will show 251 | it as `PythonFunctionContainer`[https://gradient.docs.losttech.software/Runtime/v0.4.2/LostTech.Gradient/PythonFunctionContainer.htm]. 252 | 253 | There are two ways to get an instance of it: pass TensorFlow functions back, or pass a .NET function. 254 | 255 | ### Passing TensorFlow functions back to TensorFlow 256 | TL;DR; suffix your function with `_fn`. 257 | 258 | Most NN layers expect an `activation` argument, which specifies the neuron activation function. 259 | TensorFlow defines many activation functions one would want to use in both modern and old-style APIs. 260 | The "original" one is called https://en.wikipedia.org/wiki/Sigmoid_function[sigmoid] as is available as `tf.sigmoid`. 261 | Modern networks often use some variant of https://en.wikipedia.org/wiki/Rectifier_(neural_networks)[ReLU] (`tf.nn.relu`). 262 | You can call both directly like this: `tf.sigmoid(tensor)`, but in most cases you need to pass them 263 | to `activation` parameter as `PythonFunctionContainer`. 264 | 265 | To do that you can simply get a pre-wrapped instance by adding `_fn` suffix to the function name. 266 | 267 | For example: `tf.layer.dense(activation: *tf.sigmoid_fn*)`. 268 | 269 | ### Passing C# functions to TensorFlow 270 | To get an instance of `PythonFunctionContainer` from a C# function, use static method `PythonFunctionContainer.Of(func or lambda)`. 271 | You will have to specify function argument types in place of ``. 272 | 273 | ## Python `with` blocks, C#'s `using` 274 | TL;DR; replace `with new Session(...) as sess: sess.do_stuff()` 275 | -> 276 | [source,csharp] 277 | ---- 278 | var session = new Session(...); 279 | using (session.StartUsing()) { 280 | session.do_stuff(); 281 | } 282 | ---- 283 | 284 | You can also use `new Session().UseSelf(sess => sess.DoStuff())`. 285 | 286 | TensorFlow API, being built on Python, use special *enter* and *exit* methods for the same purpose 287 | .NET has `IDisposable`. Problem is: in general they do not map directly to each other. 288 | For that reason every LostTech.TensorFlow class, that declares those special methods in TensorFlow, 289 | also exposes `.Use` and `.UseSelf` methods. In most cases it is easiest to use `.UseSelf(self => do_something(self))` 290 | as shown in the sample above. However, there might be rare special cases, when `.Use(context => do_something(context))` 291 | has to be used. The difference is that `obj.UseSelf` always passes `obj` back to the lambda, 292 | while `obj.Use` might actually generate a new object of potentially completely different type. 293 | 294 | Think of `.Use` and `.UseSelf` as LostTech.TensorFlow's best attempt at reproducing `using(var session = new Session(...)) {}` statement. 295 | 296 | ## Exceptions 297 | Most of TensorFlow exceptions have a counterpart either in LostTech.TensorFlow 298 | or in Gradient.Runtime[https://gradient.docs.losttech.software/Runtime/v0.4.2/LostTech.Gradient.Exceptions/]. 299 | 300 | If TensorFlow throws an exception, that has no counterpart, it will surface as 301 | a generic `PythonException`[https://csharpdoc.hotexamples.com/class/Python.Runtime/PythonException]. 302 | 303 | ## NumPy 304 | Since most TensorFlow samples use NumPy, LostTech.TensorFlow includes a limited subset under `numpy` namespace. 305 | It is shipped in a separate package: https://www.nuget.org/packages/LostTech.NumPy/[LostTech.NumPy]. 306 | 307 | [link=https://www.nuget.org/packages/LostTech.NumPy/] 308 | image::https://img.shields.io/nuget/v/LostTech.NumPy.svg[NuGet] 309 | 310 | ## anchor:inheritance[]Custom layers and `Model` subclassing 311 | 312 | NOTE: When subclassing `tensorflow.keras.Model`, every layer, variable or tensor 313 | must be explicitly tracked using `this.Track` method. See 314 | https://github.com/losttech/Gradient-Samples/blob/03aa035080d3a46fe6a4c8dcfd6e8f1b91a414a7/ResNetBlock/ResNetBlock.cs#L19[ResNetBlock sample]. 315 | 316 | https://www.tensorflow.org/tutorials/customization/custom_layers[The official TensorFlow tutorial] 317 | 318 | 319 | 320 | ## Recommendations 321 | * import both `tensorflow` and `numpy` namespaces: 322 | [source,csharp] 323 | ---- 324 | using tensorflow; 325 | using numpy; 326 | 327 | tf.placeholder(...); 328 | np.array(...); 329 | ---- 330 | * if you extensively use some API set under `tf.`, use `using static tf.API_HERE;` 331 | [source,csharp] 332 | ---- 333 | using static tf.keras; 334 | ... 335 | var model = models.load_model(...); 336 | new Dense(kernel_regularizer: regularizers.l2(...)); 337 | ---- 338 | * many LostTech.TensorFlow functions return `dynamic`. Whenever possible, immediately cast it to the concrete type. 339 | It will help to maintain the code. Concrete type is always known at runtime 340 | and can be seen in the debugger, or accessed via `object.GetType()` method. 341 | Most methods in `tf.` usually return `Tensor`. 342 | [source,csharp] 343 | ---- 344 | Tensor hidden = tf.layers.dense(input, hiddenSize, activation: tf.sigmoid_fn); 345 | ---- 346 | 347 | * avoid directly using classes in `Python.Runtime`. 348 | They are LostTech.TensorFlow's implementation details, which might be changed 349 | in the future major versions. 350 | 351 | # Known limitations 352 | __This section may be outdated__ 353 | 354 | ## Unloading AppDomain is not supported 355 | 356 | LostTech.TensorFlow is incompatible with `AppDomain` unloading. An attempt to 357 | unload an `AppDomain` where TensorFlow was initialized will lead to a crash 358 | in native code. 359 | 360 | This is a known problem with Unity editor, which means you can not use 361 | LostTech.TensorFlow within the editor. You must skip all TensorFlow code 362 | using the https://docs.unity3d.com/ScriptReference/Application-isEditor.html[isEditor] check. 363 | 364 | # Tips and Tricks 365 | [#csharp8] 366 | ## C# 8 367 | LostTech.TensorFlow supports the neat indexing feature of C# 8: if you are using Visual Studio 2019, 368 | you can set appropriate language level like this in the project file: `8.0`. 369 | 370 | Then you can access numpy arrays with the new syntax, for example: `arr[3..^4]`, which means "take a range from element at index 3, that includes all elements until (and including) the element with index 4 (counting from the end of the array)". 371 | 372 | # Blogs, Blog Posts & 3rd-party Samples 373 | - https://habr.com/post/453232/[GPT-2: Writing billion songs with C# and Deep Learning] 374 | - https://habr.com/post/437174/[.NET, TensorFlow, and the windmills of Kaggle — the journey begins] 375 | - http://ml.blogs.losttech.software/Reinforcement-Learning-With-Unity-ML-Agents/[Reinforcement learning with Unity ML agents] 376 | - https://lostmsu.github.io/Not-CSharp/[Not C#] - training a convolutional network to recognize programming languages 377 | 378 | 379 | 380 | # What's new 381 | ## Release Candidate: 382 | - replaced expiration with licenses 383 | - improved typing on many APIs 384 | - fixed inability to access static settings 385 | - strongly-typed wrappers for `Tensor` 386 | - enhanced `ndarray` 387 | - improved exception handling and debugging 388 | - core runtime components include source and debug symbols 389 | - LINQ enumerables and 1D .NET arrays are now automatically converted to Python 390 | lists for compatibility with bad TensorFlow APIs (can be disabled) 391 | 392 | ## Preview 7: 393 | - TensorFlow 1.15 394 | - strongly-typed accessors for `ndarray` 395 | - arithmetic, bitwise and comparison operators on Tensors (note, now to check for null `is null` must be used instead of `== null`) 396 | - `StartUsing` extension on classes like `Session`, `variable_scope`, etc to allow `using (new variable_scope(...).StartUsing()) { ... }` 397 | - improved support for enums 398 | - prepackaged TensorFlow runtime on NuGet.org for easy installation: 399 | https://www.nuget.org/packages/LostTech.TensorFlow.Python[LostTech.TensorFlow.Python] 400 | - minimal wrapper for NumPy is released in a separate package (see dependencies) 401 | - runtime initialization moved to Gradient.Runtime 402 | - bugfixes: https://github.com/losttech/Gradient/milestone/3[see Milestone] + internally reported bugs 403 | - new sample: https://github.com/losttech/Gradient-Samples/tree/master/RL-MLAgents/[reinforcement learning with Unity ML agents] 404 | explained in detail in http://ml.blogs.losttech.software/Reinforcement-Learning-With-Unity-ML-Agents/[a blog post] 405 | 406 | ## Preview 6.x: 407 | - feature: ability to <> (for example, allows 408 | to create a custom Keras `Model`, `Callback`, `Layer`, etc) 409 | - new sample: https://github.com/losttech/Gradient-Samples/tree/master/ResNetBlock[ResNetBlock] 410 | - feature: TensorFlow classes are properly marshalled when passed back to you from TensorFlow 411 | - fixed: inability to add items to collections, belonging to TensorFlow classes 412 | - fixed: crash while enumerating collections without an explicit GIL lock 413 | - fixed: crash due to use-after-free of TensorFlow objects in marshalling layer 414 | - fixed: `PythonClassContainer.Instance` failing for nested classes 415 | - fixed: `params object[]` were not passed correctly 416 | - minor: added `np.expand_dims`, reduced number of thrown and handled exceptions 417 | - expires in March 2020 418 | 419 | ## Preview 5.1: 420 | - improved passing dictionaries 421 | - setup: optionally specify Conda environment via an environment variable 422 | - setup: fixed Conda environment autodectection on Linux 423 | - improved argument types in many places 424 | - Gradient warnings are now printed to Console.Error by default, instead of Console.Out 425 | - fixed crashes on dynamic interop and multithreaded enumeration 426 | - fixed some properties not being exposed https://github.com/losttech/Gradient/issues/4 427 | 428 | ## Preview 5: 429 | - support for indexing `Tensor` objects via `dynamic` 430 | - allow using specific Python environment via `GradientSetup.UsePythonEnvironment` 431 | - numerous fixes in the interop layer 432 | - https://github.com/losttech/Gradient-Samples/tree/master/GPT-2[GPT-2 sample] 433 | 434 | ## Preview 4: 435 | - MacOS and Ubuntu support (with others possibly working too) on .NET Core 436 | - documentation included for function and parameter tooltips 437 | - fixed inability to call static class methods 438 | 439 | ## Preview 3 440 | - fixed inability to reenter TensorFlow from a callback 441 | 442 | ## Preview 2: 443 | 444 | - dynamically typed overloads, that enable fallback for tricky signatures 445 | - a common interface for tf.Variable and tf.Tensor 446 | - enabled enumeration over TensorFlow collection types 447 | 448 | # FAQ 449 | ## Why not TensorFlowSharp? 450 | |=== 451 | | | TensorFlowSharp | LostTech.TensorFlow 452 | 453 | | Load TensorFlow models 454 | | *✓* 455 | | *✓* 456 | 457 | | Train existing models 458 | | *✓* 459 | | *✓* 460 | 461 | | Create new models with low-level API 462 | | *✓* 463 | | *✓* 464 | 465 | | Create new models with high-level API 466 | | ✗ 467 | | *✓* 468 | 469 | | Dependencies 470 | | *TF* 471 | | TF + Python 472 | 473 | | TensorBoard integration 474 | | ✗ 475 | | *✓* 476 | 477 | | Estimators 478 | | ✗ 479 | | *✓* 480 | 481 | | Dataset manipulation via tf.data 482 | | ✗ 483 | | *✓* 484 | 485 | | tf.contrib 486 | | ✗ 487 | | *✓* 488 | 489 | | Commercial support 490 | | ✗ 491 | | *✓* 492 | |=== 493 | 494 | ## Why not TensorFlow.NET? 495 | ### Incomplete set of functions 496 | TensorFlow.NET does not provide full functionality of TensorFlow. As a result, 497 | https://github.com/SciSharp/TensorFlow.NET/issues/352[it is hard to implement] 498 | state of the art algorithms for **computer vision (YOLOv3)** and 499 | **language processing (GPT and BERT)** using TensorFlow.NET, especially from 500 | scratch. We have complete LostTech.TensorFlow-based samples for both: 501 | https://github.com/losttech/YOLOv4[YOLOv4] and 502 | https://github.com/losttech/Gradient-Samples/tree/master/GPT-2[GPT-2] 503 | and https://github.com/losttech/Gradient-Samples/blob/master/README.md[many more]. 504 | 505 | ### Ghost APIs 506 | TensorFlow.NET goal is to be a reimplementation of TensorFlow in C#. 507 | However, as of August 2020 only a small set of APIs actually has 508 | implementations. Many functions and classes are defined without bodies 509 | and do nothing. The state of specific APIs is not tracked, 510 | and that can create a lot of confusion. For example, there is an 511 | [line-through]#https://github.com/SciSharp/TensorFlow.NET/blob/master/src/TensorFlowNET.Core/Train/AdamOptimizer.cs[AdamOptimizer]# (they got AdamOptimizer since, but the problem is https://github.com/SciSharp/TensorFlow.NET/blob/c2138b20abc41b19a5e1d3568cdeed87bc1c7369/src/TensorFlowNET.Core/Train/GradientDescentOptimizer.cs#L38[systemic]) 512 | class, but it does not actually have any implementation, apart from 513 | the constructor, meaning it wont actually use Adam, or work at all. 514 | 515 | ### Performance 516 | LostTech.TensorFlow uses official builds of TensorFlow provided by Google, which 517 | are well-optimized. As a result, in a https://github.com/losttech/Gradient-Perf/[simple comparison] 518 | (training a CNN) LostTech.TensorFlow is about 18% faster than 519 | TensorFlow.NET. 520 | 521 | We also support TensorFlow builds, that use other accelerators, such as TPUs in 522 | Google cloud, or https://pypi.org/project/tensorflow-rocm[tensorflow-rocm] for 523 | AMD GPUs. 524 | 525 | # Credits 526 | TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc. -------------------------------------------------------------------------------- /legal/licenses/Pay-As-You-Go.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2020 Lost Tech LLC 2 | 3 | This license is only applicable to Pay-As-You-Go subscribers. 4 | Sign up at https://portal.azure.com/#create/losttechllc.tensorflow/preview 5 | 6 | 1. Permission is hereby granted, to use and copy the Software to any 7 | Internet-connected device for any purpose. Every running copy of the Software 8 | must be supplied with an appropriate key for an active subscription, and 9 | be able to access metering services. 10 | 11 | 2. A separate license must be obtained to access software source code. 12 | Decompilation and other forms of reverse engineering of the Software are 13 | prohibited. 14 | 15 | 3. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 16 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES 17 | OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 18 | IN NO EVENT SHALL THE LOST TECH LLC BE LIABLE FOR ANY CLAIM, DAMAGES 19 | OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 20 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 21 | DEALINGS IN THE SOFTWARE. -------------------------------------------------------------------------------- /legal/privacy/cloud.txt: -------------------------------------------------------------------------------- 1 | Lost Tech TensorFlow Pay-As-You-Go Privacy Policy 2 | 3 | 4 | Lost Tech TensorFlow Pay-As-You-Go collects the following information: 5 | 6 | - information about physical or virtual hardware the Software is running on 7 | - amount of time spent using the Software 8 | - name of the cloud provider and type of the instance the Software is running on 9 | - unique identifier of the cloud subscription 10 | 11 | 12 | This information is used to calculate the correct bill amount for the usage 13 | of the Software and may be processed and stored in Azure or your cloud provider. 14 | We never share this information with third parties. 15 | 16 | 17 | You can also explicitly opt-in to share the following information with Lost Tech LLC: 18 | 19 | - types of objects you create and use with TensorFlow 20 | 21 | This information is used to improve future versions of Lost Tech TensorFlow. 22 | It is never shared with third parties. 23 | 24 | Lost Tech does not collect any other personally identifiable information. 25 | --------------------------------------------------------------------------------