├── bptt.png ├── neuron.png ├── res-nets.png ├── hyperband.jpg ├── layout_grid.jpg ├── layout_legend.jpg ├── layout_random.jpg ├── lstm-formula.png ├── timeline-ai.png ├── deployment_loop.jpg ├── neural-network.png ├── artificial-neuron.png ├── rnn-architectures.png ├── supervised-learning.png ├── data_quantity_on_error.jpg ├── model_complexity_error.jpg ├── gradient_descent_updates.png ├── encoder-decoder-seq-to-seq.png ├── learning_rate_effect_on_loss.jpg ├── the-art-of-choosing-a-model.png ├── vanishing-gradient-problem.png ├── machine-learning-business-process.png ├── .gitmodules ├── .gitignore └── README.md /bptt.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/bptt.png -------------------------------------------------------------------------------- /neuron.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/neuron.png -------------------------------------------------------------------------------- /res-nets.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/res-nets.png -------------------------------------------------------------------------------- /hyperband.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/hyperband.jpg -------------------------------------------------------------------------------- /layout_grid.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/layout_grid.jpg -------------------------------------------------------------------------------- /layout_legend.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/layout_legend.jpg -------------------------------------------------------------------------------- /layout_random.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/layout_random.jpg -------------------------------------------------------------------------------- /lstm-formula.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/lstm-formula.png -------------------------------------------------------------------------------- /timeline-ai.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/timeline-ai.png -------------------------------------------------------------------------------- /deployment_loop.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/deployment_loop.jpg -------------------------------------------------------------------------------- /neural-network.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/neural-network.png -------------------------------------------------------------------------------- /artificial-neuron.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/artificial-neuron.png -------------------------------------------------------------------------------- /rnn-architectures.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/rnn-architectures.png -------------------------------------------------------------------------------- /supervised-learning.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/supervised-learning.png -------------------------------------------------------------------------------- /data_quantity_on_error.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/data_quantity_on_error.jpg -------------------------------------------------------------------------------- /model_complexity_error.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/model_complexity_error.jpg -------------------------------------------------------------------------------- /gradient_descent_updates.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/gradient_descent_updates.png -------------------------------------------------------------------------------- /encoder-decoder-seq-to-seq.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/encoder-decoder-seq-to-seq.png -------------------------------------------------------------------------------- /learning_rate_effect_on_loss.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/learning_rate_effect_on_loss.jpg -------------------------------------------------------------------------------- /the-art-of-choosing-a-model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/the-art-of-choosing-a-model.png -------------------------------------------------------------------------------- /vanishing-gradient-problem.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/vanishing-gradient-problem.png -------------------------------------------------------------------------------- /machine-learning-business-process.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Neuraxio/Machine-Learning-Figures/HEAD/machine-learning-business-process.png -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "GPU-Moores-law/microprocessor-trend-data"] 2 | path = GPU-Moores-law/microprocessor-trend-data 3 | url = https://github.com/guillaume-chevalier/microprocessor-trend-data 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # [Machine Learning Figures](https://github.com/Neuraxio/Machine-Learning-Figures) 2 | 3 | ## Timeline 4 | 5 | ![](timeline-ai.png) 6 | 7 | ## The Business Process 8 | 9 | ![](machine-learning-business-process.png) 10 | 11 | ## Deployment Loop 12 | 13 | ![](deployment_loop.jpg) 14 | 15 | ## Choosing a model 16 | 17 | ![](the-art-of-choosing-a-model.png) 18 | 19 | ## Moore's Law 20 | 21 | Note that this figure is available under the [Creative Commons Attribution 4.0 International Public (CC BY)](https://github.com/karlrupp/microprocessor-trend-data/blob/master/LICENSE.txt) License. 22 | 23 | ![](https://raw.githubusercontent.com/guillaume-chevalier/microprocessor-trend-data/7bbd582ba1376015f6cf24498f46db62811a2919/42yrs/42-years-processor-trend.png) 24 | 25 | ## Neuron 26 | 27 | ![](neuron.png) 28 | 29 | ## Artificial Neuron 30 | 31 | ![](artificial-neuron.png) 32 | 33 | ## Neural Network 34 | 35 | ![](neural-network.png) 36 | 37 | ## Gradient Descent Updates 38 | 39 | ![](gradient_descent_updates.png) 40 | 41 | ## Residual neural network (ResNet) 42 | 43 | ![](res-nets.png) 44 | 45 | ## RNN architecture 46 | 47 | ![](rnn-architectures.png) 48 | 49 | ## LSTM formula 50 | 51 | ![](lstm-formula.png) 52 | 53 | ## Encoder Decoder Seq2Seq 54 | 55 | ![](encoder-decoder-seq-to-seq.png) 56 | 57 | ## Backpropagation Through Time (BPTT) 58 | 59 | ![](bptt.png) 60 | 61 | ## Effect of data quantity on error 62 | 63 | ![](data_quantity_on_error.jpg) 64 | 65 | ## Effect of learning rate on loss 66 | 67 | ![](learning_rate_effect_on_loss.jpg) 68 | 69 | ## Effect of model complexity on loss 70 | 71 | ![](model_complexity_error.jpg) 72 | 73 | ## Grid Search v.s. Random Search 74 | 75 | ![](layout_grid.jpg) 76 | 77 | ![](layout_random.jpg) 78 | 79 | ![](layout_legend.jpg) 80 | 81 | ## Hyperband 82 | 83 | ![](hyperband.jpg) 84 | 85 | # Vanishing Gradient Problem 86 | 87 | ![](vanishing-gradient-problem.png) 88 | 89 | ## License 90 | 91 | Creative Commons License 92 | 93 | This work is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/) License. 94 | 95 | [Link to this page](https://github.com/Neuraxio/Machine-Learning-Figures) to use our figures with citation attributed to Neuraxio. 96 | --------------------------------------------------------------------------------