├── .gitignore ├── README.md ├── data └── README.md ├── pic.png ├── requirements.txt ├── requirements_data_only.txt └── src ├── __init__.py ├── examples ├── 1-shape_classifier.py ├── 2-sentiment_analysis.py └── 3-decision_trees.py ├── generators ├── random_tabular.py ├── sentiment_text.py └── shape_images.py ├── paths.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- 1 | data/*/** 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Hello Deep Learning 2 | 3 | Most introductions to deep learning are too complicated because they spend too much time trying to thrill you with details and real-world applications. 4 | 5 | ![robot waving](pic.png) 6 | 7 | This makes them too cumbersome. You already know that deep learning is amazing and that it actually works on real problems. You know that most of the hard work in industry is in the data cleaning. You don't want to set up a new environment, or play with parameters, or get dirty in the data. 8 | 9 | The first thing you want to do is to train a model, as soon as possible, and it doesn't matter how simple it is. Once you've made your very own model you'd be more than happy to find out about overfitting, data cleaning, and splitting strategies as well. But first you just want to make something yourself and see it work. 10 | 11 | *Hello Deep Learning* is the missing introduction to deep learning. It's a series of challenges, each of which gives you a task and a synthetic dataset and asks you to train and play with a trivial model. The challenges cover image generation, text classification, and tabular data, and each one: 12 | 13 | - Runs on your laptop 14 | - Trains in a few seconds 15 | - Uses perfect, noiseless, synthetic data that takes seconds to generate 16 | - Has absolutely no detail or sidebars 17 | 18 | *Hello Deep Learning* allows you to rapidly experiment with simple models and take your first steps in a calm, kindhearted environment. It gets you ready to leap into the detail and chaos of the real-world. 19 | ## Setup 20 | Install dependencies with: 21 | 22 | ``` 23 | virtualenv venv 24 | source venv/bin/activate 25 | pip3 install -r requirements.txt 26 | ``` 27 | 28 | This installs both the dependencies needed to generate the data, and the dependencies I used to solve the challenges. If you only want to install the dependencies for the data generation then you can run: 29 | 30 | ``` 31 | pip3 install -r requirements_data_only.txt 32 | ``` 33 | ## The challenges 34 | 35 | ### 1. Image classification 36 | 37 | #### Challenge 38 | 39 | Train a classifier that distinguishes between red squares and yellow circles. Your program should be able to: 40 | 41 | 1. Train a model that can distinguish between squares and circles 42 | 2. Use it to run a few individual predictions on specific images 43 | 3. Display the confusion matrix 44 | #### Data generation 45 | 46 | ``` 47 | python3 src/generators/shape_images.py 48 | ``` 49 | 50 | This generates 200 images of circles and 200 of rectangles and saves them in the `data/shapes/` directory. 51 | #### Tips 52 | 53 | - I did this using a fastai `vision_learner` based on the `resnet18` pretrained model. 54 | - I mostly copied and stitched together code snippets from [chapter 2 of the fastai book](https://github.com/fastai/fastbook/blob/master/02_production.ipynb) 55 | 56 | ### 2. Text classification 57 | 58 | #### Challenge 59 | 60 | Train a classifier that distinguishes between text inputs of positive and negative words, for example `"happy chirpy awesome"` and `"awful terrible heinous"`. Your program should be able to: 61 | 62 | 1. Train a model that can distinguish between this type of positive and negative input 63 | 2. Use it to run a few individual predictions on specific inputs 64 | #### Data generation 65 | 66 | ``` 67 | python3 src/generators/sentiment_text.py 68 | ``` 69 | 70 | This generates 1000 text files containing positive words and 1000 containing negative words and saves them in the `data/sentiment_text/` directory. 71 | #### Tips 72 | 73 | - I did this using a fastai `language_model_learner` based on the `AWD_LSTM` pretrained model, and a fastai `text_classifier_learner`. 74 | - I copied and stitched together code snippets from [chapter 10 of the fastai book](https://github.com/fastai/fastbook/blob/master/10_nlp.ipynb) 75 | ### 3. Decision trees 76 | #### Challenge 77 | 78 | Train decision trees that reverse-engineer the rules from [src/generators/random_tabular.py](src/generators/random_tabular.py) that were used to randomly generate a tabular dataset. Your program should be able to 79 | 80 | 1. Train a decision tree that reverse-engineers the rules 81 | 2. Train a random forest that reverse-engineers the rules 82 | 3. Uses these models it to run a few individual predictions on specific inputs 83 | 4. Calculates the RMS error on a validation set 84 | 5. Visualises the decision tree, using (for example) the [`dtreeviz`](https://github.com/parrt/dtreeviz) library 85 | #### Data generation 86 | 87 | ``` 88 | python3 src/generators/random_tabular.py 89 | ``` 90 | 91 | This generates 1 JSON file containing 10,000 data points and saves it in the `data/random_tabular/data.json` file.. Each data point contains: 92 | 93 | - 6 features: `a`, `b`, `c`, `d`, `e`, and `f`. Each of these is a random integer between 0 and 100. 94 | - 1 label: `y`. This label is derived deterministically from the features using simple rules contained in [src/generators/random_tabular.py](src/generators/random_tabular.py) . 95 | #### Tips 96 | 97 | - I did this using an sklearn `DecisionTreeRegressor` and `RandomForestRegressor`. 98 | - I copied and stitched together code snippets from [chapter 9 of the fastai book](https://github.com/fastai/fastbook/blob/master/09_tabular.ipynb) 99 | 100 | ## My solutions 101 | 102 | My solutions are in `src/examples/`, although they're not the only way to solve the challenges, and they're almost certainly not the best way to solve them either. 103 | -------------------------------------------------------------------------------- /data/README.md: -------------------------------------------------------------------------------- 1 | # Data 2 | 3 | Data generated by scripts in `src/generators/` will be saved here. 4 | -------------------------------------------------------------------------------- /pic.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/robert/hello-deep-learning/9fa87fabb2240670e2d75ad36a1bdd3fbec2388b/pic.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | dtreeviz 2 | fastai 3 | fastbook 4 | pandas 5 | Pillow 6 | -------------------------------------------------------------------------------- /requirements_data_only.txt: -------------------------------------------------------------------------------- 1 | Pillow 2 | -------------------------------------------------------------------------------- /src/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/robert/hello-deep-learning/9fa87fabb2240670e2d75ad36a1bdd3fbec2388b/src/__init__.py -------------------------------------------------------------------------------- /src/examples/1-shape_classifier.py: -------------------------------------------------------------------------------- 1 | from fastai.vision.all import * 2 | from PIL import Image 3 | import sys 4 | 5 | current_dir = os.path.dirname(os.path.abspath(__file__)) 6 | parent_dir = os.path.abspath(os.path.join(current_dir, "..")) 7 | sys.path.append(parent_dir) 8 | 9 | import paths 10 | 11 | if __name__ == "__main__": 12 | block = DataBlock( 13 | blocks=(ImageBlock, CategoryBlock), 14 | get_items=get_image_files, 15 | splitter=RandomSplitter(valid_pct=0.2, seed=42), 16 | get_y=parent_label, 17 | item_tfms=Resize(128) 18 | ) 19 | dls = block.dataloaders(paths.IMAGES) 20 | 21 | # Finetune the resnet vision learner 22 | learn = vision_learner(dls, resnet18, metrics=error_rate) 23 | learn.fine_tune(1) 24 | 25 | # Try some example predictions. These images are in the training set so this is 26 | # a terrible way of actually assessing anything, but good enough here. 27 | test_path1 = os.path.join(paths.IMAGES, "circles", "circle_0.png") 28 | preds1 = learn.predict(test_path1) 29 | print(preds1) 30 | 31 | test_path2 = os.path.join(paths.IMAGES, "rectangles", "rectangle_0.png") 32 | preds2 = learn.predict(test_path2) 33 | print(preds2) 34 | 35 | # Show confusion matrix to check performance, which should be perfect 36 | interp = ClassificationInterpretation.from_learner(learn) 37 | interp.plot_confusion_matrix() 38 | plt.show() 39 | -------------------------------------------------------------------------------- /src/examples/2-sentiment_analysis.py: -------------------------------------------------------------------------------- 1 | from fastai.text.all import * 2 | import sys 3 | 4 | current_dir = os.path.dirname(os.path.abspath(__file__)) 5 | parent_dir = os.path.abspath(os.path.join(current_dir, "..")) 6 | sys.path.append(parent_dir) 7 | 8 | import paths 9 | 10 | if __name__ == "__main__": 11 | # Finetune language model using sentiment data 12 | dls_lm = DataBlock( 13 | blocks=TextBlock.from_folder(paths.TEXT, is_lm=True), 14 | get_items=get_text_files, 15 | splitter=RandomSplitter(0.1), 16 | ).dataloaders(paths.TEXT, path=paths.TEXT, bs=128, seq_len=80) 17 | 18 | learn = language_model_learner( 19 | dls_lm, AWD_LSTM, drop_mult=0.3, 20 | metrics=[accuracy, Perplexity()]).to_fp16() 21 | learn.fit_one_cycle(1, 2e-2) 22 | learn.save_encoder('finetuned') 23 | 24 | # Train a classifier using the finetuned language model 25 | dls_clas = DataBlock( 26 | blocks=(TextBlock.from_folder(paths.TEXT, vocab=dls_lm.vocab), CategoryBlock), 27 | get_y=parent_label, 28 | get_items=get_text_files, 29 | splitter=RandomSplitter(0.1), 30 | ).dataloaders(paths.TEXT, path=paths.TEXT, bs=128, seq_len=72) 31 | 32 | learn2 = text_classifier_learner(dls_clas, AWD_LSTM, drop_mult=0.5, 33 | metrics=accuracy).to_fp16() 34 | learn2 = learn2.load_encoder('finetuned') 35 | learn2.fit_one_cycle(1, 2e-2) 36 | 37 | # Try a few predictions 38 | print(learn2.predict("happy great fun")) 39 | print(learn2.predict("sad bad mad")) 40 | print(learn2.predict("inferior gross garbage")) 41 | print(learn2.predict("superlative nice friendly")) 42 | -------------------------------------------------------------------------------- /src/examples/3-decision_trees.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | from fastai.tabular.all import * 3 | from fastbook import * 4 | 5 | from sklearn.ensemble import RandomForestRegressor 6 | from sklearn.tree import DecisionTreeRegressor 7 | import dtreeviz 8 | 9 | import random 10 | import sys 11 | 12 | current_dir = os.path.dirname(os.path.abspath(__file__)) 13 | parent_dir = os.path.abspath(os.path.join(current_dir, "..")) 14 | sys.path.append(parent_dir) 15 | 16 | import paths 17 | 18 | 19 | def r_mse(pred,y): 20 | return round(math.sqrt(((pred-y)**2).mean()), 6) 21 | 22 | def m_rmse(m, xs, y): 23 | return r_mse(m.predict(xs), y) 24 | 25 | 26 | if __name__ == "__main__": 27 | procs = [] 28 | cat_names = [] 29 | cont_names = ['a', 'b', 'c', 'd', 'e', 'f'] 30 | 31 | # Split data in train and validation 32 | df = pd.read_json(paths.RANDOM_TABULAR_FILE) 33 | to = TabularPandas( 34 | df, 35 | procs, 36 | cat_names, 37 | cont_names, 38 | y_names='y', 39 | splits=RandomSplitter()(range_of(df)), 40 | ) 41 | xs, y = to.train.xs, to.train.y 42 | valid_xs, valid_y = to.valid.xs, to.valid.y 43 | 44 | # Train a decision tree and try a few predictions 45 | dtree = DecisionTreeRegressor( 46 | max_leaf_nodes=20 47 | ).fit(xs.values, y) 48 | print(dtree.predict([[100, 90, 80, 70, 60, 50]])) 49 | print(m_rmse(dtree, valid_xs.values, valid_y)) 50 | 51 | # Train a random forest and try a few predictions 52 | rf = RandomForestRegressor( 53 | n_jobs=-1, 54 | n_estimators=40, 55 | max_samples=8000, 56 | max_features=0.5, 57 | min_samples_leaf=5, 58 | oob_score=True 59 | ).fit(xs.values, y) 60 | print(rf.predict([[100, 90, 80, 70, 60, 50]])) 61 | 62 | # Visualize the decision tree 63 | viz_rmodel = dtreeviz.model(model=dtree, 64 | X_train=df[cont_names], 65 | y_train=df['y'], 66 | feature_names=cont_names, 67 | target_name='y') 68 | v = viz_rmodel.view() 69 | v.show() 70 | -------------------------------------------------------------------------------- /src/generators/random_tabular.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | import random 4 | import sys 5 | 6 | current_dir = os.path.dirname(os.path.abspath(__file__)) 7 | parent_dir = os.path.abspath(os.path.join(current_dir, "..")) 8 | sys.path.append(parent_dir) 9 | 10 | import paths 11 | import utils 12 | 13 | 14 | if __name__ == "__main__": 15 | data = [] 16 | 17 | for i in range(0, 10000): 18 | a = random.randint(0, 100) 19 | b = random.randint(0, 100) 20 | c = random.randint(0, 100) 21 | d = random.randint(0, 100) 22 | e = random.randint(0, 100) 23 | f = random.randint(0, 100) 24 | 25 | y = 0 26 | if (a < 5 and b > 90): 27 | y = 1 28 | elif (b > 50 and (c > 10 and c < 30)): 29 | y = 2 30 | elif (c < 3 or c > 97): 31 | y = 3 32 | elif (d > 60 and d < 90): 33 | y = 4 34 | elif (e > 80 and e < 90): 35 | y = 5 36 | elif (f > 10 and f < 25): 37 | y = 6 38 | 39 | data.append({ 40 | 'a': a, 41 | 'b': b, 42 | 'c': c, 43 | 'd': d, 44 | 'e': e, 45 | 'f': f, 46 | 'y': y, 47 | }) 48 | 49 | utils.mkdirp(paths.RANDOM_TABULAR_DIR) 50 | 51 | fpath = os.path.join(paths.RANDOM_TABULAR_FILE, "data.json") 52 | with open(fpath, "w") as f: 53 | json.dump(data, f) 54 | -------------------------------------------------------------------------------- /src/generators/sentiment_text.py: -------------------------------------------------------------------------------- 1 | import os 2 | import random 3 | import sys 4 | 5 | current_dir = os.path.dirname(os.path.abspath(__file__)) 6 | parent_dir = os.path.abspath(os.path.join(current_dir, "..")) 7 | sys.path.append(parent_dir) 8 | 9 | import paths 10 | import utils 11 | 12 | if __name__ == "__main__": 13 | positive_words = [ 14 | "great", 15 | "excellent", 16 | "good", 17 | "fantastic", 18 | "nice", 19 | "happy", 20 | "superb", 21 | "awesome", 22 | "pleasing", 23 | "delightful", 24 | "cherubic", 25 | "spectacular", 26 | "wonderful", 27 | "pleasant", 28 | ] 29 | negative_words = [ 30 | "terrible", 31 | "horrible", 32 | "awful", 33 | "atrocious", 34 | "bad", 35 | "horrendous", 36 | "heinous", 37 | "distressing", 38 | "baleful", 39 | "nauseating", 40 | "terrifying", 41 | "stupid", 42 | "dumb", 43 | "appauling", 44 | "stupefying", 45 | ] 46 | 47 | pos_dir = os.path.join(paths.TEXT, "positive") 48 | neg_dir = os.path.join(paths.TEXT, "negative") 49 | 50 | utils.mkdirp(pos_dir) 51 | utils.mkdirp(neg_dir) 52 | 53 | for i in range(0, 1000): 54 | text = " ".join((random.choice(positive_words) for _ in range(0, 5))) 55 | fp = os.path.join(paths.TEXT, "positive", f"{i}.txt") 56 | with open(fp, "w") as f: 57 | f.write(text) 58 | 59 | for i in range(0, 1000): 60 | text = " ".join((random.choice(negative_words) for _ in range(0, 5))) 61 | fp = os.path.join(paths.TEXT, "negative", f"{i}.txt") 62 | with open(fp, "w") as f: 63 | f.write(text) 64 | -------------------------------------------------------------------------------- /src/generators/shape_images.py: -------------------------------------------------------------------------------- 1 | from PIL import Image, ImageDraw 2 | 3 | import os 4 | import random 5 | import sys 6 | 7 | current_dir = os.path.dirname(os.path.abspath(__file__)) 8 | parent_dir = os.path.abspath(os.path.join(current_dir, "..")) 9 | sys.path.append(parent_dir) 10 | 11 | import paths 12 | import utils 13 | 14 | 15 | def draw_circle(img, x, y, r): 16 | draw = ImageDraw.Draw(img) 17 | draw.ellipse((x-r, y-r, x+r, y+r), fill=(255,0,0,0)) 18 | 19 | 20 | def draw_rect(img, x, y, w, h): 21 | draw = ImageDraw.Draw(img) 22 | draw.rectangle([(x, y), (x+w, y+h)], fill ="#ffff33", outline ="red") 23 | 24 | 25 | if __name__ == "__main__": 26 | circles_dir = os.path.join(paths.IMAGES, "circles") 27 | rects_dir = os.path.join(paths.IMAGES, "rectangles") 28 | 29 | utils.mkdirp(circles_dir) 30 | utils.mkdirp(rects_dir) 31 | 32 | for i in range(0, 200): 33 | x = random.randint(150, 350) 34 | y = random.randint(150, 350) 35 | r = random.randint(10, 100) 36 | 37 | img = Image.new('RGB', (500, 500)) 38 | draw_circle(img, x, y, r) 39 | img.save(os.path.join(circles_dir, f"circle_{i}.png")) 40 | 41 | print("Saved circles") 42 | 43 | for i in range(0, 200): 44 | x = random.randint(0, 350) 45 | y = random.randint(0, 350) 46 | w = random.randint(50, 150) 47 | h = random.randint(50, 150) 48 | 49 | img = Image.new('RGB', (500, 500)) 50 | draw_rect(img, x, y, w, h) 51 | img.save(os.path.join(rects_dir, f"rectangle_{i}.png")) 52 | 53 | print("Saved rectangles") 54 | -------------------------------------------------------------------------------- /src/paths.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | fpath = os.path.dirname(__file__) 4 | datapath = os.path.join(fpath, "..", "data") 5 | 6 | IMAGES = os.path.join(datapath, "./shapes/") 7 | TEXT = os.path.join(datapath, "./sentiment_text/") 8 | RANDOM_TABULAR_DIR = os.path.join(datapath, "./random_tabular/") 9 | RANDOM_TABULAR_FILE = os.path.join(RANDOM_TABULAR_DIR, "./data.json") 10 | -------------------------------------------------------------------------------- /src/utils.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | def mkdirp(directory): 4 | if not os.path.exists(directory): 5 | os.makedirs(directory) 6 | --------------------------------------------------------------------------------