├── .gitignore ├── README.md ├── app.py ├── data ├── disaster.csv └── english_to_latex.csv ├── images ├── Venus_and_Adonis_by_Peter_Paul_Rubens.jpg └── oreilly.png ├── notebooks ├── BERT_vs_GPT_for_CLF.ipynb ├── constructing_a_vqa_system.ipynb ├── deployment_and_optimization.ipynb ├── latex_gpt2.ipynb ├── mnist.ipynb ├── rnn_and_cnn.ipynb ├── using_our_vqa.ipynb └── vgg_and_bert.ipynb └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | *.pt 3 | notebooks/clf/ 4 | notebooks/wandb/ 5 | .ipynb_checkpoints 6 | *.onnx 7 | data/MNIST/ 8 | data/art-styles/ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![oreilly-logo](images/oreilly.png) 2 | 3 | # Deep Learning for Modern AI 4 | 5 | 6 | This repository contains code for the [O'Reilly Live Online Training for Deep Learning for Modern AI](https://learning.oreilly.com/live-events/deep-learning-for-modern-ai/0642572005084) 7 | 8 | This training provides the theory and practical concepts for a comprehensive introduction to machine learning and deep learning with PyTorch —foundational knowledge needed to successfully build and train GenAI and multimodal models. By making our way through several real-world case studies including object recognition and text classification this session is an excellent crash course in deep learning with PyTorch. 9 | 10 | We use tools including large pre-trained models and model training dashboards to set up reproducible deep learning experiments and build machine learning models optimized for performance. There are several code examples throughout the training to help solidify the theoretical concepts that will be introduced. Models like Stable Diffusion, Llama 3, GPT, and BERT are highlighted as we uncover the training and optimization strategies to get the most of our models' performance, speed, and memory usage. 11 | ### Notebooks 12 | 13 | 14 | #### 1. Introduction to Deep Learning 15 | 16 | All data can be downloaded for the art classification example [here](https://drive.google.com/file/d/1jofGOHQ4PwZ50kpGuDqBeVXwDNcjPE6B/view?usp=sharing). Note it is about 6GB so it may take a bit. 17 | 18 | - [**First steps with Deep Learning with MNIST**](notebooks/mnist.ipynb) 19 | - [**RNNs and CNNs**](notebooks/rnn_and_cnn.ipynb) 20 | - [**Working with pre-trained VGG-11 and BERT models**](notebooks/vgg_and_bert.ipynb) 21 | - [**Fine-tuning BERT vs ChatGPT**](notebooks/BERT_vs_GPT_for_CLF.ipynb) 22 | - [Fine-tuning OpenAI](https://github.com/sinanuozdemir/quick-start-guide-to-llms/blob/main/notebooks/05_openai_app_review_fine_tuning.ipynb): the code to compare against BERT 23 | - [**Fine-tuning GPT-2 to convert English to LaTEX**](notebooks/latex_gpt2.ipynb) 24 | - [**Fine-tuning Llama 3 to be a chatbot**](https://colab.research.google.com/drive/1gN7jsUFQTPAj5uFrq06HcSLQSZzT7hZz?usp=sharing) 25 | 26 | #### 2. Optimizing models 27 | 28 | - [**Production Optimization**](notebooks/deployment_and_optimization.ipynb) 29 | - [**Quantizing Llama 3**](https://colab.research.google.com/drive/12RTnrcaXCeAqyGQNbWsrvcqKyOdr0NSm?usp=sharing) 30 | - [**Testing different fine-tuning configurations**](https://colab.research.google.com/drive/1fdx2XlqfAjBoyiTktkRwa8SFaRF3Ch82?usp=sharing) 31 | - [**Distilling BERT models**](https://github.com/sinanuozdemir/quick-start-guide-to-llms/blob/main/notebooks/11_distillation_example_2.ipynb) 32 | 33 | #### 3. Going Further 34 | 35 | - **[Intro to Multimodality](https://colab.research.google.com/drive/1zYSzDuYFa_cbRlti3scUjfmvradK8Sf4?usp=sharing)**: An introduction to multimodality with CLIP and SHAP-E + Diffusion 36 | 37 | - **[Whisper](https://colab.research.google.com/drive/1KxLWEEBtgix4zgP52pnxlIoJrZ8sHEYC?usp=sharing)**: An introduction to using Whisper for audio transcription 38 | 39 | - **[Llava](https://colab.research.google.com/drive/1IwNAz1Ee4YUSRNCU-SOsa7FS8Q2vmpoL?usp=sharing)**: Using an open source mult-turn multimodal engine 40 | 41 | - **[CLIP-based Stock Image Search](https://colab.research.google.com/drive/1aUz0FKQDSAyXyhRyvkkRsSy7S30mpRJc?usp=sharing)**: Using CLIP to search through a library of images 42 | 43 | - **[Dreambooth](https://colab.research.google.com/drive/1tQt1pE6l0MI79W8ZX0MMu0YVmF2I0GB3?usp=sharing)**: Fine-tuning a stable difusion model to make images of yours truly! Ever wonder what I look like blonde? Me neither but AI gave me some ideas of what it would look like. 44 | 45 | 46 | - **Visual Q/A** - This case study requires you to [download the data from my Dropbox here](https://www.dropbox.com/scl/fo/w6iyfox8gnflvm7g10n47/AB47L7tNEl2Q8eyemZa2GMA?rlkey=v9s8bv6cmjukykpilzimswar0&st=fbulzw4e&dl=0). The code snippets should download them in code if that is easier! Our goal is to emulate the process done by [Llama 3.2-Vision-Instruct](https://colab.research.google.com/drive/1r6Nab2L7rYUBV5e8K8u8EFw98adJu5uh?usp=sharing): one of Meta's latest Llama models that can take in images. 47 | 48 | - Method 1: BERT + ViT -> GPT-2 (Fusion) 49 | 50 | - Constructing and Training our model: [Local](notebooks/constructing_a_vqa_system.ipynb) and notebook in [Colab](https://colab.research.google.com/drive/1zvbruS1DvFrVgXjNouSrrF9-PphKLWWl?usp=sharing) 51 | - Using our VQA system: [Local](notebooks/using_our_vqa.ipynb) notebook and [Colab](https://colab.research.google.com/drive/16GOBndQuIBO-UfXdpPte-PXaZS2nsW1H?usp=sharing) 52 | 53 | - Method 2: BERT + ViT -> GPT-2 (Fusion) 54 | - [Train the VQA Model](https://colab.research.google.com/drive/1DSh8_yfubuu5xPVM2BQ-I_eH5rrxLKZU?usp=sharing) and [use it here](https://colab.research.google.com/drive/1AWAk7NTvgTbjktUNB6bmS6T37bgTzRgt?usp=sharing) 55 | 56 | - **[Training a Reasoning Model with Unsloth](https://colab.research.google.com/drive/1Cws1IL_T_0_cP0-cHxFA0FEsXYdiAN_8?usp=sharing)** - Advanced - See how companies like DeepSeek and Anthropic train their reasoning models. [Unsloth AI](https://unsloth.ai/) is a package aiming to make fine-tuning more streamlined, faster, and more memory efficient by handwriting things like backprop in a faster way. We salute them for their work! 57 | 58 | #### How to Use the Image Recognition Flask App 59 | 60 | `app.py` is a Flask app that uses a VGG16 model to classify the art style of an uploaded image. The app currently supports 10 different art styles: 61 | 62 | - Abstract Expressionism 63 | - Art Nouveau (Modern) 64 | - Baroque 65 | - Expressionism 66 | - Impressionism 67 | - Northern Renaissance 68 | - Post-Impressionism 69 | - Realism 70 | - Romanticism 71 | - Symbolism 72 | 73 | Start the Flask app: 74 | `python app.py` 75 | 76 | This should start the Flask app and make it available at `http://localhost:5000`. 77 | 78 | #### How to Use the App 79 | 80 | To classify an image, you can use a cURL request in the following format: 81 | 82 | 83 | ```curl -X POST -F 'image=@/path/to/your/image.jpg' http://localhost:5000/predict``` 84 | 85 | Replace `/path/to/your/image.jpg` with the path to your own image. The response will be in JSON format and will contain the predicted art style and associated confidence scores, as shown below: 86 | 87 | ``` 88 | e.g. 89 | curl -X POST -F \ 90 | 'image=@images/Venus_and_Adonis_by_Peter_Paul_Rubens.jpg' \ 91 | http://localhost:5000/predict 92 | 93 | [ 94 | ["Northern_Renaissance",0.13392961025238037], 95 | ["Realism",0.12794768810272217], 96 | ["Romanticism",0.12592236697673798], 97 | ["Post_Impressionism",0.11863630264997482], 98 | ["Baroque",0.11325731128454208], 99 | ["Symbolism",0.1120268702507019], 100 | ["Expressionism",0.08971412479877472], 101 | ["Impressionism",0.086906298995018], 102 | ["Art_Nouveau_Modern",0.05910796299576759], 103 | ["Abstract_Expressionism",0.03255145251750946]] 104 | ``` 105 | 106 | If there is an error with the request, such as no image being provided, the response will contain an error message instead: 107 | 108 | ``` 109 | { 110 | "error": "No image provided" 111 | } 112 | ``` 113 | 114 | 115 | ## Instructor 116 | 117 | **Sinan Ozdemir** is the Founder and CTO of LoopGenius where he uses State of the art AI to help people create and run their businesses. Sinan is a former lecturer of Data Science at Johns Hopkins University and the author of multiple textbooks on data science and machine learning. Additionally, he is the founder of the recently acquired Kylie.ai, an enterprise-grade conversational AI platform with RPA capabilities. He holds a master’s degree in Pure Mathematics from Johns Hopkins University and is based in San Francisco, CA. 118 | 119 | # For More 120 | 121 | - CHeck out [Deep Learning Illustrated](https://www.amazon.com/dp/0135116694?ref_=cm_sw_r_ffobk_cp_ud_dp_T500T43FCOX9F12OYRFO&peakEvent=5&dealEvent=0&bestFormat=true): A best seller by Jon Krohn, it's a very visual introduction to deep learning 122 | - [Deep Learning course: lecture slides and lab notebooks](https://m2dsupsdlclass.github.io/lectures-labs/): The course covers the basics of Deep Learning, with a focus on applications. 123 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torchvision.models as models 3 | import torchvision.transforms as transforms 4 | from PIL import Image 5 | from flask import Flask, request, jsonify 6 | from torch import nn 7 | 8 | # Initialize the Flask app 9 | app = Flask(__name__) 10 | 11 | # Define the image transformations to be applied 12 | transform = transforms.Compose([ 13 | transforms.Resize((224, 224)), # Resize the input image to 224 x 224 pixels 14 | transforms.ToTensor(), # Convert the image to a PyTorch tensor 15 | ]) 16 | 17 | # Load the pre-trained VGG16 model 18 | trained_vgg_model = models.vgg11() 19 | 20 | # Modify the model's final layer to have the correct number of output classes 21 | num_classes = 10 22 | trained_vgg_model.classifier[-1] = nn.Linear(trained_vgg_model.classifier[-1].in_features, num_classes) 23 | 24 | # Set the model to evaluation mode and load the trained weights 25 | trained_vgg_model.load_state_dict(torch.load("data/trained_vgg_model_quantized.pt")) 26 | trained_vgg_model.eval() 27 | 28 | # Define the class labels 29 | CLASSES = [ 30 | 'Abstract_Expressionism', 31 | 'Art_Nouveau_Modern', 32 | 'Baroque', 33 | 'Expressionism', 34 | 'Impressionism', 35 | 'Northern_Renaissance', 36 | 'Post_Impressionism', 37 | 'Realism', 38 | 'Romanticism', 39 | 'Symbolism' 40 | ] 41 | 42 | 43 | # Define the predict endpoint 44 | @app.route('/predict', methods=['POST']) 45 | def predict(): 46 | if request.files.get('image'): # Check if an image file was uploaded 47 | # Preprocess the image and pass it through the model 48 | image = Image.open(request.files['image'].stream).convert('RGB') 49 | input_tensor = transform(image).unsqueeze(0) 50 | with torch.no_grad(): 51 | output = trained_vgg_model(input_tensor) 52 | 53 | # Get the probabilities for all classes 54 | probabilities = torch.softmax(output, dim=1)[0].tolist() 55 | class_probabilities = {CLASSES[class_id]: prob for class_id, prob in enumerate(probabilities)} 56 | sorted_probabilities = sorted(class_probabilities.items(), key=lambda x: x[1], reverse=True) 57 | 58 | return jsonify(sorted_probabilities) 59 | 60 | return jsonify({'error': 'No image provided'}) # Return an error message if no image file was uploaded 61 | 62 | 63 | if __name__ == '__main__': 64 | app.run() # Run the Flask app if this module is executed directly (not imported) 65 | -------------------------------------------------------------------------------- /data/disaster.csv: -------------------------------------------------------------------------------- 1 | index,id,keyword,location,text,target,label 2 | 7138,10224,volcano,,@MrMikeEaton @Muazimus_Prime hill hill mountain volcano of hell mountain hill hil.,1,1 3 | 2151,3086,deaths,Blackpool,"Cancers equate for around 25% of all deaths in #Blackpool. 4 | 5 | Kowing the signs could save your life: http://t.co/5lNIdvoBff 6 | #BeClearOnCancer",1,1 7 | 4395,6247,hijacking,World,The Murderous Story Of America‰Ûªs First Hijacking: Earnest Pletch‰Ûªs cold-blooded killing of‰Û_ http://t.co/B9JAxx0vCf,1,1 8 | 2508,3602,desolation,"Birmingham, UK",The date for the release of EP03 DESOLATION is set. Stay tuned for more info while we finalise the schedule. #alt #electro #rock #comingsoon,1,1 9 | 1378,1987,bush%20fires,London/Bristol/Guildford,On holiday to relax sunbathe and drink ... Putting out bush fires? Not so much ?? #spain https://t.co/dRno7OKM21,0,0 10 | 6825,9775,trapped,????s ?? ????Ìø????Ì¡a,(?EudryLantiqua?) Hollywood Movie About Trapped Miners Released in Chile: 'The 33' Holly... http://t.co/us1DMdXZVb (?EudryLantiqua?),1,1 11 | 3877,5514,flattened,Some other mansion,"Flattened all cartoony-like. 12 | 'Whoa there Papa!' https://t.co/4zmcqRMOIs",0,0 13 | 3465,4957,exploded,,#news #science London warship exploded in 1665 because sailors were recycling artillery cartridges... http://t.co/r4WGXrA59M #life #tech,1,1 14 | 6245,8921,snowstorm,"Brooklyn, NY",'Cooler than Freddie Jackson sippin' a milkshake in a snowstorm',0,0 15 | 5664,8083,rescue,Wanderlust,Mary coming to Troy rescue. ?????? https://t.co/rosVXQeLQj,0,0 16 | 3938,5599,flood,New York,Spot Flood Combo 53inch 300W Curved Cree LED Work Light Bar 4X4 Offroad Fog Lamp - Full re‰Û_ http://t.co/fDSaoOiskJ http://t.co/2uVmq4vAfQ,0,0 17 | 3059,4388,earthquake,in the Word of God,@DArchambau THX for your great encouragement and for RT of NEW VIDEO http://t.co/cybKsXHF7d The Coming Apocalyptic US Earthquake & Tsunami,1,1 18 | 5985,8546,screams,,When you on the phone and @Worstoverdose screams 'jaileens caked up on the phone' so everyone looks at you ????????????,0,0 19 | 7186,10296,weapon,,Cat Of Nine Irons XII: This nightmarishly brutal weapon is used in ritualistic country club de http://t.co/xpFmR368uF http://t.co/nmAUMYdKe1,1,1 20 | 6493,9283,sunk,"San Jose, CA",Its like I never left. I just sunk to the background,0,0 21 | 816,1185,blizzard,columbus ohio,@StevenOnTwatter @PussyxDestroyer just order a blizzard pay then put your nuts in it say they have you ball flavored. Boom free ice cream,0,0 22 | 4127,5868,hailstorm,facebook.com/tradcatknights,"Canada: Hailstorm flash flooding slam Calgary knocks out power to 20k customers 23 | http://t.co/SkY9EokgGB http://t.co/5IyZsDA6xB",1,1 24 | 4623,6571,injury,,incident with injury:I-495 inner loop Exit 31 - MD 97/Georgia Ave Silver Spring,1,1 25 | 895,1296,bloody,Leicester,Bloody insomnia again! Grrrr!! #Insomnia,1,1 26 | 7450,10662,wounds,The American Wasteland (MV),@FEVWarrior -in the Vault that could take a look at those wounds of yours if you'd like to go to one of these places first.' Zarry has had-,0,0 27 | 3043,4366,earthquake,a box,"@AGeekyFangirl14 's things she looks in a significant other: 28 | 1. Beautiful eyes. 29 | 2. Humor. 30 | 3. Farts that creates an earthquake. 31 | 32 | ????????",0,0 33 | 422,612,arsonist,America,If you don't have anything nice to say you can come sit with me.,0,0 34 | 194,273,ambulance,Loveland Colorado,@Kiwi_Karyn Check out what's in my parking lot!! He said that until last year it was an ambulance in St Johns. http://t.co/hPvOdUD7iP,0,0 35 | 542,788,avalanche,Freeport il ,The possible new jerseys for the Avalanche next year. ???? http://t.co/nruzhR5XQu,0,0 36 | 6950,9972,tsunami,,"Crptotech tsunami and banks. 37 | http://t.co/KHzTeVeDja #Banking #tech #bitcoing #blockchain",1,1 38 | 4860,6919,mass%20murderer,,Julian Knight - @SCVSupremeCourt dismisses mass murderer's attempt to increase prisoner pay. Challenged quantum of 5% increase 2013.,1,1 39 | 3297,4725,evacuate,,Disregard my snap story there is an angry white girl riot happening as we speak. #evacuate,1,1 40 | 4977,7097,military," Quantico Marine Base, VA.",@FurTrix then find cougars who look like her even better if they're in military uniform!,0,0 41 | 411,594,arson,,Arson suspect linked to 30 fires caught in Northern California http://t.co/u1fuWrGK5U,1,1 42 | 3817,5425,first%20responders,,Carmike Cinemas on Antioch Shooting: 'We Are Grateful' for Staff and First Responders Safety Is 'Highest Priority' http://t.co/BehfHspPud,1,1 43 | 1121,1617,bombed,"Screwston, TX",'Redskins WR Roberts Belly-Bombed ' via @TeamStream http://t.co/GbcvVEvDTY,1,1 44 | 4233,6013,hazardous,"Oregon, USA",Is it possible to sneak into a hospital so I can stab myself with a hazardous needle and inject some crazy disease into my veins until I die,0,0 45 | 6397,9143,suicide%20bomber,,Suicide Bomber Kills 13 At Saudi Mosque http://t.co/oZ1DS3Xu0D #Saudi #Tripoli #Libya,1,1 46 | 4622,6570,injury,,@Sport_EN Just being linked to Arsenal causes injury.,1,1 47 | 4149,5898,harm,Kansas City,@dinallyhot Love what you picked! We're playing WORTH IT by FIFTH HARM/KID INK because of you! Listen & Vote: http://t.co/0wrATkA2jL,0,0 48 | 2927,4206,drowned,"United Kingdom,Fraserburgh","@Stephen_Georg Hey Stephen Remember that time you drowned all the yellows 49 | 50 | Read: http://t.co/0sa6Xx1oQ7",0,0 51 | 4895,6970,massacre,,"Headed to the massacre 52 | Bodies arriving everyday 53 | What were those shells you heard 54 | Picking the bones up along the way",1,1 55 | 2267,3249,deluged,balvanera,'Afterwards I had to be alone for an hour to savour and prolong the almost physical intensity of the feelings that deluged me'. Y sigue:,0,0 56 | 6027,8613,seismic,,SEISMIC AUDIO SA-15T SA15T Padded Black Speaker COVERS (2) - Qty of 1 = 1 Pair! http://t.co/2jbIbeib9G http://t.co/p5KtaqW5QG,0,0 57 | 7354,10527,wildfire,"Eddyville, Oregon 97343",Oregon's biggest wildfire slows growth http://t.co/P0GoS5URXG via @katunews,1,1 58 | 1935,2783,curfew,Uganda,@DavisKawalya I know @Mauryn143 will be saying her final goodbyes to grandpa as seen on news RiP Me? always open to ideas but may ve curfew,0,0 59 | 932,1349,blown%20up,Cobblestone,"'If a truckload of soldiers will be blown up nobody panics but when one little lion dies everyone loses their mind' 60 | http://t.co/wjNTaOkdHf",1,1 61 | 3963,5632,flooding,"Honolulu, Hawaii",If Flooding occurs in your home - How to stop mold growth via @ProudGreenHome http://t.co/KAVAovJz2V,1,1 62 | 2805,4033,disaster,,@cncpts @SOLELINKS what a disaster - can't say I'm surprised,0,0 63 | 1683,2428,collide,The Forever Girl,The Witches of the Glass Castle. Supernatural YA where sibling rivalry magic and love collide #wogc #kindle http://t.co/IzakNpJeQW,0,0 64 | 5754,8214,riot,,"@abran_caballero Discovered by @NickCannon 65 | Listen/Buy @realmandyrain #RIOT on @iTunesMusic @iTunes https://t.co/dehMym5lpk ‰Û_ #BlowMandyUp",0,0 66 | 3907,5557,flattened,,100 1' MIX NEW FLAT DOUBLE SIDED LINERLESS BOTTLE CAPS YOU CHOOSE MIX FLATTENED - Full re‰Û_ http://t.co/w00kjPrfdR http://t.co/mIXl1pFRJe,0,0 67 | 4123,5861,hailstorm,,Summer heat drives bobcats to Calgary backyards ~ 3 http://t.co/kEstuUYc4t http://t.co/PzFBb1P1mj,1,1 68 | 4842,6896,mass%20murder,"Huntsville, AL",Okay not sure the word 'mass murder' applies during this war but it was horrendous none the less. https://t.co/Sb3rjQqzIX,1,1 69 | 6892,9880,traumatised,,@wrongdejavu I'm traumatised,0,0 70 | 4122,5858,hailstorm,,IG: http://t.co/2WBiVKzJIP 'It's hailing again! #abstorm #yyc #hail #hailstorm #haildamage #yycweather #calgary #captureyyc #alberta #sto‰Û_,1,1 71 | 5093,7265,nuclear%20disaster,,3 former executives to be prosecuted in Fukushima nuclear disaster http://t.co/Gvj7slbELP,1,1 72 | 5327,7605,pandemonium,"Mumbai, India",@minhazmerchant Govt should pass the bills in the Pandemonium. UPA used to do it why cant NDA?,0,0 73 | 2348,3379,demolition,,Israel continues its demolition of Palestinian homes #gop #potus #irandeal #isis https://t.co/NMgp7iMEIi,1,1 74 | 1123,1619,bombed,My old New England home,I liked a @YouTube video http://t.co/FX7uZZXtE4 Benedict Cumberbatch Gets Video Bombed,0,0 75 | 2645,3797,destruction,denver colorado,it sure made an impact on me http://t.co/GS50DdG1JY,0,0 76 | 3220,4620,emergency%20services,"Vancouver, British Columbia",Removing tsunami debris from the West Coast: Karen Robinson Enviromental and Emergency services manager of the‰Û_ http://t.co/1MeEo3WJcO,1,1 77 | 4117,5850,hailstorm,Washington State,We The Free Hailstorm Maxi http://t.co/ERWs6IELdG,1,1 78 | 1913,2753,curfew,,@TheComedyQuote @50ShadezOfGrey the thirst has no curfew ???????????? @P45Perez,0,0 79 | 3372,4830,evacuation,,Run out evacuation hospital indexing remedial of angioplasty dissertation at power elite hospitals dismayed:‰Û_ http://t.co/VGvJGr8zoO,1,1 80 | 6382,9120,suicide%20bomb,lagos. Unilag,16yr old PKK suicide bomber who detonated bomb in Turkey Army trench released http://t.co/mGZslZz1wF,1,1 81 | 1966,2830,cyclone,"Sioux Falls, SD",Excited for Cyclone football https://t.co/Xqv6gzZMmN,0,0 82 | 150,215,airplane%20accident,"New Mexico, USA",@mickinyman @TheAtlantic That or they might be killed in an airplane accident in the night a car wreck! Politics at it's best.,0,0 83 | 1938,2786,curfew,"Elkhart, IN",Had to cancel my cats doctor appointment because she decided to go out and play and not come home by curfew ...,0,0 84 | 4735,6734,lava,Venezuela,I LAVA YOU.,0,0 85 | 4111,5842,hailstorm,,Hail:The user summons a hailstorm lasting five turns. It damages all Pokemon except the Ice type.,0,0 86 | 498,721,attacked,Peshawar,IK Only Troll His Pol Rivals Never Literally Abused Them Or Attacked Their Families. While All Of Them Literally Abuse IK. Loosers,0,0 87 | 4246,6033,hazardous,United States,DLH issues Hazardous Weather Outlook (HWO) http://t.co/a0Ad8z5Vsr #WX,1,1 88 | 4578,6511,injuries,"Sutton, London UK",@fkhanage look what Shad Forsythe has done in 1 year we won't have as many injuries as before we will inevitably have injuries like others,0,0 89 | 3532,5049,eyewitness,Pennsylvania,A true #TBT Eyewitness News WBRE WYOU http://t.co/JHVigsX5Jg,0,0 90 | 3070,4405,electrocute,,when you got an extension cord that extends from your bed to your bath tub ?? lets pray I don't electrocute myself,0,0 91 | 1829,2629,crashed,,Pakistan says army helicopter has crashed in country's restive northwest killing at least 8 http://t.co/QV1RMZI3J1,1,1 92 | 6818,9765,trapped,10 Steps Ahead. Cloud 9,Bomb head? Explosive decisions dat produced more dead children than dead bodies trapped tween buildings on that day in September there,1,1 93 | 4355,6187,hijacker,San Francisco,Governor allows parole for California school bus hijacker who kidnapped 26 children in 1976. http://t.co/hdAhLgrprl http://t.co/Z1s3T77P3L,1,1 94 | 2961,4253,drowning,Numa casa de old yellow bricks,LONDON IS DROWNING AND IIII LIVE BY THE RIVEEEEEER,1,1 95 | 4285,6088,hellfire,,Hellfire is surrounded by desires so be careful and don‰Ûªt let your desires control you! #Afterlife,0,0 96 | 6572,9405,survivors,"Dallas, TX ",Haunting memories drawn by survivors http://t.co/pRAro2OWia,1,1 97 | 5047,7195,natural%20disaster,home ,she's a natural disaster she's the last of the American girls ??,0,0 98 | 249,354,annihilation,Spain,:StarMade: :Stardate 3: :Planetary Annihilation:: http://t.co/I2hHvIUmTm via @YouTube,1,1 99 | 5382,7680,panic,,@panic awesome thanks.,0,0 100 | 2898,4163,drown,,@cameronhigdon34 I can't drown my demons they know how to swim.,0,0 101 | 2771,3983,devastation,,70 Years After Atomic Bombs Japan Still Struggles With War Past: The anniversary of the devastation wrought b... http://t.co/Targ56iGBZ,1,1 102 | 6043,8638,seismic,,A subcontractor working for French seismic survey group CGG has been kidnapped in Cairo and is held by Islamic State the company said on W‰Û_,1,1 103 | 3868,5499,flames,,If you have an opinion and you don't put it on thh internet you will furst into flames.,0,0 104 | 7452,10665,wounds,"Cleveland, OH",If time heals all wounds how come the belly button stays the same?,0,0 105 | 5434,7753,police,,Police unions retard justice & drain gov $ but cops vote 4 Rs so Rs go after teachers unions instead! @DCCC @VJ44 @Lawrence @JBouie @mmfa,0,0 106 | 6757,9681,tornado,Toronto,Environment Canada confirms 2nd tornado touched down last weekend åÈ http://t.co/x8zqbwNfO1,1,1 107 | 5874,8391,ruin,"MÌ©rida, YucatÌÁn",babe I'm gonna ruin you if you let me stay,0,0 108 | 6036,8627,seismic,"Hatteras, North Carolina",Agency seeks comments on seismic permits http://t.co/9Vd6x4WDOY,0,0 109 | 7589,10843,,,Omg earthquake,1,1 110 | 5386,7686,panic,Milwaukee WI,Someone asked me about a monkey fist about 2 feet long with a panic snap like the one pictured to be used as a... http://t.co/Yi9BBbx3FE,0,0 111 | 4604,6547,injury,"Los Angeles, California",California Law‰ÛÓNegligence and Fireworks Explosion Incidents http://t.co/d5w2zynP7b,1,1 112 | 5743,8201,riot,"Cardiff, UK",@PipRhys I predict a riot.,1,1 113 | 1015,1474,body%20bagging,401 livin',Aubrey really out here body-bagging Meek.,1,1 114 | 3425,4898,explode,they/them,"tagged by @attackonstiles 115 | 116 | millions 117 | a-punk 118 | hang em high 119 | alpha dog 120 | yeah boy and doll face 121 | little white lies 122 | explode http://t.co/lAtsSUo4wS",0,0 123 | 102,146,aftershock,Instagram - @heyimginog ,@afterShock_DeLo scuf ps live and the game... cya,0,0 124 | 5373,7669,panic,"Philadelphia, PA",Despite the crippling anxiety and overwhelming panic attacks I'd say I'm fairly well-adjusted.,0,0 125 | 3850,5479,flames,hell,@gilderoy i wish i was good enough to add flames to my nails im on fire,1,1 126 | 5945,8491,screamed,,i dont even remember slsp happening i just remember being like wtf and then the lights turned off and everyone screamed for the encore,0,0 127 | 5920,8452,screamed,,I heard the steven universe theme song from upstairs and screamed his name at the part of the song and scared my cousin,0,0 128 | 5310,7583,outbreak,Finland,Families to sue over Legionnaires: More than 40 families affected by the fatal outbreak of Legionnaires' disea... http://t.co/ZA4AXFJSVB,1,1 129 | 5955,8506,screaming,rio de janeiro | brazil,SCREAMING @MariahCarey @ArianaGrande http://t.co/xxZD1nmb1i,0,0 130 | 5454,7780,police,"Mesa, AZ",@ArizonaDOT Price Rd North bound closed from University to Rio Salado.. Lots of police.. What's crackin?,1,1 131 | 365,523,army,,One Direction Is my pick for http://t.co/q2eBlOKeVE Fan Army #Directioners http://t.co/eNCmhz6y34 x1386,0,0 132 | 5598,7988,razed,,The Latest: More Homes Razed by Northern California Wildfire - ABC News http://t.co/2872J5d4HB,1,1 133 | 5152,7348,obliterate,"Cavite, Philippines",The People's Republic Of China ( PROC ): Abandon the West Philippine Sea and all the ... https://t.co/pD14GsrfSC via @ChangePilipinas,0,0 134 | 128,184,aftershock,304,'Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose.' ‰ÛÒ Steve Jobs,0,0 135 | 4635,6588,inundated,"Zeerust, South Africa",Most of us ddnt get this English RT @ReIgN_CoCo: The World Is Inundated With Ostentatious People Stay Woke!,0,0 136 | 2519,3619,desolation,Arizona,Did Josephus get it wrong about Antiochus Epiphanes and the Abomination of Desolation? Read more: http://t.co/FWj9CcYw6k,0,0 137 | 2684,3850,detonation,,Ignition Knock (Detonation) Sensor-Senso Standard KS161 http://t.co/WadPP69LwJ http://t.co/yjTh2nABv5,0,0 138 | 4250,6038,heat%20wave,USA,Heat Advisory is In Effect From 1 PM Through 7 PM Thursday. A Building Heat Wave and Increasing Humidity... #lawx http://t.co/u0SYkowVWV,1,1 139 | 6610,9466,terrorism,,DHS Refuses to Call Chattanooga ‰Û÷Islamic Terrorism‰Ûª out of respect for MUSLIMS ... http://t.co/u8RGB51d22 via @po_st http://t.co/2tnu95VGFE,1,1 140 | 7386,10570,windstorm,Houston,New roof and hardy up..Windstorm inspection tomorrow http://t.co/kKeH8qCgc3,0,0 141 | 3921,5577,flood,New York,Spot Flood Combo 53inch 300W Curved Cree LED Work Light Bar 4X4 Offroad Fog Lamp - Full re‰Û_ http://t.co/mTmoIa0Oo0 http://t.co/Nn4ZtCmSRU,0,0 142 | 629,906,bioterrorism,,70 won 70...& some think possibility of my full transformation is impossible. I don't quite like medical mysteries. BIOTERRORISM sucks.,0,0 143 | 6986,10018,twister,,Twister was fun https://t.co/qCT6fb8wOn,0,0 144 | 7173,10278,war%20zone,,@RobertONeill31 Getting hit by a foul ball while sitting there is hardly a freak accident. It's a war zone.,0,0 145 | 2493,3582,desolate,Macclesfield,@binellithresa TY for the follow Go To http://t.co/UAN05TNkSW BRUTALLY ABUSED+DESOLATE&LOST + HER LOVELY MUM DIES..Is it Murder?,1,1 146 | 7467,10684,wounds,Charlotte,ME says many of these wounds could be fatal some rather quickly others slower and a couple not lethal at all. #KerrickTrial,0,0 147 | 1657,2394,collapsed,#ForeverWithBAP 8 ,and he almost collapsed bc he said his wish came true moderately FUCK,0,0 148 | 4737,6736,lava,Indonesia,@YoungHeroesID 4. Lava Blast Power Red #PantherAttack,0,0 149 | 5690,8120,rescued,"Winston-Salem, NC",'You can only be rescued from where you actually are and not from where you pretend you are.' Giorgio Hiatt,0,0 150 | 807,1173,blight,Kama | 18 | France ,@anellatulip there is a theory that makes way too much sense that says that the dwarves may be the actual origin of the blight,0,0 151 | 3149,4523,emergency,We are global!,SF Asian Women's Shelter crisis line (415) 751-0880. Emergency shelter/support services 4 non-English speaking Asian women & children.,1,1 152 | 3194,4585,emergency%20plan,Los Angeles,Senators calling for emergency housing: Boxer Feinstein back plan to move #homeless vets to VA campus http://t.co/Gm80X3vutf,1,1 153 | 3583,5119,fatal,Nakhon Si Thammarat,Investigators shift focus to cause of fatal Waimate fire http://t.co/c9dVDsSoFn,1,1 154 | 2380,3421,derail, Road to the Billionaires Club,@TheJenMorillo GM! I pray any attack of the enemy 2 derail ur destiny is blocked by the Lord & that He floods ur life w/heavenly Blessings,0,0 155 | 4456,6339,hostages,"Cumming, GA",C-130 specially modified to land in a stadium and rescue hostages in Iran in 1980 http://t.co/jkD7CTi2iW http://t.co/LAjN2n5e2d,1,1 156 | 2669,3831,detonate,,@WoundedPigeon http://t.co/s9soAeVcVo Detonate by @ApolloBrown ft. M.O.P.,0,0 157 | 3393,4857,evacuation,The Empire/First Order,@ariabrisard @leiaorganasolo Good. Play along with her. You may begin your operation with the death star. The evacuation is nearly complete.,0,0 158 | 3134,4503,emergency,,Emergency Flow http://t.co/lH9mrYpDrJ mp3 http://t.co/PqhuthSS3i rar http://t.co/0iW6dRf5X9,1,1 159 | 1799,2585,crash,Galatians 2:20 ,Please keep Josh the Salyers/Blair/Hall families & Jenna's friends in your prayers. She was taken far too soon. RIP http://t.co/bDN2FDPdAz,0,0 160 | 6339,9061,structural%20failure,,Investigators say a fatal Virgin Galactic spaceship crash last year was caused by structural failure after the co‰Û_,1,1 161 | 5162,7361,obliterate,Cymru araul,@McCaineNL Think how spectacular it will look when the Stonewall riots obliterate the white house.,1,1 162 | 4788,6811,loud%20bang,,Who the fuck plays music extremely loud at 9am on a Tuesday morning?? Bruv do they want me to come bang them??,0,0 163 | 2871,4127,drought,Meereen ,Pizza drought is over I just couldn't anymore...,0,0 164 | 3020,4335,dust%20storm,"Atlanta, GA",DUST IN THE WIND: @82ndABNDIV paratroopers move to a loading zone during a dust storm in support of Operation Fury: http://t.co/uGesKLCn8M,1,1 165 | 880,1275,blood,,Shed innocent blood of their sons and daughters and the land was polluted Psalms 106:38 Help stop the sin of abortion.,0,0 166 | 5138,7328,nuclear%20reactor,"Den Helder, Rijkswerf",US Navy Sidelines 3 Newest #Subs http://t.co/9WQixGMHfh,0,0 167 | 4216,5989,hazardous,,DLH issues Hazardous Weather Outlook (HWO) http://t.co/WOzuBXRi2p,1,1 168 | 1310,1892,burning,"Caracas, Venezuela.",We will be burning up like neon lights??????,1,1 169 | 3939,5600,flood,United States,Flood Advisory issued August 05 at 4:28PM EDT by NWS http://t.co/D02sbM0ojs #WxKY,1,1 170 | 6017,8595,seismic,garowe puntland somalia,Oil and Gas Exploration http://t.co/PckF0nl2yN,1,1 171 | 6105,8717,sinking,"Michigan, USA",‰Û¢‰Û¢If your lost & alone or your sinking like a stone carry onå¡å¡,0,0 172 | 937,1354,blown%20up,Nowhere Islands/Smash Manor,@TheBoyOfMasks 'Thanks again for letting me stay here since the manor was blown up..... Anyways how are you doing buddy?',0,0 173 | 314,457,armageddon,Canada,@ENews Ben Affleck......I know there's a wife/kids and other girls but I can't help it. I've loved him since Armageddon #eonlinechat,0,0 174 | 6275,8966,storm,,What tropical storm? #guillermo by hawaiianpaddlesports http://t.co/LgPgAjgomY http://t.co/FKd1mBTB68,1,1 175 | 1889,2715,crushed,#HAMont,Edwin wow. Crushed.,0,0 176 | 5550,7916,rainstorm,,#DnB #NewRelease EDGE Jimmy - Summer Rainstorm (Lapaka Sounds) http://t.co/4L8h2FKlNO via http://t.co/ZITQKDFXJY,0,0 177 | 6415,9173,suicide%20bomber,,http://t.co/9k1tqsAarM Suicide bomber kills 15 in Saudi security site mosque - Reuters http://t.co/Ev3nX9scx3,1,1 178 | 5722,8165,rescuers,,Last Second Ebay Bid RT? http://t.co/oEKUcq4ZL0 Shaolin Rescuers (dvd 2010) Shen Chan Nan Chiang Five Venoms Kung Fu ?Please Favori,0,0 179 | 5071,7229,natural%20disaster,"Suburban Detroit, Michigan",We have overrun a Natural Disaster Survival server!,1,1 180 | 4274,6072,heat%20wave,"Oklahoma City, OK",Longest Streak of Triple-Digit Heat Since 2013 Forecast in Dallas: An unrelenting and dangerous heat wave will... http://t.co/s4Srgrmqcz,1,1 181 | 1035,1501,body%20bags,Swaning Around,US ‰Û÷Institute Of Peace‰Ûª Chairman Wants Russian Body Bags http://t.co/owbUjez3q4,0,0 182 | 4316,6128,hellfire,"Denver, Colorado",@MechaMacGyver Wow bet you got blamed for that too huh?,0,0 183 | 2077,2983,dead,"Sochi, KDA, RU",@hlportal Hello! I'm looking for mod Cold Ice. I saw it on your site but link to download dead. Maybe you have it and share with me? Thanks.,0,0 184 | 3899,5545,flattened,Le Memenet,@KainYusanagi @Grummz @PixelCanuck and flattened raynor. Raynor was a balding imperfect biker marine not a emo generic western hero.,0,0 185 | 3974,5650,flooding,Global-NoLocation,#flood #disaster Burst Water Pipe Floods Apartments at NYCHA Senior Center - NY1: NY1Burst Water Pipe Floods A... http://t.co/w7SIIdujOH,1,1 186 | 7050,10101,typhoon,The Peach State,I think a Typhoon just passed through here lol,1,1 187 | 3383,4845,evacuation,"sydney, australia",my school just put the evacuation alarms on accidently with 2 different trial exams happening are you kidding me,0,0 188 | 7562,10811,wrecked,,you wrecked my whole world,0,0 189 | 2088,3001,dead,Milton Keynes ,Can't believe Ross is dead???????? @emmerdale @MikeParrActor #Emmerdale #summerfate,0,0 190 | 6011,8584,screams,5-Feb,When you go to a concert and someone screams in your ear... Does it look like I wanna loose my hearing anytime soon???,1,1 191 | 2295,3292,demolish,,I have completed the quest 'Demolish 5 Murlo...' in the #Android game The Tribez. http://t.co/pBclFsXRld #androidgames #gameinsight,0,0 192 | 4745,6749,lightning,,Thunder and lightning possible in the Pinpoint Foothill Forecast. http://t.co/CtIjdPXABk,1,1 193 | 16,24,,,I love fruits,0,0 194 | 6596,9448,terrorism,,@RobPulseNews @huyovoeTripolye Phillips should be charged for assisting terrorism. LDNR-terrorists' organizations. http://t.co/XwnJYsV9V9,1,1 195 | 514,740,attacked,"SÌ£o Paulo SP, Brasil",Christian Attacked by Muslims at the Temple Mount after Waving Israeli Flag via Pamela Geller - ... http://t.co/e4wK8Uri8A,1,1 196 | 2551,3659,destroy,,"ng2x5 mhtw4fnet 197 | 198 | Watch Michael Jordan absolutely destroy this meme-baiting camper - FOXSportscom",0,0 199 | 6184,8829,sirens,,@iK4LEN Sirens was cancelled.,0,0 200 | 7359,10536,windstorm,,Windstorm lastingness perquisite - acquiesce in a twister retreat: ZiUW http://t.co/iRt4kkgsJx,1,1 201 | 927,1343,blown%20up,Georgia,Man why hasn't @machinegunkelly blown up? He's still underground.,0,0 202 | 7238,10366,weapons,??????,#Battlefield 1942 forgotten hope secret weapons,1,1 203 | 88,130,accident,"Manchester, NH",Accident left lane blocked in #Manchester on Rt 293 NB before Eddy Rd stop and go traffic back to NH-3A delay of 4 mins #traffic,1,1 204 | 4374,6215,hijacker,,Governor allows parole for California school bus hijacker: Local... http://t.co/tAM6aoskoJ http://t.co/eL24mnFcHw,1,1 205 | 5922,8454,screamed,ljp/4,THE GIRLS NEXT TO ME SCREAMED WHAT THE FUCK IS A CHONCE I'm CRYIBG,0,0 206 | 6834,9788,trapped,å_å_Los Mina City‰ã¢,Hollywood Movie About Trapped Miners Released in Chile: 'The 33' Hollywood movie about trapped miners starring... http://t.co/x8moYeVjsJ,0,0 207 | 5842,8348,ruin,London / Birmingham,Im so anxious though because so many ppl will me watching me meet them and that makes me uncomfortable BUT I CANT LET THAT RUIN THE MOMENT,0,0 208 | 2039,2927,danger,Spinning through time.,@riverroaming 'And not too much danger please.',0,0 209 | 2942,4230,drowned,,Thank you Uplifting spirit. When Im drowned you've been an anchor,0,0 210 | 523,755,avalanche,Ireland,A little piece I wrote for the Avalanche Designs blog! I'd appreciate it greatly if you checked it out :-) https://t.co/rfvjh58eF2,0,0 211 | 6432,9203,suicide%20bombing,,< 25 Dead In Kuwait Mosque Suicide Bombing Claimed By ISIS Offshoot on http://t.co/eTITgPSrUN,1,1 212 | 925,1340,blown%20up,USA,@troylercraft YEAH ITS NOT WORTH IT BC HE ALREADY HAS SO MANY SPAMMERS & HIS TWITTER IS PROBABLY BLOWN UP EVERY SECOND,0,0 213 | 587,848,bioterror,iTunes,#world FedEx no longer to transport bioterror germs in wake of anthrax lab mishaps http://t.co/wvExJjRG6E,1,1 214 | 3661,5211,fatality,"Lowell, MA",@EBROINTHEAM jay....big L....pun....biggie...wrap over...zero question....fatality...flawless victory http://t.co/Y33QcKq7qD,0,0 215 | 6503,9297,survive,United States,AHH forgot my headphones how am I supposed to survive a day without music AYHHHHHDJJFJRJJRDJJEKS,0,0 216 | 5120,7301,nuclear%20reactor,"USA, North Dakota",Salem 2 nuclear reactor shut down over electrical circuit failure on pump: The Salem 2 nuclear reactor had bee... http://t.co/5hkGXzJLmX,1,1 217 | 7541,10782,wreckage,"New Delhi,India",Wreckage 'Conclusively Confirmed' as From MH370: Malaysia PM: Investigators and the families of those who were... http://t.co/1YIxFG1Hdy,1,1 218 | 4779,6800,loud%20bang,Kenya,tkyonly1fmk: Breaking news! Unconfirmed! I just heard a loud bang nearby. in what appears to be a blast of wind from my neighbour's ass.,0,0 219 | 2342,3369,demolition,,#download & #watch Demolition Frog (2002) http://t.co/81nEizeknm #movie,1,1 220 | 7526,10764,wreckage,"Dublin City, Ireland",Wreckage 'Conclusively Confirmed' as From MH370: Malaysia PM: Investigators and the families of those who were... http://t.co/VAZpG0ftmU,1,1 221 | 2614,3752,destruction,,Crackdown 3 Destruction Restricted to Multiplayer: Crackdown 3 impressed earlier this week with a demonstratio... http://t.co/LMWKjsYCgj,0,0 222 | 2638,3786,destruction,Jersey,That's the ultimate road to destruction,0,0 223 | 963,1393,body%20bag,,new summer long thin body bag hip A word skirt Blue http://t.co/lvKoEMsq8m http://t.co/CjiRhHh4vj,0,0 224 | 383,552,arson,"Charlotte, NC",Add Familia to the arson squad.,0,0 225 | 4522,6425,hurricane,,Stream HYPE HURRICANE,0,0 226 | 2049,2941,danger,Instagram: trillrebel_,"Guns are for protection.. 227 | That shit really shouldn't be used unless your life in danger",0,0 228 | 7532,10770,wreckage,iTunes,#science Now that a piece of wreckage from flight MH370 has been confirmed on RÌ©union Island is it possible t... http://t.co/Z2vDGIyOwf,1,1 229 | 5345,7630,pandemonium,illinois. united state ,Pandemonium In Aba As Woman Delivers Baby Without Face (Photos) @... http://t.co/JbxBi93CLu,0,0 230 | 1342,1940,burning%20buildings,,Hero's fight wars and save ppl from burning buildings etc I'm sorry but u gotta do more than pay 4 a sex change be4 I call u a hero,0,0 231 | -------------------------------------------------------------------------------- /data/english_to_latex.csv: -------------------------------------------------------------------------------- 1 | English,LaTeX 2 | integral from a to b of x squared,"\int_{a}^{b} x^2 \,dx" 3 | integral from negative 1 to 1 of x squared,"\int_{-1}^{1} x^2 \,dx" 4 | integral from negative 1 to infinity of x cubed,"\int_{-1}^{\inf} x^3 \,dx" 5 | integral from 0 to infinity of x squared,"\int_{0}^{\inf} x^2 \,dx" 6 | integral from 0 to infinity of y squared,"\int_{0}^{\inf} y^2 \,dy" 7 | integral from 1 to 2 of x over 2,"\int_{1}^{2} \frac{x}{2} \,dx" 8 | f of x equals x squared,f(x) = x^2 9 | h of x equals x squared,h(x) = x^2 10 | g of x equals x squared,g(x) = x^2 11 | g of x equals x to the eighth power,g(x) = x^8 12 | f of x equals x cubed,f(x) = x^3 13 | f of x equals x,f(x) = x 14 | h of x equals x to the fifth power,h(x) = x^5 15 | g of x equals integral from 0 to 10 of x cubed,"g(x) = \int_{0}^{10} x^3 \,dx" 16 | f of x equals x over n,f(x) = \frac{x}{n} 17 | f of x equals integral from 1 to 2 of x,"f(x) = \int_{1}^{2} x \,dx" 18 | f of x equals integral from 0 to 2 of x,"f(x) = \int_{0}^{2} x \,dx" 19 | f of x equals integral from 1 to 2 of x over 2,"f(x) = \int_{1}^{2} \frac{x}{2} \,dx" 20 | f of x equals sum from 1 to 5 of x squared,f(x) = \sum_{1}^{5} x^2 21 | x squared,x^2 22 | x cubed,x^3 23 | pi squared,\pi^2 24 | z squared,z^2 25 | z over x squared,\frac{z}{x^2} 26 | f of x equals x squared,f(x) = x^2 27 | 1 over 6,\frac{1}{6} 28 | 2 pi,2 * \pi 29 | s cubed,s^3 30 | s to the sixth power,s^6 31 | 2 pi r,2 * \pi * r 32 | pi over n,\frac{\pi}{n} 33 | f of n equals pi over n,f(n) = \frac{\pi}{n} 34 | pi times x,\pi*x 35 | pi to the fourth power,\pi^4 36 | pi to the fifth power,\pi^5 37 | f of x equals x times pi to the fifth power,f(x) = x * \pi^5 38 | g of x equals x times pi cubed,g(x) = x * \pi^3 39 | g of x equals pi cubed,g(x) = \pi^3 40 | 1 over n,\frac{1}{n} 41 | x squared over n,\frac{x^2}{n} 42 | y squared over x^2,\frac{y^2}{x^2} 43 | 1 over 7 to the seventh power,(\frac{1}{7})^7 44 | 1 over 9 to the seventh power,(\frac{1}{9})^7 45 | f of x equals x over 9 to the seventh power,(f(x) = \frac{x}{9})^7 46 | sum from i to n of X i,\sum_{i}^{n} X_i 47 | sum from 0 to n of 77 n,\sum_{0}^{n} 77 * n 48 | sum from 0 to 5 of x,\sum_{0}^{5} x 49 | sum from 1 to x of x,\sum_{1}^{x} x 50 | sum from 1 to x of x squared,\sum_{1}^{x} x^2 51 | sum from 1 to 10 of pi squared,\sum_{1}^{10} \pi^2 52 | -------------------------------------------------------------------------------- /images/Venus_and_Adonis_by_Peter_Paul_Rubens.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinanuozdemir/oreilly-pytorch-dl/3e9ccbcf5182f2e89e6db4655242f1208fa34058/images/Venus_and_Adonis_by_Peter_Paul_Rubens.jpg -------------------------------------------------------------------------------- /images/oreilly.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sinanuozdemir/oreilly-pytorch-dl/3e9ccbcf5182f2e89e6db4655242f1208fa34058/images/oreilly.png -------------------------------------------------------------------------------- /notebooks/mnist.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "876f9496-2aeb-4b3c-9e92-7d8e3c1bf051", 6 | "metadata": {}, 7 | "source": [ 8 | "# Training a DL model on the MNIST dataset" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "id": "4cfbde3a-e540-49e8-99f0-63993987a6d5", 14 | "metadata": { 15 | "jp-MarkdownHeadingCollapsed": true 16 | }, 17 | "source": [ 18 | "## Imports and Data Setup" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": 2, 24 | "id": "bdfba6c5", 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "# Import required libraries\n", 29 | "import torch\n", 30 | "import torch.nn as nn\n", 31 | "import matplotlib.pyplot as plt\n", 32 | "import torch.nn.functional as F\n", 33 | "\n", 34 | "import torchvision\n", 35 | "import numpy as np\n", 36 | "import torchvision.transforms as transforms\n", 37 | "from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score\n" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": 3, 43 | "id": "deb4a334", 44 | "metadata": {}, 45 | "outputs": [ 46 | { 47 | "data": { 48 | "text/plain": [ 49 | "" 50 | ] 51 | }, 52 | "execution_count": 3, 53 | "metadata": {}, 54 | "output_type": "execute_result" 55 | } 56 | ], 57 | "source": [ 58 | "# standardize random numbers for reproducibility\n", 59 | "torch.manual_seed(0)\n" 60 | ] 61 | }, 62 | { 63 | "cell_type": "code", 64 | "execution_count": 4, 65 | "id": "cf93c989", 66 | "metadata": {}, 67 | "outputs": [], 68 | "source": [ 69 | "img = torchvision.datasets.MNIST(root='../data', train=True, download=True)[0]" 70 | ] 71 | }, 72 | { 73 | "cell_type": "code", 74 | "execution_count": 5, 75 | "id": "41a542d4-c4c1-4640-9145-3c0836e0e5c3", 76 | "metadata": {}, 77 | "outputs": [ 78 | { 79 | "data": { 80 | "text/plain": [ 81 | "(, 5)" 82 | ] 83 | }, 84 | "execution_count": 5, 85 | "metadata": {}, 86 | "output_type": "execute_result" 87 | } 88 | ], 89 | "source": [ 90 | "img # PIL == Pillow" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": 6, 96 | "id": "0de01bf4", 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "# Load MNIST dataset\n", 101 | "batch_size = 4\n", 102 | "transform = transforms.Compose(\n", 103 | " [\n", 104 | " transforms.ToTensor(), # convert to pytorch tensor\n", 105 | " transforms.Normalize((0.1307,), (0.3081,)), # standardize the values with mean + std. Found before running code\n", 106 | " # transforms.RandomVerticalFlip()\n", 107 | " ] \n", 108 | ")\n", 109 | "\n", 110 | "train_dataset = torchvision.datasets.MNIST(root='../data', train=True, transform=transform, download=True)\n", 111 | "test_dataset = torchvision.datasets.MNIST(root='../data', train=False, transform=transform, download=True)\n", 112 | "\n", 113 | "train_loader = torch.utils.data.DataLoader(dataset=train_dataset, batch_size=batch_size, shuffle=True)\n", 114 | "test_loader = torch.utils.data.DataLoader(dataset=test_dataset, batch_size=batch_size, shuffle=False)\n" 115 | ] 116 | }, 117 | { 118 | "cell_type": "code", 119 | "execution_count": 7, 120 | "id": "e926a88c", 121 | "metadata": {}, 122 | "outputs": [ 123 | { 124 | "data": { 125 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAcTUlEQVR4nO3df3DU9b3v8dcCyQqaLI0hv0rAgD+wAvEWJWZAxJJLSOc4gIwHf3QGvF4cMXiKaPXGUZHWM2nxjrV6qd7TqURnxB+cEaiO5Y4GE441oQNKGW7blNBY4iEJFSe7IUgIyef+wXXrQgJ+1l3eSXg+Zr4zZPf75vvx69Znv9nNNwHnnBMAAOfYMOsFAADOTwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYGGG9gFP19vbq4MGDSktLUyAQsF4OAMCTc04dHR3Ky8vTsGH9X+cMuAAdPHhQ+fn51ssAAHxDzc3NGjt2bL/PD7gApaWlSZJm6vsaoRTj1QAAfJ1Qtz7QO9H/nvcnaQFat26dnnrqKbW2tqqwsFDPPfecpk+ffta5L7/tNkIpGhEgQAAw6Pz/O4ye7W2UpHwI4fXXX9eqVau0evVqffTRRyosLFRpaakOHTqUjMMBAAahpATo6aef1rJly3TnnXfqO9/5jl544QWNGjVKL774YjIOBwAYhBIeoOPHj2vXrl0qKSn5x0GGDVNJSYnq6upO27+rq0uRSCRmAwAMfQkP0Geffaaenh5lZ2fHPJ6dna3W1tbT9q+srFQoFIpufAIOAM4P5j+IWlFRoXA4HN2am5utlwQAOAcS/im4zMxMDR8+XG1tbTGPt7W1KScn57T9g8GggsFgopcBABjgEn4FlJqaqmnTpqm6ujr6WG9vr6qrq1VcXJzowwEABqmk/BzQqlWrtGTJEl1zzTWaPn26nnnmGXV2durOO+9MxuEAAINQUgK0ePFi/f3vf9fjjz+u1tZWXX311dq6detpH0wAAJy/As45Z72Ir4pEIgqFQpqt+dwJAQAGoROuWzXaonA4rPT09H73M/8UHADg/ESAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYGGG9AGAgCYzw/5/E8DGZSVhJYjQ8eElccz2jer1nxk885D0z6t6A90zr06neMx9d87r3jCR91tPpPVO08QHvmUtX1XvPDAVcAQEATBAgAICJhAfoiSeeUCAQiNkmTZqU6MMAAAa5pLwHdNVVV+m99977x0Hi+L46AGBoS0oZRowYoZycnGT81QCAISIp7wHt27dPeXl5mjBhgu644w4dOHCg3327uroUiURiNgDA0JfwABUVFamqqkpbt27V888/r6amJl1//fXq6Ojoc//KykqFQqHolp+fn+glAQAGoIQHqKysTLfccoumTp2q0tJSvfPOO2pvb9cbb7zR5/4VFRUKh8PRrbm5OdFLAgAMQEn/dMDo0aN1+eWXq7Gxsc/ng8GggsFgspcBABhgkv5zQEeOHNH+/fuVm5ub7EMBAAaRhAfowQcfVG1trT755BN9+OGHWrhwoYYPH67bbrst0YcCAAxiCf8W3KeffqrbbrtNhw8f1pgxYzRz5kzV19drzJgxiT4UAGAQS3iAXnvttUT/lRighl95mfeMC6Z4zxy8YbT3zBfX+d9EUpIyQv5z/1EY340uh5rfHk3znvnZ/5rnPbNjygbvmabuL7xnJOmnbf/VeybvP1xcxzofcS84AIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMBE0n8hHQa+ntnfjWvu6ap13jOXp6TGdSycW92ux3vm8eeWes+M6PS/cWfxxhXeM2n/ecJ7RpKCn/nfxHTUzh1xHet8xBUQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATHA3bCjYcDCuuV3H8r1nLk9pi+tYQ80DLdd5z/z1SKb3TNXEf/eekaRwr/9dqrOf/TCuYw1k/mcBPrgCAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMcDNS6ERLa1xzz/3sFu+Zf53X6T0zfM9F3jN/uPc575l4PfnZVO+ZxpJR3jM97S3eM7cX3+s9I0mf/Iv/TIH+ENexcP7iCggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMMHNSBG3jPV13jNj3rrYe6bn8OfeM1dN/m/eM5L0f2e96D3zm3+7wXsmq/1D75l4BOriu0Fogf+/WsAbV0AAABMECABgwjtA27dv10033aS8vDwFAgFt3rw55nnnnB5//HHl5uZq5MiRKikp0b59+xK1XgDAEOEdoM7OThUWFmrdunV9Pr927Vo9++yzeuGFF7Rjxw5deOGFKi0t1bFjx77xYgEAQ4f3hxDKyspUVlbW53POOT3zzDN69NFHNX/+fEnSyy+/rOzsbG3evFm33nrrN1stAGDISOh7QE1NTWptbVVJSUn0sVAopKKiItXV9f2xmq6uLkUikZgNADD0JTRAra2tkqTs7OyYx7Ozs6PPnaqyslKhUCi65efnJ3JJAIAByvxTcBUVFQqHw9GtubnZekkAgHMgoQHKycmRJLW1tcU83tbWFn3uVMFgUOnp6TEbAGDoS2iACgoKlJOTo+rq6uhjkUhEO3bsUHFxcSIPBQAY5Lw/BXfkyBE1NjZGv25qatLu3buVkZGhcePGaeXKlXryySd12WWXqaCgQI899pjy8vK0YMGCRK4bADDIeQdo586duvHGG6Nfr1q1SpK0ZMkSVVVV6aGHHlJnZ6fuvvtutbe3a+bMmdq6dasuuOCCxK0aADDoBZxzznoRXxWJRBQKhTRb8zUikGK9HAxSf/nf18Y3908veM/c+bc53jN/n9nhPaPeHv8ZwMAJ160abVE4HD7j+/rmn4IDAJyfCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYML71zEAg8GVD/8lrrk7p/jf2Xr9+Oqz73SKG24p955Je73eewYYyLgCAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMcDNSDEk97eG45g4vv9J75sBvvvCe+R9Pvuw9U/HPC71n3Mch7xlJyv/XOv8h5+I6Fs5fXAEBAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACa4GSnwFb1/+JP3zK1rfuQ988rq/+k9s/s6/xuY6jr/EUm66sIV3jOX/arFe+bEXz/xnsHQwRUQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGAi4Jxz1ov4qkgkolAopNmarxGBFOvlAEnhZlztPZP+00+9Z16d8H+8Z+I16f3/7j1zxZqw90zPvr96z+DcOuG6VaMtCofDSk9P73c/roAAACYIEADAhHeAtm/frptuukl5eXkKBALavHlzzPNLly5VIBCI2ebNm5eo9QIAhgjvAHV2dqqwsFDr1q3rd5958+appaUlur366qvfaJEAgKHH+zeilpWVqays7Iz7BINB5eTkxL0oAMDQl5T3gGpqapSVlaUrrrhCy5cv1+HDh/vdt6urS5FIJGYDAAx9CQ/QvHnz9PLLL6u6ulo/+9nPVFtbq7KyMvX09PS5f2VlpUKhUHTLz89P9JIAAAOQ97fgzubWW2+N/nnKlCmaOnWqJk6cqJqaGs2ZM+e0/SsqKrRq1aro15FIhAgBwHkg6R/DnjBhgjIzM9XY2Njn88FgUOnp6TEbAGDoS3qAPv30Ux0+fFi5ubnJPhQAYBDx/hbckSNHYq5mmpqatHv3bmVkZCgjI0Nr1qzRokWLlJOTo/379+uhhx7SpZdeqtLS0oQuHAAwuHkHaOfOnbrxxhujX3/5/s2SJUv0/PPPa8+ePXrppZfU3t6uvLw8zZ07Vz/5yU8UDAYTt2oAwKDHzUiBQWJ4dpb3zMHFl8Z1rB0P/8J7Zlgc39G/o2mu90x4Zv8/1oGBgZuRAgAGNAIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJhI+K/kBpAcPW2HvGeyn/WfkaRjD53wnhkVSPWe+dUlb3vP/NPCld4zozbt8J5B8nEFBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCY4GakgIHemVd7z+y/5QLvmclXf+I9I8V3Y9F4PPf5f/GeGbVlZxJWAgtcAQEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJrgZKfAVgWsme8/85V/8b9z5qxkvec/MuuC498y51OW6vWfqPy/wP1Bvi/8MBiSugAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAE9yMFAPeiILx3jP778yL61hPLH7Ne2bRRZ/FdayB7JG2a7xnan9xnffMt16q857B0MEVEADABAECAJjwClBlZaWuvfZapaWlKSsrSwsWLFBDQ0PMPseOHVN5ebkuvvhiXXTRRVq0aJHa2toSumgAwODnFaDa2lqVl5ervr5e7777rrq7uzV37lx1dnZG97n//vv11ltvaePGjaqtrdXBgwd18803J3zhAIDBzetDCFu3bo35uqqqSllZWdq1a5dmzZqlcDisX//619qwYYO+973vSZLWr1+vK6+8UvX19bruOv83KQEAQ9M3eg8oHA5LkjIyMiRJu3btUnd3t0pKSqL7TJo0SePGjVNdXd+fdunq6lIkEonZAABDX9wB6u3t1cqVKzVjxgxNnjxZktTa2qrU1FSNHj06Zt/s7Gy1trb2+fdUVlYqFApFt/z8/HiXBAAYROIOUHl5ufbu3avXXvP/uYmvqqioUDgcjm7Nzc3f6O8DAAwOcf0g6ooVK/T2229r+/btGjt2bPTxnJwcHT9+XO3t7TFXQW1tbcrJyenz7woGgwoGg/EsAwAwiHldATnntGLFCm3atEnbtm1TQUFBzPPTpk1TSkqKqquro481NDTowIEDKi4uTsyKAQBDgtcVUHl5uTZs2KAtW7YoLS0t+r5OKBTSyJEjFQqFdNddd2nVqlXKyMhQenq67rvvPhUXF/MJOABADK8APf/885Kk2bNnxzy+fv16LV26VJL085//XMOGDdOiRYvU1dWl0tJS/fKXv0zIYgEAQ0fAOeesF/FVkUhEoVBIszVfIwIp1svBGYy4ZJz3THharvfM4h9vPftOp7hn9F+9Zwa6B1r8v4tQ90v/m4pKUkbV7/2HenviOhaGnhOuWzXaonA4rPT09H73415wAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMBHXb0TFwDUit+/fPHsmn794YVzHWl5Q6z1zW1pbXMcayFb850zvmY+ev9p7JvPf93rPZHTUec8A5wpXQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACW5Geo4cL73Gf+b+z71nHrn0He+ZuSM7vWcGuraeL+Kam/WbB7xnJj36Z++ZjHb/m4T2ek8AAxtXQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACW5Geo58ssC/9X+ZsjEJK0mcde0TvWd+UTvXeybQE/CemfRkk/eMJF3WtsN7pieuIwHgCggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMBFwzjnrRXxVJBJRKBTSbM3XiECK9XIAAJ5OuG7VaIvC4bDS09P73Y8rIACACQIEADDhFaDKykpde+21SktLU1ZWlhYsWKCGhoaYfWbPnq1AIBCz3XPPPQldNABg8PMKUG1trcrLy1VfX693331X3d3dmjt3rjo7O2P2W7ZsmVpaWqLb2rVrE7poAMDg5/UbUbdu3RrzdVVVlbKysrRr1y7NmjUr+vioUaOUk5OTmBUCAIakb/QeUDgcliRlZGTEPP7KK68oMzNTkydPVkVFhY4ePdrv39HV1aVIJBKzAQCGPq8roK/q7e3VypUrNWPGDE2ePDn6+O23367x48crLy9Pe/bs0cMPP6yGhga9+eabff49lZWVWrNmTbzLAAAMUnH/HNDy5cv129/+Vh988IHGjh3b737btm3TnDlz1NjYqIkTJ572fFdXl7q6uqJfRyIR5efn83NAADBIfd2fA4rrCmjFihV6++23tX379jPGR5KKiookqd8ABYNBBYPBeJYBABjEvALknNN9992nTZs2qaamRgUFBWed2b17tyQpNzc3rgUCAIYmrwCVl5drw4YN2rJli9LS0tTa2ipJCoVCGjlypPbv368NGzbo+9//vi6++GLt2bNH999/v2bNmqWpU6cm5R8AADA4eb0HFAgE+nx8/fr1Wrp0qZqbm/WDH/xAe/fuVWdnp/Lz87Vw4UI9+uijZ/w+4FdxLzgAGNyS8h7Q2VqVn5+v2tpan78SAHCe4l5wAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATI6wXcCrnnCTphLolZ7wYAIC3E+qW9I//nvdnwAWoo6NDkvSB3jFeCQDgm+jo6FAoFOr3+YA7W6LOsd7eXh08eFBpaWkKBAIxz0UiEeXn56u5uVnp6elGK7THeTiJ83AS5+EkzsNJA+E8OOfU0dGhvLw8DRvW/zs9A+4KaNiwYRo7duwZ90lPTz+vX2Bf4jycxHk4ifNwEufhJOvzcKYrny/xIQQAgAkCBAAwMagCFAwGtXr1agWDQeulmOI8nMR5OInzcBLn4aTBdB4G3IcQAADnh0F1BQQAGDoIEADABAECAJggQAAAE4MmQOvWrdMll1yiCy64QEVFRfr9739vvaRz7oknnlAgEIjZJk2aZL2spNu+fbtuuukm5eXlKRAIaPPmzTHPO+f0+OOPKzc3VyNHjlRJSYn27dtns9gkOtt5WLp06Wmvj3nz5tksNkkqKyt17bXXKi0tTVlZWVqwYIEaGhpi9jl27JjKy8t18cUX66KLLtKiRYvU1tZmtOLk+DrnYfbs2ae9Hu655x6jFfdtUATo9ddf16pVq7R69Wp99NFHKiwsVGlpqQ4dOmS9tHPuqquuUktLS3T74IMPrJeUdJ2dnSosLNS6dev6fH7t2rV69tln9cILL2jHjh268MILVVpaqmPHjp3jlSbX2c6DJM2bNy/m9fHqq6+ewxUmX21trcrLy1VfX693331X3d3dmjt3rjo7O6P73H///Xrrrbe0ceNG1dbW6uDBg7r55psNV514X+c8SNKyZctiXg9r1641WnE/3CAwffp0V15eHv26p6fH5eXlucrKSsNVnXurV692hYWF1sswJclt2rQp+nVvb6/LyclxTz31VPSx9vZ2FwwG3auvvmqwwnPj1PPgnHNLlixx8+fPN1mPlUOHDjlJrra21jl38t99SkqK27hxY3SfP/3pT06Sq6urs1pm0p16Hpxz7oYbbnA//OEP7Rb1NQz4K6Djx49r165dKikpiT42bNgwlZSUqK6uznBlNvbt26e8vDxNmDBBd9xxhw4cOGC9JFNNTU1qbW2NeX2EQiEVFRWdl6+PmpoaZWVl6YorrtDy5ct1+PBh6yUlVTgcliRlZGRIknbt2qXu7u6Y18OkSZM0bty4If16OPU8fOmVV15RZmamJk+erIqKCh09etRief0acDcjPdVnn32mnp4eZWdnxzyenZ2tP//5z0arslFUVKSqqipdccUVamlp0Zo1a3T99ddr7969SktLs16eidbWVknq8/Xx5XPni3nz5unmm29WQUGB9u/fr0ceeURlZWWqq6vT8OHDrZeXcL29vVq5cqVmzJihyZMnSzr5ekhNTdXo0aNj9h3Kr4e+zoMk3X777Ro/frzy8vK0Z88ePfzww2poaNCbb75puNpYAz5A+IeysrLon6dOnaqioiKNHz9eb7zxhu666y7DlWEguPXWW6N/njJliqZOnaqJEyeqpqZGc+bMMVxZcpSXl2vv3r3nxfugZ9Lfebj77rujf54yZYpyc3M1Z84c7d+/XxMnTjzXy+zTgP8WXGZmpoYPH37ap1ja2tqUk5NjtKqBYfTo0br88svV2NhovRQzX74GeH2cbsKECcrMzBySr48VK1bo7bff1vvvvx/z61tycnJ0/Phxtbe3x+w/VF8P/Z2HvhQVFUnSgHo9DPgApaamatq0aaquro4+1tvbq+rqahUXFxuuzN6RI0e0f/9+5ebmWi/FTEFBgXJycmJeH5FIRDt27DjvXx+ffvqpDh8+PKReH845rVixQps2bdK2bdtUUFAQ8/y0adOUkpIS83poaGjQgQMHhtTr4WznoS+7d++WpIH1erD+FMTX8dprr7lgMOiqqqrcH//4R3f33Xe70aNHu9bWVuulnVMPPPCAq6mpcU1NTe53v/udKykpcZmZme7QoUPWS0uqjo4O9/HHH7uPP/7YSXJPP/20+/jjj93f/vY355xzP/3pT93o0aPdli1b3J49e9z8+fNdQUGB++KLL4xXnlhnOg8dHR3uwQcfdHV1da6pqcm999577rvf/a677LLL3LFjx6yXnjDLly93oVDI1dTUuJaWluh29OjR6D733HOPGzdunNu2bZvbuXOnKy4udsXFxYarTryznYfGxkb34x//2O3cudM1NTW5LVu2uAkTJrhZs2YZrzzWoAiQc84999xzbty4cS41NdVNnz7d1dfXWy/pnFu8eLHLzc11qamp7tvf/rZbvHixa2xstF5W0r3//vtO0mnbkiVLnHMnP4r92GOPuezsbBcMBt2cOXNcQ0OD7aKT4Ezn4ejRo27u3LluzJgxLiUlxY0fP94tW7ZsyP2ftL7++SW59evXR/f54osv3L333uu+9a1vuVGjRrmFCxe6lpYWu0UnwdnOw4EDB9ysWbNcRkaGCwaD7tJLL3U/+tGPXDgctl34Kfh1DAAAEwP+PSAAwNBEgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJj4f4W4/AnknuSPAAAAAElFTkSuQmCC", 126 | "text/plain": [ 127 | "
" 128 | ] 129 | }, 130 | "metadata": {}, 131 | "output_type": "display_data" 132 | } 133 | ], 134 | "source": [ 135 | "# transpose the array to (28, 28) format expected by Matplotlib\n", 136 | "array = np.squeeze(np.transpose(train_dataset[0][0], (1, 2, 0)))\n", 137 | "\n", 138 | "# plot the image using Matplotlib\n", 139 | "plt.imshow(array)\n", 140 | "plt.show()" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": 8, 146 | "id": "42f64815", 147 | "metadata": {}, 148 | "outputs": [ 149 | { 150 | "data": { 151 | "text/plain": [ 152 | "torch.Size([4, 1, 28, 28])" 153 | ] 154 | }, 155 | "execution_count": 8, 156 | "metadata": {}, 157 | "output_type": "execute_result" 158 | } 159 | ], 160 | "source": [ 161 | "next(iter(test_loader))[0].shape" 162 | ] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "id": "30d74d36-e002-48f2-884f-7f695fb75050", 167 | "metadata": { 168 | "jp-MarkdownHeadingCollapsed": true 169 | }, 170 | "source": [ 171 | "## Creating our Model" 172 | ] 173 | }, 174 | { 175 | "cell_type": "code", 176 | "execution_count": 9, 177 | "id": "9ef55b65", 178 | "metadata": {}, 179 | "outputs": [], 180 | "source": [ 181 | "# Define the neural network model\n", 182 | "class NeuralNet(nn.Module):\n", 183 | " def __init__(self, input_size, hidden_size, num_classes): # e.g. input_size=10, hidden_size=4, num_classes=2\n", 184 | " super(NeuralNet, self).__init__()\n", 185 | " self.fc1 = nn.Linear(input_size, hidden_size) # 1x10 input vector X 10x4 = 1x4\n", 186 | " self.relu = nn.ReLU()\n", 187 | " self.fc2 = nn.Linear(hidden_size, num_classes) # 1x4 X 4x2 = 1x2\n", 188 | " \n", 189 | " def forward(self, x):\n", 190 | " out = self.fc1(x)\n", 191 | " out = self.relu(out)\n", 192 | " out = self.fc2(out)\n", 193 | " return out" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": 10, 199 | "id": "aaae9c90-4106-41e0-8d44-4360ce1d36c5", 200 | "metadata": {}, 201 | "outputs": [ 202 | { 203 | "data": { 204 | "text/plain": [ 205 | "NeuralNet(\n", 206 | " (fc1): Linear(in_features=10, out_features=4, bias=True)\n", 207 | " (relu): ReLU()\n", 208 | " (fc2): Linear(in_features=4, out_features=2, bias=True)\n", 209 | ")" 210 | ] 211 | }, 212 | "execution_count": 10, 213 | "metadata": {}, 214 | "output_type": "execute_result" 215 | } 216 | ], 217 | "source": [ 218 | "model = NeuralNet(10, 4, 2)\n", 219 | "model" 220 | ] 221 | }, 222 | { 223 | "cell_type": "code", 224 | "execution_count": 11, 225 | "id": "96c8b840-f4ba-4991-bdee-df14528e877a", 226 | "metadata": {}, 227 | "outputs": [ 228 | { 229 | "data": { 230 | "text/plain": [ 231 | "torch.Size([4, 10])" 232 | ] 233 | }, 234 | "execution_count": 11, 235 | "metadata": {}, 236 | "output_type": "execute_result" 237 | } 238 | ], 239 | "source": [ 240 | "model.fc1.weight.shape" 241 | ] 242 | }, 243 | { 244 | "cell_type": "code", 245 | "execution_count": null, 246 | "id": "ab9aac7b-0a3c-48fd-b03e-6b552fb365ae", 247 | "metadata": {}, 248 | "outputs": [], 249 | "source": [] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": 12, 254 | "id": "27f2be33-9fdd-4bd3-8f56-25f374cf3137", 255 | "metadata": {}, 256 | "outputs": [ 257 | { 258 | "data": { 259 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQAAAAGbCAYAAAAr007mAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABlHElEQVR4nO2ddXwUx/vHP3dRQggJJCkugUCwAoFCkQS3UqDBKaRYkGJtgRYorl8cfnihQIpbBS9QoBCsxS0Q3CXudvL8/rjOZPfuogQul5v367UvuL293bnNzbMzn3lEQUQEgUBgkShN3QCBQGA6hAEQCCwYYQAEAgtGGACBwIIRBkAgsGCEARAILBhhAAQCC0YYAIHAghEGQCCwYCzSADx58gQKhQKBgYGmbkqeYtq0aVAoFAgPDzd1UzIkO+0sV64c+vXr9/4bZaZkywAEBgZCoVDA3t4eL1++NHi/adOmqF69eq41ztT8/fffUCgUUCgUuHz5ssH7/fr1g6OjY47OfejQIUybNu0dW5g3WbVqFRQKBerXr2/qphgQHByMadOm4cmTJ+/1OufOncO0adMQHR39Xq/zruRoBJCSkoK5c+fmdlvyNLndWQ8dOoTp06fn6jnzClu3bkW5cuXw77//4sGDByZtS0hICNatW8dfBwcHY/r06R/EAEyfPj1/GoBatWph3bp1ePXqVW63J0ckJydDq9W+t/PXqlULBw4cwJUrV97bNUxJQkJCrp3r8ePHOHfuHBYvXgw3Nzds3bo1S59Tq9VITU3NtXYw7OzsYGNjk+vnNRW5+bcCcmgAfvzxR2g0miyPArZs2YI6deqgQIECKFKkCHr27Innz5/Ljklvrta0aVM0bdqUv2bD8h07dmDSpEkoWbIkHBwcEBsbi8jISIwdOxY1atSAo6MjnJyc0K5dO1y/fj0nX5MzcuRIuLi4ZHkUcPjwYfj4+KBgwYIoVKgQ2rdvj9u3b/P3+/Xrh5UrVwIAn2IoFAoAgLe3Nzp37iw7X40aNaBQKHDjxg2+b+fOnVAoFLhz5w7fd/XqVbRr1w5OTk5wdHREixYtcOHCBdm52DTu1KlTGDZsGNzd3VGqVKl0v8vTp09RsWJFVK9eHW/fvs30u2/duhUuLi5o3749unbtatQAMA1m4cKFWLp0KSpUqAA7OzsEBwcDAO7evYvu3bvDzc0NBQoUQOXKlTFx4kSD80RHR6Nfv35wdnZG4cKF0b9/fyQmJsqOkf6uAgMD0a1bNwBAs2bN+H3/+++/+fGZ/e0YGbVx2rRp+P777wEA5cuX59d58uRJhvqTQqGQ/caY1hEcHIwvv/wSLi4uaNy4MX8/K/0qM6yzdfR/lC9fHl999RXWrVuH8ePHo0SJEukeO3v2bEyePBndu3dHQEAAwsLCsHz5cvj6+uLq1atwdnbOSRMwc+ZM2NraYuzYsUhJSYGtrS2Cg4Pxxx9/oFu3bihfvjzevn2Ln376CU2aNEFwcHCG7cwIJycnfPfdd5gyZQquXLkCb2/vdI/dvHkz+vbtizZt2mDevHlITEzE6tWr0bhxY1y9ehXlypXDkCFD8OrVKxw7dgybN2+Wfd7Hxwfbt2/nryMjI3H79m0olUoEBQXh448/BgAEBQXBzc0NVapUAQDcvn0bPj4+cHJywg8//AAbGxv89NNPaNq0KU6dOmUwHx82bBjc3NwwZcqUdJ8qDx8+RPPmzVGkSBEcO3YMrq6umd6rrVu3onPnzrC1tUWvXr2wevVqXLx4EZ988onBsRs3bkRycjIGDx4MOzs7FClSBDdu3ICPjw9sbGwwePBglCtXDg8fPsT+/fsxe/Zs2ee7d++O8uXL43//+x+uXLmCn3/+Ge7u7pg3b57Rtvn6+mLUqFFYtmwZfvzxR37v2L9Z+dsByLSNnTt3xr1797B9+3YsWbKE3zc3NzeEhYVleg/16datGzw9PTFnzhyw6P1c61eUDTZu3EgA6OLFi/Tw4UOytramUaNG8febNGlC1apV46+fPHlCVlZWNHv2bNl5bt68SdbW1rL9ZcuWpb59+xpcs0mTJtSkSRP++uTJkwSAPDw8KDExUXZscnIyaTQa2b7Hjx+TnZ0dzZgxQ7YPAG3cuDHD78uutXv3boqOjiYXFxfq2LEjf79v375UsGBB/jouLo6cnZ1p0KBBsvO8efOGChcuLNs/fPhwMnb7d+/eTQAoODiYiIj27dtHdnZ21LFjR+rRowc/7uOPPyY/Pz/++osvviBbW1t6+PAh3/fq1SsqVKgQ+fr68n3sb9i4cWNSq9Wya0+dOpUAUFhYGN25c4dKlChBn3zyCUVGRmZ4nxiXLl0iAHTs2DEiItJqtVSqVCn65ptvZMex++/k5EShoaGy93x9falQoUL09OlT2X6tVmvQzgEDBsiO8fPzo6JFi8r26f+u2P09efKk7Ljs/O2y0sYFCxYQAHr8+LHR727stweApk6davA9e/XqJTsuO/0qM3K8DOjh4QF/f3+sXbsWr1+/NnrMb7/9Bq1Wi+7duyM8PJxvxYoVg6enJ06ePJnTy6Nv374oUKCAbJ+dnR2USt1X0mg0iIiIgKOjIypXrvzO8/fChQvj22+/xb59+3D16lWjxxw7dgzR0dHo1auX7PtaWVmhfv36Wfq+Pj4+AIDTp08D0D3pP/nkE7Rq1QpBQUEAdEPfW7du8WM1Gg2OHj2KL774Ah4eHvxcxYsXx5dffokzZ84gNjZWdp1BgwbBysrKaBtu3bqFJk2aoFy5cvjrr7/g4uKSabsB3dP/o48+QrNmzQDohrQ9evTAjh07oNFoDI7v0qUL3Nzc+OuwsDCcPn0aAwYMQJkyZWTHsimSlKFDh8pe+/j4ICIiwuC7ZoWs/u2y28bcQP975ma/eic/gEmTJkGtVqerBdy/fx9EBE9PT7i5ucm2O3fuIDQ0NMfXLl++vME+rVaLJUuWwNPTE3Z2dnB1dYWbmxtu3LiBmJiYHF+L8c0338DZ2TldLeD+/fsAgObNmxt836NHj2bp+3700Ufw9PTknT0oKAg+Pj7w9fXFq1ev8OjRI5w9exZarZYbgLCwMCQmJqJy5coG56tSpQq0Wq3B3NDY/WN06NABhQoVwpEjR+Dk5JRpmwGdEdqxYweaNWuGx48f48GDB3jw4AHq16+Pt2/f4vjx4waf0W/Do0ePACDLS8n6HZAZqqioqCx9XkpW/3bZbWNuoH+fcrNf5UgDYHh4eKBPnz5Yu3Ytxo8fb/C+VquFQqHA4cOHjT5tpGvo6VlPjUZj9LP6T38AmDNnDiZPnowBAwZg5syZKFKkCJRKJb799ttcWSVgo4Bp06YZHQWwa2zevBnFihUzeN/aOmu3u3Hjxjh+/DiSkpJw+fJlTJkyBdWrV4ezszOCgoJw584dODo6onbt2jn+LsbuH6NLly745ZdfsHXrVgwZMiRL5ztx4gRev36NHTt2YMeOHQbvb926Fa1bt85yG7JCeiMYykGWu9z622VGRr/z9NC/T9npV5nxzt9q0qRJ2LJli1HhpUKFCiAilC9fHpUqVcrwPC4uLkbXTJ8+fSob1mbEnj170KxZM6xfv162Pzo6OksCVlb49ttvsXTpUkyfPt1AaKlQoQIAwN3dHS1btszwPBkNF318fLBx40Y+dG7YsCGUSiUaN27MDUDDhg35H9/NzQ0ODg4ICQkxONfdu3ehVCpRunTpLH/HBQsWwNraGsOGDUOhQoXw5ZdfZvqZrVu3wt3dna9uSPntt9/w+++/Y82aNRl2evZ3vnXrVpbbml3Su+9Z/dtltY3pXYeNUvR/60+fPs3wfFKy068y451dgStUqIA+ffrgp59+wps3b2Tvde7cGVZWVpg+fbqBVSYiREREyM5z4cIF2VrwgQMHsrWsYWVlZXCd3bt3G/VazClsFLB3715cu3ZN9l6bNm3g5OSEOXPmQKVSGXxWqgAXLFgQgOEPAUjTAebNm4ePP/4YhQsX5vuPHz+OS5cu8WMA3fdu3bo19u7dK3Nwefv2LbZt24bGjRtneSgP6H68a9euRdeuXdG3b1/s27cvw+OTkpLw22+/4fPPP0fXrl0NthEjRiAuLi7T87i5ucHX1xcbNmzAs2fPZO/l5KlujPTue1b/dlltY3rXcXJygqurK9d4GKtWrcryd8hOv8qMXBnXTJw4EZs3b0ZISAiqVavG91eoUAGzZs3ChAkT8OTJE3zxxRcoVKgQHj9+jN9//x2DBw/G2LFjAQABAQHYs2cP2rZti+7du+Phw4fYsmULt8xZ4fPPP8eMGTPQv39/NGzYEDdv3sTWrVuzPILIKt988w2WLFmC69ev8z80oPvjrl69Gv7+/vD29kbPnj3h5uaGZ8+e4eDBg2jUqBFWrFgBAKhTpw4AYNSoUWjTpg2srKzQs2dPAEDFihVRrFgxhISEYOTIkfz8vr6+GDduHADIDAAAzJo1C8eOHUPjxo0xbNgwWFtb46effkJKSgrmz5+f7e+oVCqxZcsWfPHFF+jevTsOHTqE5s2bGz123759iIuLQ8eOHY2+/+mnn3KnoB49emR43WXLlqFx48bw9vbG4MGDUb58eTx58gQHDx40MLg5oVatWrCyssK8efMQExMDOzs7NG/eHO7u7ln+22WljezvO3HiRPTs2RM2Njbo0KEDChYsiICAAMydOxcBAQGoW7cuTp8+jXv37mX5O2SnX2VKltcLSL4MqE/fvn0JgGwZkPHrr79S48aNqWDBglSwYEHy8vKi4cOHU0hIiOy4RYsWUcmSJcnOzo4aNWpEly5dSncZcPfu3QbXSU5OpjFjxlDx4sWpQIEC1KhRIzp//rzBOXKyDKgPW6KRLgNKP9emTRsqXLgw2dvbU4UKFahfv3506dIlfoxaraaRI0eSm5sbKRQKgyXBbt26EQDauXMn35eamkoODg5ka2tLSUlJBte9cuUKtWnThhwdHcnBwYGaNWtG586dkx2T0d9QugzISExMpCZNmpCjoyNduHDB6H3q0KED2dvbU0JCgtH3iYj69etHNjY2FB4ezu//ggULjB5769Yt8vPzI2dnZ7K3t6fKlSvT5MmTM2yn9LtJl96MLS+vW7eOPDw8yMrKymBJMCt/u6y0kYho5syZVLJkSVIqlbJ2JSYm0sCBA6lw4cJUqFAh6t69O4WGhqa7DKj/PRlZ7VcZoSASdQEEAkvFIsOBBQKBDmEABAILRhgAgcCCEQZAILBghAEQCCwYYQAEAgtGGACBwIIRBkAgsGCEARAILBhhAAQCC0YYAIHAghEGQCCwYIQBEAgsGGEABAILRhgAgcCCEQZAILBghAEQCCwYYQAEAgtGGACBwIIRBkAgsGCEARAILBhhAAQCC0YYAIHAghEGQCCwYIQBEAgsGGEABAILRhgAgcCCEQZAILBghAEQCCwYYQAEAgtGGACBwIIRBkAgsGCEARAILBhhAAQCC0YYAIHAghEGQCCwYIQBEAgsGGEABAILRhgAgcCCEQZAILBghAEQCCwYYQAEAgtGGACBwIIRBkAgsGCEARAILBhhAAQCC0YYAIHAghEGQCCwYIQBEAgsGGEABAILRhgAgcCCEQZAILBghAEQCCwYa1M3gHHndSwO33wNhUKBzz8uDs+PCpm6SQJBvkdBRGTqRvwc9AizDt7hrxUKYPYXNfBl/TImbJVAkP8x+RQgNDYZ8/68K9tHBPz4835UqOiJFy9emKhlAkH+x+QG4OzDcKg08kGIVqPG059H4NHDB2jfvr2JWiYQ5H9MbgCcC9ga7FNaWUNh6wAAuHHjBqKioj50swQCi8DkBsDH0xWlXAoY7K/Ytj//v7+//4dskkBgMZjcAFhbKfHLgHrwLuPM99UvXwRH1s3hrw8ePIjr16+boHUCQf4mT6wCMF7HJEGpUOAjJ3sAQLVq1RAcHAwA8Pb2xqVLl6BQKEzZRIEgX2HyEYCU4oUL8M4PACNGjOD/v3LlCjZv3myKZgkE+ZY8NQLQJzo6Gi4uLvy1q6sr7t+/D2dnZ9M1SiDIR+SpEYA+zs7OqFy5Mn8dHh6OqVOnmrBFAkH+Ik8bAADo27ev7PWKFSuEICgQ5BJ53gB07dpV9lqr1WL48OHIwzMXgcBsyPMGwNPTE6VKleKvFQoFzp49KwRBgSAXyPMGAAC6d+/O/8+e/N9//z2io6NN1CKBIH9gFgagY8eOstcODg4IDQ0VgqBA8I7k6WVAhlqtRpEiRRAXFyfbr1QqceXKFdSsWdNELRMIzBuzGAFYW1vj888/l+0rU6aMEAQFgnfELAwAYDgNePHiBRwcHIQgKBC8A2YxBQB0XoFFixaFVqvl+3x9fXH69Gm4u7sjJCREeAgKBNnEbEYAzs7O8PX1le27evUqKlWqJARBgSCHmI0BAIAOHTrw/9vZ2SEuLg7NmjUDIDwEBYKcYLYGICUlBQBw+PBhdOvWTQiCAkEOMCsD4OnpKQsOcnBwwLNnz9CkSRMULFhQCIICQTYxKwMAyEcB1ta6sgaBgYGYPHkyAOEhKBBkB7MzAMwfQKFQIDY2Fra2trh06RLq1q0LLy8vIQgKBNnAbJYBGWq1Gu7u7jxTcOXKlRESEoIOHTpg1KhRaNWqlfAQFAiyiNmNAKytrdGuXTv+mhUO2b9/P0qXLo3u3bsLQVAgyCJmZwCANB3AxsYGCQkJqFOnDgBgyZIlWLRokRAEBYIsYpYGoG3btrCysoJKpQKg8wkAgF9++QV2dnaYMmUKACEICgSZYZYGwNnZGT4+Pvz1P//8g5o1ayI5ORmrV6/Gt99+ywVBZgwEAoEhZmkAgLRpQKFChaDRaODt7Q1A5xGo0WiwfPlyAMDKlSuFh6BAkA5mbwASEhIAAHfu3EGZMmUQFhaGLVu2oGXLlkIQFAgywWwNAPMK1Gq1UCgUuHDhAnr37g0AWLx4MbRarRAEBYJMyBMGIDZZhal7b6HurGP4ZPZfmHkgGImp6kw/x0YBxYsX5/ucnJxw9+5dHD58GKVKlRKCoECQAXnCAAwMvIhfzj9FeHwqwuJSsP7MY3y95Qo0Gk2Gn2NegbGxsQCAPXv2ICAgAACwaNEiABCCoECQASY3AJeeROLikyjZPq1Wi+3je8La2ho7d+5M97ONGjWCi4sL4uPjYW9vj/v376NJkyawsrLCyZMnceXKFdja2gpBUCBIB5MbgKcRiYY7NSqowh4DAPz9/bnQp4/UK9DDwwMA8Ndff6FHjx4A0kYBQhAUCIxjcgNQs3Rhg31KGzvYl6sFAFCpVGjevDl3+tGH6QDx8fEAgB07duCbb74BAOzcuRPPnz8HACEICgRGMLkBqOheCN3qlDLYP2r2Sv7/f//9FwEBAUaf3Mwr8NmzZyhatCjCwsIQFhaGpk2bQqPRYNmyZQAgBEGBwBiUB9BotLTr4jPqt+EfGrDxX/rj6gsiIqpRowYB4Nv48eONfr5p06YEgHx9fQkA9ezZk/bv308AyMnJiWJiYoiIKCUlhby8vAgAjRw58oN9P4Egr5InDEB67N69W2YAAND//d//GRy3aNEiAkCffPIJASB7e3uKioqiypUrEwBavHgxP/bYsWMEgJRKJV27du1Dfh2BIM+Rpw1Aamoq2dvbywyAQqGgnTt3yo67d+8eASArKyuqVKkSAaDAwEBau3YtAaAyZcqQSqXix3fv3p0AUKNGjUir1X7oryUQ5BnytAEgIurXr5/BKMDW1pZOnDghO4497Xv27EkAqGXLlpSUlERubm4EgLZv386Pff78ORUsWJAA0C+//PKhv5JAkGcwuQiYGUOGDJG9dnZ2RmpqKjp16oRr167x/Ww1IDk5GQBw/PhxREZGYvjw4QB0qwD0n4goBEGB4D9MbYEyQ6vVUunSpWUjAE9PTwJAxYoVo0ePHhER0d9//00AqGjRotSoUSMCQAsWLKDQ0FA+jTh16hQ/rxAEBQIzGAEoFAoMHDhQti80NBRVq1bFmzdv0KZNG4SFhXGvwIiICDRo0AAAsGXLFri5ueGrr74CkOYYBEB4CAoEyAN+AFmBRfkBOoMQExODmjVrokyZMrh//z4+//xzpKSkcK/A1NRU2Nra4vr167h58yZGjx4NANi3bx9CQkL4uYSHoMDiMfUQJKvUr1/fQAzcvn07FSlShABQu3btaPPmzQSAqlatSn5+fgSAxo0bR0REHTp0IAA0ZMgQ2XmFICiwZMzGAKxYsYJ3/AIFChAA8vb2pqCgIP66V69eZGVlRQBo1apVBIBKlSpFGo2GawT29vYUGhoqO/e8efMIALm7u1NUVJRpvqBAYALMYgoAAD169ICVlRUAICkpCQUKFMCVK1dw69Yt7Nq1C1ZWVti+fTtKliwJQJcpyNnZGS9evMDp06fh6+uLOnXq8LyBUkTIsMBiMbUFyg5sGA+AihcvTgDIxcWFwsLCaP369bLpQYsWLWjQoEEEgAYMGEBERNu2bSMA5ObmRklJSbJzCw9BgSViVgZg586d3BsQAHl4eBAACggIICKiWbNmcQNgZWVFBw8e5PEAiYmJlJqaSmXKlCEAtHbtWoPzCw9BgaVhVgYgMTGRChUqxDs58/0HQOfPnyetVkvDhw/n+yZPnsw7/K5du4goLW7Ay8uLNBqN7PxCEBRYGmZlAIiI+vfvLxsFdOrUiQuCarWa1Go1dxSytramgQMHEgDq2LEjERHFxMSQk5MTAaADBw4YnF8IggJLwuwMwPHjxwkA2djYEADq0aMHFS5cmADQ6tWriYjo6NGjfBRQtGhRbgzCwsKIiGjMmDEEgJo1a2ZwfuEhKLAkzM4AqNVqKlmyJO/gNjY2fO7PBEGVSkXOzs78GDs7O740SET09OlTvlx4+fJlg2sIQVBgKZjNMiDDysqKewYWLVoUKpUKcXFxqFmzJqKiojBhwgRYW1vjs88+A6CrHJSSkgJAVzsQAMqUKWOQN1CKvoegVqv9EF9NIPjwmNoC5YQbN27wYT3+e/Kzpzb+EwS3b99OAKhChQqy0cCdO3eIiOjy5ct8teDZs2cG15AKgoGBgR/6KwoEHwSzNABERB9//DFf0wdAy5cv57kDvL29KTw8nA/zd+/eTUqlkgBQrVq1+BIfSyU2duxYo9cQgqAgv2O2BmD+/PkEgCpWrMh9Al69eiUTBFkHX7p0KY0ePZqPAlh8gLG8gVKEICjI75itAXj+/DlfCmRD/D179tDy5cv5tGD69OncKzAuLo5sbW1luQU1Go3RvIFShCAoyM+YrQEgImrevDkB4E/6Bg0akEqlopo1axIA6tq1K9cKoqOjqXfv3ga5BdPLGyhF6iGo7zwkEJgzZm0ANmzYwKcB7Ol+7tw5OnPmDO/oUk/Aw4cPy6IJbW1t6c8//zSaN1CKEAQF+RWzNgAxMTE83VfHjh0JAHXp0oWI0pKJuru7EwDy9/cnlUrFXzds2JAAUKFChWjo0KEEgOrWrZtuDIAQBAX5EbM2AEREPXr04B2czdUfPHhAb9++5YIg/vMIVKvV9M033xAA6tatGzVp0oR3auYsJM0bKEUIgoL8iNkbAKbkFytWjNq0aUMAaMSIEUREXBBkYmFQUBBdunSJJwZ5+vQprz7E4gNYzIAxhCAoyG/kGQOQmKKmI7de07HbbygpVZ3lz6WmpnJ//zlz5hAAcnBwoIiICJkgCIB++OEH0mq1/EkeGBhIL1++5DoBMxZ3795N93pCEBTkJ/KEK/DfIaGoP+cvDN58GQGbLqHh3BO48CgiS5+1sbFBz549AQC3b9/Gxx9/jMTERPz000+wtrbGypVpRUZ3794NhUKBPn36ANBlDS5RogSOHDmCIkWKAACIyKh7MENUGRbkJxREpk2Fm5CixqdzjiMuRS3b76SNh29iEJYvXw6FQpHhOS5cuIAGDRrAwcEBS5YswZAhQ1C8eHE8efIEtra2+PLLL7F9+3YAwL1792BtbQ0PDw8oFAq8ePECJUqUwPnz59G0aVOkpqbCysoKr169gru7u9HrzZ8/H+PGjYO7uztCQkLg7OycK/dCIPjQmHwE8HdImEHn16Qm4+bCXli5ciVq1KiBqKioDM9Rv359VKxYEYmJibCzs0OJEiXw+vVr3umXLl3K8wlOnDgR5cuXR+PGjUFE2LZtGwCgQYMG2LNnj+76Gg26dOmS7vVEDkFBfsHkBsAYCitrsKbdvn0bZcuWxdKlS5Gammr8eMmwfseOHRg1ahSAtHJg7u7u6NixIwDg999/R1hYmGwawOjQoQMGDx4MADhz5ky6UwFRVESQbzCpAkFE8ckqqj7lTyo77oBsqzRoKRfm2FahQgXas2eP0bX6+/fvc4X+7t273HHnyJEjREQUHBzMz+Pv708RERHceejGjRv8PKmpqbLlQ/1KxFKEICgwd0w+AihoZ43lX9aGk70131ekoC1+Gd8HNWrU4PuUSiUePnyIrl27wsfHB//884/sPBUrVsSnn34KrVaLw4cP83Ji7ClepUoVlClTBgCwefNm3Lt3D+3btwcAbN26lZ/HxsYGkydP5q/79OmDkydPGm27EAQFZo+pLRDD2DLgqVOnZCMANzc37saL/9KBseKgRGnFQ+rUqUOPHj3iIcDsCT927Fj+WW9vb9q9ezcBacVDGNK8gfjPW/Dq1atG2y08BAXmTJ4xAMbQarVUrVo1mRGoV68e+fv7c+ceW1tbGjNmDEVGRlJoaChPEhIcHMyDgfr160dEaRWE2WeXLVvGIwlPnjwpuzbLG8jel1YiliI8BAXmTJ42AERptQDw3/weAPn5+dGlS5eoRYsW/D0XFxdasmQJffbZZwSAJk6cSBcuXCBAlzfw5cuXpFKpyMXFRfYZ5kLMiocwpHkDWc4BT09Pg7JiRMJDUGC+5HkDoFKpqFSpUrzTsk45dOhQ0mg0dOjQIapatSp//6OPPiIAVK5cOdJoNNSoUSMCQBMmTCAioi+//JJPJwBwg8GKh0hhx3bu3Jl7C9arV4/i4+MN2ikEQYE5kucNABHRkiVLeAe3t7fnQ/gZM2YQkc5IrF27lnd+tq1Zs4Z+++03/rSPj4/nuQLLlSvHjytWrBgPGZYizRt44sQJWSXi1NRU2bEiZFhgjpiFAYiNjZWJcnXq1OH/X7duney4yZMn81EC/ov6K1u2LAG6vIFRUVH8/S5dushGDcYCgaR5A8+fP89FyL59+xosRwpBUGBumIUBICL64YcfZE93lt1HqVTS3r17ZcdKdQPptKFcuXKkVqt5p545c6ZszV9aPIShnzdw//79/Hzjx4+XHSsEQYG5YTYG4Pnz51zhB0BVqlThST/s7e3p7Nmz/Fhp8RCWPZht/fv350/qFi1a8JBh1qlZ8RCGsbyB0krE//d//yc7XioIprd0KBDkFczGABAR9enTh4C0Sj/z5s2j9u3b8zl+cHAwP/b777/nAt6hQ4fI1dWVd9rSpUvzJ354eLgsZLhBgwYG1zWWN5BVI2K5BaUIQVBgLpiVAbh69apsHd/R0ZHu379P9evX5x37+fPnRJRWPMTW1pYiIyMNRhBsmz17tiyHIAB68OCB7LpJSUkGeQOllYhtbW3pxIkT/HghCArMBbMyAETE1/6Zct+rVy8KCwvjw/Tq1atTZGQkEaUVD1m7di0RpVUW9vLykhmDHj16cEEQAE2dOtXgutOmTSNAnjdQrVZzZyN9b0EhCArMAbMzAIcOHSJAl/WHjQROnDhBT548oeLFixMA8vHxocTERF48xNfXl4iIbt68yefnzG2YbTY2NtwouLu7Gyj8oaGhPAGpNG9gUlISzy0o9RYUgqDAHDA7A6DVarnjT4MGDQgAVa1alVJTU+n69et8udDPz4+ePHnCjcTjx4+JiKht27YEgIYPH869AuvWrWswNdi/f7/BtQcPHmx0uTAqKornFpR6CwpBUJDXMTsDQET0888/EwAqUaIEF/cWLlxIREQnT57kYb5Dhw7lxUNmz55NRGmd0sHBgQ/fv//+ezp06BBVqVJFNiLQDz2+e/cu1yD08wZKcwtKvQWFICjIy5ilAUhKSuL5/QcNGsQFwZcvXxIR0e7du/mT/4svvuDzfq1WS1qtlmsDLKV41apViUjnUciMAtsaNWpEFy5c4Nfu0KEDAaAhQ4YYtOvOnTsG3oJCEBTkZczSABARzZw5kwBdtV+2CtCrVy/+PlvfZ09zAHT58mUiIvrll1+4ByBb/3/48CER6YwAm+tLNxZ6zCIK7e3tjQYGGfMWFIKgIK9itgYgPDycd7Q1a9bIBEHGjz/+KOvE3333HRHpBLoSJUrwkQGgqyDMYKMKpjNIQ49Hjx5NtWrVIgA0ffp0o23T9xYUgqAgr2K2BoCI6OuvvyYA1L59exo2bJhMECTSCYYDBgzgnblo0aLckWfu3Lmy5cQWLVrw8168eJF/xtnZmU6cOEEtW7bk+9iQ3t3dnZKSkoy2jdUtZN6CQhAU5EXM2gDcu3ePP53Pnz9vIAgS6Yb07dq1452R+QRERkbyjsy8AqOjo4mIZMVDANDAgQNJq9XS4cOHDRKUDB06NN16grNnz5Z5CwpBUJDXMGsDQETUqVMnAkABAQHcR18qCBIRxcfH84g/BwcH7i04atQovg+QhwMzV1+2nT9/nojSQo8dHR35ew0bNpQJhQx9b8GdO3cKQVCQpzB7AxAUFESALj7g1atXRgVBIuKlwQFdIFFkZKQsbyCgyxbMePTokcwAeHt7k1qdVrLsxYsXPCZBXyiUou8tyIqTCkFQkBcwewOg1WqpXr16BICmTJlCly9fNioIarVaWRIQHx8fSkpKom7dusk0Amknb9y4MVf8AcNIQZY3sFixYkZzFDKk3oIfffQRVahQQQiCgjyB2RsAorT4f1dXV0pMTDQqCBKl+fMzhb5z58509uxZ2VM8KCiIH79mzRoCwEOLXVxcZEt/0ryBO3bskAmFRYoUoaVLl1JKSgoRyb0F2fmEICgwNfnCAKhUKp71Z82aNRQZGWlUEJQWD2G+AUOHDuV5AwFdBWGGtHhIpUqVuCAoheUN7N27t1GhUFrMROotyByGhCAoMCX5wgAQpeUNrFSpEmk0mnQFwU8//ZQAXapwNmzv2bMn77BeXl6y8/r5+fEOri8IEsnzBj579oyIjOcoZB6FUm9BNnoQgqDAVOQbAxAbG8vTe+3du5c0Go1RQVBaPETqLVi0aFH+f+YVSET066+/EqArHtK3b1+jgqA0b6B+myZPnmxQzOTXX3+V7ROCoMBU5BsDQJSWN5CF/xoTBPWLhzBvQXYckJb6i0gn4LHiIL/++is3MlJBUD9voD4vXryg/v37y4RCPz8/2QqEEAQFpiBPGACNRku7Lj6jvhv+of4b/6Xfr7xI17kmI6RZfy5evEhEZFQQ/PzzzwnQFQ/R9xYEQDVq1JCdl7kGDxgwgI8apIKgsbyBxrh27ZpRj0JmgJgg+HdIKH295RL1XneB1p1+yEulCQS5TZ4wAGN3XTOoDjx1760cnYvlDezZsycRkVFBkK0asOIhKpWK5xZkG/MKJEqrUejk5ESxsbE8h6BUEGR5A8uWLcvdjY2RnkchAKpcuTJtPPPQ4F50W3OO1JrsG0SBIDMUREQwIQ9C49By8WnZPiJC3D+7ob2+H5cu/ovixYtn+Xw3btxAw4YNYWVlhZs3b6JMmTLYtGkThg0bBkdHR1y9ehXOzs7w8PBAXFwcjh49ioYNGyIhIQFt2rTBtWvXAADffvstZs2aBQDQarWoVq0anj9/jk2bNqF48eJo1aoVAODEiROoV68ekpOT4eXlhfDwcAQGBqJr164ZtlOtVmPz5s2YMWMGwsLC+H6FrT1KDN8Ca1t7vk8V+QKKYwtw8I9fUbNmzSzfC4EgM0xuAH69/AJjdl+X7VMlROHVCn8Ttcj0ODUJgMunX/DXL1YPgCY2FEqlEj/88AMmTpwIR0dH0zVQkG9QmroBZYs6GOxTWtsbOdJyKPRxC9lrl6b9oLSyhlarxdy5c1GpUiWsX78eGo3GRC0U5BdMbgDqliuCT8q5yPYpbe3Rc+VJeHp6GhzfvHlznD9/HvHx8elucXFx8PLyAgDMnj2b7x80aBAA8KF6iRIlAADbtm3jx0RHR8PKyopfr2PHjoiJiUF8fDxq1aoFAFi8eDHi4+OxcOFCAICLiwseP36M+Ph49O/fHwDQvn37dNsXHByMHj16GL0ftVt3h1UB+dO9csM2OHnqNGxtbQEAr1+/RkBAAGrVqoUjR45k634LBDJMqkD8R0xSKk354ybVmXmMPpl1jGbuv00JKSq6ffu2bKmMOc4oFAoaMGCAzMFHH5Y3sFSpUlz91xcEpcVDpLA8gmxjIb+LFy8mIK14iEqlMhAEpXkDQ0JCZOeNjo6mcePGyYKIpN8PAIWFR9CKE/fJZ94J+njaEfpm+xV6EaWrWnz48GF+D6RZi1q3bk3Xr1/Ptb+HwHLIEwYgI1jADTMAvr6+/LWDgwNNnTqV4uLiDD4nzRu4ZcsWvl/qIciSdLDiIYxt27YZKPQzZ86kV69e8Q7LiodIi4owD0GWN3Do0KFERJSamkrLly+XVSeqV6+eLKQYABUvXjzT+7Fp0yZ+vK+vL3dpzopRFAj0yfMGIDY2luf7Zx13w4YN1LBhQ76vWLFitG7dOpl3HlFa3sDatWtzvwJ9D0H94iFEusAd1tGlnXTdunXUunVrAuTpwFiNQuYhKM0b+Msvv/A4AvznarxhwwYeECTd9IuNpgfLMYj//A5YopHMjKJAoE+eNwBERNu3b5cNl93c3OjevXu0e/du8vDw4D/+6tWr059//sk/J80bKA0NlnoIslz/Pj4+smuy8F0A3HFIqVTSt99+S4Au/z8zKm/fvpV5CGq1WlmKcdbm1atX09u3b/l7+qXKpC7IGaHVanleAWtrazpy5AidO3eO10lgo4mff/7ZwCgKBFLMwgBotVpq1qwZAeAdrUKFCvTmzRtKTk6mJUuW8CIf+nNiad5AKcxD0NPT06B4CBHRokWLZJ6BrKyYnZ0dn3//888//HjmIVi4cGFZmTFAF2EYExNDiYmJPPJQf/jv7u6erXui0Wh4WnNHR0e6dOkSabXaTI2iQCDFLAwAEdHt27f5E5NF2Xl7e1NsbCwR6QS+0aNHG8yJg4KCeAeXVg+WCoIVK1YkIK14CJEu36C0gx46dIh7C7IQ4REjRvDjw8PDuebANubqu27dOlKpVDx9WaFChbiYx7Zhw4Zl+54kJyfzWonu7u5cl8jMKAoEDLMxAEREY8eOJUBXBZh13latWvGkG0REDx8+NJgTMz/9QYMGyc7HBEGmyrPiIQz2OdaB4uPjuX4A6GL6ExISDAQ+QBfiy0YRVapU4fEEdnZ2VLt2bYP5vzQRSXaIiYnhacrZqIiRnlEUQqGAYVYGIDY2lufzHzJkCE/m2bt3b4OkGvpzYjZflv74pYIgeyKz4iFEaQaHbTdu3KCwsDCZqMfSijMDwp7I3t7eFBkZyWsVMg1h9OjRvDOy/Y6Oju80V3/9+jWVL1+eAF2YMxsVMYwZRSEUCojMzAAQpQmC9vb2FBgYyKcFY8aMMTiWzYlZ52BinHROLBUEgbTiIUTE1Xw25O/Xrx8REe3bt0+2fu/q6kqrVq0ilUplIAhKo/+WLFnCDZh0Y4FL70JISEi6oyKGEAoF+pidAZAKgp06dZKti0vTf0lJTk6mr776StbppHNiJggyfYFF86lUKtk82sbGhjp37mzQgdu2bSvrREwQ1Bf6/P39ZdoA26TpyN+Ff//9N8NREbt/QigUMMzOABDJBcGDBw/K1sWlTj9SVCoVlS5dmg/FpXPi27dvyzICSTsDy/lnrDNLn+bSAiEqlYoLi2xurj/sl04LpKHH78rhw4czHBUxhFAoIDJTA0CUNj+vUKECJSYm8vV5ti5uDJY3sHz58rJ04A4ODtSxY0f+mrkGp6amcicfqY5w5swZIjIsHjJz5kwiIrp165bMYDC3ZEDnmiz9TLNmzXL93mRlVMQQQqFlY7YGQCoIzpgxgzQaDU/uydbFjX1GmjdQf07MhEArKyvavn27TOxj+gEAWr58ORGlFQ+RPtnnzZvHOzk7XqpB6C//SYuS5ibSUdHWrVszPV4IhZaJ2RoAIrkg+Pjx43TXxaXo5w00NifW7/Senp4EgFf48fDw4HN+VjxEP4CoSpUqdPfuXW5w0tuy6v2XXbRaLR8V2djY0NGjR7P0OSEUWhZmbQD0BUGijNfFiYznDSTSqeh16tSRdc5y5crR2bNn+Xp+06ZNeUrvX3/9lYjSiofUqFFD5gjE3meGQaFQGDz9q1Sp8l7vjzFvwawghELLwawNAJGhIEgkXxeXegsypHkDjYXoSjeFQsFde62trXl0YsOGDYlIVzyEzZ/ZMYDOSWjfvn0GnV66SYuQvC+yMirK6LPGhMIbN268xxYLPiRmbwCI5IJgUlISEelcedNbF7969SpX4KU/7qZNm9Lly5f5k0+61s/m+cuWLeN+AefOnZPVHLSysqI///yTOxcxoyKtPCTdcur9l10yGxVlRkREhEwoVCqVNHDgQCEU5gPyhQHQFwQZ//77L19zZ+viWq2W/vjjD75eDug8+Pbv38+X8aQFQ1q1aiWbExcoUIB36C5dutDUqVP5e0WLFiWNRkNhYWG8PQqFQnY+qRiYUfbg3CYzb8Gs8ODBA4PVEyEUmjf5wgAQGQqCDOm6eO/evWUJRdjx4eHhsnOFhobKhu7Hjx/nhUX1pwdSwwDowo5jY2NlegArLKLvB3Dq1KkPeo8yGhVlByEU5h/yjQEwJggyWCovaaefMGECeXl5EQBasGCBwflYDgBAV1QkMTGRTxcKFSokO9+QIUNkxUPYlKRUqVIGXn/SrWPHjh/o7qSRFW/BrCCEwvxBvjEARIaCYHoC37Jly4jIeN5Axo4dO2Q6wMKFC7lXYNeuXQ1y+X322WdcbWejhwMHDshiAYyNIPTzBn4IpKMi/XqG2UUIheZNvjIARGmCoKurq8y9t0mTJrwDs3Vxad5AfWeZxMRE2ZPe0dGRVq1aJTMK+u7B0iF+x44dZUuO6W0sb+CHRuotuGjRonc+nxAKzZN8ZQC0Wi1t375dNn+XCnzGvAWN5Q1ksCxAzEiwpzygW15k5cBcXV1leQIA0P/+9z9ujFjmYLaxVQM2HWE1Bj802fUWzApCKDQv8o0BuHjxooHAZ2NjQ/fu3ZMdp78ufvHiRaN5A4mIjh8/bnTOD4DmzJlDKSkpXO1fvXq1LPZfOlJg15OOFKSrENIEox+SnHoLZgUhFJoHZm8Anjx5Qr179zYQ+Hx8fIwKgkSG6+J9+/YlwDBvoFqt5tl7mZIv7dRERHPnzuWjACAtd4DUOUg6ImHnkU4NXF1duf/Chyan3oJZQQiFeR+zNQDGBD5/f3969uwZERn3EJQiXRevVq2a0byBRPK6BNI5vrW1NUVHR1NkZCQfQQCggIAAmSHQFwuHDBnCcwtK31u3bt0HuW/GeBdvwayef/HixTKhsE2bNkIozAOYnQFITU2lFStWyHLwMQ8+fYx5CEqRrouzyD1p3kCtVivL8Dtq1CiZIdi1axdptVpZ9N+rV6/4+y4uLkb9DiZMmEB169Y10AVyuiSXG7yrt2BWEEJh3iNPGYBX0Yn0Jsb4UJh58OkX2ZB68OmTnoegFKm3IKBz32U/fqmXHwBas2aNTOzz9/fnDkhsu379OtcCGjZsKBvqS///0UcfyfIJAqB9+/bxdoXHJdPT8IR3vKPZIze8BbNCdoXCuGQVPQ6Lp1S16QxkfiVPGIAHoXH0xcozVHbcASo77gB1W3OOnkWk/fj1BT43Nzeegy8zpB6Cjx49MnqMtOYeAJoyZQqP8gNAfn5+BOiKh1y+fJnvd3Jy4lWLqlWrRgCoXbt2/H22DMnSmPfp08dgTiy9rqenJ0UnptKQTZeo/HjdvWi+8CRdeBhutN3vg9zyFswKmQmFao2WZuy/TZUnHaKy4w5Q3VnHaNfFZ++tPZaIgogIJkSt0aLpwr/xIiqJ7yMieDhpMaVhIfz000/YuXMnAMDe3h4jRozA6NGj4eTklKXzExHat2+P06dP4/PPP8eOHTuMHrd9+3ZePdjOzg4pKSkAgPHjx6N///7w8vICEeH27dtYvHgx1q9fzz9bvnx5rFmzBm3atIFCoQARwdraGmq1GgDg6uqK8PBwbNq0CZ07d0ZKSgp+/vlnzJ07F1FRUbJ2NPl6Nh4X+hgKhYLvc7BRYnffKihX3C1L3/lduXz5Mj777DMkJCSgR48eWLduHZTK91NImojwxx9/YMqUKXj8+DEAoGrVqpg0aRJCtMWx8Ua87F4oFMDOwQ1Qr3yR99IeS8PkBuDk3VD0D7wo26dOisfLZT1N1CLTU7TXPDiWqcZfJ96/gLDfZpmwRabDvc8CFChZRbbPr3ZJLOlRyzQNyme8H7OeDaKTUg13kubDNyQPoYl5JXsdH/y3aRqSB0h9GWKwLzrRyG9GkCNMbgAaVXCFjZVCts+qgBNK9ZqJQnrD/I4dO+LatWuIj4/P9vb69WsUL14cADBp0iSD9+/evYsSJUoAAB9y2tvbIygoCPHx8Xj58iXs7e0BAFOnTpW1q127dnj69Cns7Oz4viJF5EPUefPmya738OFD9O/fXza8BQBrR1c4VW8p2+f6+VhMmL0IxYoV4/vq1KmDo0eP5uheZGebOXMmv+aGDRveyzVCQ0MxefJkFCxYUPa9qzT1g9MnnQx+M00ruxvsE+QQk6kPEtadfsgFwLLjDlC58Qdo64Wn9OzZM1lpcPynpI8aNYrCwsKyfZ30BMGIiAhesbdKlSqyOgHSdXGWNJOtGkjTgjO3YZZ6nLWV/Z/l/ktISKCZM2caxBGwbfvxS1Rp4iHZ/fjxtxuyz0pXLTp37mzg7ZibvE9vQbVaTRs3bjRaLGXbtm30OCye6s0+JrsX3daco6RU4U2YW+QJA0BEdOd1DC06cpcWHw2h+2/Tlp9u3brFveekMfaFCxem+fPnZ8uDzljIsLRib8mSJenp06cGQTxsXXz//v18n4eHB/cChETN37BhA3dOKlu2LPcbiIyMNPixs9UBtpUtW5aIdMuhq04+oLmH7xhdAXj9+jUNGTKEOxK9i1HMClnJuJxdjh07JouRkPp1LF68mB8Xk5RKm84/oTkHg+nwzVekEkuBuUqeMQAZERQUxEtyt2vXTvbDKVeuHG3fvj1dXwB9pB6C+/bt4xV7CxcuTDdv3uTHsbyB7Glbp04dOnfuHL/u9OnTDSoIFy5cmBISErhnoDSGQDoyKFeuHI9MlG4rVqzI1n25deuWLECpcOHCtGDBgvfiVpxb3oK3bt2SLZUWLlyYBg4cyA3ou4YnC7KHWRgAIqLff/+dP/EmTZpEgYGB3E8fANWrVy/LOfaYhyBz2LGzszPIzsPyBlpZWXEXVqkra58+fYhIXkHYzs6OAgMDjQ7tpR1UWiiEbUqlkmJiYnJ0b/Sfptk1ilnlXbwFX79+TYMHD5aNWr755hs6evQoN7J9+vQxqTekJWI2BoCIZM45q1evzvGcODY2lj+dFQoF/fbbb0aPY0+8L7/8kg/rmZuvg4MDxcXFyVyFAVCZMmUMnv5su3nzpiwZh3T7/PPP3+neGJtPZ8coZpXsegsa0zzY30jqdNS6dev36nQkMI5ZGQCiNPdchULBc+9nd04sNSQ2NjbpeggeOnSId2amQwBpEX1btmwhb29v/vRP78lfvXp1AnSxBMxYSZOVALrsQbnBhxAKs9JxMzNIH8rtWJAxZmcAtFotDR482OjQ3dicWF8olE4lWGIOYyHD7FpVq1bl55OKkIDO11/6WvpklxoEJjyyeS5zG5a2Uz8l2bvyvoVCYxmXGcamJDt27OBTEv2pxNu3b3OlTYLsY3YGgEhXfVcq3umHlaY3Jz59+jQXEwMCAujWrVsZhgwTEc2YMYOf548//qD58+cbPOGlsf/683oAsnDhSpUq8epCbBsxYsR7u1dZMYo5RT+3oDGBb8GCBZScnMw/875DjwXZwywNAJHx5TsparXaQChkT+COHTvyQKKMQoa1Wq0sCGnr1q2ydfHMtoYNG/Lhv/50QJoL4MKFC+/9fr0voVCaW5DpI0zg00+3rp98xFgIt+DDYrYGgEjnwMOG6FWqVKGIiAiDYxISEmjs2LGyZB6dOnXic+KMQoaZ4xB7ynl7e9OrV694CnDpHJudv27dulwArFWrFm+ffk0AtlWsWDHX1fr0MGYU30UoZHoDS34CgD755BOjeoNWq6VvvvmGj5hy06FIkHPM2gAQET179own5GjQoAElJMhj6KVefs7OzkbnxMY8BKWGYfz48XwYz6YQ0q1IkSL8/S+++MIgCxDr/Gy/9BymyAf4rkKhMYGPOTWl17mlU6dt27bl9lcS5BCzNwBEcm/BDh068OG9sWmCsTnxvHnzqEmTJnx0QJQ2NfDw8KC1a9fKkniypT7pPrYxYyMVAZmbsLHNlHPgnAiF6Ql8arU6XW9B6TRB6uUnMD35wgAQyb0FAwICKDU1NV0vPyLDH3KJEiW4RrB69Wo+7GdLVdKNxSew/H/S4b2xof6ECROMdv4GDRqY6G7JyYpQmFOBLzeLkAhyn3xjAIjkS3zS9fn0avAZmxOzYaz0Nfuxd+jQge9zdXWliIgIow4/bGPBQ+mtEqxcufID36GMMfZ0X716NQ0aNMjAg09f4GPExMRQ7dq1CdBVXMqNMmSC90e+MgBEciefjLz8pCQkJNCsWbMMhvRWVlayH/vp06f5e2PGjCEiorZt2/KOIZ37f/fdd6TRaAxWAaRte1/BO+8CM4rGIvS6dOmSJZ3g9evXXJcBQM2bNxdefnkUk+cDeN9QFhMepXcc6Yxkpp9jqcDyA0QErVZrdL8gn2E625P75GQKYCwe3dgUYP78+bKKwWwKkF5cP/LJFGDNmjUGQTwZCYVSL7+SJUuKKUAeJ98YgMxEwMy8BUuWLMl/5JmJgMxg5CcRMDOBLytCoRABzY98YQCky4BSL7/0lgH1f+zz58/PdBlQumae0TKgl5cXH3mwfXl5GdBYiG56Ah9R+h6FarU6XS+/3K5ELMg9zN4ASB2BGjZsaNQRiHnjOTs7G3VXlToCPX78mIgMHYHY6ELq18+2okWLyhyBpLn+paMC9q/0HKZwBIqPj6cZM2bIjFpWBT4i46sn2XEEyq1KxIJ3x6wNgH4uv/RcgceMGZOrrsAsGpFt7+IK7Onp+UFdgfU1j/r169OZM2dydD62eiJ1Ba5bt266rsDvK7egIOeYrQHISjCQ/o+dPZml3oKZBQOxqQF7ckl92jPbGjdubBD6W6NGDQLyRjCQNEQ3p/zyyy8GI530hML3kVtQ8G6YpQHQDwfOzMuP/dg/dDgwMzjSIX/lypUNkoG873DgzAS+nGIsHDgnQqHAdJidAcgsIUhmP/YPlRBEGvDDEoKw9/VHBc7Ozu8lIUh2Bb7skN2EINLQY6m3oEgIYlrMzgCwlGBKpZJ7+WX3xy71FrS2tubCnz4sJZijo6MsiUfhwoUJkKcEk86D9bcPmRLsXQW+rJDVlGDGQo9Pnz5NRCIlWF7BrAyAflLQnP7YY2NjuQOPNLegPsaSgjIjU7BgQYqPj6fOnTvL5r9sidCYg9CNGzfozz//fG9JQTds2JBrAl965CQp6KxZs2T3w8/Pj0JCQujevXvk5uaWoSERvF/MxgDopwV/FzU7O2nBlUolf/obSwteqVIlvs/W1jbTtODz58+n9evXG9ULcjMteG4IfPq8Sy6/9EKPpWnBhbfgh8csDIB+YZCPP/44xz/2jAqDSL0FjRUGOXv2LL9uVgqDMAMDvL/CIPqax8KFC3NF4NMnt8S727dvGwiFAwcOFN6CJiLPGIDgV2mlwe69yVppsOyq2dkpDSZ15mFPu+yWBmMiIysNlp7zDNukpcFWnrxP/zt0h86nUxrsfQp8+ryPXH4ZlQaTegvGJKXSpnOPac7BYDp0Q5QGy23yhAHIqDiofqDOu/zYjXn8ERnmFvz666/59d6lOKjUiEiLg+rPiaXbjr8uplsc9EMIfPq8z1x+6eVjAHQ+F6I46PtHQWTaGM/Q2GQ0mncCKk1aM4gImmfXEHtwPuLi4vj+Tp06Yfr06ahYsWK2rxMXF4fatWvjzZs3mDRpEsaPHy97/8WLF2jRogVevnzJ99nZ2eGvv/5C7dq1ERMTAw8PD6SkpGDatGmYNm0aP65du3ZYs2YNvLy8kJSUBAAoWrQoIiIi+DHz5s3D8OHD+eu3b99i9uzZ2LhxoyzM1sbJFcWHbpSVDddq1Ojt/AC/rFyE169fAwA++eQTzJkzBw0aNMj2vcgOS5YsweTJkwEA69evR48ePXL9GomJiVixYgUWLVqEhIQEALrw6spN/JBYz7CE+vSO1dC3Yblcb4clYnID8PvVF/hu53XZPnVCDF6u6G2iFpmewm2/gXPNVvx12N65SLx7xoQtMh3OzQaicD0/2b5mld2wsX89E7Uof2HyhCDOBWwN9imsrE3QkryDjUtx2euCVZuZqCWmx6ZEZYN9zg6GvxlBzjC5AfDxdEUplwKyfUo7BzSduRcnT55Ez549+X47OzuMGTMGr169Qnx8fJa2uLg4+Pr6AgDat2+f7nHr1q3j17G1tYVSqbs148ePR0hICH/v9u3bGDhwoKy95cuXx5EjRwCAD1etrKz4+25ubgCATZs2IT4+HhEREfjf//4HZ2dng/vRZOhsOJSuJtvnWrUBbt1/nOXv/K7bqVOn4ODgAADo0aMHYmNj39u14uLisGnTJpQrV45/32rVqmH79u2YuuUECpSsIrsXCgXQq14Zg/smyCGmFCAYD0LjyG/lGS70dF9zjp5FpIX1Xrx4URaU4+bmRitXrsyS+2x6wp8U/Yq9U6ZMkTkd+fn5EQDy8fGhS5cucaefQoUKcSGQufdKl+WYtx9T+/39/WnXrl3k4eEhEzXZ/ytWrEjRiak0dPMlKj9edy9aLPqbLhhZCXhffMiKvWfPnqUGDRrw71+8eHFav349qdU6kU+t0dKM/bep8iSdKFp31jHadfHZe2uPJZInDADjVXQivYkxXrNOq9XS3r17qXLlyvwH4+XlRfv27UvXByCjUF+G1Kcd/znzMAcX5nbMtjVr1lD9+vX5a39/f25g2Hb9+nW+/t+wYUNZB5f+/6OPPqJixYrJPrtv3z7eroj4FJkR/BB8KPfcBw8eUNeuXfn3dnBwoGnTplFcXJzR4+OSVfQ4LJ5SxRJgrpOnDEBWSE1NpZUrV8rWjZs2bWo0tDSjUF8i+dOOuaQOGjSIv6/VaqlLly78OqNGjeLr+gBo165dpNVqeUISNzc3evXqFX/fxcVFNnIBdF6HP/74I9WtW1e2v1y5cib1gvsQFXsjIiLou+++49GTSqWSAgIC6NWrV7l+LUHWMDsDwIiOjqbx48fLUm/16dOHnj3TDRGlHn/GQn2lTztp6u47d+7IjhszZgx/T5rUw9ramqKjoykyMlIW7svyBLLgIP1EIEOHDuWecFI/gXXr1n2Q+2aM5ORkat68uYHfQ26ef9GiRdyhCwC1bdvWIE+j4MNjtgaA8fTpU+62y+b648ePJx8fHwKMh/rqP+369u1LAKh9+/ay49RqNXdSYT9e5n3XokULIiLuDchGEqzjS3MESDs6O4/0fVdX11wp150T9L38cjNJh1arNdA8atSoQUeOHMm1awjeDbM3AAx9oZA9pfW95PR92i9evMif4CdOnJAde/z4cd4xpOcFQLNnz6aUlBSuMaxatUrm+69vLKT7FQqFLKGoKfICEr1fL7/MBD5B3iDfGAAi3Q96x44dMrFNKhQae9rNnDmTAFDt2rUNxETm3stiEKQKf+3atWnt2rVc7f/0009lnXzu3Llcg5AGL7H5vlQTCA0NNcn9mjdvHm9HbiXqzK7AJzAt+coAEKUJf66urgZCIctHx552SUlJvHPrd4DExERZ3T9HR0dauXKl7MkuXT3Q3zp27EjPnz83GvuvrwmYAmkuv9xI1S0EPvMkXxkAfeHPmFAIgJYtW0ZERD///DMBuiKW+j4FO3bskHX2hQsX8hDeLl26yJJ6Mv2AGQXWhgMHDlCrVq3S7fwKhYJCQkI++H3KzWIdycnJtHjxYlmuhDZt2giBz0zINwbAWKgvY9GiRbKOx4RC5lOwYMECg/NJy4BVrVqVEhMTuYCnXxF4yJAhNGjQIAJAAwYM4KMQaXXc9EYJH5p//vknV8p1pSfw/fnnn7ncYsH7JN8YgPQ8/qRPu969extdl9dPXx0aGipT7o8fP07Tpk3LcCjPEpacOHGCYmNjZbH+zHDoLwmmV7PwfZFbXn5C4Ms/5AsDkJ7Hn7HMtcyjUPpk1vcoXLZsGX+vZcuWsh97gQIFeAKRLl26yLwFixYtShqNhsLCwqh48eK8069YscLAYFhbW/PaBB+C3PDyEwJf/iNfGABjHn/Sp12rVq1kTzuW70+hUMiy/TKPQjaslc7z2dN72bJlfK3/3LlzpNVquapvZWVFf/75J3cXZtoDMxj6W1BQ0Ae5P/p+D2/evMnW54XAl38xewNgzOMvs6cdcxzq2bNnukKhtOOzp561tTU3Ng0bNiQiXecw5u9fpEgR2rdvX4arAD/88MN7vz/v4uUnBL78j1kbAGPCX2ZPO+nS3MWLF/n+kJAQqlOnjqyDli9fns6ePctFxKZNm/IRA0slzqIGq1evLstZyGoWMCcghUJhUDS0atWq7/X+5NTLTwh8loNZGwB94S8rmWt/+OEHAkC+vr5EZPzHLt3c3NzI09OTAPCRQIUKFbjg1bhxYwLAn7Jsq1KlCt25c8fAO1B/Y7kCc5ucevkJgc+yMFsDoC/8ZeVpFxsby6v67N27l86dOyf7sbORgUKhoJ07d8pCj4E0f//ly5cTEdGjR48MOvS8efNk0YGA3PNPfxSwdOnS93J/suvl9+DBA+rWrZsQ+CwMszUAUuEvMTGRP+2sra3TfdotWbKEd0h9Nbtjx478defOnYlIF3rM3IGlRoIVIJk1a5bsvZkzZxKRLpW5NH6AORwB4MaBbc2bN8/1e5MdLz99gU+hUNDAgQOFwGchmKUB0Bf+pE+7LVu2GP2MSqXinY89hdmP/datWzK3Yelcl3n/6QcE+fv786U+APT111/zZUSVSkUVKlTg77H/6/sBsLZER0fn2r2R+j2MGTMm3ePSE/iuX7+ea20R5H3MzgDoC39ZedolJyfTV199Jet40h/7sGHD+H53d3e+Pq9SqWQdxMbGRpYghG1t27aVzZGXL19OgM4tWLqU6O/vz0cc0s/v2rUrV+5NVrz8hMAnkGJ2BkAq/AUGBmb4tDP2Y3d3d5f92KU5/gDQd999x9/7+++/CUiL8e/Xrx8REe3du1fWsaU5Ct+8ecN1hlWrVlHLli35cUuWLDEodAKAevXq9c73JSQkJFMvPyHwCfQxKwMgFf4GDx6c4dNOX+Bj83fp3Faj0XCnHdahpWWvmM7ADMSNGzcoLCxMJg5KO7SXlxdfhfD29qbIyEjZKoBCoaDvvvvOwAA4Ojq+Uyd8/fo1FxqN+T0IgU+QHmZlAFiHLFOmDM+4q+/lZ+zHzjqsNN8fEfEqvcwJqEqVKrKcANKO3rp1a4qPj5clBXVxcaGEhARasWKFTEMAQIGBgdx/wMvLiwcL2draUu3atQ2MQE69AjPyexACnyAzzMYASIU/Fmgjfdql92MPCgriT3Bpvr+IiAjeaStWrEiALssPQ7/y78GDB3kuPzYlGDFiBD8+PDxc5ggkneuvW7eOVCoVr0Ts6OhosBw4fPjwbN+T9Lz8hMAnyCpmYQCkwh8bUrOnXWY/dlboUz/fHxP+KlWqxD8njSKUhhBXr16d+vXrR4AuGIiNGP755x9+PBP+nJycZEuMAOj777+nmJgYWSVi/VWFjz76KFv3xJjfgxD4BNnFLAwAE/7YPN3d3Z3u37+f6Y89PDzcaL4/qfA3ZMgQAnRFP6T4+vry87LcAEqlks/hPT09+XRBX/jTarXk5eUl6+Bubm60atUqevv2La9ErD8KyKpXoNTLj/k96GseQuATZIU8bwCkwh972m3YsEH2Yy9WrBj9/PPPBj92Y/n+pMJfr169eL6+tWvX8s9FRUVxAyF9Uq9bt45at25NgDyRJxsdeHt7k1qt5qsHdnZ2tGnTJtkow8vLi9avX2/gEASAJkyYkKV7IvV7WLRokRD4BDkmzxsAaV5+Kysr2ZM5ox+7NN+f1DmICX+Ojo507NgxPqePjIzkx2zbtk2m3AM6L79Xr17xUQibb585c4Yfe+HCBSIi6tChAwFp+f5SU1MNhMJ69eoZTAOKFy+e6f2Q+j34+voaaB4vX758p/stsCzyhAGISUqlKX/cpDozj1LdWcdoxv7blJCiotu3b8vW2/U9+DL6sRvL9ycV/hYuXEjff/89AWmuvwz9wB7m5bd48WICQA0aNCAinaNQzZo1CQANHDiQiIju3r3L26if789Y6LF+bsGw8AhaceI+NZ53nD6edoRGbb9CL6ISiUju5Sc9hxD4BDklTxiArqvP8sKgZccdoDI/7KeeK0/Khs5sa9myJV24cIHi4+PT3eLi4vgcfPbs2Xw/W4rz8vKi8PBw7sq7bds2fkx0dLSsU3bs2JFiYmIoPj6ed/bFixdTfHw8LVy4kABdyq/Hjx9TfHw8jx347LPP0m1fcHAwz1Csv9Vp253K/LBfdj8azT1Op4LOGuQsEAKf4F1REBHBhFx6Eomua87L9mlSkvBiaTcTtcj0lByxFdYFC/PXCXdOI+rgYmg0agBAsWLFMHv2bPTt21dWhlwgyC5KUzfgaUSiwT6tKsUELck7xN46IXsd9fdGaDRqKJVKTJgwAffv38eAAQNE5xe8MyY3ADVLFzbYZ12wMFx8v4KzSxHcu3cP8fHxWd7OnTsHAFAqlQgODkZ8fDxWrVoFAChYsCDu37+PsLAwODo6AgCOHj2K+Ph4vH37FjVr1uRt+Pbbb/k5Y2NjUapUKQDApk2bcOzYMX7cyZMnER8fj7CwMLi6ugIAAgMDM21ndHQ0li1bBnd3d9l3V9gUgJP3Z7J97t1nolS5Crhy5QrmzJnD2y4QvDOmnoMQEY3ddU025y077gBN3XsrR+eS5vsjMhT+iIh27txJQFpJ7tTUVO7lxzZpiO6pU6e4k09sbKyB8EdEvExY2bJlM8z2q9Vq6dChQ1StWjWD+X+lSpVo45mHBvei25pzpNZo0z2nQJBT8oQB0Gi0tOviM+q34R8asPFf+uPqixydx1i+P+bxV7VqVb4awBx7Jk6cSFqt1iDpR40aNWTnlRb9YB5/Li4uvKafRqPhcQOLFy9Ot31Xr16VRQdKS4spFAq6du0aERGdCgmlYVsuU+91F2jd6YeUlCqceQTvhzxhAHIL/Xx/Uo+/kydPEpGu6AczEsHBwfTjjz/K1vv1O3FSUhL38tuzZ4/M44+xf/9+PkKIiYkxaNeLFy+oX79+/Bq2trbUuXNn2WrDyJEj3+/NEQiMkG8MgH6+P32PPwYr0lGnTh3+NAcgc9KRuuT++uuv3J+gb9++Mo8/RtOmTQkwrLMXGxtLkyZN4u7IbGqyZ88e2T43NzeKiop67/dIINAn3xgAlu+vUqVKpNFoZB5/UochVsa7b9++/IksXZP38vKSndfPz48AXc4Bdgzz+CMiunz5MgE6n/xnz54Rkc5B6KeffpKVB2vUqBFduHCB7ty5w1OLM8emwMDAD3OTBAI98oUBUKlUVLZsWQJAa9asMSr8ERHdv3+fe9+xkN6vv/6aGjZsyDuqtFhHREQEd7Vlc3yp8EeUljOwd+/eRgW+ihUr0q+//kparZZevnxJZcqUIQDcCDRq1CjHBToFgnclXxgApuq7urpSYmKiUeGPiHiBT6YBdO7cmc6ePSsTAKWJOVjRj5IlSxoIf0RET58+5U/x7du3ywS+IkWK0P/93//xZCVRUVFUo0YNPp1ghogJfwKBKTB7A6DVaqlevXoEgKZMmWJU+GPHsacvEwqTkpJkkXRFixaVze1Z0Q9W+Vcq/BGlBSoVK1ZMJvCNHTtWNqdPSkriVYmLFSvGy5YJ4U9gaszeAAQFBRGgC455/fq1UeGPSBdIwzp61apVKSoqih49emSQtZehX/RDX/h78eIFn0ZIBb5Hjx7JrqtWq3kmYScnJx7H7+7uLoQ/gckxewPA0mwFBASkK/zFx8dzQc7BwYGeP39ORESjRo3i+wB5em79oh9M+GMCX6FChQwEPn20Wi0NHz6cjwx27tzJ1/6F8CfIC5i1Abh37x4fep87d86o8Jeamkrt2rXjnZUl/oiMjJQ54lhbW3PvP61WK0sIOnDgwHQ9+KQFQfSZPXs29zHYuXMnde/eXQh/gjyFWRsAab4/Y8KfvpdfkSJFuJvu3LlzCQAPCW7RogU/78WLF/lnnJ2d6fjx4zKBjyXycHNzo6SkJKNtY6MRALRs2TKefEQIf4K8hNkaAGm+vzVr1hgV/vS9/FjRj5SUFJ5mjOUNkBbpDAgI4J33008/lQl8Y8aM4bEA0rRgUvbv389XB8aPH08pKSl8RCGEP0FewmwNgDTfH1sFkAp/Ui8/tuzHin6wtFrFihXjIiDz/lOpVDzxhtQ9mAl8LN+fvb29bEmQce7cOW6Y+vbtS1qtlufwE8KfIK9hlgZAmu+PBepIhb9du3bxzvvFF18QkFb0Q6vV8kSgLK121apViUjX+fVTeusLfCzf35AhQwzaJfXy++yzzyg1NZWeP38uhD9BnsUsDQDL91eiRAleIYgJfydPnpR5+bF6AqzoB5uLOzg48OW577//ng4ePMjTdbPhPvPgY0jz/d29e1fWphcvXnA/g3r16lF8fDwRkRD+BHkaszMAWq2Wd1SWGpwJf9evX+eFQzp37kxPnjzhIwFW9KNt27YE6CrxsGIidevWlT31AdD+/fsNrj148GACdHkCpUi9/CpVqkRhYWFEREL4E+R5zM4AHDp0iAB5LP3Jkyfp8ePHXNH38fGhpKQkmj9/Pvf6IyK6efMm75AsKlD6xGdagZubm8HSXmhoKPcIPHXqFN+v7+XHHIGE8CcwB8zOALDqu8WKFePCn7Rib/Xq1bnQpl/0gy0Jenl58c7OBD42HQBAU6dONbguiyOoW7cuNw76Xn7Sp7wQ/gTmgFkZgKtXr/InOBP+7t+/z91/S5cuzb38rl+/zp/skZGR9Pz5c4NSXEwbkBb3ANKKfjASExPJzc2NAF3QD5Ghl5+09JgQ/gTmglkZAJbvj4l88+bN47n8XFxcKDg4mB/Lin74+fnRwYMHZQk/WOiwlZUVhYeH83V9pivow/L9lSlThjsSMVdh5uUnRQh/AnPBbAyANN8fE/5Yhh57e3s6e/YsP1atVvMQXjYNYFv//v358LxFixbcX4CNDvQj/ozl+9P38pMihD+BOWE2BoDl+2Mby9CjVCpp7969smN37NghO5YZjrJly5JareYpvGbOnMnTiLHjwsPDZefSz/en7+UnRQh/AnPDLAyANN8foMvnx/6/bt062XGTJk2SzfV79OjB1+eXL19OUVFR/H0m4DFBUX95j0ie78+Yl58UIfwJzA2zMAAs3x8gL4o5Y8YMIjKegw/QxQj89ttvXCOIj4+n7du3E6CrCcCOY5+ThgMTyfP9nThxwsDLT4oQ/gTmSJ43ACqVikqXLs07K3t6f/3116TRaAw8+NjTvGzZsqTRaKhRo0YEgCZMmEBEaTn8mCtx+/bt+RA/MTFRdm12rJ+fn1EvPylC+BOYI3neALB8f0xxB3RefpcuXeI+AYAu1Hfp0qV8VWDixIl0/vx5AkA2Njb08uVLUqlU3PuPjQrYysKAAQNk15Xm+6tYsSIBci8/KUL4E5gredoAaLVagwQc9evXpz59+hjk4IuMjDQo+sECe/r160dExCP52GeXLVvGtQVpGDFRWr4/9r7Uy0+KEP4E5kyeMQCJKWo6cus1Hbv9hpfCYjX52Obm5sbdcQHDHHzSoh/SfH83btwgIqKxY8fyz3p7e9OuXbsI0GXplQ7bY2JiZCm/9L38pAjhT2DO5AkDcPLuW6ox9U9eDLP2jKN0/mG4bA1fGpufXg4+VvRjyZIlPN9f69at+ftSLeHChQu86Me4ceNk51m4cCE/Tt/LT4oQ/gTmjskNQHyyiqpP+dOgIm6lgCWypz+bi+uH6DJY0Q8rKyu6e/cu75hHjhwhIqLg4GB+Hn9/f1nRj5s3b/LzpKam8mG/MS8/KUL4E5g71jAxf4eEIS5FLdun1ahxb8MY/trR0RGzZs3C119/DVtbW6Pn2bp1KwCgVatW+OOPP5CQkIAaNWqgVatWAICJEycCAKytrbFo0SLs3r0bKpUKNWvWRPXq1fl5RowYgZiYGADAwoUL0b17d6PX++uvv7Br1y4olUqsXLkSSqUyh3dAIDAdJjcAxiCNGiACAFSrVg1BQUFwcXFJ/3gibNmyBQDQs2dP/PjjjwCAMWPGQKFQ4O3bt9i3bx8AwM/PD25ubvz4Pn368PPs378fa9euBQA0btwYo0ePNnq91NRUjBgxAgAwfPhw1KxZ812+rkBgMhRE//U0E5GQosanc44bjAKcKB4+iWewYtkyKBSKDM9x4cIFNGjQAAULFsSSJUswePBgFC9eHE+ePIGtrS169+6Nbdu2AQDu3bsHa2treHh4QKFQ4MWLFyhRogTOnz+PZs2aISUlBUqlEq9fv4a7u7vR682fPx/jxo2Du7s7QkJC4OzsnCv3QiD40Jh83FrQzhrLv6wNJ/u0wUiRgrZYO6QlVi5fnmnnB8Cf5n5+flixYgUAYOTIkbC1tcXZs2d55y9Xrhw8PT356xYtWqBEiRK4e/cuPv/8c6SkpAAABgwYkG7nf/HiBWbMmAFAZwhE5xeYNSbWIDjGlgGzQmpqKs8LOGfOHAJ0+f4iIiJIpVLJQn1/+OEH0mq1PBV4YGCgLJcf2/Tz/UkRwp8gP5FnDEBOYdF6xYoVo9atWxMAGjFiBBGlpQZnS4hBQUF06dIlAkAFChSgp0+f8lx+TPk3FhDEEB5/gvyG2RsAltrb39+fd84HDx7QmzdvZBGErPIvK87ZrVs38vX15U48LMmINN+fFOHxJ8iPmLUBiImJ4Z6BLF9/ly5diIioX79+skg/f39/UqlUPAioYcOG3Mtv6NChBMjz/ekjPP4E+RGzNgAbNmwgAOTp6cmdes6dOyfL8cfm97t27eIlwpnRsLW1pcOHDxvk+9NHePwJ8itmbQCaN29OAHjSjgYNGsiEPxYMxCr/sixCTBfYuXOn0Xx/+gjhT5BfMVsD8Pz5cy7usbn+nj17uPDn4uJC06dPJ0CX+y8uLo7P8/FfJKCxfH/6COFPkJ8xWwPAin6wWH0PDw96+fIlNwarV6/mI4OlS5fS6NGjeednwT/6+f70EcKfIL9jtgaARQqy+fvy5cu58Oft7U3h4eE8ocfu3bt5aHCtWrW40CfN92cMIfwJ8jtmaQBY0Q+W/MPFxYUP1fFfqC/L/VehQgVZFqA7d+4QkTzf37NnzwyuIYQ/gSVgclfgnMAi/woXLgwAGDx4MMaOHQsACAgIQP369bF//34AQFhYGKKiogAA9erVg5eXFwBg0aJFAIDu3bujdOnSBtcYM2YMEhIS0KhRI/j7+7/fLyQQmApTW6DsIi36Aejy/bEqPS4uLhQWFkYqlUrmBMQyCbOiH9J8f5cvXza4hhD+BJaC2RmA48eP844P6PL+S4U/IqIjR47wzs9KgkmLfrB8f82aNTM4vxD+BJaE2RkAVuGXbZ06deLCn1qtJrVaTZ6entxIDBw4UObjHxMTQ05OTgSADhw4YHB+IfwJLAmzMgCJiYmyZJ316tWTCX9arZaGDRvG902ePFnmCUhEtGjRIgJ0JcL1nXqkwt8vv/xiiq8oEHxQzMoAsBoBzAHIw8ODAFBAQAARpVXsZfP3gwcP8nX+xMRESk1N5QZh7dq1BueXevylFxMgEOQnzMoAsIAfAFSiRAmZ8Cet2Iv/vP8GDRpEQFrRj23btvHhfVJSkuzcQvgTWCJmYwCkRT/wXzw/E/6kFXvZE37BggWyoh9arZYXFZ0+fbrs3EL4E1gqZmMAWNEPaef39vamoKAg/rpXr17c42/lypUEpBX9YFWB7O3tKTQ0VHZuIfwJLBWzcQRief8AICkpCQDw/fffo1OnTkhKSsJnn32Gzz77DFqtFlWrVsVff/0FAOjduzeUSiV3/Onbty/c3Nz4uaQ5/hYsWCBy/AksC1NboKzAin5AIgD26tXLoGIvq+Y7cuRIHvl38+ZNunv3Lv+8fr4/IfwJLJk8WRdAH+b6CwBEBGdnZ1y7dg3Pnj1DpUqVcPDgQdjZ2eHw4cMAADs7O6SmpvKiH0OGDAEAdOzYEZUrV+bn0i/ukZUMxAJBvsLUFigztFqtQdZeFgIsrdjL5vhFixalRo0acSEwNDSUZwCS5vsTwp9AYAYi4Pnz52Wd39nZma/tS5frWOVfVvBToVDQy5cvadq0aQQY5vsTwp9AYAYiICvVxYiOjoatrS3++OMPWUkuFv1nZ2cHQFf0w8XFBStXrgSQViYMEMKfQMAxtQXKiNTUVL7EB4kIqF+x9969ezzgh8UBBAYGppvvTwh/AoGOPG0Adu/eLev8gC6Xnz7Mv5/FBhQoUICioqKM5vsTHn8CQRp5YhVAqyX8euUFDt58DaVCgY41S6BTrRJ8mM4YP348Ro4cafB5Nvy3t7cHAHTq1AlnzpxBSEgInJycMHDgQACiqq9AoE+eMADjfr2B3Zdf8Ncn7obifPBT3Lx5k+/z9/fHnDlzDD4bHR2NoKAgAMDt27cB6Ep+L1y4EIAuW5CTkxMAYOnSpQgJCYG7u7uBcREILBGTi4APQuNknZ+xbNII/v969eph/fr1Rtfp//zzT2g0GpQpUwYRERFwc3ODq6sr/v77b1hbW2PUqFEAhPAnEBjD5Abg+vMYg31aVQqSn1wFAFhbW+PEiROwsbEx+nk2/Hd0dAQA9OzZE8uWLQMgz/cncvwJBIaY3ACULepguNPKBjbu5QHoYgAKFixo9LNqtZp7/z18+BAA0LJlS+zcuROArtMDwuNPIEgPkxuAuuWK4JNyLrJ9SqUSvf63A2q1Gj169Ej3s2fPnkVUVBQcHR2RkpICT09PnDp1ChqNBs2aNYO3t7cQ/gSCDDC5AQCA9f0+Qd8GZeHqaAf3QnYIaFweq/t4w8rKKsPPHThwAAC4yNetWzf8/PPPANKe/kL4EwgywNTrkO8CW+dnEYITJkyQ5fsTOf4EgozJEyOAnHD//n2EhIRAqVSCiNCgQQMeNTh69GgolUoh/AkEmWC2BoCp/0wg9PLywrNnz+Dm5gZ/f38h/AkEWcDsDUBcXBysra1x5coVAMCIESOgVCqF8CcQZAEFEZGpG5FdoqOj4erqCo1GAwBo1KgRzp49C3t7ezx79gwbN27EuHHj4O7ujpCQEOH0IxCkg1mOAJj3H3MOSk5OBqDL95eSkiI8/gSCLGKWBoAN/1UqFRwdHXH58mUAwHfffSeEP4EgG5idAZB6/wFAyZIlAQAdOnTA8+fPhfAnEGQDszMAzPuPde5Hjx4BAL755hsu/I0YMUIIfwJBFjA7A8C8/4gITk5OUKlUqFu3Li5dusQ9/qZPn27iVgoE5kGeyAeQHdj8H9BpAADQr18/jBs3DoAQ/gSC7GBWIwDm/cdISkpCmTJl8PfffwvhTyDIAWZlAKRPf5b9t127dtizZ48Q/gSCHGC2BiAlJQVOTk44fvw4ACH8CQQ5wWw8AfW9/wDA19cXp0+fFh5/AkEOMZsRAPP+Y1hbW+PixYsAhPAnEOQUszEA0uE/AJQoUQJJSUlC+BMI3gGzmAKo1WoULVoUsbGxsv1KpRJXrlwRc3+BIIeYxQjg7Nmzss7v4KBLJCqEP4Hg3TALA7Bv3z7Z68TEROHxJxDkAmZhAHbt2mWwTwh/AsG7k+c1gPv376NSpUqyfY0aNUJQUJBw+hEI3pE8PwLYs2eP7LVCoRAefwJBLpHnDcDmzZtlr0eOHCmEP4Egl8jTU4Do6Gi4uKRVDXJ1dcX9+/fF3F8gyCXy9Ahg+/btsteLFi0SnV8gyEXy1AjgdUwSlAoFPnKyBwBUq1YNwcHBAIDatWvj8uXLYu4vEOQiecIAPAyLx9jd13H1WTQAoF75IpjnVw0eHxXmx1y7dk3M/QWCXMbkGYHUGi36bvgXL6KS+L5/H0eizeAJ/HX79u1F5xcI3gMm1wCC7ofLOj/jweFA/n/9lQCBQJA7mNwARCelGuzTatSg1EQAwMcffyxbCRAIBLmHyQ1AowqusLGSC3tKK2uUC1iBChU9cejQIRO1TCDI/5jcALg72WNcWy/ZPoUCmB3QAQ/u3+OFPwQCQe6TJ1YBAODum1gcuvEaCoUCHWoWR0X3QqZukkCQ78kzBkAgEHx4TD4FEAgEpkMYAIHAghEGQCCwYIQBEAgsGGEABAILRhgAgcCCEQZAILBghAEQCCwYYQAEAgtGGACBwIIRBkAgsGCEARAILBhhAAQCC0YYAIHAghEGQCCwYIQBEAgsGGEABAILRhgAgcCCEQZAILBghAEQCCwYYQAEAgtGGACBwIL5f+h3iGBmG/r8AAAAAElFTkSuQmCC", 260 | "text/plain": [ 261 | "
" 262 | ] 263 | }, 264 | "metadata": {}, 265 | "output_type": "display_data" 266 | } 267 | ], 268 | "source": [ 269 | "import matplotlib.pyplot as plt\n", 270 | "import matplotlib.patches as patches\n", 271 | "\n", 272 | "def visualize_neural_network(input_size, hidden_size, num_classes):\n", 273 | " fig, ax = plt.subplots()\n", 274 | "\n", 275 | " # Define the layers\n", 276 | " layers = [input_size, hidden_size, num_classes]\n", 277 | " layer_positions = [(0, i) for i in range(len(layers))]\n", 278 | "\n", 279 | " # Draw the layers\n", 280 | " for i, (x, layer) in enumerate(zip(layer_positions, layers)):\n", 281 | " for j in range(layer):\n", 282 | " circle = patches.Circle((i * 2, j - layer / 2), radius=0.1, fill=True)\n", 283 | " ax.add_patch(circle)\n", 284 | "\n", 285 | " # Draw connections to the next layer\n", 286 | " if i < len(layers) - 1:\n", 287 | " for k in range(layers[i + 1]):\n", 288 | " line = plt.Line2D([i * 2 + 0.1, (i + 1) * 2 - 0.1], [j - layer / 2, k - layers[i + 1] / 2], color=\"black\")\n", 289 | " ax.add_line(line)\n", 290 | "\n", 291 | " ax.set_xlim(-0.5, len(layers) * 2)\n", 292 | " ax.set_ylim(-max(layers) / 2 - 0.5, max(layers) / 2 + 0.5)\n", 293 | " ax.set_aspect('equal')\n", 294 | " plt.axis('off')\n", 295 | " plt.title('Neural Network Architecture')\n", 296 | " plt.show()\n", 297 | "\n", 298 | "# Example parameters\n", 299 | "input_size = 10\n", 300 | "hidden_size = 4\n", 301 | "num_classes = 2\n", 302 | "\n", 303 | "# Visualize the neural network\n", 304 | "visualize_neural_network(input_size, hidden_size, num_classes)\n" 305 | ] 306 | }, 307 | { 308 | "cell_type": "code", 309 | "execution_count": 13, 310 | "id": "fca379b3", 311 | "metadata": {}, 312 | "outputs": [], 313 | "source": [ 314 | "# Some loss function options\n", 315 | "cross_entropy_loss = nn.CrossEntropyLoss()\n", 316 | "nll_loss = nn.NLLLoss()\n", 317 | "mse_loss = nn.MSELoss()\n", 318 | "\n", 319 | "optimizer = torch.optim.Adam(model.parameters(), lr=0.001)\n" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "id": "020fbccd-eb05-4ea5-a0ca-c92198d6eefa", 325 | "metadata": { 326 | "jp-MarkdownHeadingCollapsed": true 327 | }, 328 | "source": [ 329 | "## Training our model" 330 | ] 331 | }, 332 | { 333 | "cell_type": "code", 334 | "execution_count": 14, 335 | "id": "c8df8e47", 336 | "metadata": {}, 337 | "outputs": [], 338 | "source": [ 339 | "# input layer, a single hidden layer, and an output layer\n", 340 | "# Define hyperparameters\n", 341 | "input_size = 784 # 28x28 pixels\n", 342 | "hidden_size = 128\n", 343 | "num_classes = 10\n", 344 | "\n", 345 | "batch_size = 100" 346 | ] 347 | }, 348 | { 349 | "cell_type": "code", 350 | "execution_count": 31, 351 | "id": "502faaf8", 352 | "metadata": {}, 353 | "outputs": [], 354 | "source": [ 355 | "# Function to train the model\n", 356 | "import time\n", 357 | "def train_model(model, loss_function, train_loader, optimizer, num_epochs=10):\n", 358 | " # Put the model in training mode\n", 359 | " model.train()\n", 360 | " start_time = time.time()\n", 361 | " \n", 362 | " # Iterate over the number of epochs\n", 363 | " for epoch in range(num_epochs):\n", 364 | " # Initialize the running loss for this epoch to zero\n", 365 | " running_loss = 0.0\n", 366 | " \n", 367 | " # Iterate over each batch in the training loader\n", 368 | " for i, (images, labels) in enumerate(train_loader):\n", 369 | " \n", 370 | " # Reshape the images tensor to have size (batch_size, input_size)\n", 371 | " images = images.reshape(-1, input_size)\n", 372 | " # Forward pass: compute the outputs of the model given the input images\n", 373 | " outputs = model(images)\n", 374 | " # Compute the loss between the outputs and the true labels\n", 375 | " loss = loss_function(outputs, labels)\n", 376 | " # Backward pass: compute the gradients of the loss with respect to the model parameters\n", 377 | " optimizer.zero_grad()\n", 378 | " loss.backward()\n", 379 | " # Update the model parameters using the optimizer\n", 380 | " optimizer.step()\n", 381 | " \n", 382 | " # Add the current batch loss to the running loss for this epoch\n", 383 | " running_loss += loss.item()\n", 384 | "\n", 385 | " # Compute the average loss over all batches for this epoch and print it\n", 386 | " print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {running_loss/len(train_loader):.4f}')\n", 387 | " training_time = (time.time() - start_time)\n", 388 | " print(f'Training took: {round(training_time, 1)} seconds.')\n" 389 | ] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": 32, 394 | "id": "410c88bf", 395 | "metadata": {}, 396 | "outputs": [], 397 | "source": [ 398 | "# Function to evaluate the model\n", 399 | "def evaluate_model(model, test_loader):\n", 400 | " # Put the model in evaluation mode\n", 401 | " model.eval()\n", 402 | " \n", 403 | " # Initialize empty lists to store true and predicted labels\n", 404 | " y_true, y_pred = [], []\n", 405 | " \n", 406 | " # Disable gradient computation since we're only evaluating the model\n", 407 | " with torch.no_grad():\n", 408 | " # Iterate over each batch in the test loader\n", 409 | " for images, labels in test_loader:\n", 410 | " # Reshape the images tensor to have size (batch_size, input_size)\n", 411 | " images = images.reshape(-1, input_size)\n", 412 | " \n", 413 | " # Forward pass: compute the outputs of the model given the input images\n", 414 | " outputs = model(images)\n", 415 | " \n", 416 | " # Find the predicted class for each image in the batch\n", 417 | " _, predicted = torch.max(outputs.data, 1)\n", 418 | " \n", 419 | " # Append the true and predicted labels for this batch to the lists\n", 420 | " y_true.extend(labels.numpy())\n", 421 | " y_pred.extend(predicted.numpy())\n", 422 | " \n", 423 | " # Calculate evaluation metrics using the true and predicted labels\n", 424 | " accuracy, f1, precision, recall = evaluate_model_metrics(np.array(y_true), np.array(y_pred))\n", 425 | " \n", 426 | " # Return the evaluation metrics\n", 427 | " return accuracy, f1, precision, recall\n", 428 | "\n", 429 | "# Function to calculate evaluation metrics\n", 430 | "def evaluate_model_metrics(y_true, y_pred):\n", 431 | " # Compute the accuracy, F1 score, precision, and recall\n", 432 | " accuracy = accuracy_score(y_true, y_pred)\n", 433 | " f1 = f1_score(y_true, y_pred, average='macro')\n", 434 | " precision = precision_score(y_true, y_pred, average='macro')\n", 435 | " recall = recall_score(y_true, y_pred, average='macro')\n", 436 | "\n", 437 | " # Return the evaluation metrics\n", 438 | " return accuracy, f1, precision, recall\n", 439 | "\n", 440 | "# Define a dictionary of loss functions\n", 441 | "loss_functions = {\n", 442 | " 'CrossEntropyLoss': nn.CrossEntropyLoss(),\n", 443 | " 'NLLLoss': nn.NLLLoss(), # requires logsoftmax output (according to docs, so loss will be wrong without it!)\n", 444 | " 'MultiMarginLoss': nn.MultiMarginLoss(),\n", 445 | " 'KLDivLoss': nn.KLDivLoss() # requires logsoftmax output (according to docs, so loss will be wrong without it!)\n", 446 | "}" 447 | ] 448 | }, 449 | { 450 | "cell_type": "code", 451 | "execution_count": 33, 452 | "id": "559edd99-720b-47f1-b677-b8264c4e9981", 453 | "metadata": {}, 454 | "outputs": [ 455 | { 456 | "data": { 457 | "text/plain": [ 458 | "784" 459 | ] 460 | }, 461 | "execution_count": 33, 462 | "metadata": {}, 463 | "output_type": "execute_result" 464 | } 465 | ], 466 | "source": [ 467 | "input_size" 468 | ] 469 | }, 470 | { 471 | "cell_type": "code", 472 | "execution_count": 34, 473 | "id": "156ac9db", 474 | "metadata": { 475 | "scrolled": true 476 | }, 477 | "outputs": [ 478 | { 479 | "name": "stdout", 480 | "output_type": "stream", 481 | "text": [ 482 | "Training with CrossEntropyLoss:\n", 483 | "Epoch [1/5], Loss: 0.2124\n", 484 | "Epoch [2/5], Loss: 0.1234\n", 485 | "Epoch [3/5], Loss: 0.1008\n", 486 | "Epoch [4/5], Loss: 0.0905\n", 487 | "Epoch [5/5], Loss: 0.0793\n", 488 | "Training took: 74.2 seconds.\n", 489 | "Performance metrics for CrossEntropyLoss:\n", 490 | "Accuracy: 0.9716, F1-score: 0.9715, Precision: 0.9718, Recall: 0.9716\n", 491 | "\n", 492 | "Training with NLLLoss:\n", 493 | "Epoch [1/5], Loss: -2130538.3142\n", 494 | "Epoch [2/5], Loss: -12779579.6207\n", 495 | "Epoch [3/5], Loss: -32398089.4337\n", 496 | "Epoch [4/5], Loss: -60796260.3199\n", 497 | "Epoch [5/5], Loss: -97924332.6755\n", 498 | "Training took: 74.6 seconds.\n" 499 | ] 500 | }, 501 | { 502 | "name": "stderr", 503 | "output_type": "stream", 504 | "text": [ 505 | "/Users/sinanozdemir/Library/Python/3.9/lib/python/site-packages/sklearn/metrics/_classification.py:1531: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n", 506 | " _warn_prf(average, modifier, f\"{metric.capitalize()} is\", len(result))\n" 507 | ] 508 | }, 509 | { 510 | "name": "stdout", 511 | "output_type": "stream", 512 | "text": [ 513 | "Performance metrics for NLLLoss:\n", 514 | "Accuracy: 0.1135, F1-score: 0.0204, Precision: 0.0114, Recall: 0.1000\n", 515 | "\n", 516 | "Training with MultiMarginLoss:\n", 517 | "Epoch [1/5], Loss: 0.0368\n", 518 | "Epoch [2/5], Loss: 0.0201\n", 519 | "Epoch [3/5], Loss: 0.0166\n", 520 | "Epoch [4/5], Loss: 0.0140\n", 521 | "Epoch [5/5], Loss: 0.0134\n", 522 | "Training took: 70.7 seconds.\n", 523 | "Performance metrics for MultiMarginLoss:\n", 524 | "Accuracy: 0.9661, F1-score: 0.9658, Precision: 0.9663, Recall: 0.9658\n", 525 | "\n", 526 | "Training with KLDivLoss:\n" 527 | ] 528 | }, 529 | { 530 | "name": "stderr", 531 | "output_type": "stream", 532 | "text": [ 533 | "/Users/sinanozdemir/Library/Python/3.9/lib/python/site-packages/torch/nn/functional.py:2919: UserWarning: reduction: 'mean' divides the total loss by both the batch size and the support size.'batchmean' divides only by the batch size, and aligns with the KL div math definition.'mean' will be changed to behave the same as 'batchmean' in the next major release.\n", 534 | " warnings.warn(\n" 535 | ] 536 | }, 537 | { 538 | "ename": "RuntimeError", 539 | "evalue": "kl_div: Integral inputs not supported.", 540 | "output_type": "error", 541 | "traceback": [ 542 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", 543 | "\u001b[0;31mRuntimeError\u001b[0m Traceback (most recent call last)", 544 | "Cell \u001b[0;32mIn[34], line 10\u001b[0m\n\u001b[1;32m 7\u001b[0m optimizer \u001b[38;5;241m=\u001b[39m torch\u001b[38;5;241m.\u001b[39moptim\u001b[38;5;241m.\u001b[39mAdam(model\u001b[38;5;241m.\u001b[39mparameters(), lr\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0.001\u001b[39m)\n\u001b[1;32m 9\u001b[0m \u001b[38;5;66;03m# Train the model\u001b[39;00m\n\u001b[0;32m---> 10\u001b[0m \u001b[43mtrain_model\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mloss_function\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtrain_loader\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43moptimizer\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mnum_epochs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m5\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m 12\u001b[0m \u001b[38;5;66;03m# Evaluate the model\u001b[39;00m\n\u001b[1;32m 13\u001b[0m accuracy, f1, precision, recall \u001b[38;5;241m=\u001b[39m evaluate_model(model, test_loader)\n", 545 | "Cell \u001b[0;32mIn[31], line 21\u001b[0m, in \u001b[0;36mtrain_model\u001b[0;34m(model, loss_function, train_loader, optimizer, num_epochs)\u001b[0m\n\u001b[1;32m 19\u001b[0m outputs \u001b[38;5;241m=\u001b[39m model(images)\n\u001b[1;32m 20\u001b[0m \u001b[38;5;66;03m# Compute the loss between the outputs and the true labels\u001b[39;00m\n\u001b[0;32m---> 21\u001b[0m loss \u001b[38;5;241m=\u001b[39m \u001b[43mloss_function\u001b[49m\u001b[43m(\u001b[49m\u001b[43moutputs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlabels\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 22\u001b[0m \u001b[38;5;66;03m# Backward pass: compute the gradients of the loss with respect to the model parameters\u001b[39;00m\n\u001b[1;32m 23\u001b[0m optimizer\u001b[38;5;241m.\u001b[39mzero_grad()\n", 546 | "File \u001b[0;32m~/Library/Python/3.9/lib/python/site-packages/torch/nn/modules/module.py:1501\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1496\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m 1497\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m 1498\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[1;32m 1499\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m 1500\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1501\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1502\u001b[0m \u001b[38;5;66;03m# Do not call functions when jit is used\u001b[39;00m\n\u001b[1;32m 1503\u001b[0m full_backward_hooks, non_full_backward_hooks \u001b[38;5;241m=\u001b[39m [], []\n", 547 | "File \u001b[0;32m~/Library/Python/3.9/lib/python/site-packages/torch/nn/modules/loss.py:471\u001b[0m, in \u001b[0;36mKLDivLoss.forward\u001b[0;34m(self, input, target)\u001b[0m\n\u001b[1;32m 470\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mforward\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;28minput\u001b[39m: Tensor, target: Tensor) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Tensor:\n\u001b[0;32m--> 471\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mF\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mkl_div\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtarget\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mreduction\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mreduction\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlog_target\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mlog_target\u001b[49m\u001b[43m)\u001b[49m\n", 548 | "File \u001b[0;32m~/Library/Python/3.9/lib/python/site-packages/torch/nn/functional.py:2931\u001b[0m, in \u001b[0;36mkl_div\u001b[0;34m(input, target, size_average, reduce, reduction, log_target)\u001b[0m\n\u001b[1;32m 2928\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 2929\u001b[0m reduction_enum \u001b[38;5;241m=\u001b[39m _Reduction\u001b[38;5;241m.\u001b[39mget_enum(reduction)\n\u001b[0;32m-> 2931\u001b[0m reduced \u001b[38;5;241m=\u001b[39m \u001b[43mtorch\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mkl_div\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtarget\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mreduction_enum\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlog_target\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlog_target\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 2933\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m reduction \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mbatchmean\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m.\u001b[39mdim() \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m0\u001b[39m:\n\u001b[1;32m 2934\u001b[0m reduced \u001b[38;5;241m=\u001b[39m reduced \u001b[38;5;241m/\u001b[39m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m.\u001b[39msize()[\u001b[38;5;241m0\u001b[39m]\n", 549 | "\u001b[0;31mRuntimeError\u001b[0m: kl_div: Integral inputs not supported." 550 | ] 551 | } 552 | ], 553 | "source": [ 554 | "# Train and evaluate the model using different loss functions\n", 555 | "for loss_name, loss_function in loss_functions.items():\n", 556 | " print(f'Training with {loss_name}:')\n", 557 | " \n", 558 | " # Initialize a new model and optimizer for each loss function\n", 559 | " model = NeuralNet(input_size, hidden_size, num_classes)\n", 560 | " optimizer = torch.optim.Adam(model.parameters(), lr=0.001)\n", 561 | " \n", 562 | " # Train the model\n", 563 | " train_model(model, loss_function, train_loader, optimizer, num_epochs=5)\n", 564 | "\n", 565 | " # Evaluate the model\n", 566 | " accuracy, f1, precision, recall = evaluate_model(model, test_loader)\n", 567 | " print(f'Performance metrics for {loss_name}:')\n", 568 | " print(f'Accuracy: {accuracy:.4f}, F1-score: {f1:.4f}, Precision: {precision:.4f}, Recall: {recall:.4f}\\n')\n" 569 | ] 570 | }, 571 | { 572 | "cell_type": "code", 573 | "execution_count": null, 574 | "id": "0116ef96-efc8-4699-8dd2-2c3a36b96bfd", 575 | "metadata": {}, 576 | "outputs": [], 577 | "source": [ 578 | "# UH OH, KL didn't even run :(" 579 | ] 580 | }, 581 | { 582 | "cell_type": "markdown", 583 | "id": "c714633a-9df3-41d9-9c82-6163d5dfb174", 584 | "metadata": { 585 | "jp-MarkdownHeadingCollapsed": true 586 | }, 587 | "source": [ 588 | "## Making our code with with KL and NLL" 589 | ] 590 | }, 591 | { 592 | "cell_type": "code", 593 | "execution_count": 39, 594 | "id": "3efdc826-eb71-4974-8bb5-52bbad2784c7", 595 | "metadata": {}, 596 | "outputs": [], 597 | "source": [ 598 | "def one_hot_encode(labels, num_classes):\n", 599 | " # Create a tensor of zeros with shape [len(labels), num_classes]\n", 600 | " one_hot = torch.zeros(len(labels), num_classes)\n", 601 | " \n", 602 | " # Use scatter_ to assign 1s to the correct class indices\n", 603 | " one_hot.scatter_(1, labels.unsqueeze(1), 1)\n", 604 | " \n", 605 | " return one_hot\n" 606 | ] 607 | }, 608 | { 609 | "cell_type": "code", 610 | "execution_count": 40, 611 | "id": "47c8d6a0-1abb-4b5e-9e0e-7cfcd14da1b0", 612 | "metadata": {}, 613 | "outputs": [ 614 | { 615 | "data": { 616 | "text/plain": [ 617 | "tensor([[0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],\n", 618 | " [0., 0., 0., 0., 0., 0., 0., 1., 0., 0.]])" 619 | ] 620 | }, 621 | "execution_count": 40, 622 | "metadata": {}, 623 | "output_type": "execute_result" 624 | } 625 | ], 626 | "source": [ 627 | "one_hot_encode(torch.tensor([1, 7]), 10)" 628 | ] 629 | }, 630 | { 631 | "cell_type": "code", 632 | "execution_count": 43, 633 | "id": "cbb6dd60-1a9d-4063-b38e-e82135a335b3", 634 | "metadata": {}, 635 | "outputs": [], 636 | "source": [ 637 | "# Function to train the model\n", 638 | "def train_model(model, loss_function, train_loader, optimizer, num_epochs=10):\n", 639 | " # Put the model in training mode\n", 640 | " model.train()\n", 641 | " start_time = time.time()\n", 642 | " \n", 643 | " # Iterate over the number of epochs\n", 644 | " for epoch in range(num_epochs):\n", 645 | " # Initialize the running loss for this epoch to zero\n", 646 | " running_loss = 0.0\n", 647 | " \n", 648 | " # Iterate over each batch in the training loader\n", 649 | " for i, (images, labels) in enumerate(train_loader):\n", 650 | " \n", 651 | " # Reshape the images tensor to have size (batch_size, input_size)\n", 652 | " images = images.reshape(-1, input_size)\n", 653 | " # Forward pass: compute the outputs of the model given the input images\n", 654 | " outputs = model(images)\n", 655 | "\n", 656 | " #### NEW ####\n", 657 | " if 'KL' in str(loss_function) or 'NLL' in str(loss_function):\n", 658 | " outputs = F.log_softmax(outputs, dim=1)\n", 659 | " if 'KL' in str(loss_function): # KL also need one hot encoded labels: [[0, 0, 1, 0, 0], [0, 0, 0, 0, 1]] vs [2, 4]\n", 660 | " labels = one_hot_encode(labels, 10)\n", 661 | " #### NEW ####\n", 662 | " \n", 663 | " # Compute the loss between the outputs and the true labels\n", 664 | " loss = loss_function(outputs, labels)\n", 665 | " # Backward pass: compute the gradients of the loss with respect to the model parameters\n", 666 | " optimizer.zero_grad()\n", 667 | " loss.backward()\n", 668 | " # Update the model parameters using the optimizer\n", 669 | " optimizer.step()\n", 670 | " \n", 671 | " # Add the current batch loss to the running loss for this epoch\n", 672 | " running_loss += loss.item()\n", 673 | "\n", 674 | " \n", 675 | "\n", 676 | " # Compute the average loss over all batches for this epoch and print it\n", 677 | " print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {running_loss/len(train_loader):.4f}')\n", 678 | " training_time = (time.time() - start_time)\n", 679 | " print(f'Training took: {round(training_time, 1)} seconds.')\n" 680 | ] 681 | }, 682 | { 683 | "cell_type": "code", 684 | "execution_count": 44, 685 | "id": "e5c84301-45e4-497d-849b-56124f51f571", 686 | "metadata": {}, 687 | "outputs": [ 688 | { 689 | "name": "stdout", 690 | "output_type": "stream", 691 | "text": [ 692 | "Training with NLLLoss:\n", 693 | "Epoch [1/5], Loss: 0.2193\n", 694 | "Epoch [2/5], Loss: 0.1263\n", 695 | "Epoch [3/5], Loss: 0.1039\n", 696 | "Epoch [4/5], Loss: 0.0916\n", 697 | "Epoch [5/5], Loss: 0.0841\n", 698 | "Training took: 83.9 seconds.\n", 699 | "Performance metrics for NLLLoss:\n", 700 | "Accuracy: 0.9680, F1-score: 0.9679, Precision: 0.9684, Recall: 0.9678\n", 701 | "\n", 702 | "Training with KLDivLoss:\n", 703 | "Epoch [1/5], Loss: 0.2135\n", 704 | "Epoch [2/5], Loss: 0.1239\n", 705 | "Epoch [3/5], Loss: 0.1030\n", 706 | "Epoch [4/5], Loss: 0.0904\n", 707 | "Epoch [5/5], Loss: 0.0815\n", 708 | "Training took: 79.7 seconds.\n", 709 | "Performance metrics for KLDivLoss:\n", 710 | "Accuracy: 0.9679, F1-score: 0.9677, Precision: 0.9681, Recall: 0.9674\n", 711 | "\n", 712 | "Training with CrossEntropyLoss:\n", 713 | "Epoch [1/5], Loss: 0.2133\n", 714 | "Epoch [2/5], Loss: 0.1242\n", 715 | "Epoch [3/5], Loss: 0.1019\n", 716 | "Epoch [4/5], Loss: 0.0918\n", 717 | "Epoch [5/5], Loss: 0.0833\n", 718 | "Training took: 74.7 seconds.\n", 719 | "Performance metrics for CrossEntropyLoss:\n", 720 | "Accuracy: 0.9674, F1-score: 0.9670, Precision: 0.9672, Recall: 0.9669\n", 721 | "\n" 722 | ] 723 | } 724 | ], 725 | "source": [ 726 | "# Define a dictionary of loss functions\n", 727 | "loss_functions = {\n", 728 | "\n", 729 | " 'NLLLoss': nn.NLLLoss(), \n", 730 | " 'KLDivLoss': nn.KLDivLoss(reduction='batchmean'), # to avoid deprecation warning, I'm setting reduction to batchmean\n", 731 | " 'CrossEntropyLoss': nn.CrossEntropyLoss()\n", 732 | "}\n", 733 | "# Train and evaluate the model using different loss functions\n", 734 | "for loss_name, loss_function in loss_functions.items():\n", 735 | " print(f'Training with {loss_name}:')\n", 736 | " \n", 737 | " # Initialize a new model and optimizer for each loss function\n", 738 | " model = NeuralNet(input_size, hidden_size, num_classes)\n", 739 | " optimizer = torch.optim.Adam(model.parameters(), lr=0.001)\n", 740 | " \n", 741 | " # Train the model\n", 742 | " train_model(model, loss_function, train_loader, optimizer, num_epochs=5)\n", 743 | "\n", 744 | " # Evaluate the model\n", 745 | " accuracy, f1, precision, recall = evaluate_model(model, test_loader)\n", 746 | " print(f'Performance metrics for {loss_name}:')\n", 747 | " print(f'Accuracy: {accuracy:.4f}, F1-score: {f1:.4f}, Precision: {precision:.4f}, Recall: {recall:.4f}\\n')\n" 748 | ] 749 | }, 750 | { 751 | "cell_type": "code", 752 | "execution_count": 45, 753 | "id": "ff44b62b-0e6c-4a12-8e65-22ca97c1929a", 754 | "metadata": {}, 755 | "outputs": [ 756 | { 757 | "name": "stderr", 758 | "output_type": "stream", 759 | "text": [ 760 | "/Users/sinanozdemir/Library/Python/3.9/lib/python/site-packages/torchvision/datasets/mnist.py:65: UserWarning: train_labels has been renamed targets\n", 761 | " warnings.warn(\"train_labels has been renamed targets\")\n" 762 | ] 763 | }, 764 | { 765 | "data": { 766 | "text/plain": [ 767 | "" 768 | ] 769 | }, 770 | "execution_count": 45, 771 | "metadata": {}, 772 | "output_type": "execute_result" 773 | }, 774 | { 775 | "data": { 776 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGYCAYAAABcVthxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAjO0lEQVR4nO3dfVSUdf7/8dcAcqNyIxYzUKh0syVlWWk4Wlo6Gxm5efJsS1FZubIVVOqurvZV2ugG87hmmsnWdmOlpbXdWpFGu1pJqLiWqWt2K2UDtQYkJqDz+f3R8TpNeYPt4PDh93ycc53TXJ8PM+8rM59ezojLGGMEAABgkYhwDwAAAHC4CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1okK9wBtJRAIaPv27YqPj5fL5Qr3OAAAoBWMMfruu++UlpamiIgD32fpsAGzfft2paenh3sMAADwC1RXV+vYY4894HqHDZj4+HhJP/wLSEhICPM0AACgNRoaGpSenu78On4gHTZg9v2xUUJCAgEDAIBlDvX2D97ECwAArEPAAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6BAwAALAOAQMAAKxDwAAAAOsQMAAAwDoEDAAAsA4BAwAArEPAAAAA60SFe4D2pNfkV9r8NT6bntPmrwEAQEfHHRgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1okK9wAIvV6TX2nz1/hsek6bvwYAAAfCHRgAAGAd7sCgXeIuEgDgYLgDAwAArEPAAAAA6xAwAADAOocdMCtXrtSIESOUlpYml8ulF154IWjdGKOioiKlpqYqLi5OPp9PW7duDdqzY8cO5eXlKSEhQUlJSRozZox27twZtOf999/Xueeeq9jYWKWnp2vGjBmHf3UAAKBDOuw38TY2Nur000/Xddddp0svvfRn6zNmzNCcOXO0YMECZWRkaNq0acrOztamTZsUGxsrScrLy9NXX32l5cuXq6WlRddee63y8/O1aNEiSVJDQ4MuuOAC+Xw+lZaWasOGDbruuuuUlJSk/Pz8//GSgSOno7wZua2vgzdUAzhchx0ww4cP1/Dhw/e7ZozR7NmzNXXqVF1yySWSpMcff1xut1svvPCCcnNztXnzZpWVlWnNmjXq16+fJGnu3Lm66KKLNHPmTKWlpWnhwoVqbm7WI488oujoaJ1yyilav369Zs2aRcAAAIDQvgfm008/ld/vl8/nc84lJiYqKytLFRUVkqSKigolJSU58SJJPp9PERERqqysdPYMHjxY0dHRzp7s7Gxt2bJF33777X5fu6mpSQ0NDUEHAADomEIaMH6/X5LkdruDzrvdbmfN7/crJSUlaD0qKkrJyclBe/b3HD9+jZ8qKSlRYmKic6Snp//vFwQAANqlDvMppClTpqi+vt45qqurwz0SAABoIyENGI/HI0mqqakJOl9TU+OseTwe1dbWBq3v2bNHO3bsCNqzv+f48Wv8VExMjBISEoIOAADQMYU0YDIyMuTxeFReXu6ca2hoUGVlpbxeryTJ6/Wqrq5OVVVVzp4333xTgUBAWVlZzp6VK1eqpaXF2bN8+XKddNJJ6tatWyhHBgAAFjrsgNm5c6fWr1+v9evXS/rhjbvr16/Xtm3b5HK5NG7cON1555166aWXtGHDBl199dVKS0vTyJEjJUm9e/fWhRdeqLFjx2r16tV65513VFhYqNzcXKWlpUmSrrjiCkVHR2vMmDHauHGjFi9erPvuu08TJkwI2YUDAAB7HfbHqNeuXavzzz/febwvKkaPHq3HHntMkyZNUmNjo/Lz81VXV6dzzjlHZWVlzt8BI0kLFy5UYWGhhg0bpoiICI0aNUpz5sxx1hMTE7Vs2TIVFBTorLPO0lFHHaWioiI+Qg0AACT9goA577zzZIw54LrL5VJxcbGKi4sPuCc5Odn5S+sO5LTTTtNbb711uOMBwH7xlwq2Hn+xIGzQYT6FBAAA/v9x2HdgAAD4X3AXCaHAHRgAAGAdAgYAAFiHgAEAANYhYAAAgHV4Ey8AAL8Ab0YOL+7AAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6BAwAALAOAQMAAKxDwAAAAOsQMAAAwDoEDAAAsA4BAwAArEPAAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6BAwAALAOAQMAAKwTFe4BAABA+PSa/EqbPv9n03Pa5Hm5AwMAAKxDwAAAAOsQMAAAwDoEDAAAsA4BAwAArEPAAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6BAwAALAOAQMAAKxDwAAAAOsQMAAAwDoEDAAAsA4BAwAArEPAAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6IQ+YvXv3atq0acrIyFBcXJyOP/543XHHHTLGOHuMMSoqKlJqaqri4uLk8/m0devWoOfZsWOH8vLylJCQoKSkJI0ZM0Y7d+4M9bgAAMBCIQ+Ye+65R/Pnz9f999+vzZs365577tGMGTM0d+5cZ8+MGTM0Z84clZaWqrKyUl26dFF2drZ2797t7MnLy9PGjRu1fPlyLV26VCtXrlR+fn6oxwUAABaKCvUTrlq1SpdccolycnIkSb169dJTTz2l1atXS/rh7svs2bM1depUXXLJJZKkxx9/XG63Wy+88IJyc3O1efNmlZWVac2aNerXr58kae7cubrooos0c+ZMpaWlhXpsAABgkZDfgRk4cKDKy8v14YcfSpLee+89vf322xo+fLgk6dNPP5Xf75fP53O+JjExUVlZWaqoqJAkVVRUKCkpyYkXSfL5fIqIiFBlZeV+X7epqUkNDQ1BBwAA6JhCfgdm8uTJamho0Mknn6zIyEjt3btXd911l/Ly8iRJfr9fkuR2u4O+zu12O2t+v18pKSnBg0ZFKTk52dnzUyUlJbr99ttDfTkAAKAdCvkdmCVLlmjhwoVatGiR1q1bpwULFmjmzJlasGBBqF8qyJQpU1RfX+8c1dXVbfp6AAAgfEJ+B2bixImaPHmycnNzJUl9+vTR559/rpKSEo0ePVoej0eSVFNTo9TUVOframpq1LdvX0mSx+NRbW1t0PPu2bNHO3bscL7+p2JiYhQTExPqywEAAO1QyO/A7Nq1SxERwU8bGRmpQCAgScrIyJDH41F5ebmz3tDQoMrKSnm9XkmS1+tVXV2dqqqqnD1vvvmmAoGAsrKyQj0yAACwTMjvwIwYMUJ33XWXevTooVNOOUX//ve/NWvWLF133XWSJJfLpXHjxunOO+/UiSeeqIyMDE2bNk1paWkaOXKkJKl379668MILNXbsWJWWlqqlpUWFhYXKzc3lE0gAACD0ATN37lxNmzZNN954o2pra5WWlqY//OEPKioqcvZMmjRJjY2Nys/PV11dnc455xyVlZUpNjbW2bNw4UIVFhZq2LBhioiI0KhRozRnzpxQjwsAACwU8oCJj4/X7NmzNXv27APucblcKi4uVnFx8QH3JCcna9GiRaEeDwAAdAB8LyQAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANZpk4D58ssvdeWVV6p79+6Ki4tTnz59tHbtWmfdGKOioiKlpqYqLi5OPp9PW7duDXqOHTt2KC8vTwkJCUpKStKYMWO0c+fOthgXAABYJuQB8+2332rQoEHq1KmTXnvtNW3atEl//etf1a1bN2fPjBkzNGfOHJWWlqqyslJdunRRdna2du/e7ezJy8vTxo0btXz5ci1dulQrV65Ufn5+qMcFAAAWigr1E95zzz1KT0/Xo48+6pzLyMhw/tkYo9mzZ2vq1Km65JJLJEmPP/643G63XnjhBeXm5mrz5s0qKyvTmjVr1K9fP0nS3LlzddFFF2nmzJlKS0sL9dgAAMAiIb8D89JLL6lfv3767W9/q5SUFJ1xxhl66KGHnPVPP/1Ufr9fPp/POZeYmKisrCxVVFRIkioqKpSUlOTEiyT5fD5FRESosrJyv6/b1NSkhoaGoAMAAHRMIQ+YTz75RPPnz9eJJ56o119/XTfccINuvvlmLViwQJLk9/slSW63O+jr3G63s+b3+5WSkhK0HhUVpeTkZGfPT5WUlCgxMdE50tPTQ31pAACgnQh5wAQCAZ155pm6++67dcYZZyg/P19jx45VaWlpqF8qyJQpU1RfX+8c1dXVbfp6AAAgfEIeMKmpqcrMzAw617t3b23btk2S5PF4JEk1NTVBe2pqapw1j8ej2traoPU9e/Zox44dzp6fiomJUUJCQtABAAA6ppAHzKBBg7Rly5agcx9++KF69uwp6Yc39Ho8HpWXlzvrDQ0NqqyslNfrlSR5vV7V1dWpqqrK2fPmm28qEAgoKysr1CMDAADLhPxTSOPHj9fAgQN1991367LLLtPq1av14IMP6sEHH5QkuVwujRs3TnfeeadOPPFEZWRkaNq0aUpLS9PIkSMl/XDH5sILL3T+6KmlpUWFhYXKzc3lE0gAACD0AdO/f389//zzmjJlioqLi5WRkaHZs2crLy/P2TNp0iQ1NjYqPz9fdXV1Ouecc1RWVqbY2Fhnz8KFC1VYWKhhw4YpIiJCo0aN0pw5c0I9LgAAsFDIA0aSLr74Yl188cUHXHe5XCouLlZxcfEB9yQnJ2vRokVtMR4AALAc3wsJAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB12jxgpk+fLpfLpXHjxjnndu/erYKCAnXv3l1du3bVqFGjVFNTE/R127ZtU05Ojjp37qyUlBRNnDhRe/bsaetxAQCABdo0YNasWaO//e1vOu2004LOjx8/Xi+//LKeeeYZrVixQtu3b9ell17qrO/du1c5OTlqbm7WqlWrtGDBAj322GMqKipqy3EBAIAl2ixgdu7cqby8PD300EPq1q2bc76+vl4PP/ywZs2apaFDh+qss87So48+qlWrVundd9+VJC1btkybNm3Sk08+qb59+2r48OG64447NG/ePDU3N7fVyAAAwBJtFjAFBQXKycmRz+cLOl9VVaWWlpag8yeffLJ69OihiooKSVJFRYX69Okjt9vt7MnOzlZDQ4M2bty439drampSQ0ND0AEAADqmqLZ40qefflrr1q3TmjVrfrbm9/sVHR2tpKSkoPNut1t+v9/Z8+N42be+b21/SkpKdPvtt4dgegAA0N6F/A5MdXW1brnlFi1cuFCxsbGhfvoDmjJliurr652jurr6iL02AAA4skIeMFVVVaqtrdWZZ56pqKgoRUVFacWKFZozZ46ioqLkdrvV3Nysurq6oK+rqamRx+ORJHk8np99Kmnf4317fiomJkYJCQlBBwAA6JhCHjDDhg3Thg0btH79eufo16+f8vLynH/u1KmTysvLna/ZsmWLtm3bJq/XK0nyer3asGGDamtrnT3Lly9XQkKCMjMzQz0yAACwTMjfAxMfH69TTz016FyXLl3UvXt35/yYMWM0YcIEJScnKyEhQTfddJO8Xq8GDBggSbrggguUmZmpq666SjNmzJDf79fUqVNVUFCgmJiYUI8MAAAs0yZv4j2Ue++9VxERERo1apSampqUnZ2tBx54wFmPjIzU0qVLdcMNN8jr9apLly4aPXq0iouLwzEuAABoZ45IwPzrX/8KehwbG6t58+Zp3rx5B/yanj176tVXX23jyQAAgI34XkgAAMA6BAwAALAOAQMAAKxDwAAAAOsQMAAAwDoEDAAAsA4BAwAArEPAAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6BAwAALAOAQMAAKxDwAAAAOsQMAAAwDoEDAAAsA4BAwAArEPAAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6BAwAALAOAQMAAKxDwAAAAOsQMAAAwDoEDAAAsA4BAwAArEPAAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6BAwAALAOAQMAAKxDwAAAAOsQMAAAwDoEDAAAsA4BAwAArEPAAAAA6xAwAADAOgQMAACwDgEDAACsQ8AAAADrEDAAAMA6BAwAALAOAQMAAKwT8oApKSlR//79FR8fr5SUFI0cOVJbtmwJ2rN7924VFBSoe/fu6tq1q0aNGqWampqgPdu2bVNOTo46d+6slJQUTZw4UXv27An1uAAAwEIhD5gVK1aooKBA7777rpYvX66WlhZdcMEFamxsdPaMHz9eL7/8sp555hmtWLFC27dv16WXXuqs7927Vzk5OWpubtaqVau0YMECPfbYYyoqKgr1uAAAwEJRoX7CsrKyoMePPfaYUlJSVFVVpcGDB6u+vl4PP/ywFi1apKFDh0qSHn30UfXu3VvvvvuuBgwYoGXLlmnTpk1644035Ha71bdvX91xxx3685//rL/85S+Kjo4O9dgAAMAibf4emPr6eklScnKyJKmqqkotLS3y+XzOnpNPPlk9evRQRUWFJKmiokJ9+vSR2+129mRnZ6uhoUEbN27c7+s0NTWpoaEh6AAAAB1TmwZMIBDQuHHjNGjQIJ166qmSJL/fr+joaCUlJQXtdbvd8vv9zp4fx8u+9X1r+1NSUqLExETnSE9PD/HVAACA9qJNA6agoEAffPCBnn766bZ8GUnSlClTVF9f7xzV1dVt/poAACA8Qv4emH0KCwu1dOlSrVy5Uscee6xz3uPxqLm5WXV1dUF3YWpqauTxeJw9q1evDnq+fZ9S2rfnp2JiYhQTExPiqwAAAO1RyO/AGGNUWFio559/Xm+++aYyMjKC1s866yx16tRJ5eXlzrktW7Zo27Zt8nq9kiSv16sNGzaotrbW2bN8+XIlJCQoMzMz1CMDAADLhPwOTEFBgRYtWqQXX3xR8fHxzntWEhMTFRcXp8TERI0ZM0YTJkxQcnKyEhISdNNNN8nr9WrAgAGSpAsuuECZmZm66qqrNGPGDPn9fk2dOlUFBQXcZQEAAKEPmPnz50uSzjvvvKDzjz76qK655hpJ0r333quIiAiNGjVKTU1Nys7O1gMPPODsjYyM1NKlS3XDDTfI6/WqS5cuGj16tIqLi0M9LgAAsFDIA8YYc8g9sbGxmjdvnubNm3fAPT179tSrr74aytEAAEAHwfdCAgAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADWIWAAAIB1CBgAAGAdAgYAAFiHgAEAANYhYAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgnXYdMPPmzVOvXr0UGxurrKwsrV69OtwjAQCAdqDdBszixYs1YcIE3XbbbVq3bp1OP/10ZWdnq7a2NtyjAQCAMGu3ATNr1iyNHTtW1157rTIzM1VaWqrOnTvrkUceCfdoAAAgzKLCPcD+NDc3q6qqSlOmTHHORUREyOfzqaKiYr9f09TUpKamJudxfX29JKmhoaHVrxto2vULJ269w5nnl+oI19ERrkHiOlqrI1yDxHW0Vke4BonraK3DvYZ9+40xB99o2qEvv/zSSDKrVq0KOj9x4kRz9tln7/drbrvtNiOJg4ODg4ODowMc1dXVB22FdnkH5peYMmWKJkyY4DwOBALasWOHunfvLpfL1Sav2dDQoPT0dFVXVyshIaFNXqOtdYRrkDrGdXSEa5C4jvakI1yD1DGuoyNcg3RkrsMYo++++05paWkH3dcuA+aoo45SZGSkampqgs7X1NTI4/Hs92tiYmIUExMTdC4pKamtRgySkJBg9X+QUse4BqljXEdHuAaJ62hPOsI1SB3jOjrCNUhtfx2JiYmH3NMu38QbHR2ts846S+Xl5c65QCCg8vJyeb3eME4GAADag3Z5B0aSJkyYoNGjR6tfv346++yzNXv2bDU2Nuraa68N92gAACDM2m3A/O53v9PXX3+toqIi+f1+9e3bV2VlZXK73eEezRETE6PbbrvtZ390ZZOOcA1Sx7iOjnANEtfRnnSEa5A6xnV0hGuQ2td1uIw51OeUAAAA2pd2+R4YAACAgyFgAACAdQgYAABgHQIGAABYh4ABgP3g8w1A+9ZuP0YNwA5fffWV5s+fr7fffltfffWVIiIidNxxx2nkyJG65pprFBkZGe4Rf5GYmBi999576t27d7hHabVvvvlGjzzyiCoqKuT3+yVJHo9HAwcO1DXXXKOjjz46zBMCocPHqEOkurpat912mx555JFwj3JQ33//vaqqqpScnKzMzMygtd27d2vJkiW6+uqrwzRd69x000267LLLdO6554Z7lP/Z5s2b9e6778rr9erkk0/Wf/7zH913331qamrSlVdeqaFDh4Z7xINau3atfD6fTjjhBMXFxamiokJXXHGFmpub9frrryszM1NlZWWKj48P96gH9OPvofZj9913n6688kp1795dkjRr1qwjOdZhW7NmjbKzs9W5c2f5fD7n78yqqalReXm5du3apddff139+vUL86SHdv/992v16tW66KKLlJubqyeeeEIlJSUKBAK69NJLVVxcrKio9vv773Xr1qlbt27KyMiQJD3xxBMqLS3Vtm3b1LNnTxUWFio3NzfMUx6+xsZGLVmyRB999JFSU1N1+eWXOz8/wiIE3zwaxpj169ebiIiIcI9xUFu2bDE9e/Y0LpfLREREmMGDB5vt27c7636/v91fgzHGmf/EE08006dPN1999VW4R/pFXnvtNRMdHW2Sk5NNbGysee2118zRRx9tfD6fGTp0qImMjDTl5eXhHvOgBg0aZP7yl784j5944gmTlZVljDFmx44dpm/fvubmm28O13it4nK5TN++fc15550XdLhcLtO/f39z3nnnmfPPPz/cYx5SVlaWyc/PN4FA4GdrgUDA5OfnmwEDBoRhssNzxx13mPj4eDNq1Cjj8XjM9OnTTffu3c2dd95p7r77bnP00UeboqKicI95UKeddppZvny5McaYhx56yMTFxZmbb77ZzJ8/34wbN8507drVPPzww2Ge8tB69+5t/vvf/xpjjNm2bZvp1auXSUxMNP379zfJyckmJSXFfPLJJ2Gbj4BppRdffPGgx7333tvuf/EfOXKkycnJMV9//bXZunWrycnJMRkZGebzzz83xtgVMG+88Ya55ZZbzFFHHWU6depkfvOb35iXX37Z7N27N9zjtZrX6zX/93//Z4wx5qmnnjLdunUzt956q7M+efJk8+tf/zpc47VKXFyc+fjjj53He/fuNZ06dTJ+v98YY8yyZctMWlpauMZrlZKSEpORkfGzWIyKijIbN24M01SHLzY21mzevPmA65s3bzaxsbFHcKJf5vjjjzf/+Mc/jDE//MYwMjLSPPnkk876c889Z0444YRwjdcqcXFx5rPPPjPGGHPGGWeYBx98MGh94cKFJjMzMxyjHRaXy2VqamqMMcbk5eWZgQMHmrq6OmOMMd99953x+Xzm8ssvD9t8BEwr7ftdv8vlOuDR3n/xT0lJMe+//77zOBAImOuvv9706NHDfPzxx1YFzL6fVM3NzWbx4sUmOzvbREZGmrS0NHPrrbearVu3hnnKQ0tISHDm3Lt3r4mKijLr1q1z1jds2GDcbne4xmuVnj17mrffftt5vH37duNyucyuXbuMMcZ8+umnVvyiuXr1avOrX/3K/PGPfzTNzc3GGPsCplevXmbBggUHXF+wYIHp2bPnkRvoF4qLi3N+U2WMMZ06dTIffPCB8/izzz4znTt3Dsdorda9e3ezdu1aY8wP/99dv3590PpHH31k4uLiwjHaYfnx/2uPO+44s2zZsqD1d955x6Snp4djNGOMMXwKqZVSU1P13HPPKRAI7PdYt25duEc8pO+//z7oz41dLpfmz5+vESNGaMiQIfrwww/DON0v06lTJ1122WUqKyvTJ598orFjx2rhwoU66aSTwj1aq7hcLklSRESEYmNjg76FfHx8vOrr68M1WquMHDlS119/vcrKyvTPf/5TeXl5GjJkiOLi4iRJW7Zs0THHHBPmKQ+tf//+qqqq0tdff61+/frpgw8+cH5sbPGnP/1J+fn5uuWWW/TSSy+psrJSlZWVeumll3TLLbfo+uuv16RJk8I95iF5PB5t2rRJkrR161bt3bvXeSxJGzduVEpKSrjGa5Xhw4dr/vz5kqQhQ4bo2WefDVpfsmSJTjjhhHCMdtj2/TzYvXu3UlNTg9aOOeYYff311+EY6wdhSyfLjBgxwkybNu2A6+vXrzcul+sITnT4+vfvbx5//PH9rhUUFJikpCTr7sDsTyAQ+NnvFNqj0047zbz22mvO4w0bNpiWlhbn8cqVK01GRkY4Rmu17777zlx22WUmKirKuFwuM3DgwKA/E3/99dfNkiVLwjjh4XvqqaeM2+02ERERVt2BMcaYp59+2mRlZTk/Hi6Xy0RFRZmsrCyzePHicI/XKlOnTjVHH320+f3vf28yMjLM5MmTTY8ePcz8+fNNaWmpSU9PN+PHjw/3mAf15Zdfml69epnBgwebCRMmmLi4OHPOOeeYsWPHmsGDB5vo6GjzyiuvhHvMQ3K5XKZPnz7mjDPOMF27djXPPvts0PqKFSvMMcccE6bpjOFTSK301ltvqbGxURdeeOF+1xsbG7V27VoNGTLkCE/WeiUlJXrrrbf06quv7nf9xhtvVGlpqQKBwBGe7PBkZGRo7dq14X33ewiUlpYqPT1dOTk5+12/9dZbVVtbq7///e9HeLLDt3v3bu3Zs0ddu3YN9ygh8cUXX6iqqko+n09dunQJ9ziHraWlRd98840k6aijjlKnTp3CPFHrBQIBTZ8+XRUVFRo4cKAmT56sxYsXa9KkSdq1a5dGjBih+++/v93/uNTV1Wn69Ol6+eWX9cknnygQCCg1NVWDBg3S+PHjrfg02O233x70eMCAAcrOznYeT5w4UV988YWeeuqpIz2aJD5GDQAALMR7YAAAgHUIGAAAYB0CBgAAWIeAAQAA1iFgAACAdQgYAABgHQIGAABYh4ABAADW+X8/S+PVQVCRdwAAAABJRU5ErkJggg==", 777 | "text/plain": [ 778 | "
" 779 | ] 780 | }, 781 | "metadata": {}, 782 | "output_type": "display_data" 783 | } 784 | ], 785 | "source": [ 786 | "import pandas as pd\n", 787 | "\n", 788 | "# KL is better for imbalanced data, which this isn't so much\n", 789 | "pd.Series(test_dataset.train_labels).value_counts().plot(kind='bar')" 790 | ] 791 | }, 792 | { 793 | "cell_type": "code", 794 | "execution_count": 46, 795 | "id": "00bad6b3-bb53-4cf2-a3c4-a1431a7c46c3", 796 | "metadata": {}, 797 | "outputs": [ 798 | { 799 | "data": { 800 | "text/plain": [ 801 | "Dataset MNIST\n", 802 | " Number of datapoints: 10000\n", 803 | " Root location: ../data\n", 804 | " Split: Test\n", 805 | " StandardTransform\n", 806 | "Transform: Compose(\n", 807 | " ToTensor()\n", 808 | " Normalize(mean=(0.1307,), std=(0.3081,))\n", 809 | " )" 810 | ] 811 | }, 812 | "execution_count": 46, 813 | "metadata": {}, 814 | "output_type": "execute_result" 815 | } 816 | ], 817 | "source": [ 818 | "test_dataset" 819 | ] 820 | }, 821 | { 822 | "cell_type": "code", 823 | "execution_count": 47, 824 | "id": "39a4155c-8fda-44e1-9064-4dca92cd74b7", 825 | "metadata": {}, 826 | "outputs": [ 827 | { 828 | "data": { 829 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbKUlEQVR4nO3df3DU9b3v8dcCyQqYbAwh2UQCBvxBFUinFNJclMaSS4hnGFDOHVBvBxwvXGlwhNTqiaMgbeemxTno0UPxnxbqGQHLuQJHTi8djSaMbYKHKIfLtWZIJhYYklBzD9kQJATyuX9wXV1JwO+ym3eyPB8z3xmy+/3k+/br6pNvsvnG55xzAgBggA2zHgAAcH0iQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwMQI6wG+rre3VydPnlRKSop8Pp/1OAAAj5xz6uzsVE5OjoYN6/86Z9AF6OTJk8rNzbUeAwBwjY4fP65x48b1+/ygC1BKSook6W7dpxFKMp4GAODVBfXoff0+/P/z/sQtQJs2bdILL7yg1tZW5efn65VXXtHMmTOvuu6LL7uNUJJG+AgQAAw5//8Oo1f7Nkpc3oTwxhtvqLy8XOvWrdOHH36o/Px8lZSU6NSpU/E4HABgCIpLgDZu3Kjly5frkUce0Z133qlXX31Vo0aN0m9+85t4HA4AMATFPEDnz59XfX29iouLvzzIsGEqLi5WbW3tZft3d3crFApFbACAxBfzAH322We6ePGisrKyIh7PyspSa2vrZftXVlYqEAiEN94BBwDXB/MfRK2oqFBHR0d4O378uPVIAIABEPN3wWVkZGj48OFqa2uLeLytrU3BYPCy/f1+v/x+f6zHAAAMcjG/AkpOTtb06dNVVVUVfqy3t1dVVVUqLCyM9eEAAENUXH4OqLy8XEuXLtV3v/tdzZw5Uy+99JK6urr0yCOPxONwAIAhKC4BWrx4sf76179q7dq1am1t1be//W3t27fvsjcmAACuXz7nnLMe4qtCoZACgYCKtIA7IQDAEHTB9ahae9TR0aHU1NR+9zN/FxwA4PpEgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMxDxAzz//vHw+X8Q2efLkWB8GADDEjYjHJ73rrrv0zjvvfHmQEXE5DABgCItLGUaMGKFgMBiPTw0ASBBx+R7Q0aNHlZOTo4kTJ+rhhx/WsWPH+t23u7tboVAoYgMAJL6YB6igoEBbt27Vvn37tHnzZjU3N+uee+5RZ2dnn/tXVlYqEAiEt9zc3FiPBAAYhHzOORfPA5w+fVoTJkzQxo0b9eijj172fHd3t7q7u8Mfh0Ih5ebmqkgLNMKXFM/RAABxcMH1qFp71NHRodTU1H73i/u7A9LS0nT77bersbGxz+f9fr/8fn+8xwAADDJx/zmgM2fOqKmpSdnZ2fE+FABgCIl5gJ588knV1NTo008/1Z/+9Cfdf//9Gj58uB588MFYHwoAMITF/EtwJ06c0IMPPqj29naNHTtWd999t+rq6jR27NhYHwoAMITFPEA7duyI9acEACQg7gUHADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJiI+y+kw8BqX17oec34H/b9ywKv5pNTWZ7XnO/2/ltub97ufc2oE2c8r5Gk3kMfR7UOgHdcAQEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEd8NOME/9ZJvnNYtG/0d0B5sU3TLPirwv+fTC2agO9Q9/vTeqdRg4H5ya4HnN6L8PRHWsEVX1Ua3DN8MVEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABggpuRJpiXn1niec3aadH9PeSmPzvPa/7jWz7Pa5Knnfa8ZsOUNz2vkaQXsw94XvOvZ2/0vOZvRp3xvGYgfe7Oe15zoHu05zVFN/R4XqMo/h3duvi/ez+OpNurolqGb4grIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABDcjTTCj/9n7jRpH/3McBulH6gAd55VgUVTrfj7rFs9rUmsaPa/ZUHSr5zUDacTnvZ7XjD7c4nnNmP3/0/OaqclJnteM+tT7GsQfV0AAABMECABgwnOA9u/fr/nz5ysnJ0c+n0+7d++OeN45p7Vr1yo7O1sjR45UcXGxjh49Gqt5AQAJwnOAurq6lJ+fr02bNvX5/IYNG/Tyyy/r1Vdf1YEDBzR69GiVlJTo3Llz1zwsACBxeH4TQmlpqUpLS/t8zjmnl156Sc8++6wWLFggSXrttdeUlZWl3bt3a8kS77+tEwCQmGL6PaDm5ma1traquLg4/FggEFBBQYFqa2v7XNPd3a1QKBSxAQASX0wD1NraKknKysqKeDwrKyv83NdVVlYqEAiEt9zc3FiOBAAYpMzfBVdRUaGOjo7wdvz4ceuRAAADIKYBCgaDkqS2traIx9va2sLPfZ3f71dqamrEBgBIfDENUF5enoLBoKqqqsKPhUIhHThwQIWFhbE8FABgiPP8LrgzZ86osfHLW480Nzfr0KFDSk9P1/jx47V69Wr9/Oc/12233aa8vDw999xzysnJ0cKFC2M5NwBgiPMcoIMHD+ree+8Nf1xeXi5JWrp0qbZu3aqnnnpKXV1dWrFihU6fPq27775b+/bt0w033BC7qQEAQ57POeesh/iqUCikQCCgIi3QCB83EASGivb/5v3L7LXr/9Hzmo3/d7LnNfvnTvK8RpIutPT97l1c2QXXo2rtUUdHxxW/r2/+LjgAwPWJAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJjz/OgYAiW/EhFzPa/7xGe93tk7yDfe8Zuc/FHteM6al1vMaxB9XQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACW5GCuAyn6y52fOaGX6f5zX/5/znntekf3zW8xoMTlwBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmuBkpkMC6/2ZGVOs+/NsXo1jl97xi5RNPeF4z8k8feF6DwYkrIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABDcjBRLYsdLo/o55o8/7jUUfbP7PnteM2vfvntc4zyswWHEFBAAwQYAAACY8B2j//v2aP3++cnJy5PP5tHv37ojnly1bJp/PF7HNmzcvVvMCABKE5wB1dXUpPz9fmzZt6nefefPmqaWlJbxt3779moYEACQez29CKC0tVWlp6RX38fv9CgaDUQ8FAEh8cfkeUHV1tTIzM3XHHXdo5cqVam9v73ff7u5uhUKhiA0AkPhiHqB58+bptddeU1VVlX75y1+qpqZGpaWlunjxYp/7V1ZWKhAIhLfc3NxYjwQAGIRi/nNAS5YsCf956tSpmjZtmiZNmqTq6mrNmTPnsv0rKipUXl4e/jgUChEhALgOxP1t2BMnTlRGRoYaGxv7fN7v9ys1NTViAwAkvrgH6MSJE2pvb1d2dna8DwUAGEI8fwnuzJkzEVczzc3NOnTokNLT05Wenq7169dr0aJFCgaDampq0lNPPaVbb71VJSUlMR0cADC0eQ7QwYMHde+994Y//uL7N0uXLtXmzZt1+PBh/fa3v9Xp06eVk5OjuXPn6mc/+5n8fu/3lgIAJC7PASoqKpJz/d8O8A9/+MM1DQSgb8NSUjyv+eE970d1rFDvOc9rTv2PiZ7X+Lv/zfMaJA7uBQcAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATMf+V3ADi4+jzd3leszfjV1Eda8HRRZ7X+H/Pna3hDVdAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJbkYKGOj4r9/zvObw4pc9r2m60ON5jSSd+eU4z2v8aonqWLh+cQUEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjgZqTANRpxc47nNaufe8PzGr/P+3+uS/79h57XSNLY//VvUa0DvOAKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwc1Iga/wjfD+n0T+3hOe1/yXG9s9r3m9M9Pzmqznovs7Zm9UqwBvuAICAJggQAAAE54CVFlZqRkzZiglJUWZmZlauHChGhoaIvY5d+6cysrKNGbMGN14441atGiR2traYjo0AGDo8xSgmpoalZWVqa6uTm+//bZ6eno0d+5cdXV1hfdZs2aN3nrrLe3cuVM1NTU6efKkHnjggZgPDgAY2jx9x3Xfvn0RH2/dulWZmZmqr6/X7Nmz1dHRoV//+tfatm2bfvCDH0iStmzZom9961uqq6vT9773vdhNDgAY0q7pe0AdHR2SpPT0dElSfX29enp6VFxcHN5n8uTJGj9+vGpra/v8HN3d3QqFQhEbACDxRR2g3t5erV69WrNmzdKUKVMkSa2trUpOTlZaWlrEvllZWWptbe3z81RWVioQCIS33NzcaEcCAAwhUQeorKxMR44c0Y4dO65pgIqKCnV0dIS348ePX9PnAwAMDVH9IOqqVau0d+9e7d+/X+PGjQs/HgwGdf78eZ0+fTriKqitrU3BYLDPz+X3++X3+6MZAwAwhHm6AnLOadWqVdq1a5feffdd5eXlRTw/ffp0JSUlqaqqKvxYQ0ODjh07psLCwthMDABICJ6ugMrKyrRt2zbt2bNHKSkp4e/rBAIBjRw5UoFAQI8++qjKy8uVnp6u1NRUPf744yosLOQdcACACJ4CtHnzZklSUVFRxONbtmzRsmXLJEkvvviihg0bpkWLFqm7u1slJSX61a9+FZNhAQCJw+ecc9ZDfFUoFFIgEFCRFmiEL8l6HFxnfNPv8rzmX//ln+IwyeX+U0WZ5zVpr/X94w9APF1wParWHnV0dCg1NbXf/bgXHADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAExE9RtRgcFu+J23R7VuxY49MZ6kb3f+xvudrW/5p7o4TALY4QoIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDBzUiRkD750U1RrZs/KhTjSfo2rvq890XOxX4QwBBXQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACW5GikHv3PyZntdUzf/7KI82Ksp1ALziCggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMMHNSDHonZw13POa8SMG7qair3dmel6TFDrveY3zvAIY3LgCAgCYIEAAABOeAlRZWakZM2YoJSVFmZmZWrhwoRoaGiL2KSoqks/ni9gee+yxmA4NABj6PAWopqZGZWVlqqur09tvv62enh7NnTtXXV1dEfstX75cLS0t4W3Dhg0xHRoAMPR5ehPCvn37Ij7eunWrMjMzVV9fr9mzZ4cfHzVqlILBYGwmBAAkpGv6HlBHR4ckKT09PeLx119/XRkZGZoyZYoqKip09uzZfj9Hd3e3QqFQxAYASHxRvw27t7dXq1ev1qxZszRlypTw4w899JAmTJignJwcHT58WE8//bQaGhr05ptv9vl5KisrtX79+mjHAAAMUVEHqKysTEeOHNH7778f8fiKFSvCf546daqys7M1Z84cNTU1adKkSZd9noqKCpWXl4c/DoVCys3NjXYsAMAQEVWAVq1apb1792r//v0aN27cFfctKCiQJDU2NvYZIL/fL7/fH80YAIAhzFOAnHN6/PHHtWvXLlVXVysvL++qaw4dOiRJys7OjmpAAEBi8hSgsrIybdu2TXv27FFKSopaW1slSYFAQCNHjlRTU5O2bdum++67T2PGjNHhw4e1Zs0azZ49W9OmTYvLPwAAYGjyFKDNmzdLuvTDpl+1ZcsWLVu2TMnJyXrnnXf00ksvqaurS7m5uVq0aJGeffbZmA0MAEgMnr8EdyW5ubmqqam5poEAANcH7oYNfEVl+52e19SW3OJ5jWv5357XAImGm5ECAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACa4GSkGvYl/V+t5zX1/9504TNKf1gE8FpA4uAICAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgYtDdC845J0m6oB7JGQ8DAPDsgnokffn/8/4MugB1dnZKkt7X740nAQBci87OTgUCgX6f97mrJWqA9fb26uTJk0pJSZHP54t4LhQKKTc3V8ePH1dqaqrRhPY4D5dwHi7hPFzCebhkMJwH55w6OzuVk5OjYcP6/07PoLsCGjZsmMaNG3fFfVJTU6/rF9gXOA+XcB4u4Txcwnm4xPo8XOnK5wu8CQEAYIIAAQBMDKkA+f1+rVu3Tn6/33oUU5yHSzgPl3AeLuE8XDKUzsOgexMCAOD6MKSugAAAiYMAAQBMECAAgAkCBAAwMWQCtGnTJt1yyy264YYbVFBQoA8++MB6pAH3/PPPy+fzRWyTJ0+2Hivu9u/fr/nz5ysnJ0c+n0+7d++OeN45p7Vr1yo7O1sjR45UcXGxjh49ajNsHF3tPCxbtuyy18e8efNsho2TyspKzZgxQykpKcrMzNTChQvV0NAQsc+5c+dUVlamMWPG6MYbb9SiRYvU1tZmNHF8fJPzUFRUdNnr4bHHHjOauG9DIkBvvPGGysvLtW7dOn344YfKz89XSUmJTp06ZT3agLvrrrvU0tIS3t5//33rkeKuq6tL+fn52rRpU5/Pb9iwQS+//LJeffVVHThwQKNHj1ZJSYnOnTs3wJPG19XOgyTNmzcv4vWxffv2AZww/mpqalRWVqa6ujq9/fbb6unp0dy5c9XV1RXeZ82aNXrrrbe0c+dO1dTU6OTJk3rggQcMp469b3IeJGn58uURr4cNGzYYTdwPNwTMnDnTlZWVhT++ePGiy8nJcZWVlYZTDbx169a5/Px86zFMSXK7du0Kf9zb2+uCwaB74YUXwo+dPn3a+f1+t337doMJB8bXz4Nzzi1dutQtWLDAZB4rp06dcpJcTU2Nc+7Sv/ukpCS3c+fO8D5//vOfnSRXW1trNWbcff08OOfc97//fffEE0/YDfUNDPoroPPnz6u+vl7FxcXhx4YNG6bi4mLV1tYaTmbj6NGjysnJ0cSJE/Xwww/r2LFj1iOZam5uVmtra8TrIxAIqKCg4Lp8fVRXVyszM1N33HGHVq5cqfb2duuR4qqjo0OSlJ6eLkmqr69XT09PxOth8uTJGj9+fEK/Hr5+Hr7w+uuvKyMjQ1OmTFFFRYXOnj1rMV6/Bt3NSL/us88+08WLF5WVlRXxeFZWlj755BOjqWwUFBRo69atuuOOO9TS0qL169frnnvu0ZEjR5SSkmI9nonW1lZJ6vP18cVz14t58+bpgQceUF5enpqamvTMM8+otLRUtbW1Gj58uPV4Mdfb26vVq1dr1qxZmjJliqRLr4fk5GSlpaVF7JvIr4e+zoMkPfTQQ5owYYJycnJ0+PBhPf3002poaNCbb75pOG2kQR8gfKm0tDT852nTpqmgoEATJkzQ7373Oz366KOGk2EwWLJkSfjPU6dO1bRp0zRp0iRVV1drzpw5hpPFR1lZmY4cOXJdfB/0Svo7DytWrAj/eerUqcrOztacOXPU1NSkSZMmDfSYfRr0X4LLyMjQ8OHDL3sXS1tbm4LBoNFUg0NaWppuv/12NTY2Wo9i5ovXAK+Py02cOFEZGRkJ+fpYtWqV9u7dq/feey/i17cEg0GdP39ep0+fjtg/UV8P/Z2HvhQUFEjSoHo9DPoAJScna/r06aqqqgo/1tvbq6qqKhUWFhpOZu/MmTNqampSdna29Shm8vLyFAwGI14foVBIBw4cuO5fHydOnFB7e3tCvT6cc1q1apV27dqld999V3l5eRHPT58+XUlJSRGvh4aGBh07diyhXg9XOw99OXTokCQNrteD9bsgvokdO3Y4v9/vtm7d6j7++GO3YsUKl5aW5lpbW61HG1A//vGPXXV1tWtubnZ//OMfXXFxscvIyHCnTp2yHi2uOjs73UcffeQ++ugjJ8lt3LjRffTRR+4vf/mLc865X/ziFy4tLc3t2bPHHT582C1YsMDl5eW5zz//3Hjy2LrSeejs7HRPPvmkq62tdc3Nze6dd95x3/nOd9xtt93mzp07Zz16zKxcudIFAgFXXV3tWlpawtvZs2fD+zz22GNu/Pjx7t1333UHDx50hYWFrrCw0HDq2LvaeWhsbHQ//elP3cGDB11zc7Pbs2ePmzhxops9e7bx5JGGRICcc+6VV15x48ePd8nJyW7mzJmurq7OeqQBt3jxYpedne2Sk5PdzTff7BYvXuwaGxutx4q79957z0m6bFu6dKlz7tJbsZ977jmXlZXl/H6/mzNnjmtoaLAdOg6udB7Onj3r5s6d68aOHeuSkpLchAkT3PLlyxPuL2l9/fNLclu2bAnv8/nnn7sf/ehH7qabbnKjRo1y999/v2tpabEbOg6udh6OHTvmZs+e7dLT053f73e33nqr+8lPfuI6OjpsB/8afh0DAMDEoP8eEAAgMREgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJv4fx1BnJzDsp98AAAAASUVORK5CYII=", 830 | "text/plain": [ 831 | "
" 832 | ] 833 | }, 834 | "metadata": {}, 835 | "output_type": "display_data" 836 | } 837 | ], 838 | "source": [ 839 | "# transpose the array to (28, 28) format expected by Matplotlib\n", 840 | "array = np.squeeze(np.transpose(test_dataset[0][0], (1, 2, 0)))\n", 841 | "\n", 842 | "# plot the image using Matplotlib\n", 843 | "plt.imshow(array)\n", 844 | "plt.show()" 845 | ] 846 | }, 847 | { 848 | "cell_type": "code", 849 | "execution_count": 48, 850 | "id": "c6b6d5fc-a407-4704-8a0d-bce5060e8eb1", 851 | "metadata": {}, 852 | "outputs": [ 853 | { 854 | "data": { 855 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAac0lEQVR4nO3dfWxU973n8c8A9gQSe6gx9niCoYYQSENwbym4viSUFC/G0UU83Yo8dBeiCAQx2QJNk3WVhKSt5IZIaZTUBe1uC41uIA9SgA1KqYiJjdLadHFgEdvUF1tuMYttGrSeMSYYg3/7B5tpJtjQY2b89Zj3SzoSnjk/n29ORnnneIZjn3POCQCAATbMegAAwM2JAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMjrAf4sp6eHp0+fVppaWny+XzW4wAAPHLOqaOjQ6FQSMOG9X2dM+gCdPr0aeXm5lqPAQC4Qc3NzRo3blyfzw+6AKWlpUmS7tUDGqEU42kAAF5dUrc+0vvR/573JWEBqqio0EsvvaTW1lbl5+frtdde06xZs6677vMfu41Qikb4CBAAJJ3/f4fR672NkpAPIbz11lvauHGjNm3apI8//lj5+fkqLi7WmTNnEnE4AEASSkiAXn75Za1atUqPPvqovva1r2nr1q0aNWqUfv3rXyficACAJBT3AF28eFF1dXUqKir6+0GGDVNRUZFqamqu2r+rq0uRSCRmAwAMfXEP0KeffqrLly8rOzs75vHs7Gy1trZetX95ebkCgUB04xNwAHBzMP+LqGVlZQqHw9GtubnZeiQAwACI+6fgMjMzNXz4cLW1tcU83tbWpmAweNX+fr9ffr8/3mMAAAa5uF8BpaamasaMGaqsrIw+1tPTo8rKShUWFsb7cACAJJWQvwe0ceNGrVixQt/85jc1a9YsvfLKK+rs7NSjjz6aiMMBAJJQQgK0fPly/e1vf9Nzzz2n1tZWff3rX9e+ffuu+mACAODm5XPOOeshvigSiSgQCGiuFnEnBABIQpdct6q0R+FwWOnp6X3uZ/4pOADAzYkAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwMcJ6AACJ0/6fCvu17tDPtnhe87WKxz2vGf/iHz2vcZcueV6DwYkrIACACQIEADAR9wA9//zz8vl8MdvUqVPjfRgAQJJLyHtAd999tz744IO/H2QEbzUBAGIlpAwjRoxQMBhMxLcGAAwRCXkP6MSJEwqFQpo4caIeeeQRnTx5ss99u7q6FIlEYjYAwNAX9wAVFBRo+/bt2rdvn7Zs2aKmpibdd9996ujo6HX/8vJyBQKB6JabmxvvkQAAg1DcA1RSUqLvfve7mj59uoqLi/X++++rvb1db7/9dq/7l5WVKRwOR7fm5uZ4jwQAGIQS/umA0aNH684771RDQ0Ovz/v9fvn9/kSPAQAYZBL+94DOnTunxsZG5eTkJPpQAIAkEvcAPfnkk6qurtZf/vIX/eEPf9CSJUs0fPhwPfTQQ/E+FAAgicX9R3CnTp3SQw89pLNnz2rs2LG69957VVtbq7Fjx8b7UACAJOZzzjnrIb4oEokoEAhorhZphC/Fehxg0Bhxe8jzmu9X7+/XseaP6u7XOq9KptzneU1PH5+oxeBxyXWrSnsUDoeVnp7e537cCw4AYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMJHwX0gHID7OFE/wvGagbioqSd84vNzzmrHn/j0BkyBZcAUEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAE9wNGzAwbNQoz2uK//NHCZgkfvxvfsX7IufiPwiSBldAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJbkYKGOj657s8r/lp1q8SMEnvzvdc9LwmfUdtAibBUMYVEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABggpuRAgaalg63HuGa/vXE4n6sOh3vMTDEcQUEADBBgAAAJjwH6ODBg1q4cKFCoZB8Pp92794d87xzTs8995xycnI0cuRIFRUV6cSJE/GaFwAwRHgOUGdnp/Lz81VRUdHr85s3b9arr76qrVu36tChQ7r11ltVXFysCxcu3PCwAIChw/OHEEpKSlRSUtLrc845vfLKK3rmmWe0aNEiSdLrr7+u7Oxs7d69Ww8++OCNTQsAGDLi+h5QU1OTWltbVVRUFH0sEAiooKBANTU1va7p6upSJBKJ2QAAQ19cA9Ta2ipJys7Ojnk8Ozs7+tyXlZeXKxAIRLfc3Nx4jgQAGKTMPwVXVlamcDgc3Zqbm61HAgAMgLgGKBgMSpLa2tpiHm9ra4s+92V+v1/p6ekxGwBg6ItrgPLy8hQMBlVZWRl9LBKJ6NChQyosLIznoQAASc7zp+DOnTunhoaG6NdNTU06evSoMjIyNH78eK1fv14//elPNXnyZOXl5enZZ59VKBTS4sWL4zk3ACDJeQ7Q4cOHdf/990e/3rhxoyRpxYoV2r59u5566il1dnZq9erVam9v17333qt9+/bplltuid/UAICk53POOeshvigSiSgQCGiuFmmEL8V6HCAhJv9Pv+c1v7j9kOc14Z7PPK+RpH995HHPa4ZVH+nXsTD0XHLdqtIehcPha76vb/4pOADAzYkAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmPP86BgCxuh6Y6XnNL27/bwmY5GqnLvVvHXe2xkDgCggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMMHNSIEb1DYzxXqEPi3cu75f6ybrUHwHAXrBFRAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIKbkQI3KPWf/u+AHOeTi+c9r5n66qf9Otblfq0CvOEKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwc1IgS+48C+zPK85PHNLP4403POK+u4sz2su/3uj5zXAQOEKCABgggABAEx4DtDBgwe1cOFChUIh+Xw+7d69O+b5lStXyufzxWwLFiyI17wAgCHCc4A6OzuVn5+vioqKPvdZsGCBWlpaotvOnTtvaEgAwNDj+UMIJSUlKikpueY+fr9fwWCw30MBAIa+hLwHVFVVpaysLE2ZMkVr167V2bNn+9y3q6tLkUgkZgMADH1xD9CCBQv0+uuvq7KyUi+++KKqq6tVUlKiy5d7/y3z5eXlCgQC0S03NzfeIwEABqG4/z2gBx98MPrne+65R9OnT9ekSZNUVVWlefPmXbV/WVmZNm7cGP06EokQIQC4CST8Y9gTJ05UZmamGhoaen3e7/crPT09ZgMADH0JD9CpU6d09uxZ5eTkJPpQAIAk4vlHcOfOnYu5mmlqatLRo0eVkZGhjIwMvfDCC1q2bJmCwaAaGxv11FNP6Y477lBxcXFcBwcAJDfPATp8+LDuv//+6Nefv3+zYsUKbdmyRceOHdNvfvMbtbe3KxQKaf78+frJT34iv98fv6kBAEnPc4Dmzp0r51yfz//ud7+7oYEAS59ler9JaIrP+5r+eKpuqec1eTqWgEmA+OBecAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADAR91/JDSSzrsXtA3KcTy6e97xm3H9PScAkgB2ugAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAE9yMFEPS8Dsn9Wvd4Zn/1p+jeV7x23PTPK9J+aDO8xpgMOMKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwc1IMSS13Z/Vr3UpPu83Fu2PX3z4HzyvmaxDCZgEsMMVEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABggpuRYki6kOEbsGPVdV30vOauF095XnPJ8wpgcOMKCABgggABAEx4ClB5eblmzpyptLQ0ZWVlafHixaqvr4/Z58KFCyotLdWYMWN02223admyZWpra4vr0ACA5OcpQNXV1SotLVVtba3279+v7u5uzZ8/X52dndF9NmzYoPfee0/vvPOOqqurdfr0aS1dujTugwMAkpunDyHs27cv5uvt27crKytLdXV1mjNnjsLhsH71q19px44d+s53viNJ2rZtm+666y7V1tbqW9/6VvwmBwAktRt6DygcDkuSMjIyJEl1dXXq7u5WUVFRdJ+pU6dq/Pjxqqmp6fV7dHV1KRKJxGwAgKGv3wHq6enR+vXrNXv2bE2bNk2S1NraqtTUVI0ePTpm3+zsbLW2tvb6fcrLyxUIBKJbbm5uf0cCACSRfgeotLRUx48f15tvvnlDA5SVlSkcDke35ubmG/p+AIDk0K+/iLpu3Trt3btXBw8e1Lhx46KPB4NBXbx4Ue3t7TFXQW1tbQoGg71+L7/fL7/f358xAABJzNMVkHNO69at065du3TgwAHl5eXFPD9jxgylpKSosrIy+lh9fb1OnjypwsLC+EwMABgSPF0BlZaWaseOHdqzZ4/S0tKi7+sEAgGNHDlSgUBAjz32mDZu3KiMjAylp6friSeeUGFhIZ+AAwDE8BSgLVu2SJLmzp0b8/i2bdu0cuVKSdLPf/5zDRs2TMuWLVNXV5eKi4v1y1/+Mi7DAgCGDk8Bcs5dd59bbrlFFRUVqqio6PdQwI3K+s7/GbBj/Y/IP3lec/lvnyZgEiC5cC84AIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmOjXb0QFBpKvH78xd1HofyVgkt6dvXib5zWuqysBkwDJhSsgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAENyPF4Hf5sucl//WTe/t1qPX//BfPa6qa7/C85nb9b89rgKGGKyAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQ3I8Wg5y5d8rzmq/+ls1/Huqv8P3pe4zua1q9jATc7roAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABPcjBRD0uWGpn6tG//dOA8CoE9cAQEATBAgAIAJTwEqLy/XzJkzlZaWpqysLC1evFj19fUx+8ydO1c+ny9mW7NmTVyHBgAkP08Bqq6uVmlpqWpra7V//351d3dr/vz56uyM/eVfq1atUktLS3TbvHlzXIcGACQ/Tx9C2LdvX8zX27dvV1ZWlurq6jRnzpzo46NGjVIwGIzPhACAIemG3gMKh8OSpIyMjJjH33jjDWVmZmratGkqKyvT+fPn+/weXV1dikQiMRsAYOjr98ewe3p6tH79es2ePVvTpk2LPv7www9rwoQJCoVCOnbsmJ5++mnV19fr3Xff7fX7lJeX64UXXujvGACAJOVzzrn+LFy7dq1++9vf6qOPPtK4ceP63O/AgQOaN2+eGhoaNGnSpKue7+rqUldXV/TrSCSi3NxczdUijfCl9Gc0AIChS65bVdqjcDis9PT0Pvfr1xXQunXrtHfvXh08ePCa8ZGkgoICSeozQH6/X36/vz9jAACSmKcAOef0xBNPaNeuXaqqqlJeXt511xw9elSSlJOT068BAQBDk6cAlZaWaseOHdqzZ4/S0tLU2toqSQoEAho5cqQaGxu1Y8cOPfDAAxozZoyOHTumDRs2aM6cOZo+fXpC/gEAAMnJ03tAPp+v18e3bdumlStXqrm5Wd/73vd0/PhxdXZ2Kjc3V0uWLNEzzzxzzZ8DflEkElEgEOA9IABIUgl5D+h6rcrNzVV1dbWXbwkAuElxLzgAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgIkR1gN8mXNOknRJ3ZIzHgYA4NkldUv6+3/P+zLoAtTR0SFJ+kjvG08CALgRHR0dCgQCfT7vc9dL1ADr6enR6dOnlZaWJp/PF/NcJBJRbm6umpublZ6ebjShPc7DFZyHKzgPV3AerhgM58E5p46ODoVCIQ0b1vc7PYPuCmjYsGEaN27cNfdJT0+/qV9gn+M8XMF5uILzcAXn4Qrr83CtK5/P8SEEAIAJAgQAMJFUAfL7/dq0aZP8fr/1KKY4D1dwHq7gPFzBebgimc7DoPsQAgDg5pBUV0AAgKGDAAEATBAgAIAJAgQAMJE0AaqoqNBXv/pV3XLLLSooKNAf//hH65EG3PPPPy+fzxezTZ061XqshDt48KAWLlyoUCgkn8+n3bt3xzzvnNNzzz2nnJwcjRw5UkVFRTpx4oTNsAl0vfOwcuXKq14fCxYssBk2QcrLyzVz5kylpaUpKytLixcvVn19fcw+Fy5cUGlpqcaMGaPbbrtNy5YtU1tbm9HEifGPnIe5c+de9XpYs2aN0cS9S4oAvfXWW9q4caM2bdqkjz/+WPn5+SouLtaZM2esRxtwd999t1paWqLbRx99ZD1SwnV2dio/P18VFRW9Pr9582a9+uqr2rp1qw4dOqRbb71VxcXFunDhwgBPmljXOw+StGDBgpjXx86dOwdwwsSrrq5WaWmpamtrtX//fnV3d2v+/Pnq7OyM7rNhwwa99957euedd1RdXa3Tp09r6dKlhlPH3z9yHiRp1apVMa+HzZs3G03cB5cEZs2a5UpLS6NfX7582YVCIVdeXm441cDbtGmTy8/Ptx7DlCS3a9eu6Nc9PT0uGAy6l156KfpYe3u78/v9bufOnQYTDowvnwfnnFuxYoVbtGiRyTxWzpw54yS56upq59yVf/cpKSnunXfeie7zySefOEmupqbGasyE+/J5cM65b3/72+773/++3VD/gEF/BXTx4kXV1dWpqKgo+tiwYcNUVFSkmpoaw8lsnDhxQqFQSBMnTtQjjzyikydPWo9kqqmpSa2trTGvj0AgoIKCgpvy9VFVVaWsrCxNmTJFa9eu1dmzZ61HSqhwOCxJysjIkCTV1dWpu7s75vUwdepUjR8/fki/Hr58Hj73xhtvKDMzU9OmTVNZWZnOnz9vMV6fBt3NSL/s008/1eXLl5WdnR3zeHZ2tv785z8bTWWjoKBA27dv15QpU9TS0qIXXnhB9913n44fP660tDTr8Uy0trZKUq+vj8+fu1ksWLBAS5cuVV5enhobG/WjH/1IJSUlqqmp0fDhw63Hi7uenh6tX79es2fP1rRp0yRdeT2kpqZq9OjRMfsO5ddDb+dBkh5++GFNmDBBoVBIx44d09NPP636+nq9++67htPGGvQBwt+VlJRE/zx9+nQVFBRowoQJevvtt/XYY48ZTobB4MEHH4z++Z577tH06dM1adIkVVVVad68eYaTJUZpaamOHz9+U7wPei19nYfVq1dH/3zPPfcoJydH8+bNU2NjoyZNmjTQY/Zq0P8ILjMzU8OHD7/qUyxtbW0KBoNGUw0Oo0eP1p133qmGhgbrUcx8/hrg9XG1iRMnKjMzc0i+PtatW6e9e/fqww8/jPn1LcFgUBcvXlR7e3vM/kP19dDXeehNQUGBJA2q18OgD1BqaqpmzJihysrK6GM9PT2qrKxUYWGh4WT2zp07p8bGRuXk5FiPYiYvL0/BYDDm9RGJRHTo0KGb/vVx6tQpnT17dki9PpxzWrdunXbt2qUDBw4oLy8v5vkZM2YoJSUl5vVQX1+vkydPDqnXw/XOQ2+OHj0qSYPr9WD9KYh/xJtvvun8fr/bvn27+9Of/uRWr17tRo8e7VpbW61HG1A/+MEPXFVVlWtqanK///3vXVFRkcvMzHRnzpyxHi2hOjo63JEjR9yRI0ecJPfyyy+7I0eOuL/+9a/OOed+9rOfudGjR7s9e/a4Y8eOuUWLFrm8vDz32WefGU8eX9c6Dx0dHe7JJ590NTU1rqmpyX3wwQfuG9/4hps8ebK7cOGC9ehxs3btWhcIBFxVVZVraWmJbufPn4/us2bNGjd+/Hh34MABd/jwYVdYWOgKCwsNp46/652HhoYG9+Mf/9gdPnzYNTU1uT179riJEye6OXPmGE8eKykC5Jxzr732mhs/frxLTU11s2bNcrW1tdYjDbjly5e7nJwcl5qa6m6//Xa3fPly19DQYD1Wwn344YdO0lXbihUrnHNXPor97LPPuuzsbOf3+928efNcfX297dAJcK3zcP78eTd//nw3duxYl5KS4iZMmOBWrVo15P4nrbd/fklu27Zt0X0+++wz9/jjj7uvfOUrbtSoUW7JkiWupaXFbugEuN55OHnypJszZ47LyMhwfr/f3XHHHe6HP/yhC4fDtoN/Cb+OAQBgYtC/BwQAGJoIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABP/D7c1Y6KPV4dUAAAAAElFTkSuQmCC", 856 | "text/plain": [ 857 | "
" 858 | ] 859 | }, 860 | "metadata": {}, 861 | "output_type": "display_data" 862 | } 863 | ], 864 | "source": [ 865 | "# transpose the array to (28, 28) format expected by Matplotlib\n", 866 | "array = np.squeeze(np.transpose(test_dataset[5][0], (1, 2, 0)))\n", 867 | "\n", 868 | "# plot the image using Matplotlib\n", 869 | "plt.imshow(array)\n", 870 | "plt.show()" 871 | ] 872 | }, 873 | { 874 | "cell_type": "markdown", 875 | "id": "5b6ed347-5fd3-4749-861e-09cd9c532fdf", 876 | "metadata": { 877 | "jp-MarkdownHeadingCollapsed": true 878 | }, 879 | "source": [ 880 | "## Playing with Mean Squared Error Loss (MSE)" 881 | ] 882 | }, 883 | { 884 | "cell_type": "code", 885 | "execution_count": 49, 886 | "id": "9fe7de7b-a05b-463d-bbf1-b3b99d8e7ddc", 887 | "metadata": {}, 888 | "outputs": [ 889 | { 890 | "data": { 891 | "text/plain": [ 892 | "tensor(0.)" 893 | ] 894 | }, 895 | "execution_count": 49, 896 | "metadata": {}, 897 | "output_type": "execute_result" 898 | } 899 | ], 900 | "source": [ 901 | "# MSE loss of image to itself compares pixel values (perfect match for the same image)\n", 902 | "mse_loss(test_dataset[0][0], test_dataset[0][0])" 903 | ] 904 | }, 905 | { 906 | "cell_type": "code", 907 | "execution_count": 50, 908 | "id": "534493f7-b39e-44f5-9740-a0f6d99037d7", 909 | "metadata": {}, 910 | "outputs": [ 911 | { 912 | "data": { 913 | "text/plain": [ 914 | "tensor(1.0462)" 915 | ] 916 | }, 917 | "execution_count": 50, 918 | "metadata": {}, 919 | "output_type": "execute_result" 920 | } 921 | ], 922 | "source": [ 923 | "# MSE loss of image to itself compares pixel values (not a perfect match for different images)\n", 924 | "# keep this in mind for diffusion\n", 925 | "mse_loss(test_dataset[0][0], test_dataset[5][0])" 926 | ] 927 | }, 928 | { 929 | "cell_type": "code", 930 | "execution_count": null, 931 | "id": "bc77cc73-fef8-4cc0-bfd7-6d5e177d58ad", 932 | "metadata": {}, 933 | "outputs": [], 934 | "source": [] 935 | }, 936 | { 937 | "cell_type": "code", 938 | "execution_count": null, 939 | "id": "5e1b2e96-e61b-407c-8dc3-c9f4898a55dc", 940 | "metadata": {}, 941 | "outputs": [], 942 | "source": [] 943 | }, 944 | { 945 | "cell_type": "code", 946 | "execution_count": null, 947 | "id": "fcdc0cfd-ef75-4432-b675-9f0718520695", 948 | "metadata": {}, 949 | "outputs": [], 950 | "source": [] 951 | } 952 | ], 953 | "metadata": { 954 | "kernelspec": { 955 | "display_name": "Python (/usr/bin/python3)", 956 | "language": "python", 957 | "name": "my_python3" 958 | }, 959 | "language_info": { 960 | "codemirror_mode": { 961 | "name": "ipython", 962 | "version": 3 963 | }, 964 | "file_extension": ".py", 965 | "mimetype": "text/x-python", 966 | "name": "python", 967 | "nbconvert_exporter": "python", 968 | "pygments_lexer": "ipython3", 969 | "version": "3.9.6" 970 | } 971 | }, 972 | "nbformat": 4, 973 | "nbformat_minor": 5 974 | } 975 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | torch==2.0.0 2 | torchvision==0.15.1 3 | Flask==2.1.0 4 | Pillow==10.4.0 5 | transformers[torch]==4.44.0 6 | datasets==2.21.0 7 | jupyterlab 8 | ipykernel 9 | matplotlib 10 | scikit-learn==1.5.1 11 | onnx==1.16.2 12 | tiktoken 13 | wandb 14 | evaluate==0.4.2 15 | numpy<2 --------------------------------------------------------------------------------