├── Activation Functions ├── act_prelu.png ├── act_prelu_2hidden.png ├── act_relu.png ├── act_sigmoid1.png ├── act_sigmoid2.png ├── act_tanh.png ├── act_tanh_2hidden.png ├── activation_leaky.png ├── activation_prelu.png ├── activation_relu.png ├── activation_sigmoid.png ├── activation_tanh.png ├── eq_activ.png ├── eq_equiv.png ├── feature_space_1hidden.png ├── feature_space_1hidden_plates.png └── feature_space_2hidden.png ├── Architectures ├── alexnet.png ├── architecture_lenet.png ├── inception_model.png ├── inception_modules.png └── residual.png ├── Assorted ├── embed_arithmetic.png ├── image_channels.png ├── image_channels2.png ├── ngrams.png ├── ohe1.png ├── ohe2.png ├── ohe3.png ├── tensors.png ├── twodimensions.png ├── w2v_cbow.png ├── w2v_embed.png └── w2v_logits.png ├── Attention ├── aiayn_dot.png ├── aiayn_multihead.png ├── attention.png ├── attn_narrow_2heads.png ├── attn_narrow_first_head.png ├── context_translate.png ├── context_vector.png ├── cross_attn.png ├── dec_both.png ├── decoder_self.png ├── decoder_self_simplified.png ├── enc_both.png ├── enc_dec_attn_translate.png ├── encdec_attn.png ├── encdec_self_simplified.png ├── encoder_self.png ├── encoder_self_detail.png ├── encoder_self_simplified.png ├── eq09.1.png ├── eq09.11.png ├── eq09.12.png ├── eq09.13.png ├── eq09.14.png ├── eq09.15.png ├── eq09.16.png ├── eq09.17.png ├── eq09.3.png ├── eq09.4.png ├── eq09.5.png ├── eq09.8.png ├── kq_matches.png ├── multiattn.png ├── multihead_chunking.png ├── multiple_keys_context.png ├── score_alignment.png ├── score_alignment_translate.png └── translation_att.png ├── BERT ├── bert_embeddings.png ├── bert_input_embed.png ├── bert_mlm.png └── bert_nsp.png ├── BatchNorm ├── after_batchnorm.png ├── before_batchnorm.png ├── init_vs_bn.png └── layer_vs_batch_norm.png ├── Classification ├── boundary_train.png ├── boundary_val.png ├── classification_relu2.png ├── classification_softmax.png ├── flipped_log_odds.png ├── logistic_model.png ├── odds.png ├── prob_line1.png ├── prob_line2.png ├── prob_line3.png ├── thresh30.png └── thresh70.png ├── Convolutions ├── 1conv1.png ├── 1conv2.png ├── 3channel1.png ├── 3channel2.png ├── 3channel_filters1.png ├── classifier_outputs.png ├── conv1.png ├── conv1_ma.png ├── conv1d.png ├── conv1d_dilated.png ├── conv1d_edges.png ├── conv2.png ├── conv3.png ├── conv5.png ├── conv6.png ├── conv7.png ├── conv8.png ├── featurizer_inputs.png ├── featurizer_outputs.png ├── padding1.png ├── padding2.png ├── padding3.png ├── paddings.png ├── pooling1.png ├── stride1.png ├── strider2.png ├── strider3.png └── weights_filter.png ├── Decoder ├── dec_both.png ├── decoder.png ├── decoder_self.png ├── decoder_self_simplified.png └── transf_decself.png ├── Dropout ├── dropout_distrib_p50.png ├── dropout_distrib_ps.png └── dropout_paper.png ├── ELMo ├── elmo_embed.png └── elmo_lstm.png ├── Encoder ├── enc_both.png ├── encoder.png ├── encoder_self.png ├── encoder_self_detail.png ├── encoder_self_simplified.png └── transf_encself.png ├── Feed-Forward Networks ├── classification.png ├── classification_equiv.png ├── classification_relu2.png ├── classification_softmax.png ├── feature_space_1hidden.png ├── feature_space_1hidden_plates.png ├── feature_space_2hidden.png └── logistic_model.png ├── Gradient Descent ├── bad_surface.png ├── big_lr.png ├── gradient_parm.png ├── mse_parm.png ├── paths.png ├── scaled_surface.png ├── scaled_x.png ├── small_lr.png ├── surface.png ├── surface_fixedb.png ├── surface_fixedw.png └── verybig_lr.png ├── Initializations and Clipping ├── activations_init.png ├── clip_norm.png ├── clip_value.png ├── gradients_clip_vs_hook.png ├── gradients_vs_adapted.png ├── init_vs_bn.png └── vanishing_violins.png ├── LICENSE ├── LayerNorm ├── after_layer_norm.png ├── before_layer_norm.png └── layer_vs_batch_norm.png ├── Optimizers and Schedulers ├── eq06.14.png ├── eq06.16.png ├── eq06.19.png ├── scheduler_cyclical.png ├── scheduler_plateau.png ├── scheduler_step.png ├── sgd_adam_paths.png ├── sgd_momentum_paths.png ├── sgd_nesterov_paths.png ├── sgd_scheduler_paths.png └── sgds.png ├── Patch Embeddings ├── example_embed_patches.png ├── example_img.png ├── example_patches.png └── example_seq.png ├── Positional Encoding ├── dials_binary.png ├── dials_exp.png ├── encoded_distances.png ├── encoder_lost_seq.png ├── eq09.21.png ├── pe_distance.png ├── pe_sine_cosine.png ├── posenc_mod4mod8.png ├── posenc_modnorm4.png ├── posenc_modnorm_deg.png ├── posenc_modnorm_mult.png ├── posenc_modnorm_sincos.png ├── posenc_norm1k.png ├── posenc_norm4.png ├── posenc_norm4_long.png └── sincos_distance.png ├── README.md ├── RNNs ├── bidirect_rnn.png ├── eq08.1.png ├── eq08.12.png ├── eq08.13.png ├── eq08.14.png ├── eq08.5.png ├── eq08.6.png ├── eq08.7.png ├── gru_cell.png ├── journey1.png ├── lstm_cell.png ├── packed_seq_data.png ├── packed_seq_inddata.png ├── rnn1.png ├── rnn2.png ├── rnn_cell_diagram.png ├── rnn_cell_diagram_seq.png └── stacked_rnn.png ├── Seq2Seq ├── enc_dec_attn_translate.png ├── encdec_attn.png ├── encdec_self_simplified.png ├── encoder_decoder.png └── shifted_target.png └── Transformers ├── cls_hidden_state.png ├── dec_both.png ├── enc_both.png ├── eq10.3.png ├── eq10.5.png ├── eq10.6.png ├── full_transformer.png ├── full_transformer_and_class.png ├── norm_first.png ├── stacked_encdec.png ├── stacked_layers.png ├── sublayer.png ├── transf_classes.png ├── transf_decself.png ├── transf_encdecself.png ├── transf_encself.png ├── two_cls_embeds.png └── vit_model.png /Activation Functions/act_prelu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/act_prelu.png -------------------------------------------------------------------------------- /Activation Functions/act_prelu_2hidden.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/act_prelu_2hidden.png -------------------------------------------------------------------------------- /Activation Functions/act_relu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/act_relu.png -------------------------------------------------------------------------------- /Activation Functions/act_sigmoid1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/act_sigmoid1.png -------------------------------------------------------------------------------- /Activation Functions/act_sigmoid2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/act_sigmoid2.png -------------------------------------------------------------------------------- /Activation Functions/act_tanh.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/act_tanh.png -------------------------------------------------------------------------------- /Activation Functions/act_tanh_2hidden.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/act_tanh_2hidden.png -------------------------------------------------------------------------------- /Activation Functions/activation_leaky.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/activation_leaky.png -------------------------------------------------------------------------------- /Activation Functions/activation_prelu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/activation_prelu.png -------------------------------------------------------------------------------- /Activation Functions/activation_relu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/activation_relu.png -------------------------------------------------------------------------------- /Activation Functions/activation_sigmoid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/activation_sigmoid.png -------------------------------------------------------------------------------- /Activation Functions/activation_tanh.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/activation_tanh.png -------------------------------------------------------------------------------- /Activation Functions/eq_activ.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/eq_activ.png -------------------------------------------------------------------------------- /Activation Functions/eq_equiv.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/eq_equiv.png -------------------------------------------------------------------------------- /Activation Functions/feature_space_1hidden.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/feature_space_1hidden.png -------------------------------------------------------------------------------- /Activation Functions/feature_space_1hidden_plates.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/feature_space_1hidden_plates.png -------------------------------------------------------------------------------- /Activation Functions/feature_space_2hidden.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Activation Functions/feature_space_2hidden.png -------------------------------------------------------------------------------- /Architectures/alexnet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Architectures/alexnet.png -------------------------------------------------------------------------------- /Architectures/architecture_lenet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Architectures/architecture_lenet.png -------------------------------------------------------------------------------- /Architectures/inception_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Architectures/inception_model.png -------------------------------------------------------------------------------- /Architectures/inception_modules.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Architectures/inception_modules.png -------------------------------------------------------------------------------- /Architectures/residual.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Architectures/residual.png -------------------------------------------------------------------------------- /Assorted/embed_arithmetic.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/embed_arithmetic.png -------------------------------------------------------------------------------- /Assorted/image_channels.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/image_channels.png -------------------------------------------------------------------------------- /Assorted/image_channels2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/image_channels2.png -------------------------------------------------------------------------------- /Assorted/ngrams.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/ngrams.png -------------------------------------------------------------------------------- /Assorted/ohe1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/ohe1.png -------------------------------------------------------------------------------- /Assorted/ohe2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/ohe2.png -------------------------------------------------------------------------------- /Assorted/ohe3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/ohe3.png -------------------------------------------------------------------------------- /Assorted/tensors.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/tensors.png -------------------------------------------------------------------------------- /Assorted/twodimensions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/twodimensions.png -------------------------------------------------------------------------------- /Assorted/w2v_cbow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/w2v_cbow.png -------------------------------------------------------------------------------- /Assorted/w2v_embed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/w2v_embed.png -------------------------------------------------------------------------------- /Assorted/w2v_logits.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Assorted/w2v_logits.png -------------------------------------------------------------------------------- /Attention/aiayn_dot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/aiayn_dot.png -------------------------------------------------------------------------------- /Attention/aiayn_multihead.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/aiayn_multihead.png -------------------------------------------------------------------------------- /Attention/attention.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/attention.png -------------------------------------------------------------------------------- /Attention/attn_narrow_2heads.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/attn_narrow_2heads.png -------------------------------------------------------------------------------- /Attention/attn_narrow_first_head.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/attn_narrow_first_head.png -------------------------------------------------------------------------------- /Attention/context_translate.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/context_translate.png -------------------------------------------------------------------------------- /Attention/context_vector.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/context_vector.png -------------------------------------------------------------------------------- /Attention/cross_attn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/cross_attn.png -------------------------------------------------------------------------------- /Attention/dec_both.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/dec_both.png -------------------------------------------------------------------------------- /Attention/decoder_self.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/decoder_self.png -------------------------------------------------------------------------------- /Attention/decoder_self_simplified.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/decoder_self_simplified.png -------------------------------------------------------------------------------- /Attention/enc_both.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/enc_both.png -------------------------------------------------------------------------------- /Attention/enc_dec_attn_translate.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/enc_dec_attn_translate.png -------------------------------------------------------------------------------- /Attention/encdec_attn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/encdec_attn.png -------------------------------------------------------------------------------- /Attention/encdec_self_simplified.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/encdec_self_simplified.png -------------------------------------------------------------------------------- /Attention/encoder_self.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/encoder_self.png -------------------------------------------------------------------------------- /Attention/encoder_self_detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/encoder_self_detail.png -------------------------------------------------------------------------------- /Attention/encoder_self_simplified.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/encoder_self_simplified.png -------------------------------------------------------------------------------- /Attention/eq09.1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.1.png -------------------------------------------------------------------------------- /Attention/eq09.11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.11.png -------------------------------------------------------------------------------- /Attention/eq09.12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.12.png -------------------------------------------------------------------------------- /Attention/eq09.13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.13.png -------------------------------------------------------------------------------- /Attention/eq09.14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.14.png -------------------------------------------------------------------------------- /Attention/eq09.15.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.15.png -------------------------------------------------------------------------------- /Attention/eq09.16.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.16.png -------------------------------------------------------------------------------- /Attention/eq09.17.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.17.png -------------------------------------------------------------------------------- /Attention/eq09.3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.3.png -------------------------------------------------------------------------------- /Attention/eq09.4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.4.png -------------------------------------------------------------------------------- /Attention/eq09.5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.5.png -------------------------------------------------------------------------------- /Attention/eq09.8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/eq09.8.png -------------------------------------------------------------------------------- /Attention/kq_matches.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/kq_matches.png -------------------------------------------------------------------------------- /Attention/multiattn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/multiattn.png -------------------------------------------------------------------------------- /Attention/multihead_chunking.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/multihead_chunking.png -------------------------------------------------------------------------------- /Attention/multiple_keys_context.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/multiple_keys_context.png -------------------------------------------------------------------------------- /Attention/score_alignment.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/score_alignment.png -------------------------------------------------------------------------------- /Attention/score_alignment_translate.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/score_alignment_translate.png -------------------------------------------------------------------------------- /Attention/translation_att.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Attention/translation_att.png -------------------------------------------------------------------------------- /BERT/bert_embeddings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/BERT/bert_embeddings.png -------------------------------------------------------------------------------- /BERT/bert_input_embed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/BERT/bert_input_embed.png -------------------------------------------------------------------------------- /BERT/bert_mlm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/BERT/bert_mlm.png -------------------------------------------------------------------------------- /BERT/bert_nsp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/BERT/bert_nsp.png -------------------------------------------------------------------------------- /BatchNorm/after_batchnorm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/BatchNorm/after_batchnorm.png -------------------------------------------------------------------------------- /BatchNorm/before_batchnorm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/BatchNorm/before_batchnorm.png -------------------------------------------------------------------------------- /BatchNorm/init_vs_bn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/BatchNorm/init_vs_bn.png -------------------------------------------------------------------------------- /BatchNorm/layer_vs_batch_norm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/BatchNorm/layer_vs_batch_norm.png -------------------------------------------------------------------------------- /Classification/boundary_train.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/boundary_train.png -------------------------------------------------------------------------------- /Classification/boundary_val.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/boundary_val.png -------------------------------------------------------------------------------- /Classification/classification_relu2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/classification_relu2.png -------------------------------------------------------------------------------- /Classification/classification_softmax.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/classification_softmax.png -------------------------------------------------------------------------------- /Classification/flipped_log_odds.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/flipped_log_odds.png -------------------------------------------------------------------------------- /Classification/logistic_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/logistic_model.png -------------------------------------------------------------------------------- /Classification/odds.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/odds.png -------------------------------------------------------------------------------- /Classification/prob_line1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/prob_line1.png -------------------------------------------------------------------------------- /Classification/prob_line2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/prob_line2.png -------------------------------------------------------------------------------- /Classification/prob_line3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/prob_line3.png -------------------------------------------------------------------------------- /Classification/thresh30.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/thresh30.png -------------------------------------------------------------------------------- /Classification/thresh70.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Classification/thresh70.png -------------------------------------------------------------------------------- /Convolutions/1conv1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/1conv1.png -------------------------------------------------------------------------------- /Convolutions/1conv2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/1conv2.png -------------------------------------------------------------------------------- /Convolutions/3channel1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/3channel1.png -------------------------------------------------------------------------------- /Convolutions/3channel2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/3channel2.png -------------------------------------------------------------------------------- /Convolutions/3channel_filters1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/3channel_filters1.png -------------------------------------------------------------------------------- /Convolutions/classifier_outputs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/classifier_outputs.png -------------------------------------------------------------------------------- /Convolutions/conv1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv1.png -------------------------------------------------------------------------------- /Convolutions/conv1_ma.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv1_ma.png -------------------------------------------------------------------------------- /Convolutions/conv1d.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv1d.png -------------------------------------------------------------------------------- /Convolutions/conv1d_dilated.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv1d_dilated.png -------------------------------------------------------------------------------- /Convolutions/conv1d_edges.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv1d_edges.png -------------------------------------------------------------------------------- /Convolutions/conv2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv2.png -------------------------------------------------------------------------------- /Convolutions/conv3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv3.png -------------------------------------------------------------------------------- /Convolutions/conv5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv5.png -------------------------------------------------------------------------------- /Convolutions/conv6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv6.png -------------------------------------------------------------------------------- /Convolutions/conv7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv7.png -------------------------------------------------------------------------------- /Convolutions/conv8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/conv8.png -------------------------------------------------------------------------------- /Convolutions/featurizer_inputs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/featurizer_inputs.png -------------------------------------------------------------------------------- /Convolutions/featurizer_outputs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/featurizer_outputs.png -------------------------------------------------------------------------------- /Convolutions/padding1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/padding1.png -------------------------------------------------------------------------------- /Convolutions/padding2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/padding2.png -------------------------------------------------------------------------------- /Convolutions/padding3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/padding3.png -------------------------------------------------------------------------------- /Convolutions/paddings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/paddings.png -------------------------------------------------------------------------------- /Convolutions/pooling1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/pooling1.png -------------------------------------------------------------------------------- /Convolutions/stride1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/stride1.png -------------------------------------------------------------------------------- /Convolutions/strider2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/strider2.png -------------------------------------------------------------------------------- /Convolutions/strider3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/strider3.png -------------------------------------------------------------------------------- /Convolutions/weights_filter.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Convolutions/weights_filter.png -------------------------------------------------------------------------------- /Decoder/dec_both.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Decoder/dec_both.png -------------------------------------------------------------------------------- /Decoder/decoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Decoder/decoder.png -------------------------------------------------------------------------------- /Decoder/decoder_self.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Decoder/decoder_self.png -------------------------------------------------------------------------------- /Decoder/decoder_self_simplified.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Decoder/decoder_self_simplified.png -------------------------------------------------------------------------------- /Decoder/transf_decself.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Decoder/transf_decself.png -------------------------------------------------------------------------------- /Dropout/dropout_distrib_p50.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Dropout/dropout_distrib_p50.png -------------------------------------------------------------------------------- /Dropout/dropout_distrib_ps.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Dropout/dropout_distrib_ps.png -------------------------------------------------------------------------------- /Dropout/dropout_paper.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Dropout/dropout_paper.png -------------------------------------------------------------------------------- /ELMo/elmo_embed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/ELMo/elmo_embed.png -------------------------------------------------------------------------------- /ELMo/elmo_lstm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/ELMo/elmo_lstm.png -------------------------------------------------------------------------------- /Encoder/enc_both.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Encoder/enc_both.png -------------------------------------------------------------------------------- /Encoder/encoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Encoder/encoder.png -------------------------------------------------------------------------------- /Encoder/encoder_self.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Encoder/encoder_self.png -------------------------------------------------------------------------------- /Encoder/encoder_self_detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Encoder/encoder_self_detail.png -------------------------------------------------------------------------------- /Encoder/encoder_self_simplified.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Encoder/encoder_self_simplified.png -------------------------------------------------------------------------------- /Encoder/transf_encself.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Encoder/transf_encself.png -------------------------------------------------------------------------------- /Feed-Forward Networks/classification.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Feed-Forward Networks/classification.png -------------------------------------------------------------------------------- /Feed-Forward Networks/classification_equiv.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Feed-Forward Networks/classification_equiv.png -------------------------------------------------------------------------------- /Feed-Forward Networks/classification_relu2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Feed-Forward Networks/classification_relu2.png -------------------------------------------------------------------------------- /Feed-Forward Networks/classification_softmax.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Feed-Forward Networks/classification_softmax.png -------------------------------------------------------------------------------- /Feed-Forward Networks/feature_space_1hidden.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Feed-Forward Networks/feature_space_1hidden.png -------------------------------------------------------------------------------- /Feed-Forward Networks/feature_space_1hidden_plates.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Feed-Forward Networks/feature_space_1hidden_plates.png -------------------------------------------------------------------------------- /Feed-Forward Networks/feature_space_2hidden.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Feed-Forward Networks/feature_space_2hidden.png -------------------------------------------------------------------------------- /Feed-Forward Networks/logistic_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Feed-Forward Networks/logistic_model.png -------------------------------------------------------------------------------- /Gradient Descent/bad_surface.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/bad_surface.png -------------------------------------------------------------------------------- /Gradient Descent/big_lr.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/big_lr.png -------------------------------------------------------------------------------- /Gradient Descent/gradient_parm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/gradient_parm.png -------------------------------------------------------------------------------- /Gradient Descent/mse_parm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/mse_parm.png -------------------------------------------------------------------------------- /Gradient Descent/paths.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/paths.png -------------------------------------------------------------------------------- /Gradient Descent/scaled_surface.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/scaled_surface.png -------------------------------------------------------------------------------- /Gradient Descent/scaled_x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/scaled_x.png -------------------------------------------------------------------------------- /Gradient Descent/small_lr.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/small_lr.png -------------------------------------------------------------------------------- /Gradient Descent/surface.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/surface.png -------------------------------------------------------------------------------- /Gradient Descent/surface_fixedb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/surface_fixedb.png -------------------------------------------------------------------------------- /Gradient Descent/surface_fixedw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/surface_fixedw.png -------------------------------------------------------------------------------- /Gradient Descent/verybig_lr.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Gradient Descent/verybig_lr.png -------------------------------------------------------------------------------- /Initializations and Clipping/activations_init.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Initializations and Clipping/activations_init.png -------------------------------------------------------------------------------- /Initializations and Clipping/clip_norm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Initializations and Clipping/clip_norm.png -------------------------------------------------------------------------------- /Initializations and Clipping/clip_value.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Initializations and Clipping/clip_value.png -------------------------------------------------------------------------------- /Initializations and Clipping/gradients_clip_vs_hook.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Initializations and Clipping/gradients_clip_vs_hook.png -------------------------------------------------------------------------------- /Initializations and Clipping/gradients_vs_adapted.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Initializations and Clipping/gradients_vs_adapted.png -------------------------------------------------------------------------------- /Initializations and Clipping/init_vs_bn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Initializations and Clipping/init_vs_bn.png -------------------------------------------------------------------------------- /Initializations and Clipping/vanishing_violins.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Initializations and Clipping/vanishing_violins.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Attribution 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More_considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution 4.0 International Public License 58 | 59 | By exercising the Licensed Rights (defined below), You accept and agree 60 | to be bound by the terms and conditions of this Creative Commons 61 | Attribution 4.0 International Public License ("Public License"). To the 62 | extent this Public License may be interpreted as a contract, You are 63 | granted the Licensed Rights in consideration of Your acceptance of 64 | these terms and conditions, and the Licensor grants You such rights in 65 | consideration of benefits the Licensor receives from making the 66 | Licensed Material available under these terms and conditions. 67 | 68 | 69 | Section 1 -- Definitions. 70 | 71 | a. Adapted Material means material subject to Copyright and Similar 72 | Rights that is derived from or based upon the Licensed Material 73 | and in which the Licensed Material is translated, altered, 74 | arranged, transformed, or otherwise modified in a manner requiring 75 | permission under the Copyright and Similar Rights held by the 76 | Licensor. For purposes of this Public License, where the Licensed 77 | Material is a musical work, performance, or sound recording, 78 | Adapted Material is always produced where the Licensed Material is 79 | synched in timed relation with a moving image. 80 | 81 | b. Adapter's License means the license You apply to Your Copyright 82 | and Similar Rights in Your contributions to Adapted Material in 83 | accordance with the terms and conditions of this Public License. 84 | 85 | c. Copyright and Similar Rights means copyright and/or similar rights 86 | closely related to copyright including, without limitation, 87 | performance, broadcast, sound recording, and Sui Generis Database 88 | Rights, without regard to how the rights are labeled or 89 | categorized. For purposes of this Public License, the rights 90 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 91 | Rights. 92 | 93 | d. Effective Technological Measures means those measures that, in the 94 | absence of proper authority, may not be circumvented under laws 95 | fulfilling obligations under Article 11 of the WIPO Copyright 96 | Treaty adopted on December 20, 1996, and/or similar international 97 | agreements. 98 | 99 | e. Exceptions and Limitations means fair use, fair dealing, and/or 100 | any other exception or limitation to Copyright and Similar Rights 101 | that applies to Your use of the Licensed Material. 102 | 103 | f. Licensed Material means the artistic or literary work, database, 104 | or other material to which the Licensor applied this Public 105 | License. 106 | 107 | g. Licensed Rights means the rights granted to You subject to the 108 | terms and conditions of this Public License, which are limited to 109 | all Copyright and Similar Rights that apply to Your use of the 110 | Licensed Material and that the Licensor has authority to license. 111 | 112 | h. Licensor means the individual(s) or entity(ies) granting rights 113 | under this Public License. 114 | 115 | i. Share means to provide material to the public by any means or 116 | process that requires permission under the Licensed Rights, such 117 | as reproduction, public display, public performance, distribution, 118 | dissemination, communication, or importation, and to make material 119 | available to the public including in ways that members of the 120 | public may access the material from a place and at a time 121 | individually chosen by them. 122 | 123 | j. Sui Generis Database Rights means rights other than copyright 124 | resulting from Directive 96/9/EC of the European Parliament and of 125 | the Council of 11 March 1996 on the legal protection of databases, 126 | as amended and/or succeeded, as well as other essentially 127 | equivalent rights anywhere in the world. 128 | 129 | k. You means the individual or entity exercising the Licensed Rights 130 | under this Public License. Your has a corresponding meaning. 131 | 132 | 133 | Section 2 -- Scope. 134 | 135 | a. License grant. 136 | 137 | 1. Subject to the terms and conditions of this Public License, 138 | the Licensor hereby grants You a worldwide, royalty-free, 139 | non-sublicensable, non-exclusive, irrevocable license to 140 | exercise the Licensed Rights in the Licensed Material to: 141 | 142 | a. reproduce and Share the Licensed Material, in whole or 143 | in part; and 144 | 145 | b. produce, reproduce, and Share Adapted Material. 146 | 147 | 2. Exceptions and Limitations. For the avoidance of doubt, where 148 | Exceptions and Limitations apply to Your use, this Public 149 | License does not apply, and You do not need to comply with 150 | its terms and conditions. 151 | 152 | 3. Term. The term of this Public License is specified in Section 153 | 6(a). 154 | 155 | 4. Media and formats; technical modifications allowed. The 156 | Licensor authorizes You to exercise the Licensed Rights in 157 | all media and formats whether now known or hereafter created, 158 | and to make technical modifications necessary to do so. The 159 | Licensor waives and/or agrees not to assert any right or 160 | authority to forbid You from making technical modifications 161 | necessary to exercise the Licensed Rights, including 162 | technical modifications necessary to circumvent Effective 163 | Technological Measures. For purposes of this Public License, 164 | simply making modifications authorized by this Section 2(a) 165 | (4) never produces Adapted Material. 166 | 167 | 5. Downstream recipients. 168 | 169 | a. Offer from the Licensor -- Licensed Material. Every 170 | recipient of the Licensed Material automatically 171 | receives an offer from the Licensor to exercise the 172 | Licensed Rights under the terms and conditions of this 173 | Public License. 174 | 175 | b. No downstream restrictions. You may not offer or impose 176 | any additional or different terms or conditions on, or 177 | apply any Effective Technological Measures to, the 178 | Licensed Material if doing so restricts exercise of the 179 | Licensed Rights by any recipient of the Licensed 180 | Material. 181 | 182 | 6. No endorsement. Nothing in this Public License constitutes or 183 | may be construed as permission to assert or imply that You 184 | are, or that Your use of the Licensed Material is, connected 185 | with, or sponsored, endorsed, or granted official status by, 186 | the Licensor or others designated to receive attribution as 187 | provided in Section 3(a)(1)(A)(i). 188 | 189 | b. Other rights. 190 | 191 | 1. Moral rights, such as the right of integrity, are not 192 | licensed under this Public License, nor are publicity, 193 | privacy, and/or other similar personality rights; however, to 194 | the extent possible, the Licensor waives and/or agrees not to 195 | assert any such rights held by the Licensor to the limited 196 | extent necessary to allow You to exercise the Licensed 197 | Rights, but not otherwise. 198 | 199 | 2. Patent and trademark rights are not licensed under this 200 | Public License. 201 | 202 | 3. To the extent possible, the Licensor waives any right to 203 | collect royalties from You for the exercise of the Licensed 204 | Rights, whether directly or through a collecting society 205 | under any voluntary or waivable statutory or compulsory 206 | licensing scheme. In all other cases the Licensor expressly 207 | reserves any right to collect such royalties. 208 | 209 | 210 | Section 3 -- License Conditions. 211 | 212 | Your exercise of the Licensed Rights is expressly made subject to the 213 | following conditions. 214 | 215 | a. Attribution. 216 | 217 | 1. If You Share the Licensed Material (including in modified 218 | form), You must: 219 | 220 | a. retain the following if it is supplied by the Licensor 221 | with the Licensed Material: 222 | 223 | i. identification of the creator(s) of the Licensed 224 | Material and any others designated to receive 225 | attribution, in any reasonable manner requested by 226 | the Licensor (including by pseudonym if 227 | designated); 228 | 229 | ii. a copyright notice; 230 | 231 | iii. a notice that refers to this Public License; 232 | 233 | iv. a notice that refers to the disclaimer of 234 | warranties; 235 | 236 | v. a URI or hyperlink to the Licensed Material to the 237 | extent reasonably practicable; 238 | 239 | b. indicate if You modified the Licensed Material and 240 | retain an indication of any previous modifications; and 241 | 242 | c. indicate the Licensed Material is licensed under this 243 | Public License, and include the text of, or the URI or 244 | hyperlink to, this Public License. 245 | 246 | 2. You may satisfy the conditions in Section 3(a)(1) in any 247 | reasonable manner based on the medium, means, and context in 248 | which You Share the Licensed Material. For example, it may be 249 | reasonable to satisfy the conditions by providing a URI or 250 | hyperlink to a resource that includes the required 251 | information. 252 | 253 | 3. If requested by the Licensor, You must remove any of the 254 | information required by Section 3(a)(1)(A) to the extent 255 | reasonably practicable. 256 | 257 | 4. If You Share Adapted Material You produce, the Adapter's 258 | License You apply must not prevent recipients of the Adapted 259 | Material from complying with this Public License. 260 | 261 | 262 | Section 4 -- Sui Generis Database Rights. 263 | 264 | Where the Licensed Rights include Sui Generis Database Rights that 265 | apply to Your use of the Licensed Material: 266 | 267 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 268 | to extract, reuse, reproduce, and Share all or a substantial 269 | portion of the contents of the database; 270 | 271 | b. if You include all or a substantial portion of the database 272 | contents in a database in which You have Sui Generis Database 273 | Rights, then the database in which You have Sui Generis Database 274 | Rights (but not its individual contents) is Adapted Material; and 275 | 276 | c. You must comply with the conditions in Section 3(a) if You Share 277 | all or a substantial portion of the contents of the database. 278 | 279 | For the avoidance of doubt, this Section 4 supplements and does not 280 | replace Your obligations under this Public License where the Licensed 281 | Rights include other Copyright and Similar Rights. 282 | 283 | 284 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 285 | 286 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 287 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 288 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 289 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 290 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 291 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 292 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 293 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 294 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 295 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 296 | 297 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 298 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 299 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 300 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 301 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 302 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 303 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 304 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 305 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 306 | 307 | c. The disclaimer of warranties and limitation of liability provided 308 | above shall be interpreted in a manner that, to the extent 309 | possible, most closely approximates an absolute disclaimer and 310 | waiver of all liability. 311 | 312 | 313 | Section 6 -- Term and Termination. 314 | 315 | a. This Public License applies for the term of the Copyright and 316 | Similar Rights licensed here. However, if You fail to comply with 317 | this Public License, then Your rights under this Public License 318 | terminate automatically. 319 | 320 | b. Where Your right to use the Licensed Material has terminated under 321 | Section 6(a), it reinstates: 322 | 323 | 1. automatically as of the date the violation is cured, provided 324 | it is cured within 30 days of Your discovery of the 325 | violation; or 326 | 327 | 2. upon express reinstatement by the Licensor. 328 | 329 | For the avoidance of doubt, this Section 6(b) does not affect any 330 | right the Licensor may have to seek remedies for Your violations 331 | of this Public License. 332 | 333 | c. For the avoidance of doubt, the Licensor may also offer the 334 | Licensed Material under separate terms or conditions or stop 335 | distributing the Licensed Material at any time; however, doing so 336 | will not terminate this Public License. 337 | 338 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 339 | License. 340 | 341 | 342 | Section 7 -- Other Terms and Conditions. 343 | 344 | a. The Licensor shall not be bound by any additional or different 345 | terms or conditions communicated by You unless expressly agreed. 346 | 347 | b. Any arrangements, understandings, or agreements regarding the 348 | Licensed Material not stated herein are separate from and 349 | independent of the terms and conditions of this Public License. 350 | 351 | 352 | Section 8 -- Interpretation. 353 | 354 | a. For the avoidance of doubt, this Public License does not, and 355 | shall not be interpreted to, reduce, limit, restrict, or impose 356 | conditions on any use of the Licensed Material that could lawfully 357 | be made without permission under this Public License. 358 | 359 | b. To the extent possible, if any provision of this Public License is 360 | deemed unenforceable, it shall be automatically reformed to the 361 | minimum extent necessary to make it enforceable. If the provision 362 | cannot be reformed, it shall be severed from this Public License 363 | without affecting the enforceability of the remaining terms and 364 | conditions. 365 | 366 | c. No term or condition of this Public License will be waived and no 367 | failure to comply consented to unless expressly agreed to by the 368 | Licensor. 369 | 370 | d. Nothing in this Public License constitutes or may be interpreted 371 | as a limitation upon, or waiver of, any privileges and immunities 372 | that apply to the Licensor or You, including from the legal 373 | processes of any jurisdiction or authority. 374 | 375 | 376 | ======================================================================= 377 | 378 | Creative Commons is not a party to its public 379 | licenses. Notwithstanding, Creative Commons may elect to apply one of 380 | its public licenses to material it publishes and in those instances 381 | will be considered the “Licensor.” The text of the Creative Commons 382 | public licenses is dedicated to the public domain under the CC0 Public 383 | Domain Dedication. Except for the limited purpose of indicating that 384 | material is shared under a Creative Commons public license or as 385 | otherwise permitted by the Creative Commons policies published at 386 | creativecommons.org/policies, Creative Commons does not authorize the 387 | use of the trademark "Creative Commons" or any other trademark or logo 388 | of Creative Commons without its prior written consent including, 389 | without limitation, in connection with any unauthorized modifications 390 | to any of its public licenses or any other arrangements, 391 | understandings, or agreements concerning use of licensed material. For 392 | the avoidance of doubt, this paragraph does not form part of the 393 | public licenses. 394 | 395 | Creative Commons may be contacted at creativecommons.org. 396 | -------------------------------------------------------------------------------- /LayerNorm/after_layer_norm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/LayerNorm/after_layer_norm.png -------------------------------------------------------------------------------- /LayerNorm/before_layer_norm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/LayerNorm/before_layer_norm.png -------------------------------------------------------------------------------- /LayerNorm/layer_vs_batch_norm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/LayerNorm/layer_vs_batch_norm.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/eq06.14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/eq06.14.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/eq06.16.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/eq06.16.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/eq06.19.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/eq06.19.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/scheduler_cyclical.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/scheduler_cyclical.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/scheduler_plateau.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/scheduler_plateau.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/scheduler_step.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/scheduler_step.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/sgd_adam_paths.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/sgd_adam_paths.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/sgd_momentum_paths.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/sgd_momentum_paths.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/sgd_nesterov_paths.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/sgd_nesterov_paths.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/sgd_scheduler_paths.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/sgd_scheduler_paths.png -------------------------------------------------------------------------------- /Optimizers and Schedulers/sgds.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Optimizers and Schedulers/sgds.png -------------------------------------------------------------------------------- /Patch Embeddings/example_embed_patches.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Patch Embeddings/example_embed_patches.png -------------------------------------------------------------------------------- /Patch Embeddings/example_img.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Patch Embeddings/example_img.png -------------------------------------------------------------------------------- /Patch Embeddings/example_patches.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Patch Embeddings/example_patches.png -------------------------------------------------------------------------------- /Patch Embeddings/example_seq.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Patch Embeddings/example_seq.png -------------------------------------------------------------------------------- /Positional Encoding/dials_binary.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/dials_binary.png -------------------------------------------------------------------------------- /Positional Encoding/dials_exp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/dials_exp.png -------------------------------------------------------------------------------- /Positional Encoding/encoded_distances.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/encoded_distances.png -------------------------------------------------------------------------------- /Positional Encoding/encoder_lost_seq.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/encoder_lost_seq.png -------------------------------------------------------------------------------- /Positional Encoding/eq09.21.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/eq09.21.png -------------------------------------------------------------------------------- /Positional Encoding/pe_distance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/pe_distance.png -------------------------------------------------------------------------------- /Positional Encoding/pe_sine_cosine.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/pe_sine_cosine.png -------------------------------------------------------------------------------- /Positional Encoding/posenc_mod4mod8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/posenc_mod4mod8.png -------------------------------------------------------------------------------- /Positional Encoding/posenc_modnorm4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/posenc_modnorm4.png -------------------------------------------------------------------------------- /Positional Encoding/posenc_modnorm_deg.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/posenc_modnorm_deg.png -------------------------------------------------------------------------------- /Positional Encoding/posenc_modnorm_mult.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/posenc_modnorm_mult.png -------------------------------------------------------------------------------- /Positional Encoding/posenc_modnorm_sincos.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/posenc_modnorm_sincos.png -------------------------------------------------------------------------------- /Positional Encoding/posenc_norm1k.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/posenc_norm1k.png -------------------------------------------------------------------------------- /Positional Encoding/posenc_norm4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/posenc_norm4.png -------------------------------------------------------------------------------- /Positional Encoding/posenc_norm4_long.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/posenc_norm4_long.png -------------------------------------------------------------------------------- /Positional Encoding/sincos_distance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Positional Encoding/sincos_distance.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Deep Learning Visuals 2 | 3 | Shield: [![CC BY 4.0][cc-by-shield]][cc-by] 4 | 5 | This repository was inspired by the [ML Visuals](https://github.com/dair-ai/ml-visuals) repository maintained by [dair.ai](https://dair.ai/). 6 | 7 | **Deep Learning Visuals** contains **215 unique images** divided in **23 categories** (some images may appear in more than one category). All the images were originally published in my book ["Deep Learning with PyTorch Step-by-Step: A Beginner's Guide"](https://leanpub.com/pytorch). 8 | 9 | ## Can I Freely Use These Images? 10 | 11 | Sure, these images can be FREELY USED in your own blog posts, slides, presentations, or papers under the CC-BY license. 12 | 13 | ## Awesome, where are they? 14 | 15 | You can easily navigate through the pages and indices, and click on the desired image to visualize it in full size: 16 | 17 | - [Activation Functions](https://dvgodoy.github.io/dl-visuals/Activation%20Functions) 18 | - [Architectures](https://dvgodoy.github.io/dl-visuals/Architectures) 19 | - [Assorted](https://dvgodoy.github.io/dl-visuals/Assorted) 20 | - [Attention](https://dvgodoy.github.io/dl-visuals/Attention) 21 | - [Batch Norm](https://dvgodoy.github.io/dl-visuals/BatchNorm) 22 | - [BERT](https://dvgodoy.github.io/dl-visuals/BERT) 23 | - [Classification](https://dvgodoy.github.io/dl-visuals/Classification) 24 | - [Convolutions](https://dvgodoy.github.io/dl-visuals/Convolutions) 25 | - [Decoder](https://dvgodoy.github.io/dl-visuals/Decoder) 26 | - [Dropout](https://dvgodoy.github.io/dl-visuals/Dropout) 27 | - [ELMo](https://dvgodoy.github.io/dl-visuals/ELMo) 28 | - [Encoder](https://dvgodoy.github.io/dl-visuals/Encoder) 29 | - [Feed-Forward Networks](https://dvgodoy.github.io/dl-visuals/Feed-Forward%20Networks) 30 | - [Gradient Descent](https://dvgodoy.github.io/dl-visuals/Gradient%20Descent) 31 | - [Initializations and Gradient Clipping](https://dvgodoy.github.io/dl-visuals/Initializations%20and%20Clipping) 32 | - [LayerNorm](https://dvgodoy.github.io/dl-visuals/LayerNorm) 33 | - [Optimizers and Schedulers](https://dvgodoy.github.io/dl-visuals/Optimizers%20and%20Schedulers) 34 | - [Patch Embeddings](https://dvgodoy.github.io/dl-visuals/Patch%20Embeddings) 35 | - [Positional Encoding](https://dvgodoy.github.io/dl-visuals/Positional%20Encoding) 36 | - [RNNs](https://dvgodoy.github.io/dl-visuals/RNNs) 37 | - [Seq2Seq](https://dvgodoy.github.io/dl-visuals/Seq2Seq) 38 | - [Transformers](https://dvgodoy.github.io/dl-visuals/Transformers) 39 | 40 | ## How Can I Use Them? 41 | 42 | **DISCLAIMER: this is NOT legal advice, you should always read the license yourself!** 43 | 44 | In a nutshell, you're allowed to use (or adapt) these images in your own materials, even for commercial purposes, as long as you attribute it. 45 | 46 | Here is a quick guide on [Best Practices for Attribution](https://wiki.creativecommons.org/wiki/best_practices_for_attribution). 47 | 48 | Here are some examples of both images and attributions: 49 | 50 | ### Logistic Regression 51 | 52 | ![](https://raw.githubusercontent.com/dvgodoy/dl-visuals/main/Classification/logistic_model.png) 53 | 54 | Image by [dvgodoy](https://github.com/dvgodoy/dl-visuals) / [CC BY](https://creativecommons.org/licenses/by/4.0/) 55 | 56 | ### RNN 57 | 58 | ![](https://raw.githubusercontent.com/dvgodoy/dl-visuals/main/RNNs/rnn2.png) 59 | Image by [dvgodoy](https://github.com/dvgodoy/dl-visuals) / [CC BY](https://creativecommons.org/licenses/by/4.0/) 60 | 61 | ### Transformer 62 | 63 | ![](https://raw.githubusercontent.com/dvgodoy/dl-visuals/main/Transformers/full_transformer.png) 64 | Image by [dvgodoy](https://github.com/dvgodoy/dl-visuals) / [CC BY](https://creativecommons.org/licenses/by/4.0/) 65 | 66 | This work is licensed under a 67 | [Creative Commons Attribution 4.0 International License][cc-by]. 68 | 69 | [![CC BY 4.0][cc-by-image]][cc-by] 70 | 71 | [cc-by]: http://creativecommons.org/licenses/by/4.0/ 72 | [cc-by-image]: https://i.creativecommons.org/l/by/4.0/88x31.png 73 | [cc-by-shield]: https://img.shields.io/badge/License-CC%20BY%204.0-lightgrey.svg -------------------------------------------------------------------------------- /RNNs/bidirect_rnn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/bidirect_rnn.png -------------------------------------------------------------------------------- /RNNs/eq08.1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/eq08.1.png -------------------------------------------------------------------------------- /RNNs/eq08.12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/eq08.12.png -------------------------------------------------------------------------------- /RNNs/eq08.13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/eq08.13.png -------------------------------------------------------------------------------- /RNNs/eq08.14.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/eq08.14.png -------------------------------------------------------------------------------- /RNNs/eq08.5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/eq08.5.png -------------------------------------------------------------------------------- /RNNs/eq08.6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/eq08.6.png -------------------------------------------------------------------------------- /RNNs/eq08.7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/eq08.7.png -------------------------------------------------------------------------------- /RNNs/gru_cell.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/gru_cell.png -------------------------------------------------------------------------------- /RNNs/journey1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/journey1.png -------------------------------------------------------------------------------- /RNNs/lstm_cell.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/lstm_cell.png -------------------------------------------------------------------------------- /RNNs/packed_seq_data.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/packed_seq_data.png -------------------------------------------------------------------------------- /RNNs/packed_seq_inddata.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/packed_seq_inddata.png -------------------------------------------------------------------------------- /RNNs/rnn1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/rnn1.png -------------------------------------------------------------------------------- /RNNs/rnn2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/rnn2.png -------------------------------------------------------------------------------- /RNNs/rnn_cell_diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/rnn_cell_diagram.png -------------------------------------------------------------------------------- /RNNs/rnn_cell_diagram_seq.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/rnn_cell_diagram_seq.png -------------------------------------------------------------------------------- /RNNs/stacked_rnn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/RNNs/stacked_rnn.png -------------------------------------------------------------------------------- /Seq2Seq/enc_dec_attn_translate.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Seq2Seq/enc_dec_attn_translate.png -------------------------------------------------------------------------------- /Seq2Seq/encdec_attn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Seq2Seq/encdec_attn.png -------------------------------------------------------------------------------- /Seq2Seq/encdec_self_simplified.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Seq2Seq/encdec_self_simplified.png -------------------------------------------------------------------------------- /Seq2Seq/encoder_decoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Seq2Seq/encoder_decoder.png -------------------------------------------------------------------------------- /Seq2Seq/shifted_target.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Seq2Seq/shifted_target.png -------------------------------------------------------------------------------- /Transformers/cls_hidden_state.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/cls_hidden_state.png -------------------------------------------------------------------------------- /Transformers/dec_both.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/dec_both.png -------------------------------------------------------------------------------- /Transformers/enc_both.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/enc_both.png -------------------------------------------------------------------------------- /Transformers/eq10.3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/eq10.3.png -------------------------------------------------------------------------------- /Transformers/eq10.5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/eq10.5.png -------------------------------------------------------------------------------- /Transformers/eq10.6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/eq10.6.png -------------------------------------------------------------------------------- /Transformers/full_transformer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/full_transformer.png -------------------------------------------------------------------------------- /Transformers/full_transformer_and_class.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/full_transformer_and_class.png -------------------------------------------------------------------------------- /Transformers/norm_first.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/norm_first.png -------------------------------------------------------------------------------- /Transformers/stacked_encdec.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/stacked_encdec.png -------------------------------------------------------------------------------- /Transformers/stacked_layers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/stacked_layers.png -------------------------------------------------------------------------------- /Transformers/sublayer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/sublayer.png -------------------------------------------------------------------------------- /Transformers/transf_classes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/transf_classes.png -------------------------------------------------------------------------------- /Transformers/transf_decself.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/transf_decself.png -------------------------------------------------------------------------------- /Transformers/transf_encdecself.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/transf_encdecself.png -------------------------------------------------------------------------------- /Transformers/transf_encself.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/transf_encself.png -------------------------------------------------------------------------------- /Transformers/two_cls_embeds.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/two_cls_embeds.png -------------------------------------------------------------------------------- /Transformers/vit_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dvgodoy/dl-visuals/95f792b37ff7ef0c098e06a778d2f3af8205903e/Transformers/vit_model.png --------------------------------------------------------------------------------