├── .gitattributes ├── AI-models.md ├── ML&DL.md ├── NeuralNetwork.md ├── PromptEngineering.md ├── README.md ├── agi-five.png ├── attention-models.gif ├── neural-network.gif └── transformers.png /.gitattributes: -------------------------------------------------------------------------------- 1 | # Auto detect text files and perform LF normalization 2 | * text=auto 3 | -------------------------------------------------------------------------------- /AI-models.md: -------------------------------------------------------------------------------- 1 | # Generative AI Models and Applications: 2 | 3 | #### 1. Generative Adversarial Networks (GANs): 4 | 5 | - Two networks ***(generator and discriminator)*** compete to generate realistic data. 6 | - **Applications:** Image generation, deepfakes, video synthesis, style transfer 7 | 8 | #### 2. Variational Autoencoders (VAEs): 9 | 10 | - Compress and reconstruct data, generating new samples from latent space. 11 | - **Applications:** Image reconstruction, anomaly detection, data compression 12 | 13 | #### 3. [Diffusion Models:](/AI-models.md#diffusion-models) 14 | 15 | - Iteratively refine random noise to create high-quality images. 16 | - **Applications:** High-quality image generation, art creation, video synthesis 17 | 18 | #### 4. Large Language Models (LLMs): 19 | 20 | - Large language models like GPT that generate coherent text. 21 | - **Applications:** Text generation, chatbots, summarization, translation (e.g., GPT-4) 22 | 23 | #### 5. Autoregressive Models: 24 | 25 | - Generate data by predicting the next element (e.g., pixels, words) based on previous ones. 26 | - **Applications:** Audio generation, pixel-level image generation (e.g., PixelCNN) 27 | 28 | #### 6. Neural Style Transfer: 29 | 30 | - Combines content and style from different images to create new visuals. 31 | - **Applications:** Art generation by blending content and style from different images 32 | 33 | 34 | ## Diffusion Models 35 | 36 | ### Stable Diffusion Models: 37 | 38 | - Stable Diffusion builds on the Latent Diffusion Models (LDMs), a more efficient type of diffusion model that operates in a compressed latent space rather than directly on pixel data. 39 | - **Applications:** Art Creation, Image Editing, Design, AI assisted creativity 40 | - **How It Works:** 41 | - ***Encoding:*** Converts images into a lower-dimensional latent space. 42 | - ***Diffusion Process:*** Refines random noise into coherent images within the latent space. 43 | - ***Decoding:*** Converts the refined latent representation back into high-resolution images. 44 | 45 | 46 | ### Latent Diffusion Models (LDMs): 47 | 48 | - A type of diffusion model that operates in a latent space, a compressed version of the original data, which makes the model more efficient and faster. 49 | - **Applications:** Image generation, Data Compression 50 | - **How It Works:** 51 | - ***Encoding:*** Compresses data into a latent space. 52 | - ***Diffusion Process:*** Adds and removes noise within this compressed space. 53 | - ***Decoding:*** Transforms the latent data back into the original or high-resolution form. 54 | 55 | ### Denoising Diffusion Probabilistic Models (DDPMs): 56 | 57 | - A class of generative models that gradually adds noise to data during training and then learns to reverse the process to generate new data. 58 | - **Applications:** Image generation, data synthesis, Audio and Video Synthesis 59 | - **How It Works:** 60 | - ***Forward Process:*** Data is progressively corrupted by adding Gaussian noise over several steps. 61 | - ***Reverse Process:*** A neural network learns to reverse this noise addition, progressively refining the data from noise to a coherent sample. 62 | 63 | ### Score-Based Generative Models (SDEs): 64 | 65 | - A type of generative model that uses Stochastic Differential Equations (SDEs) to generate data by modeling a diffusion process. 66 | - **Applications:** Image generation, high-quality data augmentation 67 | - **How It Works:** 68 | - ***Forward Process:*** Adds noise to data using SDEs, converting it into a noise distribution. 69 | - ***Reverse Process:*** The model estimates the score function to iteratively denoise and generate new data samples. 70 | 71 | -------------------------------------------------------------------------------- /ML&DL.md: -------------------------------------------------------------------------------- 1 | # Machine Learning (ML) 2 | - Subset of Artificial Intelligence (AI) 3 | - Training algorithms learn from data to make predictions/decisions. 4 | - **Example:** Spam detection, recommendation systems, Predictive modeling, Natural Language Processing (NLP) 5 | 6 | ## ML Branches / Types 7 | 8 | #### 1. Supervised Learning 9 | - Uses labeled data (input-output pairs) 10 | - Learn a mapping between input data and output labels 11 | - **Examples:** 12 | - Image classification (e.g., cat vs. dog) 13 | - Sentiment analysis (e.g., positive vs. negative review) 14 | - Regression tasks (e.g., predicting house prices) 15 | 16 | #### 2. Unsupervised Learning 17 | - Works with unlabeled data, finds hidden patterns 18 | - Discover patterns, relationships, or structure in data 19 | - **Examples:** 20 | - Clustering customers by behavior or demographics 21 | - Dimensionality reduction (e.g., PCA, t-SNE) 22 | - Anomaly detection (e.g., fraud detection) 23 | 24 | #### 3. Reinforcement Learning 25 | - Use trial and error to learn from interactions with an environment 26 | - Learn to make decisions that maximize a reward signal 27 | - **Examples:** 28 | - Robotics (e.g., learning to walk or grasp objects) 29 | - Game playing (e.g., AlphaGo, poker) 30 | - Autonomous vehicles (e.g., learning to navigate roads) 31 | 32 | ## ML Algorithms 33 | Do not process **Raw Data** like an image or text. Data is preprocessed through steps like ***cleaning***, ***normalization***, and ***feature extraction*** to improve the model's performance. 34 | 35 | - **Linear Regression:** Predicts continuous values. 36 | - **Logistic Regression:** Binary classification. 37 | - **Decision Trees:** Tree-like model for decisions. 38 | - **k-NN (k-Nearest Neighbors):** Classifies based on closest data points. 39 | - **SVM (Support Vector Machines):** Classifies by finding the best boundary. 40 | - **Naive Bayes:** Probabilistic classification. 41 | - **Random Forest:** Ensemble of decision trees. 42 | - **K-Means:** Unsupervised clustering algorithm. 43 | 44 | ## Limitations of ML 45 | 46 | - Cannot directly work with ***unstructured, highly dimensional data*** (Required Feature Engineering) 47 | - Can learn only so much from available data (i.e performance does not increase after certain threshold even if more training data is available) 48 | 49 | # Deep Learning (DL) 50 | - Subset of ML using Artificial Neural Networks (ANN) 51 | - Works with large datasets and high computation 52 | - **Example:** Image recognition, Speech recognition, self-driving cars 53 | 54 | [Types of Neural Network](/NeuralNetwork.md) 55 | 56 | ## DL Algorithms 57 | 58 | - **Convolutional Neural Networks (CNNs):** Used for image processing. 59 | - **Recurrent Neural Networks (RNNs):** Used for sequence data (e.g., time series, language). 60 | - **Long Short-Term Memory (LSTM):** A type of RNN for long-term dependencies in sequences. 61 | - **Generative Adversarial Networks (GANs):** Used for generating data (e.g., images, videos). 62 | - **Autoencoders:** Used for data compression and noise reduction. 63 | - **Transformers:** Used for natural language processing tasks. 64 | 65 | - Linear Models 66 | - Neural Networks 67 | 68 | # TensorFlow 69 | - TensorFlow is an open-source machine learning and deep learning framework developed by Google. 70 | - It enables building, training and deploying ML and DL models. 71 | - Primarily used for deep learning tasks, it's versatile for various other types of machine learning. 72 | 73 | [Explore TensorFlow Playground](https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=4,2&seed=0.91235&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false) 74 | -------------------------------------------------------------------------------- /NeuralNetwork.md: -------------------------------------------------------------------------------- 1 | # Neural Network 2 | A Neural Network is a computational model inspired by the human brain, designed to recognize patterns, make predictions, and solve complex problems. It's a key component of machine learning and deep learning. 3 | 4 | ![ss](/neural-network.gif) 5 | 6 | ### Neural Network Basics: 7 | - **Artificial Neurons (Nodes):** Process inputs and produce outputs. 8 | - **Connections (Edges):** Transmit signals between nodes. 9 | - **Layers:** Organized nodes and connections. 10 | - **Activation Functions:** Introduce non-linearity to node outputs. 11 | 12 | ### Neural Network Components: 13 | - **Input Layer:** Receives input data. 14 | - **Hidden Layers:** Processes and transforms data. 15 | - **Output Layer:** Produces predicted outputs. 16 | - **Bias:** Adjusts node outputs. 17 | 18 | ### Training Process: 19 | - Initialize ***weights*** and ***biases***. 20 | - Perform ***forward propagation*** to generate predictions. 21 | - Calculate the loss ***(error between predicted and actual values)***. 22 | - Perform ***backward propagation*** to compute gradients. 23 | - Update weights using ***gradient descent*** or an **optimization algorithm**. 24 | - Repeat for multiple ***epochs*** until the loss is minimized. 25 | 26 | # Types of Neural Network 27 | ### 1. Feedforward Neural Network (FNN): 28 | 29 | - Basic neural network where data flows in one direction from input to output. 30 | - **Applications:** Classification, regression. 31 | - **Limitations:** 32 | - Training Time 33 | - Pixel Inequality 34 | - High Dimensionality 35 | - Limited Feauture Extraction 36 | 37 | ### 2. Convolutional Neural Network (CNN): 38 | 39 | - Specialized for processing grid-like data, particularly images. 40 | - **Applications:** Image recognition, object detection, video analysis. 41 | - **Limitations:** 42 | - Needs large data 43 | - Lacks temporal handling 44 | - High computation 45 | 46 | ### 3. [Autoencoder:](/NeuralNetwork.md#types-of-autoencoders) 47 | 48 | - A type of neural network designed for data compression, anomaly detection, dimensionally reduction and feature learning. 49 | - **Components:** Encoder, Decoder 50 | - **Applications:** Data compression, anomaly detection. 51 | - **Limitations:** 52 | - Overfitting 53 | - Lossy reconstruction 54 | - Poor data generation 55 | 56 | ### 4. [Recurrent Neural Network (RNN):](/NeuralNetwork.md#types-of-recurrent-neural-network) 57 | 58 | - Designed to recognize patterns in sequences of data such as time series or natural language. 59 | - **Applications:** Time series prediction, natural language processing (NLP). 60 | - **Limitations:** 61 | - Vanishing gradients 62 | - Slow training 63 | - Limited memory 64 | 65 | ### 5. Attention Models / Transformers: 66 | 67 | - Neural network architecture using self-attention mechanisms, widely used in NLP. 68 | - **Applications:** Language modeling (e.g., GPT, BERT), translation, summarization. 69 | 70 | ![ss](attention-models.gif) 71 | 72 | ![ss](transformers.png) 73 | 74 | ### 6. Generative Adversarial Network (GAN): 75 | 76 | - Consists of two networks (generator and discriminator) competing to produce realistic data. 77 | - **Applications:** Image generation, deepfakes, data augmentation. 78 | 79 | ### 7. Radial Basis Function Network (RBFN): 80 | 81 | - Uses radial basis functions as activation functions. 82 | - **Applications:** Function approximation, time-series prediction. 83 | 84 | ## Types of AutoEncoders 85 | 86 | ### 1. Vanilla Autoencoder 87 | - **Structure:** Basic autoencoder with a symmetrical architecture (encoder and decoder). 88 | - **Application:** Primarily used for dimensionality reduction and feature learning. 89 | - **Limitation:** Prone to learning trivial mappings, making it less effective for complex patterns. 90 | 91 | ### 2. Denoising Autoencoder (DAE) 92 | - **Purpose:** To remove noise from input data. 93 | - **Mechanism:** Trained by adding random noise to the input data and then reconstructing the original, clean data. 94 | - **Types:** Guassian noise, Salt and pepper noise 95 | - **Application:** Useful in image denoising, audio denoising, and data preprocessing. 96 | 97 | ### 3. Convolutional Autoencoder (CAE) 98 | - **Purpose:** Designed to work with spatial data like images. 99 | - **Mechanism:** Uses convolutional and pooling layers in the encoder and deconvolutional (transposed convolution) layers in the decoder. 100 | - **Application:** Image denoising, image compression, and image generation. 101 | - **Use Case:** Used extensively in computer vision for learning compact, spatially-aware representations. 102 | 103 | ### 4. Variational Autoencoder (VAE) 104 | - **Purpose:** A probabilistic autoencoder that learns a latent distribution instead of fixed encodings. 105 | - **Mechanism:** The encoder outputs the mean and variance of a Gaussian distribution, and a sample is drawn from this distribution during training. 106 | - **Application:** Generative modeling, image synthesis, and anomaly detection. 107 | - **Use Case:** VAEs are widely used for generating new data points that are similar to the input data (e.g., generating new images of faces). 108 | 109 | ## Types of Recurrent Neural Network 110 | 111 | ### 1. Simple RNN 112 | - **Definition:** A basic RNN that processes sequences by updating its hidden state at each time step. 113 | - **Use:** Good for simple time-series data but struggles with long-term dependencies. 114 | - **Limitation:** Suffers from vanishing and exploding gradient problems. 115 | 116 | ### 2. LSTM (Long Short-Term Memory) 117 | - **Definition:** An advanced RNN designed to capture long-term dependencies using memory cells and gates. 118 | - **Components:** Includes forget, input, and output gates that control information flow. 119 | - **Advantage:** Effective in capturing context from distant past inputs, making it suitable for NLP and speech recognition. 120 | 121 | ### 3. GRU (Gated Recurrent Unit) 122 | - **Definition:** A simplified version of LSTM with fewer gates (reset and update gates) which make it computationally more efficient while still handling long-term dependencies well. 123 | - **Difference:** Has fewer parameters than LSTM, making it faster and easier to train. 124 | - **Advantage:** Provides similar performance to LSTM with reduced computational complexity. 125 | 126 | ### 4. Seq2Seq RNN (Sequence to Sequence) 127 | - **Definition:** A specialized RNN used for translating sequences (e.g., text translation). 128 | - **Structure:** Consists of an encoder (processes input) and a decoder (generates output), both using LSTMs or GRUs. 129 | - **Use:** Widely applied in machine translation, chatbots, and summarization. 130 | 131 | # Activation Functions in Neural Networks 132 | 133 | The most popular activation functions in neural networks are: 134 | 135 | ### 1. ReLU (Rectified Linear Unit) 136 | - Output: 𝑓(𝑥)= max(0,𝑥) 137 | - Use: Widely used in hidden layers of deep neural networks (especially CNNs). 138 | - Reason: Simplicity, computational efficiency, and helps with faster training. 139 | 140 | ### 2. Sigmoid 141 | - Output: 𝑓(𝑥)=1/1+𝑒−𝑥 142 | - Use: Common in binary classification and output layers. 143 | - Reason: Converts inputs into probabilities between 0 and 1. 144 | 145 | ### 3. Tanh (Hyperbolic Tangent) 146 | - Output: f(x)=tanh(x) 147 | - Use: Often used in RNNs and LSTMs for better zero-centered outputs. 148 | - Reason: Zero-centered, better gradient flow than sigmoid. 149 | 150 | ### 4. Leaky ReLU 151 | - Output: f(x)=x if x>0, otherwise f(x)=0.01x 152 | - Use: Used to overcome "dying ReLU" problem where neurons become inactive. 153 | - Reason: Allows small gradients for negative values. 154 | 155 | ### 5. Softmax 156 | - Output: Converts logits into probabilities across multiple classes. 157 | - Use: Multi-class classification (output layer). 158 | - Reason: Ensures sum of probabilities equals 1 for class predictions. 159 | 160 | ### 6. Swish 161 | - Output: f(x)= x/1+e −x 162 | - Use: Newer models like deep learning architectures from Google. 163 | - Reason: Provides better performance than ReLU in some networks. 164 | -------------------------------------------------------------------------------- /PromptEngineering.md: -------------------------------------------------------------------------------- 1 | # Prompt Engineering 2 | 3 | ## Anatomy of Prompt Engineering 4 | 5 | - Simulate Persona 6 | - Task 7 | - Steps to complete Task 8 | - Context / Constarints 9 | - Goal 10 | - Format Output 11 | - Emotional touch 12 | 13 | ## Tips of effective Prompt Engineering 14 | 15 | - **Be explicit :** Clearly specify the desired format or output. 16 | - **Provide Context :** Offer relevant information or background to guide the model. 17 | - **Control Output Length :** Instruct the model to generate responses of a specific length. 18 | - **Restrict responses :** Utilize techniques like temperature adjustment to refine output quality. 19 | - **Experiment and Iterate :** Refine prompts through experimentation and feedback loops. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Artificial Intelligence 2 | 3 | ### - Natural Language Processing NLP -> Text based 4 | Natural Language Processing (NLP) is the discipline of bulding machines that can manipulate human language - or data that resembles human language - in the way that it is written, spoken and organized. 5 | 6 | - Computer Vision -> Image & Video based 7 | - Speech Recognition -> Speech & Audio based 8 | 9 | 10 | 11 | ## 1. Artificial Narrow Intelligence ANI 12 | ### 1. Descriminative AI 13 | - Learn decision boundaries to classify or predict outcomes 14 | - Identify patterns to separate classes 15 | - Primarily used for classification, prediction, and regression 16 | - **Algorithms:** 17 | - Logistic Regression 18 | - SVMs 19 | - Random Forests 20 | - Neural Networks 21 | - Decision Trees 22 | 23 | ### 2. Generative AI 24 | - Learn data distributions to generate new samples 25 | - Model underlying data generation process 26 | - Used for generation, sampling, and unsupervised learning 27 | - **Algorithms:** 28 | - Generative Adversarial Networks (GANs) 29 | - Variational Autoencoders (VAEs) 30 | - Autoregressive Models 31 | - Transformers 32 | - Diffusion Models 33 | 34 | #### Gen AI Algorithms 35 | 36 | - **Generative Adversarial Networks (GANs):** Generate new data by pitting two neural networks against each other. 37 | - **Variational Autoencoders (VAEs):** Learn to generate new data similar to training data. 38 | - **Transformer Models:** Use attention mechanisms for generating text (e.g., GPT-3, BERT). 39 | - **Diffusion Models:** Generate high-quality images by iteratively refining random noise. 40 | - **Neural Style Transfer:** Combines content and style from different images to create new ones. 41 | 42 | ## 2. Artificial General Intelligence AGI 43 | - AGI has not been achieved yet 44 | - AGI would be able to perform any intellectual task that a human can, and potentially even surpass human capabilities in many areas. 45 | - There are five steps to achieve AGI: 46 | 47 | ![SS](/agi-five.png) 48 | 49 | ## Types of AI Models 50 | ### 1. LLM (Large Language Model) / GPT (Generative Pretrained Transformers) 51 | 52 | - Neural networks trained on large text corpora to understand and generate human-like text. 53 | - **Applications:** Text generation, translation, summarization. 54 | - **Examples:** 55 | - ChatGPT 56 | - Bard / Gemini 57 | 58 | ### 2. Diffusion Models / Latent Diffusion Model 59 | 60 | - Generative models that create high-quality data samples by iteratively refining random noise. 61 | - **Applications:** Image generation, data augmentation. 62 | - **Examples:** Denoising Diffusion Probabilistic Models (DDPM) -------------------------------------------------------------------------------- /agi-five.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RidaNaz/GenAI/8630cc4dfe97f7aab0ce7fddd3c284103e4207b6/agi-five.png -------------------------------------------------------------------------------- /attention-models.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RidaNaz/GenAI/8630cc4dfe97f7aab0ce7fddd3c284103e4207b6/attention-models.gif -------------------------------------------------------------------------------- /neural-network.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RidaNaz/GenAI/8630cc4dfe97f7aab0ce7fddd3c284103e4207b6/neural-network.gif -------------------------------------------------------------------------------- /transformers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RidaNaz/GenAI/8630cc4dfe97f7aab0ce7fddd3c284103e4207b6/transformers.png --------------------------------------------------------------------------------