├── Bert ├── Bert.py └── Modules.py ├── GPT2 ├── attention.py ├── data.py ├── gpt2_train.py ├── gpt_model.py └── text_sampler.py ├── Llama ├── Rotatory.py ├── SelfAttention_GQ.py └── llama_basic.py ├── Mistral ├── Gqa_sa.py ├── mistral.py └── utils │ └── rope.py ├── Moe └── mixture_of_experts.py ├── Readme.md ├── data ├── shakespeare.txt └── the-verdict.txt └── images ├── Attentions.jpg └── Transformer.png /Bert/Bert.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Bert/Bert.py -------------------------------------------------------------------------------- /Bert/Modules.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Bert/Modules.py -------------------------------------------------------------------------------- /GPT2/attention.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/GPT2/attention.py -------------------------------------------------------------------------------- /GPT2/data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/GPT2/data.py -------------------------------------------------------------------------------- /GPT2/gpt2_train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/GPT2/gpt2_train.py -------------------------------------------------------------------------------- /GPT2/gpt_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/GPT2/gpt_model.py -------------------------------------------------------------------------------- /GPT2/text_sampler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/GPT2/text_sampler.py -------------------------------------------------------------------------------- /Llama/Rotatory.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Llama/Rotatory.py -------------------------------------------------------------------------------- /Llama/SelfAttention_GQ.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Llama/SelfAttention_GQ.py -------------------------------------------------------------------------------- /Llama/llama_basic.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Llama/llama_basic.py -------------------------------------------------------------------------------- /Mistral/Gqa_sa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Mistral/Gqa_sa.py -------------------------------------------------------------------------------- /Mistral/mistral.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Mistral/mistral.py -------------------------------------------------------------------------------- /Mistral/utils/rope.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Mistral/utils/rope.py -------------------------------------------------------------------------------- /Moe/mixture_of_experts.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Moe/mixture_of_experts.py -------------------------------------------------------------------------------- /Readme.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/Readme.md -------------------------------------------------------------------------------- /data/shakespeare.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/data/shakespeare.txt -------------------------------------------------------------------------------- /data/the-verdict.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/data/the-verdict.txt -------------------------------------------------------------------------------- /images/Attentions.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/images/Attentions.jpg -------------------------------------------------------------------------------- /images/Transformer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kmohan321/LLMs/HEAD/images/Transformer.png --------------------------------------------------------------------------------