├── .gitignore ├── LLM_Base ├── 001_Transformer.md ├── 002_Normalization.md ├── 003_Position-Embedding.md ├── 004_Activation.md ├── 005_Transformer-Other.md ├── 006_Loss-Function.md ├── 007_LLM-Frame.md ├── 008_Optimizer.md ├── 009_MoE.md ├── 010_Tokenizer.md ├── 基础内容面试点.md ├── 大模型技术栈-算法与原理.md └── 手撕内容.md ├── NLP_coding ├── BP │ ├── BP01.py │ ├── BP02.py │ └── BP03.py ├── PyTorch │ ├── t0_tensor.py │ ├── t1_pipeline.py │ ├── t2_module.py │ ├── t3_operater.py │ ├── t4_nn_element.py │ ├── t5_optim.py │ ├── t6_dataloader.py │ ├── t7_dynamic_graph.py │ └── t8_distributed-train.py ├── activation │ ├── Untitled.ipynb │ └── activations.py ├── data │ ├── iris.csv │ ├── iris_numbers.csv │ └── temp_read_data.py ├── fine-tuning │ ├── bert_cls_lora.py │ ├── demo.py │ └── my_bert_model.py ├── hf │ ├── peft.py │ └── transformer.py ├── normlization │ ├── DyT.py │ └── total_note.ipynb ├── pipeline01 │ ├── README.md │ ├── config.py │ ├── iris_data_process.py │ ├── my_metrics.py │ ├── mynet.py │ ├── save_model │ │ ├── 0_model.pth │ │ ├── 160_model.pth │ │ ├── 299_model.pth │ │ ├── 50_model.pth │ │ └── checkpoint.pth │ └── train_baseline.py ├── position_embedding │ └── total_PE.ipynb ├── test │ ├── BP.py │ ├── DyT.py │ ├── activations.py │ ├── full_mudel.py │ ├── hf_tokenizer.py │ ├── loss_func.py │ ├── pt_operator.py │ ├── pt_param_analyse.py │ └── pytorch_quick_use.py ├── transformer │ └── MHA.py └── utils │ └── __init__.py └── README.md /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/.gitignore -------------------------------------------------------------------------------- /LLM_Base/001_Transformer.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/001_Transformer.md -------------------------------------------------------------------------------- /LLM_Base/002_Normalization.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/002_Normalization.md -------------------------------------------------------------------------------- /LLM_Base/003_Position-Embedding.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/003_Position-Embedding.md -------------------------------------------------------------------------------- /LLM_Base/004_Activation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/004_Activation.md -------------------------------------------------------------------------------- /LLM_Base/005_Transformer-Other.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/005_Transformer-Other.md -------------------------------------------------------------------------------- /LLM_Base/006_Loss-Function.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/006_Loss-Function.md -------------------------------------------------------------------------------- /LLM_Base/007_LLM-Frame.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /LLM_Base/008_Optimizer.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /LLM_Base/009_MoE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/009_MoE.md -------------------------------------------------------------------------------- /LLM_Base/010_Tokenizer.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /LLM_Base/基础内容面试点.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/基础内容面试点.md -------------------------------------------------------------------------------- /LLM_Base/大模型技术栈-算法与原理.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/大模型技术栈-算法与原理.md -------------------------------------------------------------------------------- /LLM_Base/手撕内容.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/LLM_Base/手撕内容.md -------------------------------------------------------------------------------- /NLP_coding/BP/BP01.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/BP/BP01.py -------------------------------------------------------------------------------- /NLP_coding/BP/BP02.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/BP/BP02.py -------------------------------------------------------------------------------- /NLP_coding/BP/BP03.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/BP/BP03.py -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t0_tensor.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t1_pipeline.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t2_module.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t3_operater.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t4_nn_element.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t5_optim.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t6_dataloader.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t7_dynamic_graph.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/PyTorch/t8_distributed-train.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/activation/Untitled.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/activation/Untitled.ipynb -------------------------------------------------------------------------------- /NLP_coding/activation/activations.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/activation/activations.py -------------------------------------------------------------------------------- /NLP_coding/data/iris.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/data/iris.csv -------------------------------------------------------------------------------- /NLP_coding/data/iris_numbers.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/data/iris_numbers.csv -------------------------------------------------------------------------------- /NLP_coding/data/temp_read_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/data/temp_read_data.py -------------------------------------------------------------------------------- /NLP_coding/fine-tuning/bert_cls_lora.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/fine-tuning/bert_cls_lora.py -------------------------------------------------------------------------------- /NLP_coding/fine-tuning/demo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/fine-tuning/demo.py -------------------------------------------------------------------------------- /NLP_coding/fine-tuning/my_bert_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/fine-tuning/my_bert_model.py -------------------------------------------------------------------------------- /NLP_coding/hf/peft.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/hf/peft.py -------------------------------------------------------------------------------- /NLP_coding/hf/transformer.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /NLP_coding/normlization/DyT.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/normlization/DyT.py -------------------------------------------------------------------------------- /NLP_coding/normlization/total_note.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/normlization/total_note.ipynb -------------------------------------------------------------------------------- /NLP_coding/pipeline01/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/README.md -------------------------------------------------------------------------------- /NLP_coding/pipeline01/config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/config.py -------------------------------------------------------------------------------- /NLP_coding/pipeline01/iris_data_process.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/iris_data_process.py -------------------------------------------------------------------------------- /NLP_coding/pipeline01/my_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/my_metrics.py -------------------------------------------------------------------------------- /NLP_coding/pipeline01/mynet.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/mynet.py -------------------------------------------------------------------------------- /NLP_coding/pipeline01/save_model/0_model.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/save_model/0_model.pth -------------------------------------------------------------------------------- /NLP_coding/pipeline01/save_model/160_model.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/save_model/160_model.pth -------------------------------------------------------------------------------- /NLP_coding/pipeline01/save_model/299_model.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/save_model/299_model.pth -------------------------------------------------------------------------------- /NLP_coding/pipeline01/save_model/50_model.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/save_model/50_model.pth -------------------------------------------------------------------------------- /NLP_coding/pipeline01/save_model/checkpoint.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/save_model/checkpoint.pth -------------------------------------------------------------------------------- /NLP_coding/pipeline01/train_baseline.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/pipeline01/train_baseline.py -------------------------------------------------------------------------------- /NLP_coding/position_embedding/total_PE.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/position_embedding/total_PE.ipynb -------------------------------------------------------------------------------- /NLP_coding/test/BP.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/BP.py -------------------------------------------------------------------------------- /NLP_coding/test/DyT.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/DyT.py -------------------------------------------------------------------------------- /NLP_coding/test/activations.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/activations.py -------------------------------------------------------------------------------- /NLP_coding/test/full_mudel.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/full_mudel.py -------------------------------------------------------------------------------- /NLP_coding/test/hf_tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/hf_tokenizer.py -------------------------------------------------------------------------------- /NLP_coding/test/loss_func.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/loss_func.py -------------------------------------------------------------------------------- /NLP_coding/test/pt_operator.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/pt_operator.py -------------------------------------------------------------------------------- /NLP_coding/test/pt_param_analyse.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/pt_param_analyse.py -------------------------------------------------------------------------------- /NLP_coding/test/pytorch_quick_use.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/test/pytorch_quick_use.py -------------------------------------------------------------------------------- /NLP_coding/transformer/MHA.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/NLP_coding/transformer/MHA.py -------------------------------------------------------------------------------- /NLP_coding/utils/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/EthanLiu6/LLM_knowledge/HEAD/README.md --------------------------------------------------------------------------------