├── .DS_Store ├── LICENSE ├── README.md ├── TUTORIAL.md ├── assets ├── .DS_Store ├── alpacaeval.png ├── live_loss.jpg ├── logo.jpg ├── openllm.png ├── teaser_a.jpg ├── teaser_b.jpg ├── teaser_c.jpg └── teaser_d.jpg ├── conversation.py ├── minima ├── data │ ├── __init__.py │ ├── example_pb2.py │ ├── pipelines.py │ └── tfrecord_io.py ├── ds_config.json ├── ds_config_zero2.json ├── models │ ├── .DS_Store │ ├── __init__.py │ ├── llama_instruct.py │ ├── llama_lm.py │ └── sparsellama_lm.py ├── modules │ ├── .DS_Store │ ├── __init__.py │ ├── flash_attn_monkey_patch_llama.py │ ├── flash_attn_monkey_patch_sparsellama.py │ ├── modeling_llama.py │ └── modeling_sparsellama.py ├── run_building_data_llama.py ├── run_building_data_minichat.py ├── run_converting_llama.py ├── run_distillation_llama_ds.py ├── run_expanding_vocab_llama.py ├── run_instruction_llama_ds.py ├── run_instruction_llama_lora_ds.py ├── run_pretraining_llama_ds.py ├── run_sparsification_llama.py ├── scripts │ ├── .DS_Store │ ├── adapt_llama.sh │ ├── build_instruction_data.sh │ ├── build_pretraining_data.sh │ ├── build_pruning_data.sh │ ├── convert_llama.sh │ ├── distil_minima.sh │ ├── expand_vocab.sh │ ├── install_requirements.sh │ ├── instruct_minichat.sh │ ├── instruct_minichat_lora.sh │ └── prune_llama.sh └── utils.py └── mt_bench_zh ├── README.md ├── data ├── judge_prompts_zh.jsonl └── mt_bench_zh │ ├── question.jsonl │ └── reference_answer │ └── gpt-4.jsonl ├── gen_judgment.py ├── gen_model_answer.py └── show_result.py /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/.DS_Store -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/README.md -------------------------------------------------------------------------------- /TUTORIAL.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/TUTORIAL.md -------------------------------------------------------------------------------- /assets/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/.DS_Store -------------------------------------------------------------------------------- /assets/alpacaeval.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/alpacaeval.png -------------------------------------------------------------------------------- /assets/live_loss.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/live_loss.jpg -------------------------------------------------------------------------------- /assets/logo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/logo.jpg -------------------------------------------------------------------------------- /assets/openllm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/openllm.png -------------------------------------------------------------------------------- /assets/teaser_a.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/teaser_a.jpg -------------------------------------------------------------------------------- /assets/teaser_b.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/teaser_b.jpg -------------------------------------------------------------------------------- /assets/teaser_c.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/teaser_c.jpg -------------------------------------------------------------------------------- /assets/teaser_d.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/assets/teaser_d.jpg -------------------------------------------------------------------------------- /conversation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/conversation.py -------------------------------------------------------------------------------- /minima/data/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/data/__init__.py -------------------------------------------------------------------------------- /minima/data/example_pb2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/data/example_pb2.py -------------------------------------------------------------------------------- /minima/data/pipelines.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/data/pipelines.py -------------------------------------------------------------------------------- /minima/data/tfrecord_io.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/data/tfrecord_io.py -------------------------------------------------------------------------------- /minima/ds_config.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/ds_config.json -------------------------------------------------------------------------------- /minima/ds_config_zero2.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/ds_config_zero2.json -------------------------------------------------------------------------------- /minima/models/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/models/.DS_Store -------------------------------------------------------------------------------- /minima/models/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/models/__init__.py -------------------------------------------------------------------------------- /minima/models/llama_instruct.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/models/llama_instruct.py -------------------------------------------------------------------------------- /minima/models/llama_lm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/models/llama_lm.py -------------------------------------------------------------------------------- /minima/models/sparsellama_lm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/models/sparsellama_lm.py -------------------------------------------------------------------------------- /minima/modules/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/modules/.DS_Store -------------------------------------------------------------------------------- /minima/modules/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/modules/__init__.py -------------------------------------------------------------------------------- /minima/modules/flash_attn_monkey_patch_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/modules/flash_attn_monkey_patch_llama.py -------------------------------------------------------------------------------- /minima/modules/flash_attn_monkey_patch_sparsellama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/modules/flash_attn_monkey_patch_sparsellama.py -------------------------------------------------------------------------------- /minima/modules/modeling_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/modules/modeling_llama.py -------------------------------------------------------------------------------- /minima/modules/modeling_sparsellama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/modules/modeling_sparsellama.py -------------------------------------------------------------------------------- /minima/run_building_data_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_building_data_llama.py -------------------------------------------------------------------------------- /minima/run_building_data_minichat.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_building_data_minichat.py -------------------------------------------------------------------------------- /minima/run_converting_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_converting_llama.py -------------------------------------------------------------------------------- /minima/run_distillation_llama_ds.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_distillation_llama_ds.py -------------------------------------------------------------------------------- /minima/run_expanding_vocab_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_expanding_vocab_llama.py -------------------------------------------------------------------------------- /minima/run_instruction_llama_ds.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_instruction_llama_ds.py -------------------------------------------------------------------------------- /minima/run_instruction_llama_lora_ds.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_instruction_llama_lora_ds.py -------------------------------------------------------------------------------- /minima/run_pretraining_llama_ds.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_pretraining_llama_ds.py -------------------------------------------------------------------------------- /minima/run_sparsification_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/run_sparsification_llama.py -------------------------------------------------------------------------------- /minima/scripts/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/.DS_Store -------------------------------------------------------------------------------- /minima/scripts/adapt_llama.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/adapt_llama.sh -------------------------------------------------------------------------------- /minima/scripts/build_instruction_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/build_instruction_data.sh -------------------------------------------------------------------------------- /minima/scripts/build_pretraining_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/build_pretraining_data.sh -------------------------------------------------------------------------------- /minima/scripts/build_pruning_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/build_pruning_data.sh -------------------------------------------------------------------------------- /minima/scripts/convert_llama.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/convert_llama.sh -------------------------------------------------------------------------------- /minima/scripts/distil_minima.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/distil_minima.sh -------------------------------------------------------------------------------- /minima/scripts/expand_vocab.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/expand_vocab.sh -------------------------------------------------------------------------------- /minima/scripts/install_requirements.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/install_requirements.sh -------------------------------------------------------------------------------- /minima/scripts/instruct_minichat.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/instruct_minichat.sh -------------------------------------------------------------------------------- /minima/scripts/instruct_minichat_lora.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/instruct_minichat_lora.sh -------------------------------------------------------------------------------- /minima/scripts/prune_llama.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/scripts/prune_llama.sh -------------------------------------------------------------------------------- /minima/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/minima/utils.py -------------------------------------------------------------------------------- /mt_bench_zh/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/mt_bench_zh/README.md -------------------------------------------------------------------------------- /mt_bench_zh/data/judge_prompts_zh.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/mt_bench_zh/data/judge_prompts_zh.jsonl -------------------------------------------------------------------------------- /mt_bench_zh/data/mt_bench_zh/question.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/mt_bench_zh/data/mt_bench_zh/question.jsonl -------------------------------------------------------------------------------- /mt_bench_zh/data/mt_bench_zh/reference_answer/gpt-4.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/mt_bench_zh/data/mt_bench_zh/reference_answer/gpt-4.jsonl -------------------------------------------------------------------------------- /mt_bench_zh/gen_judgment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/mt_bench_zh/gen_judgment.py -------------------------------------------------------------------------------- /mt_bench_zh/gen_model_answer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/mt_bench_zh/gen_model_answer.py -------------------------------------------------------------------------------- /mt_bench_zh/show_result.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeneZC/MiniMA/HEAD/mt_bench_zh/show_result.py --------------------------------------------------------------------------------