├── .gitignore ├── EasyLM ├── __init__.py ├── bpt.py ├── checkpoint.py ├── data.py ├── jax_utils.py ├── models │ ├── __init__.py │ └── llama │ │ ├── convert_easylm_to_hf.py │ │ ├── convert_hf_to_easylm.py │ │ ├── llama_model.py │ │ ├── llama_serve.py │ │ └── llama_train.py ├── optimizers.py ├── scripts │ ├── __init__.py │ ├── benchmark_attention.py │ ├── convert_checkpoint.py │ ├── diff_checkpoint.py │ ├── lm_eval_harness.py │ └── lm_eval_json.py └── serving.py ├── LICENSE ├── README.md ├── docs ├── README.md ├── checkpointing.md ├── dataset.md ├── evaluation.md ├── koala.md ├── llama.md ├── logger.md ├── optimizer.md ├── parallelism.md └── serving.md ├── examples ├── pretrain_llama_7b.sh └── serve_llama_7b.sh └── scripts ├── gpu_environment.yml ├── tpu_commands.sh └── tpu_vm_setup.sh /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/.gitignore -------------------------------------------------------------------------------- /EasyLM/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /EasyLM/bpt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/bpt.py -------------------------------------------------------------------------------- /EasyLM/checkpoint.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/checkpoint.py -------------------------------------------------------------------------------- /EasyLM/data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/data.py -------------------------------------------------------------------------------- /EasyLM/jax_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/jax_utils.py -------------------------------------------------------------------------------- /EasyLM/models/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /EasyLM/models/llama/convert_easylm_to_hf.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/models/llama/convert_easylm_to_hf.py -------------------------------------------------------------------------------- /EasyLM/models/llama/convert_hf_to_easylm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/models/llama/convert_hf_to_easylm.py -------------------------------------------------------------------------------- /EasyLM/models/llama/llama_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/models/llama/llama_model.py -------------------------------------------------------------------------------- /EasyLM/models/llama/llama_serve.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/models/llama/llama_serve.py -------------------------------------------------------------------------------- /EasyLM/models/llama/llama_train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/models/llama/llama_train.py -------------------------------------------------------------------------------- /EasyLM/optimizers.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/optimizers.py -------------------------------------------------------------------------------- /EasyLM/scripts/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /EasyLM/scripts/benchmark_attention.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/scripts/benchmark_attention.py -------------------------------------------------------------------------------- /EasyLM/scripts/convert_checkpoint.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/scripts/convert_checkpoint.py -------------------------------------------------------------------------------- /EasyLM/scripts/diff_checkpoint.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/scripts/diff_checkpoint.py -------------------------------------------------------------------------------- /EasyLM/scripts/lm_eval_harness.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/scripts/lm_eval_harness.py -------------------------------------------------------------------------------- /EasyLM/scripts/lm_eval_json.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/scripts/lm_eval_json.py -------------------------------------------------------------------------------- /EasyLM/serving.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/EasyLM/serving.py -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/README.md -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/README.md -------------------------------------------------------------------------------- /docs/checkpointing.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/checkpointing.md -------------------------------------------------------------------------------- /docs/dataset.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/dataset.md -------------------------------------------------------------------------------- /docs/evaluation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/evaluation.md -------------------------------------------------------------------------------- /docs/koala.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/koala.md -------------------------------------------------------------------------------- /docs/llama.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/llama.md -------------------------------------------------------------------------------- /docs/logger.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/logger.md -------------------------------------------------------------------------------- /docs/optimizer.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/optimizer.md -------------------------------------------------------------------------------- /docs/parallelism.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/parallelism.md -------------------------------------------------------------------------------- /docs/serving.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/docs/serving.md -------------------------------------------------------------------------------- /examples/pretrain_llama_7b.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/examples/pretrain_llama_7b.sh -------------------------------------------------------------------------------- /examples/serve_llama_7b.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/examples/serve_llama_7b.sh -------------------------------------------------------------------------------- /scripts/gpu_environment.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/scripts/gpu_environment.yml -------------------------------------------------------------------------------- /scripts/tpu_commands.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/scripts/tpu_commands.sh -------------------------------------------------------------------------------- /scripts/tpu_vm_setup.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/young-geng/EasyLM/HEAD/scripts/tpu_vm_setup.sh --------------------------------------------------------------------------------