├── .gitignore ├── CLEX ├── __init__.py ├── clex_layer.py ├── llama │ ├── configuration_llama_clex.py │ └── modeling_llama_clex.py ├── mixtral │ ├── configuration_mixtral_clex.py │ └── modeling_mixtral_clex.py └── phi2 │ ├── configuration_phi2_clex.py │ └── modeling_phi2_clex.py ├── LICENSE ├── LICENSE_LLAMA2 ├── README.md ├── playground ├── deepspeed_zero2.json ├── deepspeed_zero3.json ├── deepspeed_zero3_offload.json └── sft_accelerate_zero3.yaml ├── requirements.txt ├── scripts ├── eval.sh ├── train_chat.sh └── train_lm.sh ├── serve ├── cli.py └── inference.py ├── setup.py └── train ├── __init__.py ├── argument.py ├── dataset.py ├── eval_lm.py ├── train_lm.py ├── train_sft.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/.gitignore -------------------------------------------------------------------------------- /CLEX/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/CLEX/__init__.py -------------------------------------------------------------------------------- /CLEX/clex_layer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/CLEX/clex_layer.py -------------------------------------------------------------------------------- /CLEX/llama/configuration_llama_clex.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/CLEX/llama/configuration_llama_clex.py -------------------------------------------------------------------------------- /CLEX/llama/modeling_llama_clex.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/CLEX/llama/modeling_llama_clex.py -------------------------------------------------------------------------------- /CLEX/mixtral/configuration_mixtral_clex.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/CLEX/mixtral/configuration_mixtral_clex.py -------------------------------------------------------------------------------- /CLEX/mixtral/modeling_mixtral_clex.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/CLEX/mixtral/modeling_mixtral_clex.py -------------------------------------------------------------------------------- /CLEX/phi2/configuration_phi2_clex.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/CLEX/phi2/configuration_phi2_clex.py -------------------------------------------------------------------------------- /CLEX/phi2/modeling_phi2_clex.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/CLEX/phi2/modeling_phi2_clex.py -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/LICENSE -------------------------------------------------------------------------------- /LICENSE_LLAMA2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/LICENSE_LLAMA2 -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/README.md -------------------------------------------------------------------------------- /playground/deepspeed_zero2.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/playground/deepspeed_zero2.json -------------------------------------------------------------------------------- /playground/deepspeed_zero3.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/playground/deepspeed_zero3.json -------------------------------------------------------------------------------- /playground/deepspeed_zero3_offload.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/playground/deepspeed_zero3_offload.json -------------------------------------------------------------------------------- /playground/sft_accelerate_zero3.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/playground/sft_accelerate_zero3.yaml -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/requirements.txt -------------------------------------------------------------------------------- /scripts/eval.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/scripts/eval.sh -------------------------------------------------------------------------------- /scripts/train_chat.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/scripts/train_chat.sh -------------------------------------------------------------------------------- /scripts/train_lm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/scripts/train_lm.sh -------------------------------------------------------------------------------- /serve/cli.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/serve/cli.py -------------------------------------------------------------------------------- /serve/inference.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/serve/inference.py -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/setup.py -------------------------------------------------------------------------------- /train/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /train/argument.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/train/argument.py -------------------------------------------------------------------------------- /train/dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/train/dataset.py -------------------------------------------------------------------------------- /train/eval_lm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/train/eval_lm.py -------------------------------------------------------------------------------- /train/train_lm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/train/train_lm.py -------------------------------------------------------------------------------- /train/train_sft.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/train/train_sft.py -------------------------------------------------------------------------------- /train/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DAMO-NLP-SG/CLEX/HEAD/train/utils.py --------------------------------------------------------------------------------