├── .gitignore ├── README.md ├── assets ├── dcolt.png └── metrics_v0.png ├── code_metrics.py ├── configs ├── accelerate │ └── fsdp.yaml └── gsm8k_64step_example.yaml ├── custom_humaneval ├── data.py ├── evaluate_functional_correctness.py ├── evaluation.py └── execution.py ├── dataloaders ├── collate_fn_math.py ├── math.py └── sampler.py ├── datasets ├── HumanEval.jsonl.gz ├── mbpp.jsonl └── mbpp_test.jsonl ├── dnnlib ├── __init__.py └── util.py ├── evaluate ├── __init__.py ├── grader.py ├── humaneval.py ├── mbpp.py └── parser.py ├── math_metrics.py ├── networks ├── configuration_llada.py ├── layers.py ├── lladou_v0.py └── modeling_llada.py ├── requirements.txt ├── scripts ├── eval_code.sh └── eval_math.sh ├── torch_utils ├── __init__.py ├── distributed.py └── misc.py ├── train.py └── training ├── training_loop_fsdp.py └── utils └── lr_scheduler.py /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | /models 3 | /datasets 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/README.md -------------------------------------------------------------------------------- /assets/dcolt.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/assets/dcolt.png -------------------------------------------------------------------------------- /assets/metrics_v0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/assets/metrics_v0.png -------------------------------------------------------------------------------- /code_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/code_metrics.py -------------------------------------------------------------------------------- /configs/accelerate/fsdp.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/configs/accelerate/fsdp.yaml -------------------------------------------------------------------------------- /configs/gsm8k_64step_example.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/configs/gsm8k_64step_example.yaml -------------------------------------------------------------------------------- /custom_humaneval/data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/custom_humaneval/data.py -------------------------------------------------------------------------------- /custom_humaneval/evaluate_functional_correctness.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/custom_humaneval/evaluate_functional_correctness.py -------------------------------------------------------------------------------- /custom_humaneval/evaluation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/custom_humaneval/evaluation.py -------------------------------------------------------------------------------- /custom_humaneval/execution.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/custom_humaneval/execution.py -------------------------------------------------------------------------------- /dataloaders/collate_fn_math.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/dataloaders/collate_fn_math.py -------------------------------------------------------------------------------- /dataloaders/math.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/dataloaders/math.py -------------------------------------------------------------------------------- /dataloaders/sampler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/dataloaders/sampler.py -------------------------------------------------------------------------------- /datasets/HumanEval.jsonl.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/datasets/HumanEval.jsonl.gz -------------------------------------------------------------------------------- /datasets/mbpp.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/datasets/mbpp.jsonl -------------------------------------------------------------------------------- /datasets/mbpp_test.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/datasets/mbpp_test.jsonl -------------------------------------------------------------------------------- /dnnlib/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/dnnlib/__init__.py -------------------------------------------------------------------------------- /dnnlib/util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/dnnlib/util.py -------------------------------------------------------------------------------- /evaluate/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/evaluate/__init__.py -------------------------------------------------------------------------------- /evaluate/grader.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/evaluate/grader.py -------------------------------------------------------------------------------- /evaluate/humaneval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/evaluate/humaneval.py -------------------------------------------------------------------------------- /evaluate/mbpp.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/evaluate/mbpp.py -------------------------------------------------------------------------------- /evaluate/parser.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/evaluate/parser.py -------------------------------------------------------------------------------- /math_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/math_metrics.py -------------------------------------------------------------------------------- /networks/configuration_llada.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/networks/configuration_llada.py -------------------------------------------------------------------------------- /networks/layers.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/networks/layers.py -------------------------------------------------------------------------------- /networks/lladou_v0.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/networks/lladou_v0.py -------------------------------------------------------------------------------- /networks/modeling_llada.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/networks/modeling_llada.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/requirements.txt -------------------------------------------------------------------------------- /scripts/eval_code.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/scripts/eval_code.sh -------------------------------------------------------------------------------- /scripts/eval_math.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/scripts/eval_math.sh -------------------------------------------------------------------------------- /torch_utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/torch_utils/__init__.py -------------------------------------------------------------------------------- /torch_utils/distributed.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/torch_utils/distributed.py -------------------------------------------------------------------------------- /torch_utils/misc.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/torch_utils/misc.py -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/train.py -------------------------------------------------------------------------------- /training/training_loop_fsdp.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/training/training_loop_fsdp.py -------------------------------------------------------------------------------- /training/utils/lr_scheduler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/maple-research-lab/LLaDOU/HEAD/training/utils/lr_scheduler.py --------------------------------------------------------------------------------