├── .gitignore ├── LICENSE ├── README.md ├── batch.py ├── benchmark.py ├── demo.py ├── demo2.py ├── eval ├── GSM8K_100sample.jsonl ├── MATH500_100sample.jsonl ├── calibration_data_v5_rc.txt ├── lambada_test.jsonl └── mmlu_test_dataset │ ├── data-00000-of-00001.arrow │ ├── dataset_info.json │ └── state.json ├── faster_251101 ├── LICENSE ├── README.md ├── batch.py ├── benchmark.py ├── demo.py ├── demo2.py ├── eval │ ├── GSM8K_100sample.jsonl │ ├── MATH500_100sample.jsonl │ ├── calibration_data_v5_rc.txt │ ├── lambada_test.jsonl │ └── mmlu_test_dataset │ │ ├── data-00000-of-00001.arrow │ │ ├── dataset_info.json │ │ └── state.json ├── reference │ ├── __init__.py │ ├── cuda │ │ ├── rwkv7_state_fwd_fp16.cpp │ │ └── rwkv7_state_fwd_fp16.cu │ ├── rwkv7.py │ ├── rwkv_vocab_v20230424.txt │ └── utils.py └── rollout.py ├── reference ├── __init__.py ├── cuda │ ├── rwkv7_state_fwd_fp16.cpp │ └── rwkv7_state_fwd_fp16.cu ├── rwkv7.py ├── rwkv_vocab_v20230424.txt └── utils.py └── rollout.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/README.md -------------------------------------------------------------------------------- /batch.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/batch.py -------------------------------------------------------------------------------- /benchmark.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/benchmark.py -------------------------------------------------------------------------------- /demo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/demo.py -------------------------------------------------------------------------------- /demo2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/demo2.py -------------------------------------------------------------------------------- /eval/GSM8K_100sample.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/eval/GSM8K_100sample.jsonl -------------------------------------------------------------------------------- /eval/MATH500_100sample.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/eval/MATH500_100sample.jsonl -------------------------------------------------------------------------------- /eval/calibration_data_v5_rc.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/eval/calibration_data_v5_rc.txt -------------------------------------------------------------------------------- /eval/lambada_test.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/eval/lambada_test.jsonl -------------------------------------------------------------------------------- /eval/mmlu_test_dataset/data-00000-of-00001.arrow: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/eval/mmlu_test_dataset/data-00000-of-00001.arrow -------------------------------------------------------------------------------- /eval/mmlu_test_dataset/dataset_info.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/eval/mmlu_test_dataset/dataset_info.json -------------------------------------------------------------------------------- /eval/mmlu_test_dataset/state.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/eval/mmlu_test_dataset/state.json -------------------------------------------------------------------------------- /faster_251101/LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/LICENSE -------------------------------------------------------------------------------- /faster_251101/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/README.md -------------------------------------------------------------------------------- /faster_251101/batch.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/batch.py -------------------------------------------------------------------------------- /faster_251101/benchmark.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/benchmark.py -------------------------------------------------------------------------------- /faster_251101/demo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/demo.py -------------------------------------------------------------------------------- /faster_251101/demo2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/demo2.py -------------------------------------------------------------------------------- /faster_251101/eval/GSM8K_100sample.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/eval/GSM8K_100sample.jsonl -------------------------------------------------------------------------------- /faster_251101/eval/MATH500_100sample.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/eval/MATH500_100sample.jsonl -------------------------------------------------------------------------------- /faster_251101/eval/calibration_data_v5_rc.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/eval/calibration_data_v5_rc.txt -------------------------------------------------------------------------------- /faster_251101/eval/lambada_test.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/eval/lambada_test.jsonl -------------------------------------------------------------------------------- /faster_251101/eval/mmlu_test_dataset/data-00000-of-00001.arrow: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/eval/mmlu_test_dataset/data-00000-of-00001.arrow -------------------------------------------------------------------------------- /faster_251101/eval/mmlu_test_dataset/dataset_info.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/eval/mmlu_test_dataset/dataset_info.json -------------------------------------------------------------------------------- /faster_251101/eval/mmlu_test_dataset/state.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/eval/mmlu_test_dataset/state.json -------------------------------------------------------------------------------- /faster_251101/reference/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /faster_251101/reference/cuda/rwkv7_state_fwd_fp16.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/reference/cuda/rwkv7_state_fwd_fp16.cpp -------------------------------------------------------------------------------- /faster_251101/reference/cuda/rwkv7_state_fwd_fp16.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/reference/cuda/rwkv7_state_fwd_fp16.cu -------------------------------------------------------------------------------- /faster_251101/reference/rwkv7.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/reference/rwkv7.py -------------------------------------------------------------------------------- /faster_251101/reference/rwkv_vocab_v20230424.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/reference/rwkv_vocab_v20230424.txt -------------------------------------------------------------------------------- /faster_251101/reference/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/reference/utils.py -------------------------------------------------------------------------------- /faster_251101/rollout.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/faster_251101/rollout.py -------------------------------------------------------------------------------- /reference/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /reference/cuda/rwkv7_state_fwd_fp16.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/reference/cuda/rwkv7_state_fwd_fp16.cpp -------------------------------------------------------------------------------- /reference/cuda/rwkv7_state_fwd_fp16.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/reference/cuda/rwkv7_state_fwd_fp16.cu -------------------------------------------------------------------------------- /reference/rwkv7.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/reference/rwkv7.py -------------------------------------------------------------------------------- /reference/rwkv_vocab_v20230424.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/reference/rwkv_vocab_v20230424.txt -------------------------------------------------------------------------------- /reference/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/reference/utils.py -------------------------------------------------------------------------------- /rollout.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/Albatross/HEAD/rollout.py --------------------------------------------------------------------------------