├── .github └── FUNDING.yml ├── .gitignore ├── 20B_tokenizer.json ├── API_DEMO.py ├── API_DEMO_CHAT.py ├── API_DEMO_WORLD.py ├── ChatRWKV-strategy.png ├── ChatRWKV.png ├── LICENSE ├── README.md ├── RWKV-eval.png ├── RWKV-paper.png ├── RWKV-v5-benchmark-1.png ├── RWKV_in_150_lines.py ├── RWKV_v5_demo.py ├── RWKV_v6_demo.py ├── RWKV_v6_demo_cuda_bf16.py ├── chat.py ├── misc ├── lambada_test.jsonl ├── sample-1.png ├── sample-2.png ├── sample-3.png ├── sample-4.png ├── sample-5.png ├── sample-6.png └── sample-7.png ├── music ├── run.py ├── sample-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid ├── sample2-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid ├── sample3-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid └── tokenizer-midi.json ├── requirements.txt ├── run_lm_eval.py ├── rwkv-x060.png ├── rwkv_pip_package ├── LICENSE ├── MANIFEST.in ├── README.md ├── pyproject.toml └── src │ └── rwkv │ ├── __init__.py │ ├── cuda │ ├── gemm_fp16_cublas.cpp │ ├── operators.cu │ ├── rwkv5.cu │ ├── rwkv5_op.cpp │ ├── rwkv6.cu │ ├── rwkv6_op.cpp │ ├── rwkv7.cu │ ├── rwkv7_op.cpp │ └── wrapper.cpp │ ├── model.py │ ├── rwkv_tokenizer.py │ ├── rwkv_vocab_v20230424.txt │ └── utils.py ├── rwkv_state_merger.py ├── src ├── model_run.py └── utils.py ├── tokenizer ├── rwkv_tokenizer.py └── rwkv_vocab_v20230424.txt └── v2 ├── 20B_tokenizer.json ├── benchmark.py ├── benchmark_more.py ├── benchmark_world.py ├── chat.py ├── convert_model.py └── prompt └── default ├── Chinese-1.py ├── Chinese-2.py ├── English-0.py ├── English-1.py ├── English-2.py ├── English-3.py ├── English-4.py ├── English-5.py ├── Japanese-1.py └── Japanese-2.py /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | ko_fi: rwkv_lm 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/.gitignore -------------------------------------------------------------------------------- /20B_tokenizer.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/20B_tokenizer.json -------------------------------------------------------------------------------- /API_DEMO.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/API_DEMO.py -------------------------------------------------------------------------------- /API_DEMO_CHAT.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/API_DEMO_CHAT.py -------------------------------------------------------------------------------- /API_DEMO_WORLD.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/API_DEMO_WORLD.py -------------------------------------------------------------------------------- /ChatRWKV-strategy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/ChatRWKV-strategy.png -------------------------------------------------------------------------------- /ChatRWKV.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/ChatRWKV.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/README.md -------------------------------------------------------------------------------- /RWKV-eval.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/RWKV-eval.png -------------------------------------------------------------------------------- /RWKV-paper.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/RWKV-paper.png -------------------------------------------------------------------------------- /RWKV-v5-benchmark-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/RWKV-v5-benchmark-1.png -------------------------------------------------------------------------------- /RWKV_in_150_lines.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/RWKV_in_150_lines.py -------------------------------------------------------------------------------- /RWKV_v5_demo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/RWKV_v5_demo.py -------------------------------------------------------------------------------- /RWKV_v6_demo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/RWKV_v6_demo.py -------------------------------------------------------------------------------- /RWKV_v6_demo_cuda_bf16.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/RWKV_v6_demo_cuda_bf16.py -------------------------------------------------------------------------------- /chat.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/chat.py -------------------------------------------------------------------------------- /misc/lambada_test.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/misc/lambada_test.jsonl -------------------------------------------------------------------------------- /misc/sample-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/misc/sample-1.png -------------------------------------------------------------------------------- /misc/sample-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/misc/sample-2.png -------------------------------------------------------------------------------- /misc/sample-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/misc/sample-3.png -------------------------------------------------------------------------------- /misc/sample-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/misc/sample-4.png -------------------------------------------------------------------------------- /misc/sample-5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/misc/sample-5.png -------------------------------------------------------------------------------- /misc/sample-6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/misc/sample-6.png -------------------------------------------------------------------------------- /misc/sample-7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/misc/sample-7.png -------------------------------------------------------------------------------- /music/run.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/music/run.py -------------------------------------------------------------------------------- /music/sample-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/music/sample-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid -------------------------------------------------------------------------------- /music/sample2-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/music/sample2-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid -------------------------------------------------------------------------------- /music/sample3-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/music/sample3-RWKV-4-MIDI-120M-v1-20230714-ctx4096.mid -------------------------------------------------------------------------------- /music/tokenizer-midi.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/music/tokenizer-midi.json -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | tokenizers>=0.13.2 2 | prompt_toolkit 3 | -------------------------------------------------------------------------------- /run_lm_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/run_lm_eval.py -------------------------------------------------------------------------------- /rwkv-x060.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv-x060.png -------------------------------------------------------------------------------- /rwkv_pip_package/LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/LICENSE -------------------------------------------------------------------------------- /rwkv_pip_package/MANIFEST.in: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/MANIFEST.in -------------------------------------------------------------------------------- /rwkv_pip_package/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/README.md -------------------------------------------------------------------------------- /rwkv_pip_package/pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/pyproject.toml -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/gemm_fp16_cublas.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/gemm_fp16_cublas.cpp -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/operators.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/operators.cu -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/rwkv5.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/rwkv5.cu -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/rwkv5_op.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/rwkv5_op.cpp -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/rwkv6.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/rwkv6.cu -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/rwkv6_op.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/rwkv6_op.cpp -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/rwkv7.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/rwkv7.cu -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/rwkv7_op.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/rwkv7_op.cpp -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/cuda/wrapper.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/cuda/wrapper.cpp -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/model.py -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/rwkv_tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/rwkv_tokenizer.py -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/rwkv_vocab_v20230424.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/rwkv_vocab_v20230424.txt -------------------------------------------------------------------------------- /rwkv_pip_package/src/rwkv/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_pip_package/src/rwkv/utils.py -------------------------------------------------------------------------------- /rwkv_state_merger.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/rwkv_state_merger.py -------------------------------------------------------------------------------- /src/model_run.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/src/model_run.py -------------------------------------------------------------------------------- /src/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/src/utils.py -------------------------------------------------------------------------------- /tokenizer/rwkv_tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/tokenizer/rwkv_tokenizer.py -------------------------------------------------------------------------------- /tokenizer/rwkv_vocab_v20230424.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/tokenizer/rwkv_vocab_v20230424.txt -------------------------------------------------------------------------------- /v2/20B_tokenizer.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/20B_tokenizer.json -------------------------------------------------------------------------------- /v2/benchmark.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/benchmark.py -------------------------------------------------------------------------------- /v2/benchmark_more.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/benchmark_more.py -------------------------------------------------------------------------------- /v2/benchmark_world.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/benchmark_world.py -------------------------------------------------------------------------------- /v2/chat.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/chat.py -------------------------------------------------------------------------------- /v2/convert_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/convert_model.py -------------------------------------------------------------------------------- /v2/prompt/default/Chinese-1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/Chinese-1.py -------------------------------------------------------------------------------- /v2/prompt/default/Chinese-2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/Chinese-2.py -------------------------------------------------------------------------------- /v2/prompt/default/English-0.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/English-0.py -------------------------------------------------------------------------------- /v2/prompt/default/English-1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/English-1.py -------------------------------------------------------------------------------- /v2/prompt/default/English-2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/English-2.py -------------------------------------------------------------------------------- /v2/prompt/default/English-3.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/English-3.py -------------------------------------------------------------------------------- /v2/prompt/default/English-4.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/English-4.py -------------------------------------------------------------------------------- /v2/prompt/default/English-5.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/English-5.py -------------------------------------------------------------------------------- /v2/prompt/default/Japanese-1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/Japanese-1.py -------------------------------------------------------------------------------- /v2/prompt/default/Japanese-2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/BlinkDL/ChatRWKV/HEAD/v2/prompt/default/Japanese-2.py --------------------------------------------------------------------------------