├── .gitignore ├── KV_process.py ├── KVcache_manager.py ├── LICENSE ├── README.md ├── calib_config.py ├── calibration.py ├── eval_longbench.py ├── eval_needle.py ├── eval_ppl.py ├── experiments ├── modeling_llama_skvq.py ├── modeling_mistral_skvq.py └── utils.py ├── kernels ├── csrc │ ├── pybind.cc │ ├── quant.cuh │ ├── skvq_quant.cu │ └── skvq_quant.h └── setup.py ├── longbench_config ├── dataset2maxlen.json ├── dataset2prompt.json ├── model2maxlen.json └── model2path.json ├── media ├── overview.jpg └── score.jpg ├── metrics.py ├── requirements.txt ├── score_longbench.py └── viz-needle.ipynb /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/.gitignore -------------------------------------------------------------------------------- /KV_process.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/KV_process.py -------------------------------------------------------------------------------- /KVcache_manager.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/KVcache_manager.py -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/README.md -------------------------------------------------------------------------------- /calib_config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/calib_config.py -------------------------------------------------------------------------------- /calibration.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/calibration.py -------------------------------------------------------------------------------- /eval_longbench.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/eval_longbench.py -------------------------------------------------------------------------------- /eval_needle.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/eval_needle.py -------------------------------------------------------------------------------- /eval_ppl.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/eval_ppl.py -------------------------------------------------------------------------------- /experiments/modeling_llama_skvq.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/experiments/modeling_llama_skvq.py -------------------------------------------------------------------------------- /experiments/modeling_mistral_skvq.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/experiments/modeling_mistral_skvq.py -------------------------------------------------------------------------------- /experiments/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/experiments/utils.py -------------------------------------------------------------------------------- /kernels/csrc/pybind.cc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/kernels/csrc/pybind.cc -------------------------------------------------------------------------------- /kernels/csrc/quant.cuh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/kernels/csrc/quant.cuh -------------------------------------------------------------------------------- /kernels/csrc/skvq_quant.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/kernels/csrc/skvq_quant.cu -------------------------------------------------------------------------------- /kernels/csrc/skvq_quant.h: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/kernels/csrc/skvq_quant.h -------------------------------------------------------------------------------- /kernels/setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/kernels/setup.py -------------------------------------------------------------------------------- /longbench_config/dataset2maxlen.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/longbench_config/dataset2maxlen.json -------------------------------------------------------------------------------- /longbench_config/dataset2prompt.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/longbench_config/dataset2prompt.json -------------------------------------------------------------------------------- /longbench_config/model2maxlen.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/longbench_config/model2maxlen.json -------------------------------------------------------------------------------- /longbench_config/model2path.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/longbench_config/model2path.json -------------------------------------------------------------------------------- /media/overview.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/media/overview.jpg -------------------------------------------------------------------------------- /media/score.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/media/score.jpg -------------------------------------------------------------------------------- /metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/metrics.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/requirements.txt -------------------------------------------------------------------------------- /score_longbench.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/score_longbench.py -------------------------------------------------------------------------------- /viz-needle.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cat538/SKVQ/HEAD/viz-needle.ipynb --------------------------------------------------------------------------------