├── .gitattributes ├── .gitignore ├── CITATION.cff ├── LICENSE ├── README.md ├── data └── BurstGPT_1.csv ├── example ├── .gitignore ├── LICENSE ├── README.md ├── framework_inference_call │ ├── __init__.py │ ├── framework_lightllm.py │ └── framework_vllm.py ├── preprocess_data │ ├── README.md │ ├── ShareGPT_Distribution.png │ ├── __init__.py │ ├── load_data.py │ ├── pre_shareGPT.py │ └── shareGPT.json ├── profile_server │ ├── __init__.py │ ├── config.py │ ├── logger.py │ ├── prompt_set.py │ └── server.py ├── profile_vllm_server.py ├── profile_vllm_trace.sh ├── requirements.txt └── start_vllm.sh └── img ├── Fig1-2.png ├── Fig10-13.png ├── Fig3-4.png ├── req-res.png └── workload_generator2.png /.gitattributes: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | data/*_2.csv 2 | -------------------------------------------------------------------------------- /CITATION.cff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/CITATION.cff -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/README.md -------------------------------------------------------------------------------- /data/BurstGPT_1.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/data/BurstGPT_1.csv -------------------------------------------------------------------------------- /example/.gitignore: -------------------------------------------------------------------------------- 1 | **/__pycache__ 2 | -------------------------------------------------------------------------------- /example/LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/LICENSE -------------------------------------------------------------------------------- /example/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/README.md -------------------------------------------------------------------------------- /example/framework_inference_call/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/framework_inference_call/__init__.py -------------------------------------------------------------------------------- /example/framework_inference_call/framework_lightllm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/framework_inference_call/framework_lightllm.py -------------------------------------------------------------------------------- /example/framework_inference_call/framework_vllm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/framework_inference_call/framework_vllm.py -------------------------------------------------------------------------------- /example/preprocess_data/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/preprocess_data/README.md -------------------------------------------------------------------------------- /example/preprocess_data/ShareGPT_Distribution.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/preprocess_data/ShareGPT_Distribution.png -------------------------------------------------------------------------------- /example/preprocess_data/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/preprocess_data/__init__.py -------------------------------------------------------------------------------- /example/preprocess_data/load_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/preprocess_data/load_data.py -------------------------------------------------------------------------------- /example/preprocess_data/pre_shareGPT.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/preprocess_data/pre_shareGPT.py -------------------------------------------------------------------------------- /example/preprocess_data/shareGPT.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/preprocess_data/shareGPT.json -------------------------------------------------------------------------------- /example/profile_server/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/profile_server/__init__.py -------------------------------------------------------------------------------- /example/profile_server/config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/profile_server/config.py -------------------------------------------------------------------------------- /example/profile_server/logger.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/profile_server/logger.py -------------------------------------------------------------------------------- /example/profile_server/prompt_set.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/profile_server/prompt_set.py -------------------------------------------------------------------------------- /example/profile_server/server.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/profile_server/server.py -------------------------------------------------------------------------------- /example/profile_vllm_server.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/profile_vllm_server.py -------------------------------------------------------------------------------- /example/profile_vllm_trace.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/profile_vllm_trace.sh -------------------------------------------------------------------------------- /example/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/requirements.txt -------------------------------------------------------------------------------- /example/start_vllm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/example/start_vllm.sh -------------------------------------------------------------------------------- /img/Fig1-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/img/Fig1-2.png -------------------------------------------------------------------------------- /img/Fig10-13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/img/Fig10-13.png -------------------------------------------------------------------------------- /img/Fig3-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/img/Fig3-4.png -------------------------------------------------------------------------------- /img/req-res.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/img/req-res.png -------------------------------------------------------------------------------- /img/workload_generator2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HPMLL/BurstGPT/HEAD/img/workload_generator2.png --------------------------------------------------------------------------------