├── .containerignore ├── .gitignore ├── LICENSE ├── README.md ├── app.py ├── assets ├── guidellm.png ├── llmcompressor.png ├── start-vllm.gif ├── stop-vllm.gif ├── vllm-playground.png ├── vllm.jpeg ├── vllm.png └── vllm_only.png ├── benchmarks.json ├── cli_demo ├── CLI_DEMO_SUMMARY.md ├── README.md ├── WORKFLOW_VISUAL.txt ├── demo.env ├── docs │ ├── CLI_DEMO_GUIDE.md │ ├── CLI_QUICK_REFERENCE.md │ └── USAGE_EXAMPLES.md └── scripts │ ├── README.md │ ├── benchmark_guidellm.sh │ ├── demo_full_workflow.sh │ └── test_vllm_serving.sh ├── config ├── example_configs.json └── vllm_cpu.env ├── container_manager.py ├── containers ├── Containerfile.cpu ├── Containerfile.mac ├── Containerfile.vllm-playground └── README.md ├── deployments ├── README.md ├── deploy-to-openshift.sh ├── kubernetes-deployment.yaml ├── openshift-deployment-dev.yaml ├── openshift-deployment.yaml └── undeploy-from-openshift.sh ├── docs ├── CLI_QUICK_REFERENCE.md ├── CPU_MODELS_QUICKSTART.md ├── FEATURES.md ├── GATED_MODELS_GUIDE.md ├── GPU_SETUP_GUIDE.md ├── INSTALLATION_GUIDE.md ├── LOCAL_MODEL_GUIDE.md ├── MACOS_CPU_GUIDE.md ├── PERFORMANCE_METRICS.md ├── QUICKSTART.md ├── QUICK_REFERENCE.md └── TROUBLESHOOTING.md ├── env.example ├── index.html ├── openshift ├── Containerfile ├── QUICK_START.md ├── README.md ├── build.sh ├── deploy.sh ├── kubernetes_container_manager.py ├── manifests │ ├── 01-namespace.yaml │ ├── 02-rbac.yaml │ ├── 03-configmap.yaml │ ├── 04-webui-deployment.yaml │ ├── 05-pvc-optional.yaml │ └── README.md ├── requirements-k8s.txt └── undeploy.sh ├── requirements.txt ├── run.py ├── scripts ├── README.md ├── kill_playground.py ├── restart_playground.sh ├── run_cpu.sh ├── start.sh ├── verify_chat_routing.sh └── verify_setup.py └── static ├── css └── style.css └── js └── app.js /.containerignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/.containerignore -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/README.md -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/app.py -------------------------------------------------------------------------------- /assets/guidellm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/assets/guidellm.png -------------------------------------------------------------------------------- /assets/llmcompressor.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/assets/llmcompressor.png -------------------------------------------------------------------------------- /assets/start-vllm.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/assets/start-vllm.gif -------------------------------------------------------------------------------- /assets/stop-vllm.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/assets/stop-vllm.gif -------------------------------------------------------------------------------- /assets/vllm-playground.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/assets/vllm-playground.png -------------------------------------------------------------------------------- /assets/vllm.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/assets/vllm.jpeg -------------------------------------------------------------------------------- /assets/vllm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/assets/vllm.png -------------------------------------------------------------------------------- /assets/vllm_only.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/assets/vllm_only.png -------------------------------------------------------------------------------- /benchmarks.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/benchmarks.json -------------------------------------------------------------------------------- /cli_demo/CLI_DEMO_SUMMARY.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/CLI_DEMO_SUMMARY.md -------------------------------------------------------------------------------- /cli_demo/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/README.md -------------------------------------------------------------------------------- /cli_demo/WORKFLOW_VISUAL.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/WORKFLOW_VISUAL.txt -------------------------------------------------------------------------------- /cli_demo/demo.env: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/demo.env -------------------------------------------------------------------------------- /cli_demo/docs/CLI_DEMO_GUIDE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/docs/CLI_DEMO_GUIDE.md -------------------------------------------------------------------------------- /cli_demo/docs/CLI_QUICK_REFERENCE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/docs/CLI_QUICK_REFERENCE.md -------------------------------------------------------------------------------- /cli_demo/docs/USAGE_EXAMPLES.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/docs/USAGE_EXAMPLES.md -------------------------------------------------------------------------------- /cli_demo/scripts/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/scripts/README.md -------------------------------------------------------------------------------- /cli_demo/scripts/benchmark_guidellm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/scripts/benchmark_guidellm.sh -------------------------------------------------------------------------------- /cli_demo/scripts/demo_full_workflow.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/scripts/demo_full_workflow.sh -------------------------------------------------------------------------------- /cli_demo/scripts/test_vllm_serving.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/cli_demo/scripts/test_vllm_serving.sh -------------------------------------------------------------------------------- /config/example_configs.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/config/example_configs.json -------------------------------------------------------------------------------- /config/vllm_cpu.env: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/config/vllm_cpu.env -------------------------------------------------------------------------------- /container_manager.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/container_manager.py -------------------------------------------------------------------------------- /containers/Containerfile.cpu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/containers/Containerfile.cpu -------------------------------------------------------------------------------- /containers/Containerfile.mac: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/containers/Containerfile.mac -------------------------------------------------------------------------------- /containers/Containerfile.vllm-playground: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/containers/Containerfile.vllm-playground -------------------------------------------------------------------------------- /containers/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/containers/README.md -------------------------------------------------------------------------------- /deployments/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/deployments/README.md -------------------------------------------------------------------------------- /deployments/deploy-to-openshift.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/deployments/deploy-to-openshift.sh -------------------------------------------------------------------------------- /deployments/kubernetes-deployment.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/deployments/kubernetes-deployment.yaml -------------------------------------------------------------------------------- /deployments/openshift-deployment-dev.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/deployments/openshift-deployment-dev.yaml -------------------------------------------------------------------------------- /deployments/openshift-deployment.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/deployments/openshift-deployment.yaml -------------------------------------------------------------------------------- /deployments/undeploy-from-openshift.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/deployments/undeploy-from-openshift.sh -------------------------------------------------------------------------------- /docs/CLI_QUICK_REFERENCE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/CLI_QUICK_REFERENCE.md -------------------------------------------------------------------------------- /docs/CPU_MODELS_QUICKSTART.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/CPU_MODELS_QUICKSTART.md -------------------------------------------------------------------------------- /docs/FEATURES.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/FEATURES.md -------------------------------------------------------------------------------- /docs/GATED_MODELS_GUIDE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/GATED_MODELS_GUIDE.md -------------------------------------------------------------------------------- /docs/GPU_SETUP_GUIDE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/GPU_SETUP_GUIDE.md -------------------------------------------------------------------------------- /docs/INSTALLATION_GUIDE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/INSTALLATION_GUIDE.md -------------------------------------------------------------------------------- /docs/LOCAL_MODEL_GUIDE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/LOCAL_MODEL_GUIDE.md -------------------------------------------------------------------------------- /docs/MACOS_CPU_GUIDE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/MACOS_CPU_GUIDE.md -------------------------------------------------------------------------------- /docs/PERFORMANCE_METRICS.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/PERFORMANCE_METRICS.md -------------------------------------------------------------------------------- /docs/QUICKSTART.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/QUICKSTART.md -------------------------------------------------------------------------------- /docs/QUICK_REFERENCE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/QUICK_REFERENCE.md -------------------------------------------------------------------------------- /docs/TROUBLESHOOTING.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/docs/TROUBLESHOOTING.md -------------------------------------------------------------------------------- /env.example: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/env.example -------------------------------------------------------------------------------- /index.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/index.html -------------------------------------------------------------------------------- /openshift/Containerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/Containerfile -------------------------------------------------------------------------------- /openshift/QUICK_START.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/QUICK_START.md -------------------------------------------------------------------------------- /openshift/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/README.md -------------------------------------------------------------------------------- /openshift/build.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/build.sh -------------------------------------------------------------------------------- /openshift/deploy.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/deploy.sh -------------------------------------------------------------------------------- /openshift/kubernetes_container_manager.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/kubernetes_container_manager.py -------------------------------------------------------------------------------- /openshift/manifests/01-namespace.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/manifests/01-namespace.yaml -------------------------------------------------------------------------------- /openshift/manifests/02-rbac.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/manifests/02-rbac.yaml -------------------------------------------------------------------------------- /openshift/manifests/03-configmap.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/manifests/03-configmap.yaml -------------------------------------------------------------------------------- /openshift/manifests/04-webui-deployment.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/manifests/04-webui-deployment.yaml -------------------------------------------------------------------------------- /openshift/manifests/05-pvc-optional.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/manifests/05-pvc-optional.yaml -------------------------------------------------------------------------------- /openshift/manifests/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/manifests/README.md -------------------------------------------------------------------------------- /openshift/requirements-k8s.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/requirements-k8s.txt -------------------------------------------------------------------------------- /openshift/undeploy.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/openshift/undeploy.sh -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/requirements.txt -------------------------------------------------------------------------------- /run.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/run.py -------------------------------------------------------------------------------- /scripts/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/scripts/README.md -------------------------------------------------------------------------------- /scripts/kill_playground.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/scripts/kill_playground.py -------------------------------------------------------------------------------- /scripts/restart_playground.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/scripts/restart_playground.sh -------------------------------------------------------------------------------- /scripts/run_cpu.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/scripts/run_cpu.sh -------------------------------------------------------------------------------- /scripts/start.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/scripts/start.sh -------------------------------------------------------------------------------- /scripts/verify_chat_routing.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/scripts/verify_chat_routing.sh -------------------------------------------------------------------------------- /scripts/verify_setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/scripts/verify_setup.py -------------------------------------------------------------------------------- /static/css/style.css: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/static/css/style.css -------------------------------------------------------------------------------- /static/js/app.js: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/micytao/vllm-playground/HEAD/static/js/app.js --------------------------------------------------------------------------------