├── .gitignore ├── LICENSE.md ├── README.md ├── artifact_evaluation.md ├── config_tuner.py ├── data └── ttft_AI21-Jamba-1.5-Mini.pickle ├── environment.yml ├── plotting ├── cache_usage_breakdown.py ├── context_window_over_time.py ├── fine_grained_analysis.py ├── microbenchmark_arrivalrate.py ├── microbenchmark_contention.py ├── microbenchmark_dstate.py ├── microbenchmark_layer_composition.py ├── plotting.py ├── sglang_comparison.py ├── state_size_comparison.py ├── token_hit_rate.py └── ttft.py ├── policy_exploration.py ├── radix_cache_hybrid.py ├── radix_cache_vllm.py ├── run_all_experiments.sh ├── toy_example.py ├── utils.py └── utils ├── analyze_trace.py ├── extract_wins.py ├── extract_wins_df.py └── generate_trace.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/LICENSE.md -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/README.md -------------------------------------------------------------------------------- /artifact_evaluation.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/artifact_evaluation.md -------------------------------------------------------------------------------- /config_tuner.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/config_tuner.py -------------------------------------------------------------------------------- /data/ttft_AI21-Jamba-1.5-Mini.pickle: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/data/ttft_AI21-Jamba-1.5-Mini.pickle -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/environment.yml -------------------------------------------------------------------------------- /plotting/cache_usage_breakdown.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/cache_usage_breakdown.py -------------------------------------------------------------------------------- /plotting/context_window_over_time.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/context_window_over_time.py -------------------------------------------------------------------------------- /plotting/fine_grained_analysis.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/fine_grained_analysis.py -------------------------------------------------------------------------------- /plotting/microbenchmark_arrivalrate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/microbenchmark_arrivalrate.py -------------------------------------------------------------------------------- /plotting/microbenchmark_contention.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/microbenchmark_contention.py -------------------------------------------------------------------------------- /plotting/microbenchmark_dstate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/microbenchmark_dstate.py -------------------------------------------------------------------------------- /plotting/microbenchmark_layer_composition.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/microbenchmark_layer_composition.py -------------------------------------------------------------------------------- /plotting/plotting.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/plotting.py -------------------------------------------------------------------------------- /plotting/sglang_comparison.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/sglang_comparison.py -------------------------------------------------------------------------------- /plotting/state_size_comparison.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/state_size_comparison.py -------------------------------------------------------------------------------- /plotting/token_hit_rate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/token_hit_rate.py -------------------------------------------------------------------------------- /plotting/ttft.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/plotting/ttft.py -------------------------------------------------------------------------------- /policy_exploration.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/policy_exploration.py -------------------------------------------------------------------------------- /radix_cache_hybrid.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/radix_cache_hybrid.py -------------------------------------------------------------------------------- /radix_cache_vllm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/radix_cache_vllm.py -------------------------------------------------------------------------------- /run_all_experiments.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/run_all_experiments.sh -------------------------------------------------------------------------------- /toy_example.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/toy_example.py -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/utils.py -------------------------------------------------------------------------------- /utils/analyze_trace.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/utils/analyze_trace.py -------------------------------------------------------------------------------- /utils/extract_wins.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/utils/extract_wins.py -------------------------------------------------------------------------------- /utils/extract_wins_df.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/utils/extract_wins_df.py -------------------------------------------------------------------------------- /utils/generate_trace.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ruipeterpan/marconi/HEAD/utils/generate_trace.py --------------------------------------------------------------------------------