├── .DS_Store ├── .gitignore ├── LICENSE ├── README.md ├── data └── pg_distractors.json ├── experiments ├── longmemeval │ ├── README.md │ ├── evaluate │ │ ├── evaluate_longmemeval.py │ │ └── visualize.py │ └── run │ │ └── run_longmemeval.py ├── models │ ├── README.md │ ├── base_provider.py │ ├── llm_judge.py │ └── providers │ │ ├── anthropic.py │ │ ├── google.py │ │ └── openai.py ├── niah_extension │ ├── README.md │ ├── evaluate │ │ ├── analyze_distractors.py │ │ ├── evaluate_niah_extension.py │ │ └── visualize.py │ └── run │ │ ├── create_haystacks.py │ │ └── run_niah_extension.py └── repeated_words │ ├── README.md │ ├── evaluate │ └── evaluate_repeated_words.py │ └── run │ └── run_repeated_words.py ├── images └── image.png ├── requirements.txt └── results ├── gpt_4_1_heatmap.png ├── gpt_4_1_longmemeval.png ├── gpt_4_1_longmemeval_focused_evaluated.csv ├── gpt_4_1_longmemeval_focused_results.csv ├── gpt_4_1_longmemeval_full_evaluated.csv ├── gpt_4_1_longmemeval_full_results.csv ├── gpt_4_1_niah_evaluated.csv └── gpt_4_1_niah_results.csv /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/.DS_Store -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .env 2 | venv/ -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/README.md -------------------------------------------------------------------------------- /data/pg_distractors.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/data/pg_distractors.json -------------------------------------------------------------------------------- /experiments/longmemeval/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/longmemeval/README.md -------------------------------------------------------------------------------- /experiments/longmemeval/evaluate/evaluate_longmemeval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/longmemeval/evaluate/evaluate_longmemeval.py -------------------------------------------------------------------------------- /experiments/longmemeval/evaluate/visualize.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/longmemeval/evaluate/visualize.py -------------------------------------------------------------------------------- /experiments/longmemeval/run/run_longmemeval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/longmemeval/run/run_longmemeval.py -------------------------------------------------------------------------------- /experiments/models/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/models/README.md -------------------------------------------------------------------------------- /experiments/models/base_provider.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/models/base_provider.py -------------------------------------------------------------------------------- /experiments/models/llm_judge.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/models/llm_judge.py -------------------------------------------------------------------------------- /experiments/models/providers/anthropic.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/models/providers/anthropic.py -------------------------------------------------------------------------------- /experiments/models/providers/google.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/models/providers/google.py -------------------------------------------------------------------------------- /experiments/models/providers/openai.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/models/providers/openai.py -------------------------------------------------------------------------------- /experiments/niah_extension/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/niah_extension/README.md -------------------------------------------------------------------------------- /experiments/niah_extension/evaluate/analyze_distractors.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/niah_extension/evaluate/analyze_distractors.py -------------------------------------------------------------------------------- /experiments/niah_extension/evaluate/evaluate_niah_extension.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/niah_extension/evaluate/evaluate_niah_extension.py -------------------------------------------------------------------------------- /experiments/niah_extension/evaluate/visualize.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/niah_extension/evaluate/visualize.py -------------------------------------------------------------------------------- /experiments/niah_extension/run/create_haystacks.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/niah_extension/run/create_haystacks.py -------------------------------------------------------------------------------- /experiments/niah_extension/run/run_niah_extension.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/niah_extension/run/run_niah_extension.py -------------------------------------------------------------------------------- /experiments/repeated_words/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/repeated_words/README.md -------------------------------------------------------------------------------- /experiments/repeated_words/evaluate/evaluate_repeated_words.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/repeated_words/evaluate/evaluate_repeated_words.py -------------------------------------------------------------------------------- /experiments/repeated_words/run/run_repeated_words.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/experiments/repeated_words/run/run_repeated_words.py -------------------------------------------------------------------------------- /images/image.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/images/image.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/requirements.txt -------------------------------------------------------------------------------- /results/gpt_4_1_heatmap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/results/gpt_4_1_heatmap.png -------------------------------------------------------------------------------- /results/gpt_4_1_longmemeval.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/results/gpt_4_1_longmemeval.png -------------------------------------------------------------------------------- /results/gpt_4_1_longmemeval_focused_evaluated.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/results/gpt_4_1_longmemeval_focused_evaluated.csv -------------------------------------------------------------------------------- /results/gpt_4_1_longmemeval_focused_results.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/results/gpt_4_1_longmemeval_focused_results.csv -------------------------------------------------------------------------------- /results/gpt_4_1_longmemeval_full_evaluated.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/results/gpt_4_1_longmemeval_full_evaluated.csv -------------------------------------------------------------------------------- /results/gpt_4_1_longmemeval_full_results.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/results/gpt_4_1_longmemeval_full_results.csv -------------------------------------------------------------------------------- /results/gpt_4_1_niah_evaluated.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/results/gpt_4_1_niah_evaluated.csv -------------------------------------------------------------------------------- /results/gpt_4_1_niah_results.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chroma-core/context-rot/HEAD/results/gpt_4_1_niah_results.csv --------------------------------------------------------------------------------