├── Qwen0.5B-FullKV-128k_0.446.png
├── Qwen0.5B-InfiniRetri_1m_1.000.png
├── README.md
└── imgs
    ├── Qwen0.5B-FullKV-128k_0.446.png
    └── Qwen0.5B-InfiniRetri_1m_1.000.png


/Qwen0.5B-FullKV-128k_0.446.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MrYxJ/InfiniRetri/fe4993846b80f0dd70882bb3f60908a1e144085c/Qwen0.5B-FullKV-128k_0.446.png


--------------------------------------------------------------------------------
/Qwen0.5B-InfiniRetri_1m_1.000.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MrYxJ/InfiniRetri/fe4993846b80f0dd70882bb3f60908a1e144085c/Qwen0.5B-InfiniRetri_1m_1.000.png


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # InfiniRetri
 2 | Implementation of currently Submitting Paper: "**Infinite Retrieval: Attention Enhanced LLMs in Long Context Processing**", which can apply any Transformer-based LLMs(Large Language Moidels) to handle long-context without training.
 3 | 
 4 | Considering that the work is still under submission, the InfiniRetri was moved by the author to another project for presentation, at [InfiniRetri2](https://github.com/CapitalCode2020/InfiniRetri2).
 5 | 
 6 | Notably，our method **InfiniRretri** can enbale the 0.5B(Qwen2.5-0.5B-Instruct), which originally had a maximum context length of 32K, to Haystack(retrieval) up over 1M tokens on Needle-In-a-Haystack(NIH) test, and even theoretically **infinite-length**.
 7 | 
 8 | ![Using Origin](Qwen0.5B-FullKV-128k_0.446.png)
 9 | 
10 | 
11 | ![Using InfiniRetri](Qwen0.5B-InfiniRetri_1m_1.000.png)
12 | 
13 | 
14 | 
15 | 


--------------------------------------------------------------------------------
/imgs/Qwen0.5B-FullKV-128k_0.446.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MrYxJ/InfiniRetri/fe4993846b80f0dd70882bb3f60908a1e144085c/imgs/Qwen0.5B-FullKV-128k_0.446.png


--------------------------------------------------------------------------------
/imgs/Qwen0.5B-InfiniRetri_1m_1.000.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/MrYxJ/InfiniRetri/fe4993846b80f0dd70882bb3f60908a1e144085c/imgs/Qwen0.5B-InfiniRetri_1m_1.000.png


--------------------------------------------------------------------------------