├── .gitignore ├── .python-version ├── .vscode ├── launch.json └── settings.json ├── README.md ├── mla ├── __init__.py └── transformer.py ├── pyproject.toml ├── tests ├── conftest.py ├── test_mha.py ├── test_mla.py ├── test_rope.py └── test_transformer.py └── uv.lock /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/.gitignore -------------------------------------------------------------------------------- /.python-version: -------------------------------------------------------------------------------- 1 | 3.12 2 | -------------------------------------------------------------------------------- /.vscode/launch.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/.vscode/launch.json -------------------------------------------------------------------------------- /.vscode/settings.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/.vscode/settings.json -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/README.md -------------------------------------------------------------------------------- /mla/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /mla/transformer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/mla/transformer.py -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/pyproject.toml -------------------------------------------------------------------------------- /tests/conftest.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/tests/conftest.py -------------------------------------------------------------------------------- /tests/test_mha.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/tests/test_mha.py -------------------------------------------------------------------------------- /tests/test_mla.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/tests/test_mla.py -------------------------------------------------------------------------------- /tests/test_rope.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/tests/test_rope.py -------------------------------------------------------------------------------- /tests/test_transformer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/tests/test_transformer.py -------------------------------------------------------------------------------- /uv.lock: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hkproj/multi-latent-attention/HEAD/uv.lock --------------------------------------------------------------------------------