├── .gitignore ├── LICENSE ├── README.md ├── configs ├── MAGMA_v1.yml └── MAGMA_v2.yml ├── example_inference.py ├── examples ├── magma_oracle.png ├── magma_present.jpg ├── magma_social.png ├── magma_treasure.png ├── magma_tree.jpg └── model.jpg ├── magma ├── __init__.py ├── adapters.py ├── config.py ├── datasets │ ├── __init__.py │ ├── convert_datasets.py │ └── dataset.py ├── image_encoders.py ├── image_input.py ├── image_prefix.py ├── language_model.py ├── magma.py ├── sampling.py ├── train_loop.py ├── transforms.py └── utils.py ├── requirements.txt ├── requirements_for_setup.txt ├── setup.py └── train.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/README.md -------------------------------------------------------------------------------- /configs/MAGMA_v1.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/configs/MAGMA_v1.yml -------------------------------------------------------------------------------- /configs/MAGMA_v2.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/configs/MAGMA_v2.yml -------------------------------------------------------------------------------- /example_inference.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/example_inference.py -------------------------------------------------------------------------------- /examples/magma_oracle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/examples/magma_oracle.png -------------------------------------------------------------------------------- /examples/magma_present.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/examples/magma_present.jpg -------------------------------------------------------------------------------- /examples/magma_social.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/examples/magma_social.png -------------------------------------------------------------------------------- /examples/magma_treasure.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/examples/magma_treasure.png -------------------------------------------------------------------------------- /examples/magma_tree.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/examples/magma_tree.jpg -------------------------------------------------------------------------------- /examples/model.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/examples/model.jpg -------------------------------------------------------------------------------- /magma/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/__init__.py -------------------------------------------------------------------------------- /magma/adapters.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/adapters.py -------------------------------------------------------------------------------- /magma/config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/config.py -------------------------------------------------------------------------------- /magma/datasets/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/datasets/__init__.py -------------------------------------------------------------------------------- /magma/datasets/convert_datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/datasets/convert_datasets.py -------------------------------------------------------------------------------- /magma/datasets/dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/datasets/dataset.py -------------------------------------------------------------------------------- /magma/image_encoders.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/image_encoders.py -------------------------------------------------------------------------------- /magma/image_input.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/image_input.py -------------------------------------------------------------------------------- /magma/image_prefix.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/image_prefix.py -------------------------------------------------------------------------------- /magma/language_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/language_model.py -------------------------------------------------------------------------------- /magma/magma.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/magma.py -------------------------------------------------------------------------------- /magma/sampling.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/sampling.py -------------------------------------------------------------------------------- /magma/train_loop.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/train_loop.py -------------------------------------------------------------------------------- /magma/transforms.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/transforms.py -------------------------------------------------------------------------------- /magma/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/magma/utils.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/requirements.txt -------------------------------------------------------------------------------- /requirements_for_setup.txt: -------------------------------------------------------------------------------- 1 | torchtyping 2 | typeguard 3 | gdown 4 | tqdm 5 | timm 6 | deepspeed 7 | wandb -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/setup.py -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Aleph-Alpha-Research/magma/HEAD/train.py --------------------------------------------------------------------------------