├── .gitignore ├── LICENSE.txt ├── README.md ├── configs └── LYNX.yaml ├── data ├── Open_VQA_images.jsonl └── Open_VQA_videos.jsonl ├── dataset ├── __init__.py ├── eval_datasets.py ├── tokenizers │ └── __init__.py └── utils.py ├── environment.yml ├── generate.py ├── generate.sh ├── images ├── ablation.png ├── logo_plus.png ├── lynx.png ├── open_vqa_image_result.png └── result_other.png ├── licenses ├── LICENSE-BAAI-VISION.txt ├── LICENSE-adapter-transformers.txt ├── LICENSE-flamingo-pytorch.txt └── LICENSE-transformers.txt ├── models ├── __init__.py ├── adapter_modeling.py ├── bridges │ ├── __init__.py │ └── resampler.py ├── llms │ ├── __init__.py │ └── llama │ │ ├── __init__.py │ │ ├── configuration_llama.py │ │ ├── convert_llama_weights_to_hf.py │ │ ├── modeling_llama.py │ │ └── tokenization_llama.py ├── lynx.py └── vits │ ├── __init__.py │ └── eva_vit.py ├── requirements.txt └── utils.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/LICENSE.txt -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/README.md -------------------------------------------------------------------------------- /configs/LYNX.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/configs/LYNX.yaml -------------------------------------------------------------------------------- /data/Open_VQA_images.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/data/Open_VQA_images.jsonl -------------------------------------------------------------------------------- /data/Open_VQA_videos.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/data/Open_VQA_videos.jsonl -------------------------------------------------------------------------------- /dataset/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/dataset/__init__.py -------------------------------------------------------------------------------- /dataset/eval_datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/dataset/eval_datasets.py -------------------------------------------------------------------------------- /dataset/tokenizers/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/dataset/tokenizers/__init__.py -------------------------------------------------------------------------------- /dataset/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/dataset/utils.py -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/environment.yml -------------------------------------------------------------------------------- /generate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/generate.py -------------------------------------------------------------------------------- /generate.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/generate.sh -------------------------------------------------------------------------------- /images/ablation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/images/ablation.png -------------------------------------------------------------------------------- /images/logo_plus.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/images/logo_plus.png -------------------------------------------------------------------------------- /images/lynx.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/images/lynx.png -------------------------------------------------------------------------------- /images/open_vqa_image_result.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/images/open_vqa_image_result.png -------------------------------------------------------------------------------- /images/result_other.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/images/result_other.png -------------------------------------------------------------------------------- /licenses/LICENSE-BAAI-VISION.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/licenses/LICENSE-BAAI-VISION.txt -------------------------------------------------------------------------------- /licenses/LICENSE-adapter-transformers.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/licenses/LICENSE-adapter-transformers.txt -------------------------------------------------------------------------------- /licenses/LICENSE-flamingo-pytorch.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/licenses/LICENSE-flamingo-pytorch.txt -------------------------------------------------------------------------------- /licenses/LICENSE-transformers.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/licenses/LICENSE-transformers.txt -------------------------------------------------------------------------------- /models/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /models/adapter_modeling.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/adapter_modeling.py -------------------------------------------------------------------------------- /models/bridges/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/bridges/__init__.py -------------------------------------------------------------------------------- /models/bridges/resampler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/bridges/resampler.py -------------------------------------------------------------------------------- /models/llms/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /models/llms/llama/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/llms/llama/__init__.py -------------------------------------------------------------------------------- /models/llms/llama/configuration_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/llms/llama/configuration_llama.py -------------------------------------------------------------------------------- /models/llms/llama/convert_llama_weights_to_hf.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/llms/llama/convert_llama_weights_to_hf.py -------------------------------------------------------------------------------- /models/llms/llama/modeling_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/llms/llama/modeling_llama.py -------------------------------------------------------------------------------- /models/llms/llama/tokenization_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/llms/llama/tokenization_llama.py -------------------------------------------------------------------------------- /models/lynx.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/lynx.py -------------------------------------------------------------------------------- /models/vits/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /models/vits/eva_vit.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/models/vits/eva_vit.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/requirements.txt -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bytedance/lynx-llm/HEAD/utils.py --------------------------------------------------------------------------------