├── .gitmodules ├── LLaMA-3-V ├── builder.py ├── conversation.py ├── llava_llama.py └── train.py ├── Phi-3-V ├── builder.py ├── conversation.py ├── llava_phi3.py ├── main__init__.py ├── model__init__.py └── train.py ├── README.md ├── images ├── LLaVA-pp-results.png ├── lava++_radar_plot.png └── logos │ ├── IVAL_logo.png │ ├── MBZUAI_logo.png │ ├── Oryx_logo.png │ └── face.png └── scripts ├── LLaMA3-V_finetune_lora.sh ├── LLaMA3-V_full_finetune.sh ├── LLaMA3-V_pretrain.sh ├── Phi3-V_finetune_lora.sh ├── Phi3-V_full_finetune.sh ├── Phi3-V_pretrain.sh └── merge_lora_weights.py /.gitmodules: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/.gitmodules -------------------------------------------------------------------------------- /LLaMA-3-V/builder.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/LLaMA-3-V/builder.py -------------------------------------------------------------------------------- /LLaMA-3-V/conversation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/LLaMA-3-V/conversation.py -------------------------------------------------------------------------------- /LLaMA-3-V/llava_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/LLaMA-3-V/llava_llama.py -------------------------------------------------------------------------------- /LLaMA-3-V/train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/LLaMA-3-V/train.py -------------------------------------------------------------------------------- /Phi-3-V/builder.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/Phi-3-V/builder.py -------------------------------------------------------------------------------- /Phi-3-V/conversation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/Phi-3-V/conversation.py -------------------------------------------------------------------------------- /Phi-3-V/llava_phi3.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/Phi-3-V/llava_phi3.py -------------------------------------------------------------------------------- /Phi-3-V/main__init__.py: -------------------------------------------------------------------------------- 1 | from .model import LlavaPhiForCausalLM -------------------------------------------------------------------------------- /Phi-3-V/model__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/Phi-3-V/model__init__.py -------------------------------------------------------------------------------- /Phi-3-V/train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/Phi-3-V/train.py -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/README.md -------------------------------------------------------------------------------- /images/LLaVA-pp-results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/images/LLaVA-pp-results.png -------------------------------------------------------------------------------- /images/lava++_radar_plot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/images/lava++_radar_plot.png -------------------------------------------------------------------------------- /images/logos/IVAL_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/images/logos/IVAL_logo.png -------------------------------------------------------------------------------- /images/logos/MBZUAI_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/images/logos/MBZUAI_logo.png -------------------------------------------------------------------------------- /images/logos/Oryx_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/images/logos/Oryx_logo.png -------------------------------------------------------------------------------- /images/logos/face.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/images/logos/face.png -------------------------------------------------------------------------------- /scripts/LLaMA3-V_finetune_lora.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/scripts/LLaMA3-V_finetune_lora.sh -------------------------------------------------------------------------------- /scripts/LLaMA3-V_full_finetune.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/scripts/LLaMA3-V_full_finetune.sh -------------------------------------------------------------------------------- /scripts/LLaMA3-V_pretrain.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/scripts/LLaMA3-V_pretrain.sh -------------------------------------------------------------------------------- /scripts/Phi3-V_finetune_lora.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/scripts/Phi3-V_finetune_lora.sh -------------------------------------------------------------------------------- /scripts/Phi3-V_full_finetune.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/scripts/Phi3-V_full_finetune.sh -------------------------------------------------------------------------------- /scripts/Phi3-V_pretrain.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/scripts/Phi3-V_pretrain.sh -------------------------------------------------------------------------------- /scripts/merge_lora_weights.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbzuai-oryx/LLaVA-pp/HEAD/scripts/merge_lora_weights.py --------------------------------------------------------------------------------