├── .gitattributes ├── LICENSE ├── README.md ├── data └── test_data.jsonl ├── figs └── model.png ├── requirements.txt ├── scripts ├── construct_dpo_data.bash ├── inference.sh └── train.sh └── src ├── eval ├── MMTab_evaluation.ipynb └── eval_data.json ├── generate_data ├── generate_dpo_data.py ├── image_generate_response.py ├── markdown_generate_response.py ├── multi_modal_generate_response.py └── prompt │ ├── __init__.py │ ├── image_prompt.py │ ├── markdown_prompt.py │ └── multi_modal_prompt.py └── inference ├── multi_modal_inference.py └── prompt ├── __init__.py └── multi_modal_prompt.py /.gitattributes: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/.gitattributes -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/README.md -------------------------------------------------------------------------------- /data/test_data.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/data/test_data.jsonl -------------------------------------------------------------------------------- /figs/model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/figs/model.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/requirements.txt -------------------------------------------------------------------------------- /scripts/construct_dpo_data.bash: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/scripts/construct_dpo_data.bash -------------------------------------------------------------------------------- /scripts/inference.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/scripts/inference.sh -------------------------------------------------------------------------------- /scripts/train.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/scripts/train.sh -------------------------------------------------------------------------------- /src/eval/MMTab_evaluation.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/eval/MMTab_evaluation.ipynb -------------------------------------------------------------------------------- /src/eval/eval_data.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/eval/eval_data.json -------------------------------------------------------------------------------- /src/generate_data/generate_dpo_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/generate_data/generate_dpo_data.py -------------------------------------------------------------------------------- /src/generate_data/image_generate_response.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/generate_data/image_generate_response.py -------------------------------------------------------------------------------- /src/generate_data/markdown_generate_response.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/generate_data/markdown_generate_response.py -------------------------------------------------------------------------------- /src/generate_data/multi_modal_generate_response.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/generate_data/multi_modal_generate_response.py -------------------------------------------------------------------------------- /src/generate_data/prompt/__init__.py: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /src/generate_data/prompt/image_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/generate_data/prompt/image_prompt.py -------------------------------------------------------------------------------- /src/generate_data/prompt/markdown_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/generate_data/prompt/markdown_prompt.py -------------------------------------------------------------------------------- /src/generate_data/prompt/multi_modal_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/generate_data/prompt/multi_modal_prompt.py -------------------------------------------------------------------------------- /src/inference/multi_modal_inference.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/inference/multi_modal_inference.py -------------------------------------------------------------------------------- /src/inference/prompt/__init__.py: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /src/inference/prompt/multi_modal_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NEUIR/HIPPO/HEAD/src/inference/prompt/multi_modal_prompt.py --------------------------------------------------------------------------------