├── .gitignore ├── .vscode └── settings.json ├── INSTALL.md ├── README.md ├── datasets ├── coco │ └── readme.md └── detail_23k.json ├── eval_configs ├── minigpt4_eval.yaml └── minigpt4_llama2_eval.yaml ├── models ├── blip2_candidate_answer_generator.py ├── consistency_crosscheck.py ├── consistency_crosscheck_object.py ├── evaluation_utils.py ├── instructblip_candidate_answer_generator.py ├── minigpt4_candidate_answer_generator.py ├── question_generator.py ├── refine_dataset.py └── seesaw_counterfactual_generation.py ├── refined_datasets ├── LLaVA+_dataset.json └── expanded_LLaVA++_dataset.json ├── requirements.txt ├── results └── LLaVA_question_checks.json ├── train_configs ├── minigpt4_stage1_pretrain.yaml └── minigpt4_stage2_finetune.yaml └── utils ├── 5000_update_expandcoco_all_val2017_vg_coco_annotations.json ├── coco_category.json ├── coco_object_co_occur.json ├── cropimage2image.py ├── grounded_sam.py ├── hallucination_eval_coco_imageids.json ├── prompt_generation.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/.gitignore -------------------------------------------------------------------------------- /.vscode/settings.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/.vscode/settings.json -------------------------------------------------------------------------------- /INSTALL.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/INSTALL.md -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/README.md -------------------------------------------------------------------------------- /datasets/coco/readme.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/datasets/coco/readme.md -------------------------------------------------------------------------------- /datasets/detail_23k.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/datasets/detail_23k.json -------------------------------------------------------------------------------- /eval_configs/minigpt4_eval.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/eval_configs/minigpt4_eval.yaml -------------------------------------------------------------------------------- /eval_configs/minigpt4_llama2_eval.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/eval_configs/minigpt4_llama2_eval.yaml -------------------------------------------------------------------------------- /models/blip2_candidate_answer_generator.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/blip2_candidate_answer_generator.py -------------------------------------------------------------------------------- /models/consistency_crosscheck.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/consistency_crosscheck.py -------------------------------------------------------------------------------- /models/consistency_crosscheck_object.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/consistency_crosscheck_object.py -------------------------------------------------------------------------------- /models/evaluation_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/evaluation_utils.py -------------------------------------------------------------------------------- /models/instructblip_candidate_answer_generator.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/instructblip_candidate_answer_generator.py -------------------------------------------------------------------------------- /models/minigpt4_candidate_answer_generator.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/minigpt4_candidate_answer_generator.py -------------------------------------------------------------------------------- /models/question_generator.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/question_generator.py -------------------------------------------------------------------------------- /models/refine_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/refine_dataset.py -------------------------------------------------------------------------------- /models/seesaw_counterfactual_generation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/models/seesaw_counterfactual_generation.py -------------------------------------------------------------------------------- /refined_datasets/LLaVA+_dataset.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/refined_datasets/LLaVA+_dataset.json -------------------------------------------------------------------------------- /refined_datasets/expanded_LLaVA++_dataset.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/refined_datasets/expanded_LLaVA++_dataset.json -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/requirements.txt -------------------------------------------------------------------------------- /results/LLaVA_question_checks.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/results/LLaVA_question_checks.json -------------------------------------------------------------------------------- /train_configs/minigpt4_stage1_pretrain.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/train_configs/minigpt4_stage1_pretrain.yaml -------------------------------------------------------------------------------- /train_configs/minigpt4_stage2_finetune.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/train_configs/minigpt4_stage2_finetune.yaml -------------------------------------------------------------------------------- /utils/5000_update_expandcoco_all_val2017_vg_coco_annotations.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/utils/5000_update_expandcoco_all_val2017_vg_coco_annotations.json -------------------------------------------------------------------------------- /utils/coco_category.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/utils/coco_category.json -------------------------------------------------------------------------------- /utils/coco_object_co_occur.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/utils/coco_object_co_occur.json -------------------------------------------------------------------------------- /utils/cropimage2image.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/utils/cropimage2image.py -------------------------------------------------------------------------------- /utils/grounded_sam.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/utils/grounded_sam.py -------------------------------------------------------------------------------- /utils/hallucination_eval_coco_imageids.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/utils/hallucination_eval_coco_imageids.json -------------------------------------------------------------------------------- /utils/prompt_generation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/utils/prompt_generation.py -------------------------------------------------------------------------------- /utils/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Yuqifan1117/HalluciDoctor/HEAD/utils/utils.py --------------------------------------------------------------------------------