├── .gitignore ├── Images ├── MICCAI_benchmark_table.png ├── category_comp.png ├── data_pipeline_with_stats.png └── hal-aware.png ├── README.md ├── dataset └── Gut-VLM │ ├── Images │ └── README.md │ ├── VQA_format_all_annotations.json │ ├── all_annotations.json │ └── train_test_split │ ├── VQA_format_testset_only.json │ ├── VQA_fromat_trainset_only.json │ ├── classwise_test_split │ ├── cecum_test.json │ ├── dlp_test.json │ ├── drm_test.json │ ├── eso_test.json │ ├── pol_test.json │ ├── pylorus_test.json │ ├── uc_test.json │ └── zline_test.json │ ├── classwise_train_split │ ├── cecum_train.json │ ├── dlp_train.json │ ├── drm_train.json │ ├── eso_train.json │ ├── pol_train.json │ ├── pylorus_train.json │ ├── uc_train.json │ └── zline_train.json │ ├── test.json │ └── train.json ├── eval_scripts ├── caption2vqa.py ├── evaluate_caption.py ├── evaluate_vqa.py └── parse_vqa.py ├── results_example ├── final_qwen_caption_hal_aware_results.json └── groundtruth_test_captions.json └── training_style └── hallucinated_aware_train.json /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/.gitignore -------------------------------------------------------------------------------- /Images/MICCAI_benchmark_table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/Images/MICCAI_benchmark_table.png -------------------------------------------------------------------------------- /Images/category_comp.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/Images/category_comp.png -------------------------------------------------------------------------------- /Images/data_pipeline_with_stats.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/Images/data_pipeline_with_stats.png -------------------------------------------------------------------------------- /Images/hal-aware.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/Images/hal-aware.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/README.md -------------------------------------------------------------------------------- /dataset/Gut-VLM/Images/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/Images/README.md -------------------------------------------------------------------------------- /dataset/Gut-VLM/VQA_format_all_annotations.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/VQA_format_all_annotations.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/all_annotations.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/all_annotations.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/VQA_format_testset_only.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/VQA_format_testset_only.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/VQA_fromat_trainset_only.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/VQA_fromat_trainset_only.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_test_split/cecum_test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_test_split/cecum_test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_test_split/dlp_test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_test_split/dlp_test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_test_split/drm_test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_test_split/drm_test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_test_split/eso_test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_test_split/eso_test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_test_split/pol_test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_test_split/pol_test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_test_split/pylorus_test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_test_split/pylorus_test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_test_split/uc_test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_test_split/uc_test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_test_split/zline_test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_test_split/zline_test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_train_split/cecum_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_train_split/cecum_train.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_train_split/dlp_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_train_split/dlp_train.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_train_split/drm_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_train_split/drm_train.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_train_split/eso_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_train_split/eso_train.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_train_split/pol_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_train_split/pol_train.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_train_split/pylorus_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_train_split/pylorus_train.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_train_split/uc_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_train_split/uc_train.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/classwise_train_split/zline_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/classwise_train_split/zline_train.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/test.json -------------------------------------------------------------------------------- /dataset/Gut-VLM/train_test_split/train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/dataset/Gut-VLM/train_test_split/train.json -------------------------------------------------------------------------------- /eval_scripts/caption2vqa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/eval_scripts/caption2vqa.py -------------------------------------------------------------------------------- /eval_scripts/evaluate_caption.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/eval_scripts/evaluate_caption.py -------------------------------------------------------------------------------- /eval_scripts/evaluate_vqa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/eval_scripts/evaluate_vqa.py -------------------------------------------------------------------------------- /eval_scripts/parse_vqa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/eval_scripts/parse_vqa.py -------------------------------------------------------------------------------- /results_example/final_qwen_caption_hal_aware_results.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/results_example/final_qwen_caption_hal_aware_results.json -------------------------------------------------------------------------------- /results_example/groundtruth_test_captions.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/results_example/groundtruth_test_captions.json -------------------------------------------------------------------------------- /training_style/hallucinated_aware_train.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhattarailab/Hallucination-Aware-VLM/HEAD/training_style/hallucinated_aware_train.json --------------------------------------------------------------------------------