├── README.md ├── aggregate_eval_stat.py ├── config ├── accelerate_config.yaml ├── aggregate_eval_stat.yaml ├── eval.yaml ├── finetune.yaml ├── forget.yaml ├── forget_lora.yaml └── model_config.yaml ├── data_generation └── illustration_generate.py ├── data_loader.py ├── data_module.py ├── dataset ├── README.md ├── full.json ├── prompt.json └── split.json ├── eval ├── eval_mme.py ├── eval_pope.py └── result.csv ├── evaluate_util.py ├── finetune.py ├── forget.py ├── gpt_eval.py ├── inference.py ├── overview.png ├── pyproject.toml ├── requirements.txt ├── results_collect.py ├── scripts ├── aggregate_eval_stat.bash ├── eval_everything.bash ├── finetune.bash ├── forget.bash └── forget_lora.bash ├── utils.py └── visualization.ipynb /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/README.md -------------------------------------------------------------------------------- /aggregate_eval_stat.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/aggregate_eval_stat.py -------------------------------------------------------------------------------- /config/accelerate_config.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/config/accelerate_config.yaml -------------------------------------------------------------------------------- /config/aggregate_eval_stat.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/config/aggregate_eval_stat.yaml -------------------------------------------------------------------------------- /config/eval.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/config/eval.yaml -------------------------------------------------------------------------------- /config/finetune.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/config/finetune.yaml -------------------------------------------------------------------------------- /config/forget.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/config/forget.yaml -------------------------------------------------------------------------------- /config/forget_lora.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/config/forget_lora.yaml -------------------------------------------------------------------------------- /config/model_config.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/config/model_config.yaml -------------------------------------------------------------------------------- /data_generation/illustration_generate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/data_generation/illustration_generate.py -------------------------------------------------------------------------------- /data_loader.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/data_loader.py -------------------------------------------------------------------------------- /data_module.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/data_module.py -------------------------------------------------------------------------------- /dataset/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | license: apache-2.0 3 | --- 4 | -------------------------------------------------------------------------------- /dataset/full.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/dataset/full.json -------------------------------------------------------------------------------- /dataset/prompt.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/dataset/prompt.json -------------------------------------------------------------------------------- /dataset/split.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/dataset/split.json -------------------------------------------------------------------------------- /eval/eval_mme.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/eval/eval_mme.py -------------------------------------------------------------------------------- /eval/eval_pope.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/eval/eval_pope.py -------------------------------------------------------------------------------- /eval/result.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/eval/result.csv -------------------------------------------------------------------------------- /evaluate_util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/evaluate_util.py -------------------------------------------------------------------------------- /finetune.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/finetune.py -------------------------------------------------------------------------------- /forget.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/forget.py -------------------------------------------------------------------------------- /gpt_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/gpt_eval.py -------------------------------------------------------------------------------- /inference.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/inference.py -------------------------------------------------------------------------------- /overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/overview.png -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/pyproject.toml -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/requirements.txt -------------------------------------------------------------------------------- /results_collect.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/results_collect.py -------------------------------------------------------------------------------- /scripts/aggregate_eval_stat.bash: -------------------------------------------------------------------------------- 1 | python ./aggregate_eval_stat.py 2 | -------------------------------------------------------------------------------- /scripts/eval_everything.bash: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/scripts/eval_everything.bash -------------------------------------------------------------------------------- /scripts/finetune.bash: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/scripts/finetune.bash -------------------------------------------------------------------------------- /scripts/forget.bash: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/scripts/forget.bash -------------------------------------------------------------------------------- /scripts/forget_lora.bash: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/scripts/forget_lora.bash -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/utils.py -------------------------------------------------------------------------------- /visualization.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SaFo-Lab/FIUBench/HEAD/visualization.ipynb --------------------------------------------------------------------------------