├── .gitignore ├── README.md ├── assets ├── finance_duration.png ├── finance_sentiment.png ├── finance_type.png ├── mtbench.png ├── weather_duration.png └── weather_event.png ├── data_preparation ├── finance │ ├── create_correlation.py │ ├── create_mcqa.py │ ├── create_pairs.py │ ├── run_create_correlation.sh │ ├── run_create_mcqa.sh │ ├── run_create_pairs.sh │ └── utils.py └── weather │ └── create_pairs.py ├── download_processed_dataset.py ├── download_raw_dataset.py ├── evaluation ├── api_call.py ├── finance │ ├── correlation_prediction.py │ ├── mcqa.py │ ├── meta_prompt.py │ ├── run_indicator_prediction.sh │ ├── run_question_answering.sh │ ├── run_trend_classification.sh │ ├── run_ts_forecast.sh │ ├── trend_classification.py │ └── value_prediction.py ├── utils.py └── weather │ ├── __init__.py │ ├── indicator_prediction.py │ ├── mcqa.py │ ├── meta_prompt.py │ ├── run_indicator_prediction.sh │ ├── run_question_answering.sh │ ├── run_temperature_forecast.sh │ ├── run_trend_classification.sh │ ├── temperature_forecast.py │ └── trend_classification.py └── requirement.txt /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/.gitignore -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/README.md -------------------------------------------------------------------------------- /assets/finance_duration.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/assets/finance_duration.png -------------------------------------------------------------------------------- /assets/finance_sentiment.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/assets/finance_sentiment.png -------------------------------------------------------------------------------- /assets/finance_type.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/assets/finance_type.png -------------------------------------------------------------------------------- /assets/mtbench.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/assets/mtbench.png -------------------------------------------------------------------------------- /assets/weather_duration.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/assets/weather_duration.png -------------------------------------------------------------------------------- /assets/weather_event.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/assets/weather_event.png -------------------------------------------------------------------------------- /data_preparation/finance/create_correlation.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data_preparation/finance/create_mcqa.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data_preparation/finance/create_pairs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/data_preparation/finance/create_pairs.py -------------------------------------------------------------------------------- /data_preparation/finance/run_create_correlation.sh: -------------------------------------------------------------------------------- 1 | python create_correlation.py -------------------------------------------------------------------------------- /data_preparation/finance/run_create_mcqa.sh: -------------------------------------------------------------------------------- 1 | python create_mcqa.py -------------------------------------------------------------------------------- /data_preparation/finance/run_create_pairs.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/data_preparation/finance/run_create_pairs.sh -------------------------------------------------------------------------------- /data_preparation/finance/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/data_preparation/finance/utils.py -------------------------------------------------------------------------------- /data_preparation/weather/create_pairs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/data_preparation/weather/create_pairs.py -------------------------------------------------------------------------------- /download_processed_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/download_processed_dataset.py -------------------------------------------------------------------------------- /download_raw_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/download_raw_dataset.py -------------------------------------------------------------------------------- /evaluation/api_call.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/api_call.py -------------------------------------------------------------------------------- /evaluation/finance/correlation_prediction.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/correlation_prediction.py -------------------------------------------------------------------------------- /evaluation/finance/mcqa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/mcqa.py -------------------------------------------------------------------------------- /evaluation/finance/meta_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/meta_prompt.py -------------------------------------------------------------------------------- /evaluation/finance/run_indicator_prediction.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/run_indicator_prediction.sh -------------------------------------------------------------------------------- /evaluation/finance/run_question_answering.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/run_question_answering.sh -------------------------------------------------------------------------------- /evaluation/finance/run_trend_classification.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/run_trend_classification.sh -------------------------------------------------------------------------------- /evaluation/finance/run_ts_forecast.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/run_ts_forecast.sh -------------------------------------------------------------------------------- /evaluation/finance/trend_classification.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/trend_classification.py -------------------------------------------------------------------------------- /evaluation/finance/value_prediction.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/finance/value_prediction.py -------------------------------------------------------------------------------- /evaluation/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/utils.py -------------------------------------------------------------------------------- /evaluation/weather/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /evaluation/weather/indicator_prediction.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/indicator_prediction.py -------------------------------------------------------------------------------- /evaluation/weather/mcqa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/mcqa.py -------------------------------------------------------------------------------- /evaluation/weather/meta_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/meta_prompt.py -------------------------------------------------------------------------------- /evaluation/weather/run_indicator_prediction.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/run_indicator_prediction.sh -------------------------------------------------------------------------------- /evaluation/weather/run_question_answering.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/run_question_answering.sh -------------------------------------------------------------------------------- /evaluation/weather/run_temperature_forecast.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/run_temperature_forecast.sh -------------------------------------------------------------------------------- /evaluation/weather/run_trend_classification.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/run_trend_classification.sh -------------------------------------------------------------------------------- /evaluation/weather/temperature_forecast.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/temperature_forecast.py -------------------------------------------------------------------------------- /evaluation/weather/trend_classification.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/evaluation/weather/trend_classification.py -------------------------------------------------------------------------------- /requirement.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Graph-and-Geometric-Learning/MTBench/HEAD/requirement.txt --------------------------------------------------------------------------------