├── GPT_evaluation └── prompts │ ├── adherence.txt │ ├── db_quality.txt │ ├── input_quality.txt │ └── sql_quality.txt ├── OmniSQL_prediction_results └── README.md ├── README.md ├── assets ├── example_questions.png ├── framework.png └── main_results.png ├── data_synthesis ├── README.md ├── cot_synthesis │ ├── README.md │ ├── generate_cot_synthesis_prompts.py │ ├── post_process_cot.py │ ├── prompt_templates │ │ └── cot_synthesis_prompt_template.txt │ └── synthesize_cot.py ├── database_synthesis │ ├── README.md │ ├── build_sqlite_databases.py │ ├── enhance_schema.py │ ├── generate_schema_enhancement_prompts.py │ ├── generate_schema_synthesis_prompts.py │ ├── generate_tables_json.py │ ├── prompt_templates │ │ ├── enhance_prompt.txt │ │ └── schema_prompt.txt │ ├── sqlite_schema_parser.py │ ├── synthesize_schema.py │ └── web_tables.json.zip ├── question_synthesis │ ├── README.md │ ├── generate_question_synthesis_prompts.py │ ├── post_process_questions.py │ ├── prompt_templates │ │ └── question_synthesis_prompt.txt │ └── synthesize_question.py └── sql_synthesis │ ├── README.md │ ├── generate_sql_synthesis_prompts.py │ ├── post_process_sqls.py │ ├── prompt_templates │ ├── sql_synthesis_prompt.txt │ └── sqlite_funcs.json │ └── synthesize_sql.py ├── examples ├── example_1.txt ├── example_2.txt ├── example_3.txt └── example_4.txt └── train_and_evaluate ├── README.md ├── accelerate_config_14b.yaml ├── accelerate_config_32b.yaml ├── accelerate_config_7b.yaml ├── auto_evaluation.py ├── build_contents_index.py ├── eval_open_source_models.py ├── evaluate_bird.py ├── evaluate_spider.py ├── evaluate_spider2.py ├── infer.py ├── merge_lora_adapter.py ├── monkey_patch_packing.py ├── nltk_downloader.py ├── process_dataset.py ├── process_dataset.sh ├── train.py ├── train_omnisql_14b.sh ├── train_omnisql_32b.sh ├── train_omnisql_7b.sh └── utils ├── load_sft_dataset.py └── lr_scheduler.py /GPT_evaluation/prompts/adherence.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/GPT_evaluation/prompts/adherence.txt -------------------------------------------------------------------------------- /GPT_evaluation/prompts/db_quality.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/GPT_evaluation/prompts/db_quality.txt -------------------------------------------------------------------------------- /GPT_evaluation/prompts/input_quality.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/GPT_evaluation/prompts/input_quality.txt -------------------------------------------------------------------------------- /GPT_evaluation/prompts/sql_quality.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/GPT_evaluation/prompts/sql_quality.txt -------------------------------------------------------------------------------- /OmniSQL_prediction_results/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/OmniSQL_prediction_results/README.md -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/README.md -------------------------------------------------------------------------------- /assets/example_questions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/assets/example_questions.png -------------------------------------------------------------------------------- /assets/framework.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/assets/framework.png -------------------------------------------------------------------------------- /assets/main_results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/assets/main_results.png -------------------------------------------------------------------------------- /data_synthesis/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/README.md -------------------------------------------------------------------------------- /data_synthesis/cot_synthesis/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/cot_synthesis/README.md -------------------------------------------------------------------------------- /data_synthesis/cot_synthesis/generate_cot_synthesis_prompts.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/cot_synthesis/generate_cot_synthesis_prompts.py -------------------------------------------------------------------------------- /data_synthesis/cot_synthesis/post_process_cot.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/cot_synthesis/post_process_cot.py -------------------------------------------------------------------------------- /data_synthesis/cot_synthesis/prompt_templates/cot_synthesis_prompt_template.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/cot_synthesis/prompt_templates/cot_synthesis_prompt_template.txt -------------------------------------------------------------------------------- /data_synthesis/cot_synthesis/synthesize_cot.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/cot_synthesis/synthesize_cot.py -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/README.md -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/build_sqlite_databases.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/build_sqlite_databases.py -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/enhance_schema.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/enhance_schema.py -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/generate_schema_enhancement_prompts.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/generate_schema_enhancement_prompts.py -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/generate_schema_synthesis_prompts.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/generate_schema_synthesis_prompts.py -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/generate_tables_json.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/generate_tables_json.py -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/prompt_templates/enhance_prompt.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/prompt_templates/enhance_prompt.txt -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/prompt_templates/schema_prompt.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/prompt_templates/schema_prompt.txt -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/sqlite_schema_parser.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/sqlite_schema_parser.py -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/synthesize_schema.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/synthesize_schema.py -------------------------------------------------------------------------------- /data_synthesis/database_synthesis/web_tables.json.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/database_synthesis/web_tables.json.zip -------------------------------------------------------------------------------- /data_synthesis/question_synthesis/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/question_synthesis/README.md -------------------------------------------------------------------------------- /data_synthesis/question_synthesis/generate_question_synthesis_prompts.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/question_synthesis/generate_question_synthesis_prompts.py -------------------------------------------------------------------------------- /data_synthesis/question_synthesis/post_process_questions.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/question_synthesis/post_process_questions.py -------------------------------------------------------------------------------- /data_synthesis/question_synthesis/prompt_templates/question_synthesis_prompt.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/question_synthesis/prompt_templates/question_synthesis_prompt.txt -------------------------------------------------------------------------------- /data_synthesis/question_synthesis/synthesize_question.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/question_synthesis/synthesize_question.py -------------------------------------------------------------------------------- /data_synthesis/sql_synthesis/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/sql_synthesis/README.md -------------------------------------------------------------------------------- /data_synthesis/sql_synthesis/generate_sql_synthesis_prompts.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/sql_synthesis/generate_sql_synthesis_prompts.py -------------------------------------------------------------------------------- /data_synthesis/sql_synthesis/post_process_sqls.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/sql_synthesis/post_process_sqls.py -------------------------------------------------------------------------------- /data_synthesis/sql_synthesis/prompt_templates/sql_synthesis_prompt.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/sql_synthesis/prompt_templates/sql_synthesis_prompt.txt -------------------------------------------------------------------------------- /data_synthesis/sql_synthesis/prompt_templates/sqlite_funcs.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/sql_synthesis/prompt_templates/sqlite_funcs.json -------------------------------------------------------------------------------- /data_synthesis/sql_synthesis/synthesize_sql.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/data_synthesis/sql_synthesis/synthesize_sql.py -------------------------------------------------------------------------------- /examples/example_1.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/examples/example_1.txt -------------------------------------------------------------------------------- /examples/example_2.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/examples/example_2.txt -------------------------------------------------------------------------------- /examples/example_3.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/examples/example_3.txt -------------------------------------------------------------------------------- /examples/example_4.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/examples/example_4.txt -------------------------------------------------------------------------------- /train_and_evaluate/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/README.md -------------------------------------------------------------------------------- /train_and_evaluate/accelerate_config_14b.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/accelerate_config_14b.yaml -------------------------------------------------------------------------------- /train_and_evaluate/accelerate_config_32b.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/accelerate_config_32b.yaml -------------------------------------------------------------------------------- /train_and_evaluate/accelerate_config_7b.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/accelerate_config_7b.yaml -------------------------------------------------------------------------------- /train_and_evaluate/auto_evaluation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/auto_evaluation.py -------------------------------------------------------------------------------- /train_and_evaluate/build_contents_index.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/build_contents_index.py -------------------------------------------------------------------------------- /train_and_evaluate/eval_open_source_models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/eval_open_source_models.py -------------------------------------------------------------------------------- /train_and_evaluate/evaluate_bird.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/evaluate_bird.py -------------------------------------------------------------------------------- /train_and_evaluate/evaluate_spider.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/evaluate_spider.py -------------------------------------------------------------------------------- /train_and_evaluate/evaluate_spider2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/evaluate_spider2.py -------------------------------------------------------------------------------- /train_and_evaluate/infer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/infer.py -------------------------------------------------------------------------------- /train_and_evaluate/merge_lora_adapter.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/merge_lora_adapter.py -------------------------------------------------------------------------------- /train_and_evaluate/monkey_patch_packing.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/monkey_patch_packing.py -------------------------------------------------------------------------------- /train_and_evaluate/nltk_downloader.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/nltk_downloader.py -------------------------------------------------------------------------------- /train_and_evaluate/process_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/process_dataset.py -------------------------------------------------------------------------------- /train_and_evaluate/process_dataset.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/process_dataset.sh -------------------------------------------------------------------------------- /train_and_evaluate/train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/train.py -------------------------------------------------------------------------------- /train_and_evaluate/train_omnisql_14b.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/train_omnisql_14b.sh -------------------------------------------------------------------------------- /train_and_evaluate/train_omnisql_32b.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/train_omnisql_32b.sh -------------------------------------------------------------------------------- /train_and_evaluate/train_omnisql_7b.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/train_omnisql_7b.sh -------------------------------------------------------------------------------- /train_and_evaluate/utils/load_sft_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/utils/load_sft_dataset.py -------------------------------------------------------------------------------- /train_and_evaluate/utils/lr_scheduler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/RUCKBReasoning/OmniSQL/HEAD/train_and_evaluate/utils/lr_scheduler.py --------------------------------------------------------------------------------