├── CONTRIBUTING.md ├── LICENSE-3rd-party.txt ├── LICENSE.txt ├── README.md ├── data ├── test │ ├── when2call_test_llm_judge.jsonl │ └── when2call_test_mcq.jsonl └── train │ ├── when2call_train_pref.jsonl │ └── when2call_train_sft.jsonl ├── evaluation ├── llm_as_a_judge │ ├── aggregate_llm_as_a_judge_results.py │ ├── run_openai_inference.py │ └── run_openai_judge.py └── mcq │ └── lm_eval_harness │ └── when2call │ ├── additional_metrics.py │ ├── utils.py │ ├── when2call-functionary.yaml │ ├── when2call-hermes.yaml │ ├── when2call-llama3_2.yaml │ ├── when2call-qwen2_5.yaml │ ├── when2call-xlam.yaml │ └── when2call.yaml ├── gitattributes.txt ├── requirements.txt └── synthetic_data_gen ├── convert_raw_train_data_to_pref.py ├── convert_raw_train_data_to_sft.py ├── create_eval_data.py ├── create_raw_train_data.py └── utils.py /CONTRIBUTING.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/CONTRIBUTING.md -------------------------------------------------------------------------------- /LICENSE-3rd-party.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/LICENSE-3rd-party.txt -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/LICENSE.txt -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/README.md -------------------------------------------------------------------------------- /data/test/when2call_test_llm_judge.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/data/test/when2call_test_llm_judge.jsonl -------------------------------------------------------------------------------- /data/test/when2call_test_mcq.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/data/test/when2call_test_mcq.jsonl -------------------------------------------------------------------------------- /data/train/when2call_train_pref.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/data/train/when2call_train_pref.jsonl -------------------------------------------------------------------------------- /data/train/when2call_train_sft.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/data/train/when2call_train_sft.jsonl -------------------------------------------------------------------------------- /evaluation/llm_as_a_judge/aggregate_llm_as_a_judge_results.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/llm_as_a_judge/aggregate_llm_as_a_judge_results.py -------------------------------------------------------------------------------- /evaluation/llm_as_a_judge/run_openai_inference.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/llm_as_a_judge/run_openai_inference.py -------------------------------------------------------------------------------- /evaluation/llm_as_a_judge/run_openai_judge.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/llm_as_a_judge/run_openai_judge.py -------------------------------------------------------------------------------- /evaluation/mcq/lm_eval_harness/when2call/additional_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/mcq/lm_eval_harness/when2call/additional_metrics.py -------------------------------------------------------------------------------- /evaluation/mcq/lm_eval_harness/when2call/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/mcq/lm_eval_harness/when2call/utils.py -------------------------------------------------------------------------------- /evaluation/mcq/lm_eval_harness/when2call/when2call-functionary.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/mcq/lm_eval_harness/when2call/when2call-functionary.yaml -------------------------------------------------------------------------------- /evaluation/mcq/lm_eval_harness/when2call/when2call-hermes.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/mcq/lm_eval_harness/when2call/when2call-hermes.yaml -------------------------------------------------------------------------------- /evaluation/mcq/lm_eval_harness/when2call/when2call-llama3_2.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/mcq/lm_eval_harness/when2call/when2call-llama3_2.yaml -------------------------------------------------------------------------------- /evaluation/mcq/lm_eval_harness/when2call/when2call-qwen2_5.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/mcq/lm_eval_harness/when2call/when2call-qwen2_5.yaml -------------------------------------------------------------------------------- /evaluation/mcq/lm_eval_harness/when2call/when2call-xlam.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/mcq/lm_eval_harness/when2call/when2call-xlam.yaml -------------------------------------------------------------------------------- /evaluation/mcq/lm_eval_harness/when2call/when2call.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/evaluation/mcq/lm_eval_harness/when2call/when2call.yaml -------------------------------------------------------------------------------- /gitattributes.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/gitattributes.txt -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | openai==1.75.0 -------------------------------------------------------------------------------- /synthetic_data_gen/convert_raw_train_data_to_pref.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/synthetic_data_gen/convert_raw_train_data_to_pref.py -------------------------------------------------------------------------------- /synthetic_data_gen/convert_raw_train_data_to_sft.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/synthetic_data_gen/convert_raw_train_data_to_sft.py -------------------------------------------------------------------------------- /synthetic_data_gen/create_eval_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/synthetic_data_gen/create_eval_data.py -------------------------------------------------------------------------------- /synthetic_data_gen/create_raw_train_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/synthetic_data_gen/create_raw_train_data.py -------------------------------------------------------------------------------- /synthetic_data_gen/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NVIDIA/When2Call/HEAD/synthetic_data_gen/utils.py --------------------------------------------------------------------------------