├── .gitignore ├── LICENSE ├── README.md ├── abr ├── run_abr.sh └── run_eval.py ├── api ├── run_api.sh └── run_eval.py ├── ctranslate2 ├── run_eval.py └── run_whisper.sh ├── data ├── sample_4469669.wav └── sample_ami-es2015b.wav ├── granite ├── run_eval.py └── run_granite.sh ├── kyutai ├── run_eval.py └── run_kyutai.sh ├── liteASR ├── run_eval.py └── run_liteasr.sh ├── moonshine ├── run_eval.py └── run_moonshine.sh ├── nemo_asr ├── run_canary.sh ├── run_eval.py ├── run_eval_ml.py ├── run_eval_salm.py ├── run_nemo_ml.sh ├── run_parakeet.sh └── run_salm.sh ├── normalizer ├── __init__.py ├── data_utils.py ├── english_abbreviations.py ├── eval_utils.py └── normalizer.py ├── phi ├── run_eval.py └── run_phi4_multimodal.sh ├── requirements ├── requirements-api.txt ├── requirements.txt ├── requirements_abr.txt ├── requirements_ctranslate2.txt ├── requirements_granite.txt ├── requirements_kyutai.txt ├── requirements_moonshine.txt ├── requirements_nemo.txt ├── requirements_phi.txt ├── requirements_speechbrain.txt └── requirements_trtllm.txt ├── speechbrain ├── run_conformer.sh ├── run_conformersmall.sh ├── run_crdnn_rnnlm.sh ├── run_crdnn_transformerlm.sh ├── run_eval.py ├── run_transformer_transformerlm.sh ├── run_wav2vec2_commonvoice.sh └── run_wav2vec2_librispeech.sh ├── tensorrtllm ├── run_eval.py ├── run_whisper.sh └── whisper_utils.py └── transformers ├── run_data2vec.sh ├── run_eval.py ├── run_hubert.sh ├── run_mms.sh ├── run_wav2vec2.sh ├── run_wav2vec2_conformer.sh └── run_whisper.sh /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/README.md -------------------------------------------------------------------------------- /abr/run_abr.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/abr/run_abr.sh -------------------------------------------------------------------------------- /abr/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/abr/run_eval.py -------------------------------------------------------------------------------- /api/run_api.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/api/run_api.sh -------------------------------------------------------------------------------- /api/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/api/run_eval.py -------------------------------------------------------------------------------- /ctranslate2/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/ctranslate2/run_eval.py -------------------------------------------------------------------------------- /ctranslate2/run_whisper.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/ctranslate2/run_whisper.sh -------------------------------------------------------------------------------- /data/sample_4469669.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/data/sample_4469669.wav -------------------------------------------------------------------------------- /data/sample_ami-es2015b.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/data/sample_ami-es2015b.wav -------------------------------------------------------------------------------- /granite/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/granite/run_eval.py -------------------------------------------------------------------------------- /granite/run_granite.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/granite/run_granite.sh -------------------------------------------------------------------------------- /kyutai/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/kyutai/run_eval.py -------------------------------------------------------------------------------- /kyutai/run_kyutai.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/kyutai/run_kyutai.sh -------------------------------------------------------------------------------- /liteASR/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/liteASR/run_eval.py -------------------------------------------------------------------------------- /liteASR/run_liteasr.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/liteASR/run_liteasr.sh -------------------------------------------------------------------------------- /moonshine/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/moonshine/run_eval.py -------------------------------------------------------------------------------- /moonshine/run_moonshine.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/moonshine/run_moonshine.sh -------------------------------------------------------------------------------- /nemo_asr/run_canary.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/nemo_asr/run_canary.sh -------------------------------------------------------------------------------- /nemo_asr/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/nemo_asr/run_eval.py -------------------------------------------------------------------------------- /nemo_asr/run_eval_ml.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/nemo_asr/run_eval_ml.py -------------------------------------------------------------------------------- /nemo_asr/run_eval_salm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/nemo_asr/run_eval_salm.py -------------------------------------------------------------------------------- /nemo_asr/run_nemo_ml.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/nemo_asr/run_nemo_ml.sh -------------------------------------------------------------------------------- /nemo_asr/run_parakeet.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/nemo_asr/run_parakeet.sh -------------------------------------------------------------------------------- /nemo_asr/run_salm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/nemo_asr/run_salm.sh -------------------------------------------------------------------------------- /normalizer/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/normalizer/__init__.py -------------------------------------------------------------------------------- /normalizer/data_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/normalizer/data_utils.py -------------------------------------------------------------------------------- /normalizer/english_abbreviations.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/normalizer/english_abbreviations.py -------------------------------------------------------------------------------- /normalizer/eval_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/normalizer/eval_utils.py -------------------------------------------------------------------------------- /normalizer/normalizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/normalizer/normalizer.py -------------------------------------------------------------------------------- /phi/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/phi/run_eval.py -------------------------------------------------------------------------------- /phi/run_phi4_multimodal.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/phi/run_phi4_multimodal.sh -------------------------------------------------------------------------------- /requirements/requirements-api.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/requirements/requirements-api.txt -------------------------------------------------------------------------------- /requirements/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/requirements/requirements.txt -------------------------------------------------------------------------------- /requirements/requirements_abr.txt: -------------------------------------------------------------------------------- 1 | sentencepiece==0.2.1 2 | torchcodec 3 | -------------------------------------------------------------------------------- /requirements/requirements_ctranslate2.txt: -------------------------------------------------------------------------------- 1 | datasets 2 | evaluate 3 | faster-whisper>=0.8.0 4 | jiwer 5 | librosa 6 | -------------------------------------------------------------------------------- /requirements/requirements_granite.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/requirements/requirements_granite.txt -------------------------------------------------------------------------------- /requirements/requirements_kyutai.txt: -------------------------------------------------------------------------------- 1 | torch>=2.6.0 2 | julius 3 | moshi>=0.2.6 4 | -------------------------------------------------------------------------------- /requirements/requirements_moonshine.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/requirements/requirements_moonshine.txt -------------------------------------------------------------------------------- /requirements/requirements_nemo.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/requirements/requirements_nemo.txt -------------------------------------------------------------------------------- /requirements/requirements_phi.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/requirements/requirements_phi.txt -------------------------------------------------------------------------------- /requirements/requirements_speechbrain.txt: -------------------------------------------------------------------------------- 1 | torch 2 | speechbrain 3 | evaluate 4 | datasets -------------------------------------------------------------------------------- /requirements/requirements_trtllm.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/requirements/requirements_trtllm.txt -------------------------------------------------------------------------------- /speechbrain/run_conformer.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/speechbrain/run_conformer.sh -------------------------------------------------------------------------------- /speechbrain/run_conformersmall.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/speechbrain/run_conformersmall.sh -------------------------------------------------------------------------------- /speechbrain/run_crdnn_rnnlm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/speechbrain/run_crdnn_rnnlm.sh -------------------------------------------------------------------------------- /speechbrain/run_crdnn_transformerlm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/speechbrain/run_crdnn_transformerlm.sh -------------------------------------------------------------------------------- /speechbrain/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/speechbrain/run_eval.py -------------------------------------------------------------------------------- /speechbrain/run_transformer_transformerlm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/speechbrain/run_transformer_transformerlm.sh -------------------------------------------------------------------------------- /speechbrain/run_wav2vec2_commonvoice.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/speechbrain/run_wav2vec2_commonvoice.sh -------------------------------------------------------------------------------- /speechbrain/run_wav2vec2_librispeech.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/speechbrain/run_wav2vec2_librispeech.sh -------------------------------------------------------------------------------- /tensorrtllm/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/tensorrtllm/run_eval.py -------------------------------------------------------------------------------- /tensorrtllm/run_whisper.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/tensorrtllm/run_whisper.sh -------------------------------------------------------------------------------- /tensorrtllm/whisper_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/tensorrtllm/whisper_utils.py -------------------------------------------------------------------------------- /transformers/run_data2vec.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/transformers/run_data2vec.sh -------------------------------------------------------------------------------- /transformers/run_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/transformers/run_eval.py -------------------------------------------------------------------------------- /transformers/run_hubert.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/transformers/run_hubert.sh -------------------------------------------------------------------------------- /transformers/run_mms.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/transformers/run_mms.sh -------------------------------------------------------------------------------- /transformers/run_wav2vec2.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/transformers/run_wav2vec2.sh -------------------------------------------------------------------------------- /transformers/run_wav2vec2_conformer.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/transformers/run_wav2vec2_conformer.sh -------------------------------------------------------------------------------- /transformers/run_whisper.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/huggingface/open_asr_leaderboard/HEAD/transformers/run_whisper.sh --------------------------------------------------------------------------------