[Paper 리뷰] M2R-Whisper: Multi-Stage and Multi-Scale Retrieval Augmentation for Enhancing Whisper
M2R-Whisepr: Multi-Stage and Multi-Scale Retrieval Augmentation for Enhancing WhisperWhisper는 다양한 subdialect를 acculately recognize 하는데 한계가 있음M2R-WhisperIn-Context Learning과 Retrieval-Augmented technique을 Whisper에 도입Pre-processing stage에서 sentence-level in-context learning을 적용하고 post-processing stage에서는 token-level $k$-Nearest Neighbor를 적용논문 (ICASSP 2025) : Paper Link1. IntroductionWhisper는 Autom..
[Paper 리뷰] Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASR
Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASRLarge Transformer-based model은 self-attention mechanism으로 인해 computationally intensive 함Whisper-MedusaWhisper architecture를 extend 하여 iteration 마다 multiple token을 predictWord Error Rate에 대한 영향을 최소화하면서 latency를 50% 절감논문 (ICASSP 2025) : Paper Link1. IntroductionWhisper와 같은 Transformer-based supervised model은 Automatic ..
[Paper 리뷰] Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decoding
Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware DecodingCode-Switching Automatic Speech Recognition은 여전히 seamless language switch 측면에서 한계가 있음CS-WhisperWhisper를 기반으로 encoder의 intra-sentence switching을 향상하기 위해 Encoder Refiner를 도입각 decoder layer에서 language-specific decoding information을 얻기 위해 서로 다른 language prompt를 가진 Language-Aware Adapter를 활용논문 (ICASSP 2025) : Pap..
[Paper 리뷰] Multilingual DistilWhisper: Efficient Distillation of Multi-Task Speech Models via Language-Specific Experts
Multilingual DistilWhisper: Efficient Distillation of Multi-Task Speech Models via Language-Specific ExpertsWhisper는 under-represented language에 대해 여전히 낮은 성능을 보임Multilingual DistilWhisperWhisper-Large-V2에 대한 knowledge distillation을 적용Language-specific expert를 통한 lightweight modular ASR fine-tuning논문 (ICASSP 2024) : Paper Link1. IntroductionAutomatic Speech Recognition (ASR) task에서 Whisper는 강력한 성..
[Paper 리뷰] LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR
LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASRMultilingual Automatic Speech Recognition을 위해서는 language interference와 성능 저하 없는 new language incorporation이 필요함LoRA-WhisperWhisper에 LoRA matrix를 incorporate 하여 language interference를 완화LoRA와 language 간의 similarity를 활용하여 new language에 대한 성능을 개선논문 (ICASSP 2024) : Paper Link1. IntroductionAutomatic Speech Recognition (ASR)은 speech를 wr..
[Paper 리뷰] CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptions
CrisperWhisper: Accurate Timestamps on Verbatim Speech TranscriptionsWhisper의 tokenizer를 adjust 하여 word-level timestamps precision을 향상할 수 있음CrisperWhisperWhisper decoder의 cross-attention score에 dynamic time warping을 적용추가적인 fine-tuning을 통해 robustness를 향상논문 (INTERSPEECH 2024) : Paper Link1. IntroductionAutomatic Speech Recognition (ASR)에서 large-scale, weakly supervised learning은 뛰어난 성능을 보이고 있음특히 S..