MME_STT_Model

Hindi speech to Roman-script Hinglish transcription (lowercase, no punctuation). The model was fine-tuned with LoRA and the adapter was merged into the weights, so it loads as a normal Transformers model with no PEFT.

Usage

import torch, librosa
from transformers import WhisperForConditionalGeneration, WhisperProcessor

model = WhisperForConditionalGeneration.from_pretrained("MmeOG/MME_STT_Model", torch_dtype=torch.float16).to("cuda")
processor = WhisperProcessor.from_pretrained("MmeOG/MME_STT_Model")

audio, _ = librosa.load("sample.wav", sr=16000)   # 16 kHz mono, up to 30 seconds
feats = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to("cuda", torch.float16)
ids = model.generate(feats, language="hi", task="transcribe")
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

Evaluation

Phonetic WER (pWER, lower is better) on six public Hindi sets (1,100 clips). It is a lenient metric used to compare systems on the same clips, not an absolute accuracy figure.

Dataset	This model	Before fine-tuning
fleurs	18.45	20.15
commonvoice	26.42	29.16
indictts	13.32	13.72
kathbath	19.79	21.73
kathbath_noisy	21.91	23.37
mucs	19.65	20.84
all	19.64	21.21

Limitations

• Not validated on real phone calls.
• Not a streaming model; it transcribes after the speaker finishes.
• Accuracy drops in noise: about 5 to 7 points at 10 dB, and strong noise (0 dB) breaks it.
• Occasional repetition loops (about 0.5% of clips); a repetition guard at decoding time is recommended.
• Very short clips (under 2 seconds) have high error rates.
• Outputs have no punctuation or capitalisation, and there is no speaker diarisation.