Benchmark · frontier-ASR baseline
How a frontier ASR hears Bantu-accented English.
Every code-switch take is time-segmented at the Bantu→English boundary, so the
English half is isolated, transcribed by a frontier ASR (Google Cloud Speech-to-Text),
and scored against the gold reference. The result is a measured, reference-backed
accent-robustness benchmark — run your own model on the same audio and
compare to this baseline.
The systematic failure mode
The accent breaks function words before it breaks meaning.
The substantive English is mostly recovered (content recall 81%),
but the discourse marker means is mis-transcribed in
69% of takes — heard as “men's”, “main”, even
“Main Street”. A high-frequency word a frontier ASR gets wrong almost every time on this
accent. That is exactly the gap an accent benchmark is built to surface.
Listen + compare
Gold reference vs the ASR hypothesis.
Play the English segment (what the STT was given), then read the gold reference against Cloud STT's transcript.
akanya nga fwaya ukuya kucimbushu
“means” misheard
akanya nga fwaya ukuya kucimbushu
means she will defecate when she feels like going to the toilet
Audio pending.
Gold refmeans she will defecate when she feels like going to the toilet
Cloud STTmen's shoes defecate when she feels like going to the toilet
confidence 0.84
content recall 89%
WER 0.30
Speaker BEM-A
akanya nga fwaya ukuya kucimbushu
not detected as English
akanya nga fwaya ukuya kucimbushu
means he will defecate when he feels like going to the toilet
Audio pending.
Gold refmeans he will defecate when he feels like going to the toilet
Cloud STTyou defecate
confidence 0.48
content recall 11%
WER 0.90
Speaker BEM-A
Akabomba ku ofesi namumushi nga afika
not detected as English
Akabomba ku ofesi namumushi nga afika
means he will work at the office tomorrow when he arrives
Audio pending.
Gold refmeans he will work at the office tomorrow when he arrives
Cloud STTmemes
confidence 0.45
content recall 0%
WER 1.00
Speaker BEM-A
Alelanda ne mfumu nga bafika ku ng'anda
not detected as English
Alelanda ne mfumu nga bafika ku ng'anda
means he was speaking with the chief when they arrived at the house
Audio pending.
Gold refmeans he was speaking with the chief when they arrived at the house
Cloud STTmeans
confidence 0.59
content recall 0%
WER 1.00
Speaker BEM-A
Abantu babomba ku ofesi lyonse
not detected as English
Abantu babomba ku ofesi lyonse
means the people work at the office every day
Audio pending.
Gold refmeans the people work at the office every day
Cloud STTmeans
confidence 0.55
content recall 0%
WER 1.00
Speaker BEM-A
Bafwile mu nshita ya nsala ya myaka iyibili
content review
Bafwile mu nshita ya nsala ya myaka iyibili
means they died in the famine of two years ago
Audio pending.
Gold refmeans they died in the famine of two years ago
Cloud STTmeans
confidence 0.64
content recall 0%
WER 1.00
Speaker BEM-A
akanya umunani
content review
akanya umunani
means share the relish
Audio pending.
Gold refmeans share the relish
Cloud STTmeans share the radish
confidence 0.66
content recall 50%
WER 0.50
Speaker BEM-A
ulya akanya!
content review
ulya akanya!
means that one will face severe consequences
Audio pending.
Gold refmeans that one will face severe consequences
Cloud STTmeans that one who said severe consequences
confidence 0.60
content recall 67%
WER 0.33
Speaker BEM-A
akanya kalelila
content review
akanya kalelila
means the baby is crying
Audio pending.
Gold refmeans the baby is crying
Cloud STTmeans the babies crying
confidence 0.74
content recall 33%
WER 0.67
Speaker BEM-A