Meet Oak: medical transcription that follows the conversation across languages and speakers

More than 74 million people in the US (2024) speak a language other than English at home. Spanish alone accounts for almost 45 million.

The clinical workforce reflects that multilingual reality. A national study of first-year US resident physicians found that 71.6% reported speaking a non-English language at some level, with Spanish the most prevalent advanced or native language in 34 of the 36 specialties studied.

And it’s not just the US. A similar pattern is seen in England and Australia, and throughout the world.

Healthcare already happens across languages, yet much of the technology documenting it still assumes the conversation will stay in one.

That is the gap Oak, our new medical speech-to-text model, is built to close.

Introducing Oak: built for multilingual consultations

Available today through the Speechmatics API, Oak is designed specifically for clinical conversations that move between speakers and languages.

It follows those changes without requiring a language to be selected in advance, while preserving the drug names, conditions and other clinical terms that carry the most meaning.

In our evaluation against eight other speech providers, Oak ranked first for medical term recall in all 11 languages we benchmarked. Across those languages, it captured 91% of medical terms on average. The other eight providers captured 74% on average, which means roughly one drug name, condition or side effect in four was missed.

Those results reflect the problem Oak was designed around: keeping clinical meaning intact as conversations move between speakers, languages and specialist terminology.

What Oak does

A medical consultation can involve a patient, clinician, nurse, interpreter or relative, and the conversation may shift between speakers and languages as it develops. Oak is built to follow that movement without asking clinical teams to simplify the conversation for the model.

It follows language changes. Oak switches between languages mid-conversation, as often as the speakers do, and returns language metadata alongside the transcript, so you know which language each part was spoken in. There’s no language to choose up front, and no need to route audio between models.

It follows speakers. Consultations often include a relative, an interpreter or both. Oak tracks who said what, preserving the distinction between a symptom reported by a patient, an observation added by somebody else and an instruction given by the clinician. That attribution can then flow into a clinical summary or downstream application.

It is built for clinical terminology. Drug names, diagnoses, procedures and side effects carry disproportionate importance in a medical transcript, so Oak is designed to preserve the terms that matter most rather than treating every word as equally consequential.

It supports medical speech across more than 55 languages. For this release, we benchmarked English alongside our customers’ most commonly used languages: Arabic, Danish, Dutch, Finnish, French, German, Norwegian, Polish, Spanish and Swedish.

It deploys where clinical data needs to stay. Oak can run in the cloud, private cloud or on-premises, giving healthcare organizations more control over where sensitive audio is processed and how it fits within existing security and data requirements.

Why medical accuracy needs a different measure

Those capabilities solve part of the problem. The next question is how to judge whether a medical transcript is actually accurate where it matters.

Speech recognition is often measured using word error rate, or WER. It is a useful measure of overall transcription quality, but in medical it can hide an important distinction.

WER treats every word equally. Mistranscribe “the” and the transcript receives one error. Mistranscribe “amlodipine” and it receives one error too, even though the clinical consequences of those mistakes are very different.

That is why we also measure medical term recall: how reliably a model captures the drug names, conditions, procedures and side effects spoken in a recording.

The distinction matters because the transcript is rarely the final product. In a medical scribe, it becomes the raw material for a clinical note, summary and, eventually, something a clinician may review and sign. An error introduced at transcription can travel through everything built on top of it.

A model therefore needs to do more than produce readable text. It needs to preserve the clinical meaning that downstream systems depend on.

How we tested

We compared Oak with AWS, AssemblyAI, Deepgram, ElevenLabs, Google, Microsoft, OpenAI and Soniox across 19 medical test sets in 11 languages.

Every provider received the same audio and was scored using the same methodology. We used default settings, with no custom vocabulary or keyterm prompting, and selected each provider’s best available model for each language, including dedicated medical or pharma models or domain where they existed and covered the language.

The English sets cover drug names, conditions, side effects, substances, and product and company names, alongside recorded clinical conversations. The ten non-English sets contain medical speech in each language.

We scored medical term recall using the Whisper normalizer and averaged the results without weighting, so a single large test set could not dominate the overall score.

Medical term recall: first in all 11 languages

Oak ranked first for medical term recall in every language we tested, capturing 91% of medical terms on average across all 11. The next best provider missed 2.1 times as many terms as Oak.

Medical term recall (%), averaged across test sets in each language. Higher is better. Includes dedicated medical, pharma or domain models wherever available. Speechmatics internal evaluation, October 2026. Full results also include Danish, Dutch, Finnish, Polish and Swedish.

Some of the largest differences appear in languages where the wider field struggles. In Spanish, Oak captured 96.3% of medical terms, meaning it missed around four in every 100 drug names, conditions and side effects, compared with around 13 missed by the other providers on average.

In Norwegian, Oak captured 93.3% of medical terms, compared with 74.5% for the next best provider.

Those individual results matter, but for teams operating across several markets there is another important question: how much does performance change when the language does?

Consistent from one language to the next

Most models that perform strongly in English lose substantial ground elsewhere.

Deepgram’s medical term recall fell from 84.0% in English to 26.1% in Danish, while AssemblyAI moved from 85.6% in English to 55.7% in Norwegian. Across the providers we tested, the difference between English performance and each provider’s weakest language ranged from 9 to 58 percentage points.

To put this into perspective, Oak’s lowest-scoring language in the benchmark was Danish, at 85.7%, where it still outperformed the next best provider.

For a team deploying across markets, that consistency can matter as much as the highest individual score. Expanding into new languages shouldn’t require recalibrating your trust in the transcript.

Oak versus specialist medical models

Our wider benchmark includes general-purpose speech models, so we also compared Oak with dedicated medical models wherever providers offered them for the language being tested.

In English, Oak scored 88.5% for medical term recall, compared with 87.7% for ElevenLabs Scribe v2 Medical and 85.6% for AssemblyAI’s medical model.

The gaps are larger in several other languages. In German, Oak captured 91.4% of medical terms compared with 88.1% and 83.3% respectively. In French, Oak scored 93.9% against 85.1% and 80.9%. In Spanish, Oak captured 96.3% against 86.4% for AssemblyAI’s medical model.

Code-switching is a separate problem

The benchmark above measures medical speech in each language independently. A consultation that moves between languages presents another challenge because the model has to recognize that a switch has happened and follow it correctly.

We have evaluated that capability separately.

In our published code-switching evaluation, conducted without language hints, Arabic and English produced a mixed error rate of 15.1%, less than half the 33.2% recorded by the next best model.

Mandarin and English also came out ahead, at 14.1% against 14.5%. Mixed error rate counts the share of words wrong across the whole transcript, so lower is better. These are examples of the pairs we have measured, not the limit of what the model supports.

Built for how clinical teams work

Benchmark results become useful when they translate into fewer workarounds for the teams building and using clinical products.

For multilingual medical scribes, one model can cover the supported languages used by clinicians and patients, reducing the need for a language picker, a separate routing layer or a different speech provider for each market. If the consultation changes language, the transcription can continue without restarting the job around a new language setting.

For multi-person visits, speaker attribution preserves the structure of the conversation. When a family member answers for a patient or an interpreter joins the consultation, a downstream note or summary can retain whether information was reported by the patient, observed by somebody else or instructed by the clinician.

For international deployments, consistency across languages means a new market does not automatically begin with a weaker transcription model. Teams can apply the same evaluation and metric across the languages they serve.

There is also an operational constraint that matters particularly in healthcare: where patient audio is processed and stored.

In production, and built for patient data

Oak builds on Speechmatics’ existing work across clinical documentation, patient intake and healthcare call handling.

Sully.ai uses Speechmatics for clinical scribes and AI receptionists serving more than 500 provider customers. Humetrix used Speechmatics for multilingual patient intake for people from 97 countries during the Paris 2024 Olympics, while in the UK, tens of millions of NHS 111 calls each year are transcribed through our partner Content Guru.

Working with clinical audio also brings a different set of requirements around security, privacy and where data is processed. Speechmatics is HIPAA-compliant, SOC 2 Type II certified, ISO/IEC 27001:2022 accredited and GDPR-compliant. Zero data retention is available by default, alongside audit trail support.

Oak can run in the cloud, private cloud and on-premises, giving healthcare organizations more control over where patient audio is processed and how the service fits within their own infrastructure, regulatory and data sovereignty requirements.

For batch workflows, speed matters too. An hour of multilingual audio can be transcribed in around 18 seconds. For a medical scribe, that means a draft note can be ready while the consultation is still fresh, so the clinician can review and sign it before moving on to the next patient.

Getting started

Oak is available now through the Speechmatics API, with configuration details in the Speechmatics documentation and runnable examples available through Speechmatics Academy.

Healthcare teams can also send us a sample of their own audio to benchmark Oak against the provider they use today, or talk to our healthcare team about deployment options and data requirements.

Our results give you a place to start. The more useful test is how Oak performs with your patients, your languages and the recordings your product actually has to handle.