120 languages, and what makes each one hard
Coverage is easy to claim and hard to deliver. Every language below has a production path we have documented, and a note on the specific difficulty that decides whether a collection project in that language succeeds.
Highest demand
Languages where we have measured the strongest buyer search demand. 15 languages
-
Hindi
Hindi and Urdu are mutually intelligible at the spoken level, but one is written in Devanagari and the other in Perso-Arabic script. Which script a recording is transcribed into directly determines whether the data can feed a model targeting Urdu.
-
Arabic (MSA)
Almost nobody speaks Modern Standard Arabic as a native, everyday language. The practical result of recording standard Arabic is that every speaker carries their own dialect accent: unless the speaker's dialect background and the tolerated deviation are defined up front, annotation consistency collapses.
-
Indonesian
In everyday speech, Indonesians mix in local languages (Javanese, Sundanese) and English loanwords heavily. Three language components inside a single sentence is normal, so word segmentation and annotation rules have to be fixed before collection begins.
-
Thai
Thai has five tones, and Thai script carries no word spacing. A sentence is one long run of continuous characters. There is no single right answer for where word boundaries go, which lets different annotators produce mutually incompatible corpora.
-
Turkish
Turkish is an agglutinative language: one root can carry a long chain of suffixes, so the word list explodes. Istanbul and eastern accents also differ noticeably, if speaker origin is not recorded clearly, model generalization suffers.
-
Vietnamese
Vietnamese has six tones, marked in writing by a heavy load of diacritics. The three major dialect regions (north, central, and south), realize tones quite differently, and mixing them in one batch gives the acoustic model contradictory tone cues.
-
Filipino
Filipinos speak Taglish in daily life: switching between Tagalog and English sentence by sentence, sometimes word by word. This is how native speakers naturally talk, not a sign of substandard speech. Anyone who wants authentic data has to treat this mixing as the target, not as noise.
-
Persian
Written Persian and spoken Tehrani Persian are far apart: verb endings, pronouns, and connected speech all shift in the colloquial form. Transcribing spoken recordings against the written form produces text that does not match the audio.
-
Tamil
Written and spoken Tamil are two separate systems, and colloquial speech drops or merges a large share of sounds. Accents also differ across Tamil Nadu in India, Sri Lanka, and Malaysia, so speaker region has to be annotated on every record.
-
Telugu
Telugu has four major dialect regions (Coastal Andhra, Rayalaseema, Telangana, and the south) that differ noticeably, and the Telangana dialect carries a large share of Urdu loanwords. Without region labels, you cannot tell dialect variation from annotation error.
-
Malayalam
Malayalam is spoken fast, with dense connected speech, and public speech corpora are far smaller than for Indian languages of comparable size. The pool of qualified annotators is small, so a single project easily reuses the same people: speaker deduplication is mandatory.
-
Nepali
Usable Nepali speech data is surprisingly scarce, and the accent layers of speakers across borders (Nepal, Sikkim in India, Bhutan) are poorly mapped. Speaker metadata (nationality, home region, language of education), has to be recorded item by item.
-
Sinhala
Everyday Sinhala conversation is heavily embedded with English words, especially in the Colombo urban belt. Treating this mixed speech as pure Sinhala strips out high-frequency loanwords as if they were errors.
-
Bangla
The Bangla of Bangladesh and of West Bengal in India has diverged in vocabulary and accent. The everyday word for the same meaning is often different. In search terms and annotation alike, bangla tracks real usage better than bengali.
-
Khmer
Khmer script has no spaces between words, and it contains many letters that are written but not pronounced. The transcription convention has to be set first: write as spelled (orthographic) or write as spoken (phonemic). Data produced under the two conventions cannot be mixed.
Established demand
Languages with consistent but narrower buyer demand, including several major dialect variants. 13 languages
-
Arabic (Egyptian)
Thanks to a large film and television industry, Egyptian Arabic is the most widely understood dialect in the Arab world, but speech within Egypt still runs on two sets of realizations. Upper Egypt and Lower Egypt. This matters especially for customer-service speech.
-
Arabic (Gulf)
Gulf Arabic differs noticeably across Saudi Arabia, the UAE, and Kuwait, yet buyers usually search for saudi directly. Recording and annotating by country from the start is far less work than splitting a mixed batch afterward.
-
Arabic (Moroccan)
Moroccan Darija mixes in heavy French and Berber vocabulary, and the spoken form has almost no unified written standard. The same word can come out with three different spellings from three annotators. The transcription convention has to be set project by project.
-
Kannada
Kannada has more than a dozen regional dialects, but only a few have a written tradition. That rules out transcribing dialect speech against a standard script, so the specific dialects to be collected have to be fixed first.
-
Kazakh
Kazakh is switching from Cyrillic to Latin script, and both writing systems circulate at the same time. Existing corpora are inconsistent in script, so a normalization pass is required before any training use.
-
Uzbek
Uzbek is officially written in Latin script, but a large share of existing corpora is in Cyrillic. The conversion rules between the two leave a few letters without a clean one-to-one mapping, so conversion introduces errors.
-
Punjabi
Punjabi is the classic case of one spoken language with two scripts: Gurmukhi on the Indian side and Shahmukhi (a Perso-Arabic system) on the Pakistani side. The same passage written on either side cannot be cross-searched against the other.
-
Burmese
Burmese script has no word spacing, and there is an unresolved Unicode encoding dispute. The same visual glyph has multiple equivalent encodings in the system. Without normalization before delivery, downstream parsing fails outright.
-
Amharic
Amharic uses the Ge'ez (Ethiopic) script, in which each consonant has seven vowel-variant glyphs. Which variant is written and how the sound is actually pronounced in speech often disagree, so the annotation guideline has to state which side governs.
-
Ukrainian
Everyday Ukrainian speech is a mix of Ukrainian and Russian (Surzhyk), in proportions that vary from speaker to speaker. On top of that, many people searching this term are language learners. A dataset page has to target the dataset intent only and not drift toward learning demand.
-
Yoruba
Yoruba has three tones, and they change in connected speech. Diacritics mark tone in writing, but everyday text frequently drops them: text that does not match the actual pronunciation is the norm in existing corpora.
-
Swahili
Swahili has few native speakers; the large majority of its users speak it as a second language, with accents shaped heavily by their first languages. Standard Tanzanian Swahili and the Kenyan coastal dialects also differ, so speaker background has to be documented.
-
Kurdish
The two main branches of Kurdish (Sorani and Kurmanji) have low mutual intelligibility, close to two separate languages, and they use different scripts. Public resources are so thin that even TTS datasets are rare. A textbook scarce language.
Narrower demand
Languages we can source where buyer demand is thin but real. 3 languages
-
Odia
Odia is one of India's official languages, but public speech corpora are far smaller than for Hindi or Bangla at the same official tier. Qualified annotators are concentrated in Odisha, so the speaker pool is limited by geography.
-
Hausa
Hausa is written in two scripts at once: Latin (Boko) dominates, while Arabic script (Ajami) is used in religious contexts. Running both in parallel means the script-to-speech mapping has to be defined separately for each.
-
Pashto
Pashto splits into two major dialect groups (western/Kandahar and eastern/Peshawar) with substantially different vowel systems. It is also spoken across borders, so each speaker's region and dialect affiliation must be recorded in the metadata.
Full coverage
The rest of our documented coverage. Lower search demand, and often exactly where a requirement lands. 89 languages
-
Spanish (Spain)
Peninsular Spanish divides between ceceo and seseo pronunciation habits, and northern and southern accents differ noticeably. Mixing them in one batch gives the acoustic model contradictory sibilant cues.
-
Spanish (Latin America)
Latin American Spanish is not one accent: the differences among Mexican, Colombian, Argentine, and Chilean speech run deeper than the internal dialect splits of many countries. Deliver a batch that mixes all four and the buyer will most likely send it back.
-
French
French spelling and pronunciation are far apart. A large share of word-final letters are silent, and liaison changes word boundaries. How to segment words once connected speech has run them together needs an explicit convention.
-
German
The German dialect continuum runs from Low German to Bavarian with enormous range, and standard German (Hochdeutsch) is mostly learned rather than native. If the buyer wants standard German only, speakers have to be screened during collection.
-
Portuguese (Brazil)
Brazilian Portuguese varies widely internally (São Paulo, Rio, the Northeast, the South), and the nasal vowels of speech are realized in ways far more complex than the written descriptions suggest.
-
Portuguese (Portugal)
European Portuguese reduces unstressed vowels so heavily that many syllables are nearly swallowed. Annotating it with Brazilian Portuguese habits produces systematic omissions.
-
Italian
Italy's regional languages differ sharply from standard Italian, and some (Neapolitan, Sicilian) are treated by linguists as separate languages. When a buyer says Italian, the first question is which one they mean.
-
Russian
Russian stress placement is unpredictable and changes vowel quality, yet writing never marks it. The same string of letters is pronounced differently depending on where the stress falls, so annotation has to be done by ear.
-
Dutch
Dutch accents differ clearly between the Netherlands and Belgium (Flemish), and the dialect layers inside the Netherlands are complex. The target market has to be pinned down at purchase time.
-
Polish
Polish is full of consonant clusters and nasal vowels, and diacritics are what separate the letters. A dropped diacritic in transcription turns the word into a different one.
-
Greek
Greek carries the legacy of diglossia: written and spoken forms were separated for a long time, and even after Modern Standard Greek was settled, some vocabulary still runs on two tracks. Corpora have to be unified onto a single convention.
-
Hebrew
Hebrew writing usually omits vowels (niqqud), while speech has them all. Whether transcription should supply the vowels has no default answer, so it must be fixed at the start of the project.
-
Czech
Czech distinguishes short and long vowels, and that distinction carries meaning, but everyday writing often drops the diacritics. Miss one mark in annotation and it is a different word.
-
Romanian
Romanian goes by different names and different written norms in Romania and Moldova (Moldova used Cyrillic for a long period). The script version of a corpus has to be stated explicitly.
-
Hungarian
Hungarian belongs to the Uralic family and has no kinship with the Indo-European languages around it, so a model cannot transfer from its neighbors. Usable corpora are scarce, which means collecting essentially from scratch.
-
Danish
Danish spelling-to-sound correspondence is highly irregular, and the stød (a glottal catch) is a meaning-bearing feature. Combined with the small corpus base, it is the hardest of the Nordic languages to work with.
-
Finnish
Spoken and written Finnish are almost two separate systems: speech abbreviates and swallows sounds heavily, and the written form barely appears in real conversation. If you want conversational data, written Finnish cannot be the transcription baseline.
-
Norwegian
Norwegian has two official written standards running in parallel (Bokmål and Nynorsk), and the spoken dialects are extremely fragmented. Which written standard pairs with which dialect has to be agreed project by project.
-
Swedish
Swedish tonal accents (grave and acute) distinguish meaning, and writing shows nothing of them. Public speech corpora are also limited in scale, which pushes annotation cost up.
-
Catalan
Catalan splits into eastern and western dialect groups; Central Catalan (Barcelona) and Valencian differ noticeably, and the linguistic status of Valencian is itself disputed. Region labels have to be precise.
-
Mandarin Chinese
Mandarin has four tones plus the neutral tone, its tone-sandhi rules in connected speech are complex (especially strings of third tone), and Han characters carry no sound. Transcription has to choose between characters and pinyin, and the two serve entirely different purposes.
-
Cantonese
Cantonese has six to nine tones, and many colloquial words have no standard character at all: annotators end up improvising characters or falling back on romanization. The lack of a unified written standard is the biggest trap in this kind of corpus.
-
Japanese
Japanese mixes three scripts, and there is no hard rule for whether a given word is written in kanji or kana. When the choice of script in transcription is inconsistent, both search and training suffer.
-
Korean
Korean has a full set of sound-change rules (liaison, tensing, nasalization), so actual pronunciation differs substantially from letter-by-letter spelling. Annotation has to choose spelling-based or pronunciation-based, and the two cannot be mixed.
-
Malay
Malaysian and Indonesian Malay are very close, but their word choices have diverged, and local speech mixes in English, Chinese, and Tamil loanwords. Drawing the boundary between language components within one batch is genuinely hard.
-
Javanese
Javanese has explicit register levels (ngoko, krama, and others): the same meaning takes different words depending on who is being addressed. The register has to be annotated, or the data is unusable for dialogue systems.
-
Sundanese
Sundanese has its own register system, and the traditional Sundanese script has fallen almost entirely out of use in daily life, with Latin letters dominant. Both script and register have to be recorded as metadata.
-
Cebuano
Cebuano has many native speakers, but an even larger number of Filipinos speak it as a second language with accents shaped by Tagalog. Collection has to separate native speakers from second-language speakers.
-
Lao
Lao has six tones, and Lao script shares an origin with Thai script but looks different, so corpora from the two countries cannot be used interchangeably. Spelling practice in everyday writing is not unified either.
-
Hmong
Hmong has seven or eight tones, marked in the Latin-based romanization by consonant letters at the end of a syllable. Those letters are not pronounced, and they are easily mistaken for actual consonants.
-
Tetum
Tetum is an official language of Timor-Leste, but usable speech corpora are extremely thin, and it carries loanwords from both Portuguese and Indonesian. This is a textbook case of collection from zero.
-
Ilocano
Ilocano has many speakers, but they are scattered across the Philippines and overseas communities, and speech corpora from the home region are scarce. Speaker recruitment takes coordination across regions.
-
Bicolano
Bicolano is not one language but a group, with at least six variants that are not fully mutually intelligible. Which branch to collect has to be decided before the project starts, or the data cannot be merged.
-
Urdu
Urdu and Hindi are mutually intelligible in speech, but their writing systems are entirely different (Perso-Arabic versus Devanagari). Which script a batch of audio is transcribed into decides which side it can serve.
-
Gujarati
Gujarati writing does not distinguish short from long vowels, but in speech the length distinction carries meaning. This mismatch between script and speech means the annotation guideline has to state which side governs.
-
Marathi
Marathi spans a wide dialect range: the standard is based on Pune, but Mumbai speech and the Konkan and Vidarbha dialects differ noticeably. Region labels are not optional.
-
Assamese
Assamese is written in a script close to Bengali, but it is a different language. Public speech corpora are very thin, and in a data pipeline Assamese is easily mis-merged with Bengali.
-
Maithili
Maithili is spoken across borders (Bihar in India, Nepal), was long treated as a dialect of Hindi, and has very little standalone corpus. Annotating speakers' language identity gets confusing fast.
-
Bhojpuri
Bhojpuri has tens of millions of speakers, but a weak written tradition and few standardized corpora, and it is heavily mixed with Hindi in existing data sources. Language-identification cleanup is required.
-
Sindhi
Sindhi is written in Perso-Arabic script in Pakistan and in Devanagari in India. One language, two scripts, and corpora from either side cannot be exchanged directly.
-
Dhivehi
Dhivehi uses Thaana script, written right to left, and it is very small: fewer than half a million native speakers. Public corpora are close to zero, so this is collection entirely from scratch.
-
Kashmiri
Kashmiri is written in Perso-Arabic or Devanagari depending on the region, and given the political sensitivity, handling region and identity information during collection requires particular care.
-
Azerbaijani
Azerbaijani uses Latin script in Azerbaijan and Perso-Arabic script in Iran, so corpora from the two sides are written incompatibly. Speech on the Iranian side is more deeply influenced by Persian.
-
Georgian
Georgian has its own script and a heavy load of consonant clusters, sometimes six or seven consonants in a row. The transcription convention has to deal with how those clusters are segmented.
-
Armenian
Armenian splits into Eastern Armenian (in Armenia itself) and Western Armenian (in diaspora communities), with low mutual intelligibility. Which branch is wanted has to be established at purchase time.
-
Kyrgyz
Kyrgyz uses Cyrillic plus several letters of its own, which standard keyboards do not fully support. Combined with scarce corpora, the text-processing toolchain has to be built in-house.
-
Tajik
Tajik is a form of Persian written in Cyrillic. The Persian corpora it could borrow from are nearly impossible to reuse directly, because the script is different.
-
Turkmen
Turkmen's Latin alphabet includes several special symbols with limited keyboard and font support. Usable speech corpora are very scarce, so collection starts from zero.
-
Mongolian
Mongolian is written in Cyrillic in Mongolia and in the traditional vertical script in Inner Mongolia, China. Two scripts and two accent bases mean the corpora cannot simply be merged.
-
Uyghur
Uyghur uses Perso-Arabic script, written right to left, and it has vowel reduction. Public speech corpora are scarce, and toolchain support is limited.
-
Tatar
Tatar is written in Cyrillic but needs extra letters for certain phonemes, and it carries a high proportion of Russian loanwords. The language components in a corpus have to be distinguished first.
-
Chechen
Chechen belongs to the Nakh language family, has no kinship with the languages around it, and has a large and complex consonant system. Usable corpora are near zero, leaving only from-scratch collection.
-
Levantine Arabic
Levantine Arabic covers Lebanon, Syria, Jordan, and Palestine, with accents that differ from country to country and no unified written standard. The dialect boundary has to be defined up front.
-
Arabic (Iraqi)
Iraqi Arabic shows influence from Turkish, Persian, and Kurdish, and Baghdad speech differs substantially from the southern accents. Speaker region has to be annotated record by record.
-
Arabic (Sudanese)
Sudanese Arabic has relatively low mutual intelligibility within the Arab world, with distinctive sounds and vocabulary. Public corpora are very thin. A scarce dialect.
-
Arabic (Tunisian)
Tunisian Arabic mixes in heavy French vocabulary and is spoken fast, with severe syllable compression. The same passage can come out with transcriptions that differ in length by thirty percent across annotators.
-
Arabic (Yemeni)
Yemeni Arabic preserves some phonetic features of Old Arabic and diverges strongly from other dialects. Usable corpora are scarce, and collection conditions are constrained by the situation on the ground.
-
Tamazight (Berber)
The Berber languages run three writing systems in parallel (Tifinagh, Latin, and Arabic) and branch into many dialects. Fixing the script before the dialect is the standard order for this kind of project.
-
Kabyle
Kabyle is the Berber branch with relatively more corpus, written mainly in Latin script. But in a pipeline it is easily conflated with corpora from other Tamazight branches.
-
Assyrian Neo-Aramaic
Assyrian Neo-Aramaic is written in Syriac script, and because its communities are dispersed across Iraq, Iran, Syria, and diaspora in Europe and North America, the accent layers are complex and speakers are scattered. A very small language.
-
Dari
Dari is the Afghan variety of Persian, differing from Iranian Persian in both vocabulary and accent. Mix the two in one corpus and the model's output tilts toward one side.
-
Igbo
Igbo has many dialects with limited mutual intelligibility, and standard written Igbo is a deliberately assembled compromise. The everyday speech of most native speakers does not match it.
-
Twi (Akan)
Twi is a tonal language, and tones change in connected speech. Tone marking in writing is inconsistent, so a single corpus often mixes texts that mark tone with texts that do not.
-
Zulu
Zulu is tonal, but writing never marks tone, and it also has click consonants. Those clicks are written as letter combinations in Latin script, which are easy to misread.
-
Xhosa
Xhosa has more click types than Zulu, and its writing likewise leaves tone unmarked. Non-native annotators can barely transcribe these sounds accurately.
-
Afrikaans
Afrikaans shares its roots with Dutch but has long since split from it, and its speech shows clear influence from English and African languages. Mixing it with Dutch corpora confuses the model.
-
Somali
Somali has a tonal accent that carries meaning, marked in writing sometimes and not others. Annotation consistency within a single corpus is a real problem.
-
Oromo
Oromo is among the most widely spoken languages in Ethiopia, but it branches into many dialects, its standard written form was settled relatively late, and existing corpora are poorly standardized.
-
Tigrinya
Tigrinya uses the Ge'ez script, and many consonants change in speech while the writing stays fixed. The mapping between script and pronunciation needs dedicated documentation.
-
Wolof
Wolof is the lingua franca of Senegal, everyday speech mixes in French heavily, and the share of non-native users is very high. Native speakers and second-language speakers have to be annotated separately.
-
Fula (Fulfulde)
Fula stretches from West Africa to Central Africa across nearly twenty countries, forming a very long dialect continuum whose local varieties have low mutual intelligibility. The collection region has to be fixed before the project starts.
-
Bambara
Bambara is tonal, and tone marking in writing is inconsistent. It is also the lingua franca of Mali with a large number of non-native speakers, so speaker background needs to be recorded.
-
Kinyarwanda
Kinyarwanda has a large native-speaker population, yet public speech corpora are far smaller than for African languages of comparable size. Available annotation resources are concentrated inside Rwanda.
-
Luganda
Luganda is tonal and writing does not mark tone, and it also contrasts short and long vowels. A distinction shown in writing by doubling the vowel, which is frequently omitted.
-
Shona
Shona branches into many internal dialects (Karanga, Zezuru, and others), and everyday writing follows the orthography loosely. Corpora need a spelling normalization pass first.
-
Chichewa
Chichewa is spoken across Malawi, Zambia, and Mozambique, with different spelling habits in each. Corpora are also scarce, so the conventions have to be defined in-house.
-
Sesotho
Sesotho belongs to the same language group as Setswana and Northern Sotho, and corpora are easily confused across them. Tone is unmarked in writing, and public speech resources are few.
-
Setswana
Setswana is tonal but tone is not written, and it has a complex noun-class system that affects pronunciation. Annotation consistency is a standing problem.
-
Malagasy
Malagasy belongs to the Austronesian family, unrelated to the languages of the African mainland, so a model cannot transfer from nearby languages. Within the island, the highland and coastal dialects differ clearly.
-
Quechua
Quechua is a language family rather than a single language, its branches have low mutual intelligibility, and it has multiple official orthographies in Peru, Bolivia, and Ecuador.
-
Guarani
Everyday speech in Paraguay is a deep blend of Spanish and Guarani (Jopara), and conversation in pure Guarani is actually uncommon. The mixing ratio has to be recorded.
-
Nahuatl
Nahuatl variants have low mutual intelligibility, and orthography is not unified across communities. Mexico officially recognizes around thirty variants.
-
Yucatec Maya
Yucatec Maya has tones (high and low) and a full set of glottalized consonants. Tone is usually left unmarked in writing, so annotation depends on listening.
-
Aymara
Aymara has many long consonants and aspirated versus glottalized contrasts, marked in the orthography with apostrophes and similar symbols that are applied loosely. Usable speech corpora are very few.
-
Haitian Creole
Haitian Creole takes most of its vocabulary from French, but its grammar is entirely different, and the two cannot be inferred from each other. The French-derived spellings have been reworked, which calls for a dedicated orthographic convention.
-
Navajo
Navajo is tonal and has a complex consonant system, and its native-speaker population keeps declining. The speaker pool is very small, so a project easily ends up recording the same people repeatedly.
-
Maori
Maori long vowels carry meaning, marked in writing with a macron that is often dropped. Fluent native speakers are also limited in number, so the speaker pool is small.
-
Samoan
Samoan distinguishes a formal register from an everyday one, with substantially different vocabulary (similar to the register levels of Javanese). The register has to be specified at collection time.
-
Fijian
Standard Fijian is based on the eastern Bau dialect and differs clearly from the western dialects. In writing, the letters b, d, g, and q stand for sounds completely different from what English letters suggest.
Submit a sourcing request
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.