Languages of DialectBench

On this page
Language cluster Dialect name Glottocode mBERT seen UDP POS NER SDQA RCMC NLI TC SC DI MT
albanian albanian alba1267 ✓ ✓ ✓ - - - - - - - -
gheg albanian gheg1238 - ✓ ✓ - - - - - - - -
anglic african american vernacular english afri1276 - ✓ - - - - - - - - -
australian english aust1314 - - - - ✓ - - - - - -
english stan1293 ✓ ✓ ✓ ✓ - ✓ ✓ ✓ - ✓ -
english (a:scotland) stan1293 - - - - ✓ - - - - - -
english (a:uk) stan1293 - - - - - - - - - ✓ -
english (o:controlled) stan1293 - - - ✓ - - - - - - -
indian english (a:north) indi1255 - - - - ✓ - - - - - -
indian english (a:south) indi1255 - - - - ✓ - - - - - -
irish english iris1255 - - - - ✓ - - - - - -
jamaican creole english jama1262 - - - ✓ - - - - - - -
kenyan english keny1281 - - - - ✓ - - - - - -
new zealand english newz1240 - - - - ✓ - - - - - -
nigerian english nige1260 - - - - ✓ - - - - - -
north american english nort3314 - - - - - - - - - ✓ -
old english (ca. 450-1100) olde1238 - - - ✓ - - - - - - -
philippine english phil1246 - - - - ✓ - - - - - -
singlish sing1272 - ✓ ✓ - - - - - - - -
southeast american english sout3300 - - - - ✓ - - - - - -
southern african english sout3331 - - - - ✓ - - - - - -
arabic aleppo alep1241 - - - - - - - - - ✓ ✓
algerian arabic alge1239 - - - - ✓ - - - ✓ ✓ ✓
arabian peninsula arabic (a:yemen) arab1393 - - - - - - - - - ✓ ✓
arabic (a:bahrain) - - - - - ✓ - - - - - -
arabic (a:jordan) - - - - - ✓ - - - ✓ - -
arabic (a:saudi-arabia) - - - - - ✓ - - - ✓ - -
egyptian arabic egyp1253 - - ✓ ✓ ✓ ✓ ✓ ✓ ✓ - -
egyptian arabic (a:alx) egyp1253 - - - - - - - - - ✓ ✓
egyptian arabic (a:asw) egyp1253 - - - - - - - - - ✓ ✓
egyptian arabic (a:cai) egyp1253 - - - - - - - - - ✓ ✓
egyptian arabic (a:kha) egyp1253 - - - - - - - - - ✓ ✓
fez. meknes fezm1238 - - - - - - - - - ✓ ✓
gilit mesopotamian arabic meso1252 - - - - - - - - - ✓ ✓
gulf arabic gulf1241 - - ✓ - - - - - - - -
gulf arabic (a:doh) gulf1241 - - - - - - - - - ✓ ✓
gulf arabic (a:jed) gulf1241 - - - - - - - - - ✓ ✓
gulf arabic (a:mus) gulf1241 - - - - - - - - - ✓ ✓
gulf arabic (a:riy) gulf1241 - - - - - - - - - ✓ ✓
levantine arabic nort3139 - - ✓ - - - - - - - -
levantine arabic (a:north) nort3139 - - - - - ✓ ✓ ✓ - - -
levantine arabic (a:north-dam) nort3139 - - - - - - - - - ✓ ✓
libyan arabic liby1240 - - - - - - - - - - -
libyan arabic (a:ben) liby1240 - - - - - - - - - ✓ ✓
moroccan arabic moro1292 - - - - ✓ ✓ ✓ ✓ ✓ - -
najdi arabic najd1235 - - - - - ✓ ✓ ✓ - - -
north african arabic nort3191 - ✓ ✓ - - - - - - - -
north mesopotamian arabic nort3142 - - - - - ✓ ✓ ✓ - - -
north mesopotamian arabic (a:bas) nort3142 - - - - - - - - - ✓ ✓
north mesopotamian arabic (a:mos) nort3142 - - - - - - - - - ✓ ✓
rabat-casablanca arabic raba1252 - - - - - - - - - ✓ ✓
sfax sfax1238 - - - - - - - - - ✓ ✓
south levantine arabic sout3123 - ✓ ✓ - - - ✓ ✓ ✓ - -
south levantine arabic (a:south-amm) sout3123 - - - - - - - - - ✓ ✓
south levantine arabic (a:south-jer) sout3123 - - - - - - - - - ✓ ✓
south levantine arabic (a:south-sal) sout3123 - - - - - - - - - ✓ ✓
standard arabic stan1318 ✓ ✓ ✓ ✓ - ✓ ✓ ✓ ✓ ✓ -
sunni beiruti arabic sunn1238 - - - - - - - - - ✓ ✓
ta'izzi-adeni arabic taiz1242 - - - - - - ✓ ✓ - - -
tripolitanian arabic trip1239 - - - - - - - - - ✓ ✓
tunisian arabic tuni1259 - - - - ✓ - ✓ ✓ ✓ - -
tunisian arabic (a:tun) tuni1259 - - - - - - - - - ✓ ✓
tunisian arabic (r:casual) tuni1259 - - - - - - - - ✓ - -
basque basque (a:amenduze) basq1248 ✓ - - - - - - - - - ✓
basque (a:azkaine) basq1248 - - - - - - - - - - ✓
basque (a:baigorri) basq1248 - - - - - - - - - - ✓
basque (a:barkoxe) basq1248 - - - - - - - - - - ✓
basque (a:donibane) basq1248 - - - - - - - - - - ✓
basque (a:garruze) basq1248 - - - - - - - - - - ✓
basque (a:iholdi) basq1248 - - - - - - - - - - ✓
basque (a:jatsu) basq1248 - - - - - - - - - - ✓
basque (a:jutsi) basq1248 - - - - - - - - - - ✓
basque (a:larzabale) basq1248 - - - - - - - - - - ✓
basque (a:luhuso) basq1248 - - - - - - - - - - ✓
basque (a:sara) basq1248 - - - - - - - - - - ✓
basque (a:senpere) basq1248 - - - - - - - - - - ✓
basque (a:suhuskune) basq1248 - - - - - - - - - - ✓
basque (a:uharte) basq1248 - - - - - - - - - - ✓
navarro-labourdin basque (a:behorlegi) basq1249 - - - - - - - - - - ✓
navarro-labourdin basque (a:bidarrai) basq1249 - - - - - - - - - - ✓
navarro-labourdin basque (a:helete) basq1249 - - - - - - - - - - ✓
navarro-labourdin basque (a:mugerre) basq1249 - - - - - - - - - - ✓
navarro-labourdin basque (a:urruna) basq1249 - - - - - - - - - - ✓
souletin (a:maule) basq1250 - - - - - - - - - - ✓
bengali vanga (a:barisal) vang1242 - - - - - - - - - - ✓
vanga (a:dhaka) vang1242 ✓ - - - ✓ - - - - - ✓
vanga (a:jessore) vang1242 - - - - - - - - - - ✓
vanga (a:khulna) vang1242 - - - - - - - - - - ✓
vanga (a:kushtia) vang1242 - - - - - - - - - - ✓
vanga (a:west bengal) vang1242 ✓ - - - ✓ - - - - - -
circassian adyghe adyg1241 - - - ✓ - - - - - - -
kabardian kaba1278 - - - ✓ - - - - - - -
common turkic bashkir bash1264 ✓ - - - - - - - - - ✓
central oghuz azer1255 - - - - - - - - - - ✓
central oghuz (m:spoken) azer1255 - - - - - - ✓ ✓ - - -
crimean tatar crim1257 - - - ✓ - - - - - - -
kara-kalpak kara1467 - - - - - - - - - - ✓
kazakh kaza1248 ✓ - - - - - - - - - ✓
kirghiz kirg1245 ✓ - - - - - - - - - ✓
north azerbaijani nort2697 ✓ - - ✓ - - ✓ ✓ - - -
sakha yaku1245 - - - - - - - - - - ✓
south azerbaijani sout2697 ✓ - - ✓ - - ✓ ✓ - - -
turkish nucl1301 ✓ - - ✓ - - - - - - ✓
uzbek uzbe1247 ✓ - - - - - - - - - ✓
eastern romance aromanian arom1237 - - - ✓ - - - - - - -
moldavian mold1248 - - - ✓ - - - - - - -
romanian roma1327 ✓ - - ✓ - - - - - - -
eastern-western armenian eastern armenian nucl1235 ✓ ✓ ✓ - - - - - - - -
western armenian homs1234 - ✓ ✓ - - - - - - - -
farsic dari dari1249 - - - - - - - - - - ✓
frisian ems-weser frisian sate1242 - - - ✓ - - - - - - -
northern frisian nort2626 - - - ✓ - - - - - - -
western frisian west2354 ✓ - - ✓ - - - - - - -
gallo-italian emiliano-romagnolo emil1243 - - - ✓ - - - - - - -
ligurian ligu1248 - ✓ ✓ ✓ - - ✓ ✓ - - -
lombard lomb1257 ✓ - - ✓ - - ✓ ✓ - - -
piemontese piem1238 ✓ - - ✓ - - - - - - -
venetian vene1258 - - - ✓ - - ✓ ✓ - - -
gallo-rhaetian anglo-norman angl1258 - - - ✓ - - - - - - -
arpitan fran1260 - - - ✓ - - - - - - -
french stan1290 ✓ ✓ ✓ ✓ - - - - - - -
french (a:paris) stan1290 - ✓ ✓ - - - - - - - -
friulian friu1240 - - - ✓ - - ✓ ✓ - - -
old french (842-ca. 1400) oldf1239 - ✓ ✓ - - - - - - - -
romansh roma1326 - - - ✓ - - - - - - -
walloon wall1255 - - - ✓ - - - - - - -
greater panjabic eastern panjabi panj1256 ✓ - - ✓ - - - - - - -
western panjabi west2386 ✓ - - ✓ - - - - - - -
greek apulian greek apul1237 - - - - - - - - - - ✓
cretan cret1244 - - - - - - - - - - ✓
cypriot greek cypr1249 - - - - - - - - - - -
cypriot greek (r:casual, m:written, i:fb) cypr1249 - - - - - - - - - ✓ -
cypriot greek (r:casual, m:written, i:other) cypr1249 - - - - - - - - - ✓ -
cypriot greek (r:casual, m:written, i:twitter) cypr1249 - - - - - - - - - ✓ -
modern greek mode1248 ✓ - - ✓ - - - - - - -
modern greek (r:casual, m:written, i:fb) mode1248 - - - - - - - - - ✓ -
modern greek (r:casual, m:written, i:other) mode1248 - - - - - - - - - ✓ -
modern greek (r:casual, m:written, i:twitter) mode1248 - - - - - - - - - ✓ -
pontic pont1253 - - - ✓ - - - - - - -
high german bavarian bava1246 ✓ - - ✓ - - - - - - -
central alemannic swis1247 - - - ✓ - - - - - - -
central alemannic (a:ag) swis1247 - - - - - - - - - - ✓
central alemannic (a:ai) swis1247 - - - - - - - - - - ✓
central alemannic (a:ar) swis1247 - - - - - - - - - - ✓
central alemannic (a:be) swis1247 - - - - - - - - - ✓ ✓
central alemannic (a:bl) swis1247 - - - - - - - - - - ✓
central alemannic (a:bs) swis1247 - - - - - - - - - ✓ ✓
central alemannic (a:fr) swis1247 - - - - - - - - - - ✓
central alemannic (a:gl) swis1247 - - - - - - - - - - ✓
central alemannic (a:gr) swis1247 - - - - - - - - - - ✓
central alemannic (a:lu) swis1247 - - - - - - - - - ✓ ✓
central alemannic (a:nw) swis1247 - - - - - - - - - - ✓
central alemannic (a:ow) swis1247 - - - - - - - - - - ✓
central alemannic (a:sg) swis1247 - - - - - - - - - - ✓
central alemannic (a:sh) swis1247 - - - - - - - - - - ✓
central alemannic (a:so) swis1247 - - - - - - - - - - ✓
central alemannic (a:sz) swis1247 - - - - - - - - - - ✓
central alemannic (a:tg) swis1247 - - - - - - - - - - ✓
central alemannic (a:ur) swis1247 - - - - - - - - - - ✓
central alemannic (a:vs) swis1247 - - - - - - - - - - ✓
central alemannic (a:zg) swis1247 - - - - - - - - - - ✓
central alemannic (a:zh) swis1247 - ✓ ✓ - - - - - - ✓ ✓
central bavarian cent1967 - - - - - - - - - - ✓
german stan1295 ✓ ✓ ✓ ✓ - - - - - - -
kölsch kols1241 - - - ✓ - - - - - - -
limburgan limb1263 - - - ✓ - - ✓ ✓ - - -
luxemburgish luxe1243 ✓ - - ✓ - - ✓ ✓ - - -
pennsylvania german penn1240 - - - ✓ - - - - - - -
pfaelzisch-lothringisch pala1330 - - - ✓ - - - - - - -
upper saxon uppe1465 - - - - - - - - - - ✓
hindustani fiji hindi fiji1242 - - - ✓ - - - - - - -
hindi hind1269 ✓ - - ✓ - - - - - - -
inuit alaskan inupiaq inup1234 - - - ✓ - - - - - - -
kalaallisut kala1399 - - - ✓ - - - - - - -
italian romance continental southern italian neap1235 - ✓ ✓ ✓ - - - - - - -
italian ital1282 ✓ ✓ ✓ ✓ - - ✓ ✓ - - -
italian (a:abruzzo) ital1282 - - - - - - - - - - ✓
italian (a:basilicata) ital1282 - - - - - - - - - - ✓
italian (a:calabria) ital1282 - - - - - - - - - - ✓
italian (a:campania) ital1282 - - - - - - - - - - ✓
italian (a:emilia-romagna) ital1282 - - - - - - - - - - ✓
italian (a:friuli-venezia giulia) ital1282 - - - - - - - - - - ✓
italian (a:lazio) ital1282 - - - - - - - - - - ✓
italian (a:liguria) ital1282 - - - - - - - - - - ✓
italian (a:lombardia) ital1282 - - - - - - - - - - ✓
italian (a:marche) ital1282 - - - - - - - - - - ✓
italian (a:molise) ital1282 - - - - - - - - - - ✓
italian (a:piemonte) ital1282 - - - - - - - - - - ✓
italian (a:puglia) ital1282 - - - - - - - - - - ✓
italian (a:sardegna) ital1282 - - - - - - - - - - ✓
italian (a:sicilia) ital1282 - - - - - - - - - - ✓
italian (a:toscana) ital1282 - - - - - - - - - - ✓
italian (a:trentino-alto adige/südtirol) ital1282 - - - - - - - - - - ✓
italian (a:umbria) ital1282 - - - - - - - - - - ✓
italian (a:unknown) ital1282 - - - - - - - - - - ✓
italian (a:veneto) ital1282 - - - - - - - - - - ✓
italian (r:casual, m:written, i:tweet) ital1282 ✓ ✓ ✓ - - - - - - - -
italian (r:formal, m:written, i:essay) ital1282 ✓ ✓ ✓ - - - - - - - -
italian romance (a:apulia, m:spoken, i:tarantino) - - - - ✓ - - - - - - -
sicilian sici1248 ✓ - - ✓ - - ✓ ✓ - - -
komi komi komi1267 - - - ✓ - - - - - - -
komi-permyak komi1269 - ✓ ✓ ✓ - - - - - - -
komi-zyrian (m:spoken) komi1268 - ✓ ✓ - - - - - - - -
komi-zyrian (m:written) komi1268 - ✓ ✓ - - - - - - - -
korean korean (a:south-eastern, m:spoken) kore1280 - - - - ✓ - - - - - -
seoul (m:spoken) seou1239 ✓ - - - ✓ - - - - - -
kurdish central kurdish cent1972 - - - ✓ - - ✓ ✓ - - -
kurdish kurd1259 - - - ✓ - - - - - - -
northern kurdish nort2641 - - - - - - ✓ ✓ - - -
sine'i sine1239 - - - - - - - - - - ✓
sorani sora1257 - - - - - - - - - - ✓
latvian east latvian east2282 - - - ✓ - - ✓ ✓ - - -
latvian latv1249 ✓ - - ✓ - - ✓ ✓ - - -
mari eastern mari east2328 - - - ✓ - - - - - - -
western mari west2392 - - - ✓ - - - - - - -
modern dutch dutch dutc1256 ✓ - - ✓ - - ✓ ✓ - - -
western flemish vlaa1240 - - - ✓ - - - - - - -
zeeuws zeeu1238 - - - ✓ - - - - - - -
neva central and north pohjanmaa (a:ostrobothnian) cent1985 - - ✓ - - - - - - - -
estonian esto1258 ✓ - ✓ - - - - - - - -
finnish finn1318 ✓ - ✓ - - - - - - - -
häme (a:tavastian) hame1240 - - ✓ - - - - - - - -
neva (a:south-west trans) - - - ✓ - - - - - - - -
savo (a:savonian) savo1254 - - ✓ - - - - - - - -
southeastern finnish (a:south-east) sout1743 - - ✓ - - - - - - - -
southwestern finnish (a:south-west) sout2677 - - ✓ - - - - - - - -
norwegian norwegian norw1258 ✓ - - - - - - - - - -
norwegian (a:eastern) norw1258 - - - - - - - - - - ✓
norwegian (a:setesdal) norw1258 - - - - - - - - - - ✓
norwegian (a:southwestern) norw1258 - - - - - - - - - - ✓
norwegian (m:written, i:samnorsk) - - - - ✓ - - - - - - -
norwegian bokmål (m:written) norw1259 ✓ ✓ ✓ ✓ - - ✓ ✓ - - -
norwegian nynorsk (m:written) norw1262 ✓ ✓ ✓ ✓ - - ✓ ✓ - - -
norwegian nynorsk (m:written, i:old) norw1262 - ✓ ✓ - - - - - - - -
saami north saami nort2671 - ✓ ✓ - - - - - - - -
skolt saami skol1241 - ✓ ✓ - - - - - - - -
sabellic umbrian umbr1253 - ✓ ✓ - - - - - - - -
sardo-corsican corsican cors1241 - - - ✓ - - - - - - -
sardinian sard1257 - - - ✓ - - ✓ ✓ - - -
serbian-croatian-bosnian bosnian standard bosn1245 ✓ - - ✓ - - - - - - -
croatian standard croa1245 ✓ - - ✓ - - - - - - -
serbian standard serb1264 ✓ - - ✓ - - - - - - -
serbian-croatian-bosnian sout1528 ✓ - - ✓ - - - - - - -
sinitic cantonese cant1236 - - - ✓ - - ✓ ✓ - - -
classical chinese lite1248 - ✓ ✓ ✓ - - - - - - -
classical-middle-modern sinitic (a:hongkong, o:traditional) clas1255 - ✓ ✓ - - - - - - - -
classical-middle-modern sinitic (o:simplified) clas1255 ✓ ✓ ✓ - - ✓ ✓ ✓ - - -
classical-middle-modern sinitic (o:traditional) clas1255 ✓ - - - - ✓ ✓ ✓ - - -
hakka chinese hakk1236 - - - ✓ - - - - - - -
mandarin chinese mand1415 - - - ✓ - - - - - - -
mandarin chinese (a:mainland, o:simplified) mand1415 - - - - - - - - - ✓ -
mandarin chinese (a:mainland, o:traditional, i:synthetic) mand1415 - - - - - - - - - ✓ -
mandarin chinese (a:taiwan, o:simplified) mand1415 - - - - - - - - - ✓ -
mandarin chinese (a:taiwan, o:traditional, i:synthetic) mand1415 - - - - - - - - - ✓ -
min nan chinese minn1241 - - - ✓ - - - - - - -
wu chinese wuch1236 - - - ✓ - - - - - - -
sorbian lower sorbian lowe1385 - - - ✓ - - - - - - -
upper sorbian uppe1395 - - - ✓ - - - - - - -
sotho-tswana (s.30) northern sotho nort3233 - - - ✓ - ✓ ✓ ✓ - - -
southern sotho sout2807 - - - ✓ - ✓ ✓ ✓ - - -
southwestern shifted romance brazilian portuguese braz1246 - ✓ ✓ - - - - - - ✓ -
extremaduran extr1243 - - - ✓ - - - - - - -
galician gali1258 ✓ - - ✓ - - ✓ ✓ - - -
latin american spanish amer1254 - - - - - - - - - ✓ -
mirandese mira1251 - - - ✓ - - - - - - -
occitan occi1239 ✓ - ✓ ✓ - - ✓ ✓ - - ✓
portuguese (a:european) port1283 - ✓ ✓ - - - ✓ ✓ - ✓ -
portuguese (i:mix) port1283 - ✓ ✓ - - - - - - - -
portuguese (m:written) port1283 ✓ ✓ ✓ - - - - - - ✓ -
spanish stan1288 ✓ - - ✓ - - ✓ ✓ - ✓ -
spanish (a:europe) stan1288 - - - - - - - - - ✓ -
swahili swahili swah1253 ✓ - - - - - - - - - -
swahili (a:kenya) swah1253 - - - - ✓ - - - - - -
swahili (a:tanzania) swah1253 - - - - ✓ - - - - - -
tigrinya tigrinya (a:eritrea) tigr1271 - - - - - - - - - - ✓
tigrinya (a:ethiopia) tigr1271 - - - - - - - - - - ✓
tupi-guarani subgroup i.a mbyá guaraní (a:brazil) mbya1239 - ✓ ✓ - - - - - - - -
mbyá guaraní (a:paraguay) mbya1239 - ✓ ✓ - - - - - - - -
old guarani oldp1258 - ✓ ✓ - - - - - - - -
west low german west low german west2357 ✓ ✓ ✓ ✓ - - - - - - -
yoruba yoruba (a:central nigeria) yoru1245 ✓ - - - - - - - - - ✓