Skip to content

Latest commit

 

History

History
285 lines (282 loc) · 89.9 KB

highest.md

File metadata and controls

285 lines (282 loc) · 89.9 KB

Tatoeba Challenge Data - v2021-08-07

This is the "highest" sub-set of the Tatoeba data. Download the data files from the link in the table below. There is a total of

  • 275 language pairs in this sub-set
lang-pair test dev train
Afrikaans - English afr-eng 1374 1007 11997779
Arabic - German ara-deu 1209 1029 35746632
Arabic - Modern Greek (1453-) ara-ell 426 28374699
Arabic - English ara-eng 10305 19529 126590524
Arabic - French ara-fra 1569 36 80204592
Arabic - Hebrew ara-heb 1208 91 26812456
Arabic - Italian ara-ita 235 11 36352000
Arabic - Japanese ara-jpn 1336 6 10789186
Arabic - Polish ara-pol 207 36355468
Arabic - Russian ara-rus 3715 1209 60875366
Arabic - Spanish ara-spa 1511 1031 96933291
Arabic - Turkish ara-tur 1263 52 37609476
Bengali - English ben-eng 2500 2648 17730303
Bulgarian - Bulgarian bul-bul 219 14416303
Bulgarian - German bul-deu 314 46013912
Bulgarian - English bul-eng 10000 7800 109171589
Bulgarian - French bul-fra 446 44520076
Bulgarian - Italian bul-ita 2500 4006 44334610
Bulgarian - Russian bul-rus 1247 1000 49589532
Bulgarian - Spanish bul-spa 286 52378053
Bulgarian - Turkish bul-tur 834 1 36521635
Bulgarian - Ukrainian bul-ukr 1020 10889657
Bulgarian - Chinese bul-zho 416 16197275
Catalan - English cat-eng 1631 228 29496611
Catalan - French cat-fra 700 2 25575384
Catalan - Italian cat-ita 298 12842701
Catalan - Portuguese cat-por 747 3 14773751
Catalan - Spanish cat-spa 1534 1294 155036058
Czech - German ces-deu 3490 1127 73708754
Czech - English ces-eng 13824 16189 171935414
Czech - French ces-fra 438 5 82332401
Czech - Hungarian ces-hun 1911 114 57434476
Czech - Italian ces-ita 1099 1 54876556
Czech - Polish ces-pol 568 3 82803948
Czech - Russian ces-rus 2934 5466 49955853
Czech - Slovenian ces-slv 489 17 36545653
Czech - Spanish ces-spa 1807 134 92655594
Danish - German dan-deu 9998 6921 66857588
Danish - English dan-eng 10795 20089 132853159
Danish - Finnish dan-fin 2665 1743 38934546
Danish - French dan-fra 1731 45 63194534
Danish - Italian dan-ita 284 4 50887929
Danish - Dutch dan-nld 1643 36 56709329
Danish - Norwegian dan-nor 1311 1017 39038808
Danish - Portuguese dan-por 873 4 64218680
Danish - Russian dan-rus 1713 1020 19286598
Danish - Spanish dan-spa 5000 5136 66865377
Danish - Swedish dan-swe 1549 1440 56372864
Danish - Turkish dan-tur 758 4 20743995
German - Modern Greek (1453-) deu-ell 2500 2801 62266768
German - English deu-eng 17565 289779 582899212
German - Estonian deu-est 244 1 34366655
German - Persian deu-fas 3185 1025 13516810
German - Finnish deu-fin 2647 7142 55548870
German - French deu-fra 12418 98158 242670995
German - Serbo-Croatian deu-hbs 1959 9 63556080
German - Hebrew deu-heb 3090 1125 24149594
German - Hungarian deu-hun 15342 54083 57470609
German - Italian deu-ita 10094 12198 149228324
German - Japanese deu-jpn 11428 29868 22071648
German - Korean deu-kor 1104 9 14391527
German - Lithuanian deu-lit 1115 355 29078892
German - Malay (macrolanguage) deu-msa 544 2 34554754
German - Dutch deu-nld 10218 26387 136798789
German - Norwegian deu-nor 3651 1001 46122783
German - Polish deu-pol 5000 5701 93422130
German - Portuguese deu-por 10000 7045 138805696
German - Romanian deu-ron 1141 1013 54396800
German - Russian deu-rus 12800 100323 70387523
German - Slovenian deu-slv 492 7 40879856
German - Spanish deu-spa 10521 75061 212444622
German - Swedish deu-swe 3410 1126 83271324
German - Turkish deu-tur 5000 7277 44783818
German - Ukrainian deu-ukr 10319 11721 11376817
German - Vietnamese deu-vie 400 19580930
German - Chinese deu-zho 3944 6838 27292037
Modern Greek (1453-) - Modern Greek (1453-) ell-ell 533 45907853
Modern Greek (1453-) - English ell-eng 10899 12920 147932024
Modern Greek (1453-) - French ell-fra 1506 10 93398426
Modern Greek (1453-) - Italian ell-ita 424 58470238
Modern Greek (1453-) - Dutch ell-nld 922 67 58147304
Modern Greek (1453-) - Portuguese ell-por 885 23 103939019
Modern Greek (1453-) - Russian ell-rus 2500 2532 33094232
Modern Greek (1453-) - Spanish ell-spa 1829 1005 102869700
Modern Greek (1453-) - Swedish ell-swe 253 38640662
Modern Greek (1453-) - Turkish ell-tur 1469 109 43847517
English - English eng-eng 12062 96608 540486918
English - Esperanto eng-epo 17675 241675 17630279
English - Estonian eng-est 1359 1096 59607889
English - Basque eng-eus 1060 1001 12196006
English - Persian eng-fas 3762 1031 40786833
English - Finnish eng-fin 10690 69896 103531791
English - French eng-fra 12681 251749 747876036
English - Galician eng-glg 1015 3 14723137
English - Serbo-Croatian eng-hbs 10017 14206 159611854
English - Hebrew eng-heb 10519 153571 62390511
English - Hindi eng-hin 5000 5944 26203459
English - Hungarian eng-hun 13037 97144 113408329
English - Icelandic eng-isl 2503 6938 18935522
English - Italian eng-ita 17320 472974 332003631
English - Japanese eng-jpn 13862 194065 60849617
English - Korean eng-kor 2400 1041 34482796
English - Latvian eng-lav 1631 3 41742167
English - Lithuanian eng-lit 2528 5643 49895503
English - Macedonian eng-mkd 10010 70319 17720716
English - Malay (macrolanguage) eng-msa 5000 6893 115696959
English - Dutch eng-nld 12696 62387 272141200
English - Norwegian eng-nor 5000 5707 128202505
English - Polish eng-pol 10099 44189 187862997
English - Portuguese eng-por 13222 204462 400271909
English - Romanian eng-ron 5508 9661 137409800
English - Russian eng-rus 19425 504174 217612504
English - Slovenian eng-slv 2495 1610 67362056
English - Spanish eng-spa 16583 197532 952526014
English - Albanian eng-sqi 1109 2 29479902
English - Swedish eng-swe 10362 15558 162176758
English - Thai eng-tha 1154 1 15527865
English - Turkish eng-tur 13907 658320 108218954
English - Ukrainian eng-ukr 13127 159487 33956793
English - Vietnamese eng-vie 2500 3135 66952989
English - Chinese eng-zho 10390 43075 129323178
Estonian - Russian est-rus 613 19 18651327
Basque - Spanish eus-spa 1850 1003 13722778
Persian - French fas-fra 376 5 15013068
Finnish - French fin-fra 1920 1056 73683315
Finnish - Hebrew fin-heb 212 4 24661220
Finnish - Hungarian fin-hun 1297 8 40976084
Finnish - Italian fin-ita 1039 1003 45912067
Finnish - Dutch fin-nld 317 4 50151979
Finnish - Norwegian fin-nor 2488 1148 23982457
Finnish - Polish fin-pol 609 45755362
Finnish - Portuguese fin-por 477 2 72256117
Finnish - Russian fin-rus 3643 1009 26020504
Finnish - Spanish fin-spa 2513 7399 76584113
Finnish - Swedish fin-swe 2841 7284 53558867
Finnish - Turkish fin-tur 1796 3 30273762
Finnish - Chinese fin-zho 382 2 14045578
French - French fra-fra 1000 2118 31022622
French - Serbo-Croatian fra-hbs 474 85147249
French - Hebrew fra-heb 3281 1064 45796218
French - Hungarian fra-hun 2494 4848 83244777
French - Italian fra-ita 10091 68868 231333370
French - Japanese fra-jpn 10169 28594 22898818
French - Korean fra-kor 321 2 15182595
French - Malay (macrolanguage) fra-msa 1003 3 41191970
French - Dutch fra-nld 11548 16735 120823312
French - Norwegian fra-nor 477 42348379
French - Polish fra-pol 3087 1005 101164681
French - Portuguese fra-por 10518 17062 236220480
French - Romanian fra-ron 1925 7 101594168
French - Russian fra-rus 11490 183050 108801349
French - Slovenian fra-slv 448 42945228
French - Spanish fra-spa 10294 40595 432661466
French - Swedish fra-swe 1407 1127 77591133
French - Turkish fra-tur 2582 6718 58533150
French - Ukrainian fra-ukr 10035 18252 11611752
French - Vietnamese fra-vie 1299 8 23484571
French - Chinese fra-zho 9994 7558 57926170
Galician - Portuguese glg-por 433 3 10326178
Galician - Spanish glg-spa 2121 1012 52632730
Serbo-Croatian - Italian hbs-ita 534 61656215
Serbo-Croatian - Japanese hbs-jpn 400 10932369
Serbo-Croatian - Norwegian hbs-nor 5000 6149 27599589
Serbo-Croatian - Polish hbs-pol 417 86028120
Serbo-Croatian - Russian hbs-rus 2500 4177 59206165
Serbo-Croatian - Spanish hbs-spa 607 2 94876911
Serbo-Croatian - Ukrainian hbs-ukr 942 7 11236257
Serbo-Croatian - Chinese hbs-zho 236 30682886
Hebrew - Hebrew heb-heb 1000 732 27106602
Hebrew - Hungarian heb-hun 417 11 31509647
Hebrew - Italian heb-ita 1706 30 21327922
Hebrew - Japanese heb-jpn 241 4 11189194
Hebrew - Dutch heb-nld 2500 4357 32771321
Hebrew - Polish heb-pol 5000 7757 35274514
Hebrew - Portuguese heb-por 719 24 61463570
Hebrew - Russian heb-rus 2500 3568 26109586
Hebrew - Spanish heb-spa 1849 1077 50478131
Hebrew - Turkish heb-tur 1377 30 34308349
Hebrew - Chinese heb-zho 709 5 15832590
Hungarian - Hungarian hun-hun 1064 20053026
Hungarian - Italian hun-ita 5000 7159 53544788
Hungarian - Dutch hun-nld 1629 35 52573654
Hungarian - Polish hun-pol 1934 15 59572847
Hungarian - Portuguese hun-por 2500 3517 88148731
Hungarian - Russian hun-rus 2687 6098 34839281
Hungarian - Spanish hun-spa 2500 4194 88624314
Hungarian - Swedish hun-swe 1614 525 36082251
Hungarian - Turkish hun-tur 1002 10 41149669
Hungarian - Chinese hun-zho 434 1 16371663
Italian - Italian ita-ita 1000 350 34550078
Italian - Japanese ita-jpn 2660 1005 16282820
Italian - Lithuanian ita-lit 224 21783427
Italian - Malay (macrolanguage) ita-msa 426 24858158
Italian - Dutch ita-nld 2578 6940 89569959
Italian - Norwegian ita-nor 939 11 32063415
Italian - Polish ita-pol 1294 1002 63922556
Italian - Portuguese ita-por 3066 6371 160983108
Italian - Romanian ita-ron 1005 8 67043699
Italian - Russian ita-rus 10045 66133 45793091
Italian - Spanish ita-spa 5000 9247 234944470
Italian - Swedish ita-swe 715 16 55658265
Italian - Turkish ita-tur 10000 5703 41846022
Italian - Vietnamese ita-vie 250 15543964
Italian - Chinese ita-zho 2943 1004 20214003
Japanese - Korean jpn-kor 622 3 11991800
Japanese - Malay (macrolanguage) jpn-msa 2616 1060 15638976
Japanese - Dutch jpn-nld 3430 1057 18652849
Japanese - Polish jpn-pol 9999 14809 17723193
Japanese - Portuguese jpn-por 1940 1133 18955445
Japanese - Russian jpn-rus 10142 15675 17761727
Japanese - Spanish jpn-spa 10742 23420 23630883
Japanese - Swedish jpn-swe 896 5 13928449
Japanese - Turkish jpn-tur 811 4 18727012
Japanese - Chinese jpn-zho 2497 2894 14128610
Korean - Russian kor-rus 221 11518705
Korean - Spanish kor-spa 940 1 15861123
Latvian - Russian lav-rus 274 12923057
Lithuanian - Polish lit-pol 1787 23786757
Lithuanian - Russian lit-rus 3598 5230 18115912
Lithuanian - Spanish lit-spa 454 1 37797104
Lithuanian - Turkish lit-tur 1472 11214365
Malay (macrolanguage) - Malay (macrolanguage) msa-msa 958 16 26590210
Malay (macrolanguage) - Spanish msa-spa 232 56886826
Malay (macrolanguage) - Chinese msa-zho 372 8 16343358
Dutch - Dutch nld-nld 1000 1031 35210723
Dutch - Norwegian nld-nor 203 1 35533677
Dutch - Polish nld-pol 1193 1006 80521905
Dutch - Portuguese nld-por 2500 4882 116758221
Dutch - Romanian nld-ron 2269 1048 53291318
Dutch - Russian nld-rus 2550 6526 48300917
Dutch - Spanish nld-spa 10113 18068 124489314
Dutch - Turkish nld-tur 2500 3880 51207660
Dutch - Chinese nld-zho 1654 56 19621838
Norwegian - Polish nor-pol 281 22912084
Norwegian - Portuguese nor-por 481 3 40056683
Norwegian - Russian nor-rus 1279 1 13388553
Norwegian - Spanish nor-spa 960 6 48733384
Norwegian - Swedish nor-swe 566 4 43150787
Polish - Portuguese pol-por 705 121613981
Polish - Russian pol-rus 3543 1015 61733651
Polish - Spanish pol-spa 2544 5075 109213032
Polish - Swedish pol-swe 1392 11 58775791
Polish - Turkish pol-tur 892 54918111
Polish - Ukrainian pol-ukr 2519 6898 10070070
Polish - Chinese pol-zho 1004 2 19688440
Portuguese - Portuguese por-por 2500 4668 295005671
Portuguese - Romanian por-ron 681 31 100215551
Portuguese - Russian por-rus 10000 10050 73050883
Portuguese - Spanish por-spa 10947 56853 333782720
Portuguese - Swedish por-swe 320 13 81841206
Portuguese - Turkish por-tur 1794 9 91166415
Portuguese - Chinese por-zho 1033 3 37752011
Romanian - Russian ron-rus 782 7 22601817
Romanian - Spanish ron-spa 1959 4 110068108
Romanian - Turkish ron-tur 2460 1010 43254953
Russian - Slovenian rus-slv 657 45 25884431
Russian - Spanish rus-spa 10506 86903 117477558
Russian - Swedish rus-swe 1282 52 32070553
Russian - Turkish rus-tur 5000 6257 36690600
Russian - Ukrainian rus-ukr 10000 7427 33899654
Russian - Vietnamese rus-vie 313 1 15233193
Russian - Chinese rus-zho 2500 4844 46782394
Slovenian - Chinese slv-zho 825 11534402
Spanish - Spanish spa-spa 2500 2566 88064723
Spanish - Swedish spa-swe 1351 204 79447560
Spanish - Turkish spa-tur 10615 18213 67971005
Spanish - Ukrainian spa-ukr 10115 12969 12819049
Spanish - Vietnamese spa-vie 594 8 41113753
Spanish - Chinese spa-zho 2705 6922 63850602
Swedish - Turkish swe-tur 203 6 35260202
Swedish - Chinese swe-zho 1049 2 15036092
Turkish - Turkish tur-tur 2500 1187 56734692
Turkish - Chinese tur-zho 986 9 20798951
Vietnamese - Chinese vie-zho 440 4 11160560
Chinese - Chinese zho-zho 3114 5810 17413844