Adding support for Thai Language

[korakot]

I listened to the recent PyTorch Dev Conference. Yuhao Zhang said that a new release with 83 languages is coming. Do these new 30 languages include Thai as well?

There is this PUD-Thai treebank (1000 sentences). But there are some mistakes in annotations. If you use it to train, I want to help correct it first.

Also, I know of a new Thai treebank, containing 2.4k sentences and 140k words to be released in a few months. I have access to it, but it's not finished yet. Maybe I can help prepare some parts and use it to train the Thai model.

Please suggest if we could cooperate.

[qipeng]

@korakot I think the best way is to get in touch with the Universal Dependencies community first, to start building that treebank and correcting Thai-PUD. If you could give us a work-in-progress version of the larger treebank, we can try to train some models to see if the performance is acceptable. :)

[AngledLuffa]

@korakot, did you ever make more progress on getting the Thai treebank? I was looking to add more models to stanza, and one of the things I found was that there is a Thai sentiment dataset available. It would only make sense to provide that in the context of a larger set of Thai models, though, so now is a good time to revisit building models for Thai.

[korakot]

There are some progress. A new constituency treebank came out, so need conversion. The Thai PUD needs update, no progress yet. TNC treebank still has only head info, but no relation-type label, I need to check the progress.

Can you provide a timeline and what type of Thai corpora do you need. I'm probably the one who know most about these free Thai corpora.

[AngledLuffa]

In terms of seeing if the Thai sentiment dataset is working for us, getting data for a segmentation model soon would be great. I've read that there was a corpus called Orchid which included both segmentation and POS, but the links I can find for that are out of date. There's also a corpus for InterBEST, which should be available at thailang.nectec.or.th, but I get a 404 when I try to register an account there.

In terms of models beyond that, at a very minimum I think we would need a POS model as well. stanza doesn't currently have a constituency parser, so that dataset might not be helpful now, but we could always try to use it in corenlp or possibly one day include a constituency parser with stanza. Dependencies would be great, of course, since we generally have that model in other language. For all of these, there is no rush - when it arrives, we'll evaluate our Thai models and see if they can be added.

Thanks for the help!

[korakot]

For BEST, probably the same as InterBest, (they rename it a few times). Here's my list of direct links to them.

https://gist.github.com/korakot/abf6c18c71cefe7b9107689dd904751f

For orchid, you can get it here.

https://www.nectec.or.th/corpuso/phocadownload/dl_text_thai-eng/orchid_corpus.zip

All these (and other) Thai segmentation datasets use different criteria. I plan to normalize them to the same criteria. So, you could use them as is now. And I will update you with them and other segmentation datasets, once I finish normalization.

POS tagsets are also the same. We have 2-3 tagsets which I plan to normalize them all to Chula-version of PUD.

[AngledLuffa]

Thank you, this is extremely helpful. If I understand correctly, there is segmentation and POS available in Orchid, and segmentation and NER available in Best. However, you say the segmentation standards are different. Different enough that you would recommend not mixing them? In that case, we wouldn't be able to provide a unified segmentation-POS-NER chain unless we cheat on it somehow and train the NER on Orchid segmentation, for example. I could also just train a model on both datasets at the same time and see what happens Also, I notice a serious lack of test data. Orchid doesn't seem to have any, and the Best data doesn't have gold segmentation or labeled NER that I can see. Is there a standardized way to handle that, or should we just split it up however we think best?

[korakot]

There are many problems about the current state of Thai datasets.

I can confirm that

• Orchid has SEG and POS
• Best has SEG and NER
• Their SEGs are different, but no research on how much different.
• Orchid is old (1997) and has no standardized test set
• Best has an old guideline for SEG and NER, but it's in Thai. There is a new updated guideline, but it's not publicly released.
• You can probably split it however you like.

The root cause is probably that we have few NLP researchers and activities. I can help solve these problems one at a time, whichever is a priority.

[AngledLuffa]

Sounds interesting to me. I'll talk with my PI tomorrow and see what we can do. At a very minimum we can train two different segmenters and see what differences they have. One limitation is, as far as I know, no one in our reseach group speaks Thai

…

On Sat, May 30, 2020, 8:24 PM Korakot Chaovavanich ***@***.***> wrote: There are many problems about the current state of Thai datasets. I can confirm that - Orchid has SEG and POS - Best has SEG and NER - Their SEGs are different, but no research on how much different. - Orchid is old (1997) and has no standardized test set - Best has an old guideline <https://www.nectec.or.th/corpuso/phocadownload/dl_text_thai-eng/BEST/BEST_Guideline_Release1.pdf> for SEG and NER, but it's in Thai. There is a new updated guideline, but it's not publicly released. - You can probably split it however you like. The root cause is probably lack of NLP researchers and activities. I can help you solve these problems one at a time, whichever is a priority. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#148 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2AYWLW4KMBF332XKZC4ATRUHEXXANCNFSM4JBX6FXQ> .

[korakot]

Today I start to check the quality of BEST segmentation. I found a few errors in even the first files.

This week I will compare BEST with ORCHID, and probably TNC and Wisesight. I hope to get a clearer idea of their differences. And I will choose one (probably TNC) and convert the rest. So we can have a consistent dataset to train word segmentation.

[AngledLuffa]

Did Wisesight have a seg standard? I saw the sentiment, but it looks unsegmented.

…

On Sun, May 31, 2020, 11:33 AM Korakot Chaovavanich < ***@***.***> wrote: Today I start to check the quality of BEST segmentation. I found a few errors in even the first files. This week I will compare BEST with ORCHID, and probably TNC and Wisesight. I hope to get a clearer idea of their differences. And I will choose one (probably TNC) and convert the rest. So we can have a consistent dataset to train word segmentation. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#148 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2AYWOKPRR2XT6E3AKKHXTRUKPHHANCNFSM4JBX6FXQ> .

[korakot]

We have a portion of it segmented too. You can get it here.

https://github.com/PyThaiNLP/wisesight-sentiment/tree/master/word-tokenization

[AngledLuffa]

I spoke with my PI, and he says that if you're up for collaborating on these datasets, we're interested. Perhaps one place to start would be to test segmentation models built on the different seg datasets

[korakot]

I am happy to collaborate on these datasets.

I guess you can evaluate the 3 segmentation datasets (Orchid, Best, Wisesight)
on a downstream task (sentiment) and compare them.
I will help clean up each of them. And I may convert them to the same standard.
I am thinking of TNC treebank which is in my original post. If we use TNC, we will
have less work when we need to train dependency parsing.

Please tell me if you have any suggestion on how we should proceed.

[korakot]

For Orchid, the original format is a bit hard to work with.
Someone (K. Vee) has converted it to XML, so that it's easier to parse out words and sentences. His webhost had gone down, but I did make a copy here.

https://github.com/korakot/thainlp/blob/master/xmlchid.xml