Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about acos task on the Chinese dataset #311

Open
twotwoiscute opened this issue Apr 27, 2023 · 3 comments
Open

Question about acos task on the Chinese dataset #311

twotwoiscute opened this issue Apr 27, 2023 · 3 comments

Comments

@twotwoiscute
Copy link

twotwoiscute commented Apr 27, 2023

Hi,Thanks for the great work, I have a question on the task of acos
I've downloaded the dataset from here and there's a task called acos which I am interested in.

Observation

I try a quick train on all the dataset in acos_datasets and found out the fact that the model does not do well on dataset 505.Chinese_Zhang, the fine-tuned model always output the empty string even the value of both training ands val loss decrease during process.

So I check the paper and realize the dataset paper used is shown below:

Dataset: SemEval 2014,15 and 16 datasets are
used for our experimentation. The dataset is used
as a benchmark for ABSA tasks and has customer
reviews from three domains; laptops (Lapt14), hotels (Hotel15), and restaurants (Rest14, Rest15, and
Rest16)
which migh bring out the fact that the pretrained model it used in [training script] would not be so good at Chinese. (

model_checkpoint = "kevinscaria/ate_tk-instruct-base-def-pos-neg-neut-combined"
)

What I have done

I am new to NLP, I believe there's always the step that deal with tokenizer like the codeAutoTokenizer.from_pretrained before training and here's how I init the model, I downloaded it here directly.

self.tokenizer = AutoTokenizer.from_pretrained(checkpoint)
self.model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)

Question

Is it okay to apply the original pipeline that the model is trained on english I/O directly to the model that takes chinese input I/O? if not what should I do before get into training. Thanks!

@twotwoiscute
Copy link
Author

I also try train dataset 505.Chinese_Zhang only, the training process looks to go me.
However when it comes to predict, the model still output empty string as described above.

@yangheng95
Copy link
Owner

Sorry for my late reply as I am on a vacation. Saddly, the ACOS task is only tested for English, while any other language is enabled, I will release a new version. BTW, Chinese is supported by ATEPC task.

@twotwoiscute
Copy link
Author

Thanks for the reply , from your reply I can tell that you’ve known all it takes to get the things done. Can you tell me the pipeline of how to achieve it? I would like to try it on my own.Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants