add back self.max_position_embeddings = config.max_position_embeddings

[chengchengpei]

What does this PR do?

Fix hiyouga/LLaMA-Factory#5461

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[chengchengpei]

@zucchini-nlp ^^ can you take a look at this change? Thanks ^^

[zucchini-nlp]

@chengchengpei I am not sure why the error is being raised, as the new RoPE doesn't need any position embeddings in its init. Am I missing smth?

[chengchengpei]

@chengchengpei I am not sure why the error is being raised, as the new RoPE doesn't need any position embeddings in its init. Am I missing smth?

@zucchini-nlp thanks for your reply :)

I am new to transformers.

i used llama-factory to fine tune. last week, it works. but yesterday, i saw some errors described in hiyouga/LLaMA-Factory#5461

i tracked the code changes and found self.max_position_embeddings = config.max_position_embeddings was removed currently. i can run the finetune again after i added it back.
if self.max_position_embeddings = config.max_position_embeddings is not required, how to fix the errors i saw in hiyouga/LLaMA-Factory#5461 ? thanks for your help.

[zucchini-nlp]

I see, just for my own understanding wanted to know where exactly the attribute is being accessed. I'll try to reproduce it later :)

Thanks for fixing this! It is in line with Llama and seems to be causing bugs while tuning, so we can have it back

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

Okay, cc @gante to take into account if we refactor

[chengchengpei]

@zucchini-nlp

thanks.

when this PR can be merged?

[zucchini-nlp]

Yes, sorry. I was off for a while, merging :)