add trust_remote_code in tokenizer to fix baichuan issue

[sywangyi]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[sywangyi]

^C2024-11-05T04:02:07.490749Z INFO text_generation_launcher: Terminating webserver
2024-11-05T04:02:07.490796Z INFO text_generation_launcher: Waiting for webserver to gracefully shutdown
2024-11-05T04:02:07.491197Z INFO text_generation_router::server: router/src/server.rs:2479: signal received, starting graceful shutdown
thread 'tokio-runtime-worker' panicked at router/src/validation.rs:464:14:
Failure in python tokenizer worker: PyErr { type: <class 'ValueError'>, value: ValueError('The repository for baichuan-inc/Baichuan2-7B-Chat contains custom co de which must be executed to correctly load the model. You can inspect the repository content at https://hf.co/baichuan-inc/Baichuan2-7B-Chat.\nPlease pass the argument trust_remote_code=True to allow custom code to be run.'), traceback: Some(<traceback object at 0x7f0a1c3c0400>) }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at router/src/validation.rs:425:55:
called Result::unwrap() on an Err value: SendError { .. }
thread 'tokio-runtime-worker' panicked at /usr/src/router/src/validation.rs:121:48:
called Result::unwrap() on an Err value: RecvError(())

[sywangyi]

client: curl: (52) Empty reply from server

[sywangyi]

@Narsil and @danieldk please help review.

[sywangyi]

It's platform independent issue. you could use baichuan-inc/Baichuan2-7B-Chat to reproduce

[Narsil]

Thanks a lot for this. This was oversight/bad rebase on my part.

Cheers !