Chainlit bonus material fixes

[d-kleine]

• moved to device
  • idx and model were on different devices when using CUDA
• fixed path
• fixed readme command
• added .files folder from chainlit to .gitignore
  • .files directory is used by Chainlit to store temporary files during runtime, which should not be included in version control

[d-kleine]

Please double-check if this works fine for you too 🙂

I think it would be good to add a welcome message for the user to see that the chatbot is working before typing, currently there is none. What do you also think about adding with torch.no_grad(): to the inference code?

[rasbt]

Looks good to me, thanks for the PR! There was only a minor thing where the device movement can be simplified (i.e., the cloning and detaching shouldn't be necessary)

[rasbt]

I think it would be good to add a welcome message for the user to see that the chatbot is working before typing

Thanks for suggesting, but I would just stay with simplicity here.

with torch.no_grad():

This is a good suggestion, but it's already handled in the def generate() function we are calling there.

[d-kleine]

This is a good suggestion, but it's already handled in the def generate() function we are calling there.

Okay, I see. I think it would be helpful to add this info as a code comment as this not directly obvious from the code, and it would be on par with your written code that uses with torch.no_grad(): when generating the output text

[rasbt]

Re: The warning you are getting without cloning and detaching

Oh interesting, I am not getting the warning. Maybe it's an older PyTorch version or a Windows thing?

[d-kleine]

Oh interesting, I am not getting the warning. Maybe it's an older PyTorch version or a Windows thing?

I ran the code both on Linux as well as Windows. I think this has been the fix suggested by the error message, so that's why I added it to the code. Just tested your changes: without clone().detach() it works for me too, without any warning:

2024-09-18 14:37:33 - Your app is available at http://localhost:8000
2024-09-18 14:37:35 - Translation file for de not found. Using default translation en-US.
2024-09-18 14:37:36 - Translated markdown file for de not found. Defaulting to chainlit.md.
2024-09-18 14:37:44 - Translation file for de not found. Using default translation en-US.

LGTM 🙂