it is too slow to chat with the chatbot

[jensenw-pki]

I deployed the DocsGPT in AWS environment. But it is too slow to chat with it, it needs several minutes to response an answer for each question.
what is the potential problem? it seems work perfectly in the official website https://docsgpt.arc53.com/

[dartpain]

How did you deploy it?
Are you using sagemaker?
Streaming responses shoud be just as fast as anywhere else.

Unless you are using llama.cpp

[jensenw-pki]

I just create an EC2 instance.

[dartpain]

ok, so you used the ec2 guide here: https://docs.docsgpt.co.uk/Deploying/Hosting-the-app
right?

And for inference you are using OpenAI?