OpenAI API timeout limit

[jamieaa64]

We have a question for Chatgpt 4 and set the max tokens to 4096 where we want it to generate YAML for Drupal. It works through chat.openai.com but we have to piece together the yaml from about 3 or 4 responses. We're trying to do the same thing via the API and it works with 1000 token limit (taking about 3 minutes) but when we increase it any further it will timeout after 10minutes.

Is 10 minutes a hard limit for the OpenAI API? Or is it something we can set to be longer for big questions?

If it is, is this something we can do via the OpenAI php API? Looking through the official python library it appears that there is a setting called TIMEOUT_SECS which is set to 600 by default. We can't see anything referencing the word "timeout" in the openai php library.

[jamieaa64]

If I go to

vendor/openai-php/client/src/OpenAI.php:29

I can write this code:

$client = new GuzzleClient(["timeout"=>13]);

This works at changing the timeout to 13 seconds. If I put in 660 into it, it will still timeout after 10 minutes. This suggests to me that there is a hard limit of 10 minutes for chatgpt4 and the Open AI API

[jamieaa64]

Ok it looks like this is resolved by streaming the api response rather then doing it as one thing.