Add curl Api Docs examples

[freddyaboulton]

• I have searched to see if a similar issue already exists.

Is your feature request related to a problem? Please describe.

We should add some curl examples to the ApiDocs like we used to have.

Some users don’t want to use python or JavaScript clients.

Describe the solution you'd like
The problem of course is that curl won’t respect the queue. So what should be do about apps that have queueing enabled?

1. Use a command line tool for querying websockets https://github.com/vi/websocat
2. Don’t show curl docs for routes that are placed on the queue when queuing is enabled.
3. Show all routes and its on the app developer to set api_open=False to prevent the queue being skipped.

Additional context
Add any other context

[abidlabs]

I strongly agree, we need to be able to support cURL -- not only for those users who want to use cURL but because cURL often serves as a template that people adapt into whatever language they are working with (Swift, Android Java, etc.).

Because queuing is enabled by default in Spaces, and most interesting API routes are going to have queuing enabled, I would like to propose another option that would work with queueing:

• We expose another endpoint /api/submit/, which can be used to submit a job over regular HTTP POST request. This endpoint adds a job to the queue and returns a job ID (random hash)
• For jobs submitted through /api/submit/, the Gradio app holds onto the results for a given period of time (say 15 minutes), and then allows the results to be retrieved through a regular HTTP GET request at /api/result/ with the job ID

[freddyaboulton]

That's a neat idea! I'm hesitant to hold onto the results in the server because that would break with replication (very viral demos) and is not REST-ful. Another crazy option is that we can refactor the queue to work with server-sent events and not websockets. Curl natively supports server-sent events. The good thing (outside of this issue) is that every request (regardless of queueing) is restful HTTPS but we would break two-way communication between client and server. However, all the communication we send from the client can be sent as soon as it connects. It should be possible. Anyway, just mentioning this since this kind of change could be possible with 4.0.

For posterity, this is what websocat would look like for calculator with queuing enabled.

❯ websocat --text ws://127.0.0.1:7860/queue/join
{"msg":"send_hash"}
{"fn_index": 0, "session_hash": "12312"}
{"msg":"estimation","rank":0,"queue_size":1,"avg_event_process_time":0.0,"avg_event_concurrent_process_time":null,"rank_eta":null,"queue_eta":1.0}
{"msg":"send_data"}
{"data": ["0", "add", "5"], "fn_index": 0}
{"msg":"process_starts"}
{"msg":"process_completed","output":{"data":[5.0],"is_generating":false,"duration":0.00023818016052246094,"average_duration":0.00023818016052246094},"success":true}

[pngwn]

Server sent events are a worse solution imo. They have more overhead and more limitations. They don't support binary data (we don't currently need this but removing the option when our system supports it seems odd to me), number of open connections is limited (depending on protocol, tends to be more aggressive on mobile), they result in larger payloads in both directions (post and sse stream) and we actually sent a lot of updates, it would be difficult to figure out when to remove people from the queue. And all two way communication would need to be sent out of band and remapped by the server potentially increasing complexity. SSE isn't super simple to set up either you have to post an event and then setup some kind of subscription to an event source in whatever way that works for your language. You also need to do more work to map requests back to responses.

I don't think this usecase is common enough for us to warrant rearchitecting, considering those limitations. I just think we should document the WebSocket contract and encourage people to create clients.

I'm not against a 'submit' route but it would have limitations.

[julien-c]

(FWIW we use SSE for all other HF projects 😅 )

[pngwn]

I don't think a submit route and a result route violate REST though. Submit is a request to start a job and result/id is a request for a resource. I would consider the result data that could be stored in a db and fetched by the get result route. Whether that is an in memory db or something is just an implementation detail.

What I don't understand about /submit is how do they know when the job is complete?

[pngwn]

FWIW we use SSE for all other HF projects 😅

Are there any projects that implement a queue or handle binary data and use SSE? I'd be curious to know how those problems had been solved elsewhere.

[abidlabs]

What I don't understand about /submit is how do they know when the job is complete?

Yeah this is the main issue. Potentially /submit could also provide an ETA if one is available. But the user would have to keep pinging /result until they have a result

For posterity, this is what websocat would look like for calculator with queuing enabled.

The websocat solution could work, but it requires installing a new command line tool (and more importantly imo), its much harder for developers to translate it to other languages that they are working in. E.g. maybe would be good to get @pcuenca's thoughts on the relative difficulty of websocket requests vs. post requests from Swift.

We've discussed SSE before but it would require a massive refactor (plus might not support everything as @pngwn pointed out) so I think it should be done only if none of the other paths are viable :)

[pcuenca]

There's native support for WebSockets inside Apple's networking framework, although I don't have direct experience with it.

I think that cURL requests are more readable and can serve as documentation on their own, which is very valuable. However, if the cURL requests do not really represent the way things work (as shown by having to break things up into two new endpoints), then we may be adding more complexity and maintainability burden.

If documentation is our main goal, I'd maybe use cURL requests for illustration purposes, stating that they are not compatible with the queue system, and direct users to the websockets spec to create apps in other languages. We can potentially keep an uncurated collection of community clients somewhere.

I don't know the specifics or constraints surrounding this task, though.

[296651310]

我强烈同意，我们需要能够支持 cURL——不仅是为了那些想要使用 cURL 的用户，而且因为 cURL 通常充当人们适应他们正在使用的任何语言（Swift、Android Java 等）的模板。 ）。

由于 Spaces 中默认启用排队，并且大多数有趣的 API 路由都将启用排队，因此我想提出另一个与排队一起使用的选项：

• 我们公开另一个端点/api/submit/，它可用于通过常规 HTTP POST 请求提交作业。该端点将作业添加到队列并返回作业ID（随机哈希）
• 对于通过 提交的作业/api/submit/，Gradio 应用程序会在给定的时间段（例如 15 分钟）内保留结果，然后允许使用/api/result/作业 ID通过常规 HTTP GET 请求检索结果

How do I use /api/submit and /api/result? I couldn't find any examples in the documentation. Can you provide an example to help me?

[mathematicalmichael]

Hi! I'm strongly in favor of this, and as I understand it, the switch to SSE with v4 should make this a lot easier. I'd be happy to contribute to the docs if I can actually figure out the usage patterns for the backend API.

@abidlabs any chance these exist somewhere that I'm not seeing so far?

[abidlabs]

Hi @mathematicalmichael we actually started implementing this before we realized that the curl syntax is not well-suited for sending json in SSE. The reason for this is SSE basically requires you to use a GET request to send the first message to establish the SSE, and so you need to encode the payload through GET's query parameters. This technically doable, but the syntax is just quite great with JSON, e.g.

curl "http://localhost:7860/queue/join?fn_index=0&session_hash=1&data=%5B%22hello%22%5D"

to send "hello" to a simple "hello world" gradio example:

import gradio as gr

gr.Interface(lambda x:x, "textbox", "textbox").launch()

In short, we won't add curl examples to all the Gradio demos since the syntax is quite ugly, particularly for more complex examples, but it is doable if you want to rig it up yourself for a particular demo.

[yosun]

How do you queue join from curl API when you are sending a json payload?

{
"detail": "Not Found"
}

[yosun]

hey i'm checking back since... gradio api on HF still doesn't work for me - hasn't since november or so.

[ryanshrott]

@yosun @freddyaboulton Any solution to this? I want to run a curl command. I get:
Failed to retrieve data: 404 {"detail":"Not Found"}

[freddyaboulton]

Hi @ryanshrott , we have not prepared docs for Curl usage yet.

[shivmsingh]

Any fix for this?

{
"detail": "This API endpoint does not accept direct HTTP POST requests. Please join the queue to use this API."
}

[freddyaboulton]

@ryanshrott @shivmsingh I wrote a short curl how-to guide that should hold you over until we get the official docs in place

https://www.freddyboulton.com/blog/gradio-curl

[shivmsingh]

@freddyaboulton Thanks alot, really appreciate it.

[ryanshrott]

@freddyaboulton When will the official docs be in place? I'd prefer just to do a simple post request if possible? Will the official docs be simpler to use?

[ryanshrott]

@abidlabs Do you know the timeline for adding back CURL examples to the gradio API docs?

[abidlabs]

@ryanshrott in the next month or so