Gcoap can not send separate responses

[chrysn]

Description

Gcoap does not have any facilities to send separate responses (ie. to send an empty ACK to an incoming CON, and then send the response in a CON). This is needed for:

• Testing gcoap: Process CON responses #14178 -- otherwise, Gcoap needs to react to behavior it can not produce by itself.
• Implementing a proxy (gcoap: add simple forward-proxy #13790) -- or any resource really that takes time to render (but barely anything in embedded other than a network retransmission takes time in the order of magnitude of retransmission).

As this is complex machinery, it should be noted that anything that comes of this will be purely opt-in for the handler. No machine code should change due to this for servers that produce their responses fast. (Passing the remote to the handlers would in changed code, but that'll be needed for other applications as well anyway.)

Towards resolution

Once #14169 is resolved (probably with #14178), much is already there, as the code in gcoap_req_init can be used as a template for response. (In particular, all the retransmission mechanism of requests works for responses just as well.)

There's generally two ways to go about this, and I'd prefer to pursue the latter (or even none if this is all too special a case ... but proxies will need this):

• Allow the handler to send the ACK from the handler callback before blocking in there. This locks up the complete Gcoap thread, and while this particular client won't retransmit (provided it got the ACK), other hosts' requests (and retransmissions) would queue up unconfirmed.

  The traditional web server mitigation would be to have multiple worker threads draining the event queue, but I don't think we want to go there (and even then it'd need some synchronization between them to ACK what a different worker is busy on).

• Allow the handler to return with a plain ACK (or even no response yet) after extracting something like a "server memo" and starting whatever needs to be done. Callbacks from there would eventually send out a response.

  This is what the proxy PR (well, a branch on it) is doing, and what my little demo does as well.

  The big blockers her are:

  • Currently, a server handler has no way whatsoever to determine which remote a request came from. This should already be an issue when using DTLS for client authentication. I don't see how (without hacks like passing data in via globals) this can be done without changing coap_handler_t. (gcoap: add a handler for request matching #14029 might open up a way to have different kind of CoAP handlers, but maybe a well-planned signature transition is easier than coping with the long-term effects of widening up the possibilities).

    It's not so much an issue for the proxy as it's using gcoap: add a handler for request matching #14029 to get its own unique handler signature anyway.

  • Creation of a response message. Can almost be done manually by the user with some hdr->ver_t_tkl fiddling, the only part that's missing is access to the global MID counter. A gcoap_res_init that does that right and takes the token from the memo would be in order.

  • Helpers for the memo (so not every implementer needs to do the shared parts again). The memo at very least needs to store the sock_udp_ep_t remote and the token. It should also contain the message ID (to identify whether a request is a duplicate and thus can be ACKed again if the first ACK got lost). The memo could optionally contain an xtimer to send the empty ACK only on demand (which also needs the MID, and sets a flag in the memo to indicate the response can't be ACK any more). Possibly passing in such a memoy should be the only way to call gcoap_res_init.

  Allocation of the memos would be tied to the application, as the memo would also hold all intermediate state of the handler (eg. a client memo in the proxy case, a lock on a bus, or what so not). Failure to allocate would be communicated right from the handler by returning 5.03, possibly accompanied by a Max-Age that gives an estimate of when to retry.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.