Request Event Driven (non-blocking) WiFi socket API

[drmpf]

The request is for a method call which returns true iff a call to WiFiClient.write(buffer) will NOT block.
as in

if (client.canWriteBufferWithoutBlocking()) {
   client.write(buffer);  // this will NOT block
}

The low level API has a callback framework, so this could be implemented by a boolean which is cleared on a call to client.write() and set by the callback that is executed when the buffer has been sent and ACKed.

(Also as another request, can client.flush() be a noop, so incoming data not lost, but kept until read or client stopped.)

Background to this Request ==========
(see #917)

ESP can only handle one outgoing TCP packet at a time. It needs to keep the last packet for retransmission until it is ACKed from the other side. For Windows clients the ACK delay is 0.2 secs.

While ESP is waiting for ACK, it cannot accept another client.write(buffer)
So currently "client.write() blocks until either data is sent and ACKed, or timeout occurs (currently hard-coded to 5 seconds). Also client.flush() doesn't "force" sending data after client.write(), because there is no output buffer which can be flushed. It does discard incoming data though."

If the client is connected then you could expect that after a maximum of 0.2 sec blocking, the next data buffer will be written. However if the connection is lost, (i.e. no return ACK) then the sketch will block for 5 sec.

In many cases both the 0.2 sec and 5 sec blocking is too long.

In my particular case of a UART to WiFi bridge sketch, 0.2 sec at 115200 baud is about 2K of incoming serial data that is being ignored / lost while the sketch is blocked in client.write();

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[igrr]

I think what you need is a non-blocking write function, similar to POSIX send, which would return EWOULDBLOCK if previous packet has not been acked yet.

[drmpf]

That would work, but would not be familiar to novice Arduino users.
I suggest something more like how Serial is used.
Even something as simple as
WiFiClient.writeAvailable() returning 0 when would block would work and return 1460 when packet was acked
Actually this would give a lot more information to the average user.
EDIT:
Serial uses
Serial.availableForWrite()
for this, so I suggest WiFiClient use the same function name but will return only 0 or 1460 depending.

[drmpf]

Here is my rough outline of what I think is needed.
Not a working example and not up to you standard igrr

bool tcpSendBlocked = false;
static void ICACHE_FLASH_ATTR sentCb(void *arg) {
  tcpSendBlocked = false;
}

void setup() {
  // put your setup code here, to run once:
  espconn_regist_sentcb(conn, sentCb);
}

int clientAvailableForWrite() {
  if (tcpSendBlocked) {
    return 0;
  } else {
    return 1460;
  }
}

void clientSendBuffer(byte *buf) {
  tcpSendBlocked = true;
  client.write(buf);
}

void loop() {
  // put your main code here, to run repeatedly:

  if (clientAvailableForWrite()>0) {
    clientSendBuffer(buf); // will not block
  }
......
}

[drmpf]

I checked the Arduino WiFiClient.flush() and it also just consumes any buffered incoming data :-(
So for consistency I suppose we need to leave flush() as it is. But I still find this method's action very un-expected. I think of flush() as something that happens on output not input.

[me-no-dev]

the WiFi library is not using espconn underneath, so the callbacks will not work.

[drmpf]

igrr mentioned LwIP library which has a call back called tcp_sent( ) that seems to do the job.

[me-no-dev]

sure, I just read the code you posted above and saw espconn_regist_sentcb(conn, sentCb);

[drmpf]

I stole that from elsewhere, but

        err_t   _sent(tcp_pcb* pcb, uint16_t len) {
            DEBUGV(":sent %d\r\n", len);
            _size_sent -= len;
            if(_size_sent == 0 && _send_waiting) esp_schedule();
            return ERR_OK;
        }

in ClientContext.h almost seems to do the job.
Except
i) _send_waiting is private
ii) _sent() does not seem to set _send_waiting back to false?
I assume I am missing something here.
But looks very close, if we added an accessor method for _send_waiting

[igrr]

You seem to be missing the point that no user code gets executed until _sent() callback is called.

Sketch execution is suspended in write method (link), and resumed in _sent method which you have quoted.

So adding an accessor for _send_waiting is not sufficient. write method has to be made non-blocking. This is a trivial change, but it would break compatibility with WiFi Shield library. I think it would be better to create a separate library with an event-driven API.

[drmpf]

OK, my mistake. That suggestion will not work.
You clearly stated that the write() call blocks for at least the time it takes to send the data and get the ACK.
An event-driven API is the way to go for compatibility.

[drmpf]

I have come up a few small mods for WiFiClient which allow the user to keep the sketch alive while
sending WiFi data.
As noted previously sending a packet to Windows blocks the entire sketch for 200mS

The mods keep the spirit of the original blocking api while adding a
isSendWaiting() method to allow uses to check if WiFiClient.write() will block

Ignoring this new method the user's code is unchanged.

The operation is almost the same except that the first call to WiFiClient.write() return immediately.
Subsequent calls will block for upto 5 sec if the client is waiting an ACK to the previous packet.

If the 5 sec timeout occures the current connection is considered broken and is aborted.
The 5 sec timeout could be make larger to allow for retries if necessary.

With these changes the user's code will only block if the WiFiClient is busy sending a packet and
cannot accept more data. This is similar to HardwareSerial which does not block until it cannot accept
more data. WiFiClient::isSendWaiting() is similar to HardwareSerial::availableForWrite()

Now the user can optionally write code like

if (!client.isSendWaiting()) {
   client.write(buff,size); // will not block
} else {
   // don't call write, keep looping and accumulating bytes in buff ready for next write
}

This solves the problem in a Uart to WiFi bridge of lost incoming Serial data while the previous packet
is waiting for ACK. In general sketches it ensures the loop() keeps running to do its other work.

Code changes
in ClientContext.h

    size_t write(const char* data, size_t size) {
      if (!_pcb) {
        DEBUGV(":wr !_pcb\r\n");
        return 0;
      }

      if (size == 0) {
        return 0;
      }
      size_t counter = 0;
      while (isSendWaiting() && (counter < 5000) ) {
        delay(1);
        counter++;
      }
      if (counter >= 5000) {
        // timed out waiting for last send to complete
        _send_waiting = false;
        return (size_t)-1; // abort connection
      }
      size_t room = tcp_sndbuf(_pcb);
      size_t will_send = (room < size) ? room : size;
      err_t err = tcp_write(_pcb, data, will_send, 0);
      if (err != ERR_OK) {
        DEBUGV(":wr !ERR_OK\r\n");
        return 0;
      }

      _size_sent = will_send;
      DEBUGV(":wr\r\n");
      tcp_output( _pcb );
      _send_waiting = true;
      delay(0);
      //delay(5000); // max send timeout
      //_send_waiting = false;
      DEBUGV(":ww\r\n");
      return will_send - _size_sent;
    }

    bool isSendWaiting() {
      return _send_waiting;
    }
private:
    err_t _sent(tcp_pcb* pcb, uint16_t len) {
      DEBUGV(":sent %d\r\n", len);
      _size_sent -= len;
      if (_size_sent == 0 && _send_waiting) {
        _send_waiting = false;
        esp_schedule();
      }
      return ERR_OK;
    }

in WiFiClient.cpp

bool WiFiClient::isSendWaiting() {
  if (!_client) {
    return false;
  }
  return _client->isSendWaiting();
}

size_t WiFiClient::write(const uint8_t *buf, size_t size) {
  if (!_client || !size) {
    return 0;
  }
  //return _client->write(reinterpret_cast<const char*>(buf), size);
  size_t bytesWritten =  _client->write(reinterpret_cast<const char*>(buf), size);
  if (bytesWritten == ((size_t) - 1)) {
    // error abort connection
    if (!_client) {
      return 0;
    }
    _client->abort();
    _client->unref();
    _client = 0;
  }
  return bytesWritten;
}

in WiFiClient.h

  bool isSendWaiting();