Host side tests (Phase 2)

[igrr]

This issue is tracking implementation of a set of mocks which enable testing parts of the core on the PC, and writing relevant tests.

Easy testing targets include core classes (String, Stream, Print), heap management functions, and filesystem APIs.

Testing WiFiClient, WiFiClientSecure, WiFiServer, ESP8266WebServer, ESP8266httpClient is also possible, but requires a bit more work to build Linux port of LwIP and link against it.

To do list:

Phase 1 (#1716)

• initial batch of mocks
• some filesystem tests
• add makefile, run tests on Travis
• correct license headers in all mock files
• measure coverage for host based tests

Phase 2

• mock for HardwareSerial
• more filesystem tests
• optional replacement of malloc/free/new/delete in test environment — to test against simulated 40k heap. Need to figure out how to support different behaviour of operator new and operator delete in parts of code under test and in test runner.
• (new) unify host-tests and emulation-on-host APIs

Phase 3

• tests for number formatting functions in core_esp8266_noniso.c
• tests for String class

Phase 4

• LwIP port for Linux and infrastructure to link tests against it
• tests for WiFiClient/WiFiServer
  (update: can be done by adding more device tests now also runnable on host)

Phase 5

• AxTLS BearSSL Linux build
  (done with emulation-on-host)
• tests for WiFiClientSecure
  (device test test_sw_http_client is working on host with https w/ BearSSL using WiFiClientSecure)

Phase 6 (edit: done via running device tests on host)

• tests for ESP8266WebServer
• tests for ESP8266HTTPClient

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[devyte]

Progress has been done, but more work remains. Pushing back for next steps.

[devyte]

@earlephilhower @d-a-v is there anything further to add here before moving on? If not, I'll push back.

[d-a-v]

There'll always be something pending in this feature request.
Let's just not close it and keep regularly pushing it back so we can discuss on its state.

Currenly as @earlephilhower stated a short while ago, we have

• host tests (history)
• emulation on host (quite recent)
• device tests (recent)

Device tests use emulation on host, but host tests uses a different API for a subset of function.
This needs to be unified.