Vast improvements to native emitter, new nrf port, unified documentation

In this release there are a wide range of improvements and additions to both the core and the ports. In the core the main improvement was to the native emitter to have much more comprehensive support for general Python features, such as generators and complex exception handling, and the generated machine code is smaller and retains its efficiency. Elsewhere, fuzzy testing was used to find and eliminate some corner-case bugs, user classes were optimised when they don't use special accessors, underscores in numeric literals are now supported (PEP515), and the uio.IOBase class was added to allow user defined streams.

For the extended modules there is now a VfsPosix filesystem component, a new ucryptolib module with AES support, addition of ure.sub() and of uhashlib.md5, and the lwIP socket implementation now has working TCP listen/accept backlog.

Compared to the last release the minimal baseline core code size is reduced by about 2.2%, down by roughly 1500 bytes for bare-arm port and 3500 bytes for minimal x86 port. Most other ports have increased in size due to the addition of new features (eg ucryptolib, ure.sub).

The stm32 port sees the introduction of a new bootloader -- mboot -- which supports DFU upload via USB FS, USB HS, as well as a custom I2C protocol, and also support to program external SPI flash. There is significant refactoring of the USB device driver, improved VCP throughput, and support for 2x VCP interfaces on the one USB device. lwIP has been integrated, and support added for SDRAM. Cortex-M0 CPUs are now supported along with STM32F0 MCUs.

For the esp8266 port the heap is increased by 2kbytes, the radio is automatically put to sleep if no WLAN interfaces are active, and the UART can now be disconnected from the REPL and its RX buffer length configured. Upon soft-reset sockets are now cleaned up.

The esp32 port has added support for external SPI RAM, PPPoS functionality, improved performance and stability when using threads, and other general bug fixes.

There is a new nrf port for Nordic MCUs, currently supporting nRF51x and nRF52x chips.

The docs have now been unified so there is just one set of documentation covering all ports. And initial documentation for the esp32 port is added.

There are two changes at the Python API level that are not backwards with previous versions:

• uos.dupterm now requires that a stream passed to it is derived from uio.IOBase (or is a native stream object).
• The esp32 neopixel driver has had its default timing changed from 400kHz to 800kHz; existing code that didn't explicitly specify the "timing" parameter in the NeoPixel constructor may need to be updated to specify this as "timing=0" to get 400kHz timing.

A detailed list of changes follows.

py core:

• mpconfig.h: be stricter when autodetecting machine endianness
• mpstate.h: adjust start of root pointer section to exclude non-ptrs
• nlrx86: use naked attribute on nlr_push for gcc 8.0 and higher
• vm: adjust #if logic for gil_divisor so braces are balanced
• objfun: fix variable name in DECODE_CODESTATE_SIZE() macro
• vm: use enum names instead of magic numbers in multi-opcode dispatch
• vm: improve performance of opcode dispatch when using switch stmt
• repl: fix handling of unmatched brackets and unfinished quotes
• misc.h: add MP_STATIC_ASSERT macro to do static assertions
• compile: change comment about ITER_BUF_NSLOTS to a static assertion
• emitbc: avoid undefined behavior calling memset() with NULL 1st arg
• parsenum: use int instead of mp_int_t for parsing float exponent
• parsenum: avoid undefined behavior parsing floats with large exponents
• objfloat: fix undefined shifting behavior in high-quality float hash
• mpz: avoid undefined behavior at integer overflow in mpz_hash
• objfloat: fix undefined integer behavior hashing negative zero
• gc: when GC threshold is hit don't unnecessarily collect twice
• parsenum: adjust braces so they are balanced
• modbuiltins: add support for rounding integers
• objgenerator: save state in old_globals instead of local variable
• objgenerator: protect against reentering a generator
• emit: combine fast and deref into one function for load/store/delete
• emit: combine yield value and yield-from emit funcs into one
• emit: combine build tuple/list/map emit funcs into one
• emit: combine name and global into one func for load/store/delete
• emit: combine load/store/delete subscr into one emit function
• emit: combine load/store/delete attr into one emit function
• emit: combine break_loop and continue_loop into one emit function
• emit: combine import from/name/star into one emit function
• emit: merge build set/slice into existing build emit function
• emit: combine setup with/except/finally into one emit function
• objtype: fix assertion failures in mp_obj_new_type by checking types
• objtype: fix assertion failures in super_attr by checking type
• stream: move definition of mp_stream_p_t from obj.h to stream.h
• reader: allow MICROPY_VFS_POSIX to work with MICROPY_READER_POSIX
• mpconfig.h: add default MICROPY_VFS_FAT config value
• obj.h: introduce a "flags" entry in mp_obj_type_t
• objtype: don't expose mp_obj_instance_attr()
• objtype: optimise instance get/set/del by skipping special accessors
• gc: add gc_sweep_all() function to run all remaining finalisers
• lexer: add support for underscores in numeric literals
• modio: add uio.IOBase class to allow to define user streams
• mkrules.mk: regenerate all qstrs when config files change
• stream: introduce and use efficient mp_get_stream to access stream_p
• objarray: replace 0x80 with new MP_OBJ_ARRAY_TYPECODE_FLAG_RW macro
• add checks for stream objects in print() and sys.print_exception()
• compile: combine break and continue compile functions
• compile: combine subscript_2 and subscript_3 into one function
• compile: combine global and nonlocal statement compile functions
• compile: combine or_test and and_test compile functions
• compile: combine expr, xor_expr and and_expr into one function
• compile: handle return/break/continue correctly in async with
• objgenerator: eliminate need for mp_obj_gen_wrap wrapper instances
• obj.h: fix broken build for object repr C when float disabled
• simplify some cases of accessing the map of module and type dict
• objdict: make mp_obj_dict_get_map an inline function
• objmodule: make mp_obj_module_get_globals an inline function
• obj.h: give compile error if using obj repr D with single-prec float
• malloc: give a compile warning if using finaliser without GC
• objgenerator: implement name with normal fun attr accessor code
• emitnative: optimise for iteration asm code for non-debug build
• runtime: use mp_obj_new_int_from_ll when return int is not small
• stream: introduce MP_STREAM_GET_FILENO ioctl request
• objstr: in format error message, use common string with %s for type
• asmthumb: optimise native code calling runtime glue functions
• mpconfig.h: introduce MICROPY_DEBUG_PRINTER for debugging output
• fix compiling with debug enabled and make more use of DEBUG_printf
• emitnative: factor common code for native jump helper
• emitnative: fix x86 native zero checks by comparing full word
• asmx86: use generic emit function to simplify cmp emit function
• emitnative: fix native locals stack to start at correct location
• emitnative: simplify handling of exception objects from nlr_buf_t
• emitnative: allocate space for local stack info as it's needed
• compile: for dynamic compiler, widen literal 1 to get correct shift
• py.mk: don't hardcode path to libaxtls.a
• gc: in gc_alloc, reset n_free var right before search for free mem
• objarray: allow to build again when bytearray is disabled
• stream: adjust mp_stream_posix_XXX to take void*, not mp_obj_t
• emitnative: use small tables to simplify handling of local regs
• asmxtensa: handle function entry/exit when stack use larger than 127
• asm*: support assembling code to jump to a register, and get PC+off
• emitnative: optimise and improve exception handling in native code
• asmxtensa: fix bug with order of regs in addi encoding
• asmxtensa: optimise loading local addr and support larger offsets
• emitnative: fix bug with store of 16 and 32 values in viper ARM mode
• asmxtensa: use narrow version of add instr to reduce native code size
• emitnx86: fix number of args passed to mp_setup_code_state, 4 not 5
• vm: fix handling of finally-return with complex nested finallys
• emitnative: cancel caught exception once handled to prevent reraise
• emitnative: add support for return/break/continue in try and with
• compile: factor code that compiles start/end of exception handler
• py.mk: build axtls library directly from its source files
• runtime: fix incorrect test for MICROPY_PORT_DEINIT_FUNC
• objarray: bytearray: allow 2nd/3rd arg to constructor
• emitnative: fix try-finally in outer scope, so finally is cancelled
• fix native functions so they run with their correct globals context
• optimise call to mp_arg_check_num by compressing fun signature
• asmx64: fix bug in assembler when creating disp with r13 and 0 offset
• {asmx86,asmx64}: extend test_r8_with_r8 to accept all 8 lower regs
• emit: move MP_EMIT_OPT_xxx enums from compile.h to emitglue.h
• emit: remove need to call set_native_type to set native/viper mode
• emit: remove need to call set_native_type to set viper return type
• emit: completely remove set_native_type, arg type is set in compiler
• compile: merge viper annotation and normal param compilation stages
• compile: factor code that compiles viper type annotations
• emitnative: reuse mp_native_type_from_qstr when searching for a cast
• make viper functions have the same entry signature as native
• emitnative: support arbitrary number of arguments to viper functions
• emitnative: use macros instead of raw offsetof for slot locations
• emitnative: make viper f...

Parser size reduced, new Python stack, stm32 improvements, new esp32 port

This release brings some significant size reductions to the parser, as well as removal of unused code and additional tests to improve coverage of the core. A new optional, internal Python stack is introduced for scoped allocation and can be used instead of alloca to provide more efficient and flexible temporary memory; see MICROPY_ENABLE_PYSTACK. There have been many improvements and internal code refactors of the stm32 port, and the port to the Espressif ESP32 SoC has been merged from its development repository.

Compared to the last release the change in code size is (in bytes, using gcc 7.3):

bare-arm: -1832
minimal x86: -2608
unix x64: -5129
unix nanbox: +1024
stm32: +2120
cc3200: -928
esp8266: +336

The decrease is mainly due to the reduced size of the parser, where the table of rule pointers was compressed to a table of offsets. The increase in the stm32 and esp8266 ports is due to additional features, such as more colour formats in the framebuf module and the addition of ujson.dump().

Code coverage (measured by gcov) has improved since the last version (v1.9.3 on the left, v1.9.4 on the right):

py: 15202/15447 = 98.4% -> 15391/15513 = 99.2%
extmod: 2227/ 2363 = 94.2% -> 2291/ 2430 = 94.3%

Changes and additions seen by the Python user include: improvement of dir() and tab-completion to list all available attributes of an object; addition of efficient ucollections.deque type with fixed size; better handling and polling support of sockets that get into an error state; implementation of key and cert keyword arguments in ussl.wrap_socket (for axtls); uos.ilistdir now return 4-tuples with the file size in the fourth position.

Changes at the C level include: removal of "make_qstr_if_not_already" argument from mp_obj_new_str; rename of mp_exc_recursion_depth to mp_raise_recursion_depth; removal of mp_const_MemoryError_obj; switching of stream close operation from method to ioctl; refactoring of how native emitter code is compiled with a file per architecture.

The stm32 port has seen a lot of improvements and additions, as well as some significant internal refactoring to better support configuring custom boards. STM32F7 support is improved, USB HS is supported for F723 and F733, the CAN class has improvements to handle bus errors, ctrl-C can now interrupt running code when the REPL is over UART, and the ADC class added the read_timed_multi static method. Board configuration has seen some internal changes, in particular with setting LD_FILES, TEXT0_ADDR, and TEXT1_ADDR in mpconfigboard.mk (previously LD_FILE, FLASH_ADDR, TEXT_ADDR). The pin_X and pyb_pin_X identifiers have been changed to be pointers to objects rather than objects, and main is renamed to stm32_main.

A detailed list of changes follows.
py core:

• compile: use alloca instead of qstr_build when compiling import name
• mpconfig: introduce reusable MP_HTOBE32(), etc. macros
• objtype: mp_obj_new_type: name base types related vars more clearly
• objnamedtuple: add _asdict function if OrderedDict is supported
• emitnative: clean up asm macro names so they have dest as first arg
• mkenv.mk: use $(PYTHON) consistently when calling Python tools
• objstr: remove "make_qstr_if_not_already" arg from mp_obj_new_str
• objstr: make mp_obj_new_str_of_type check for existing interned qstr
• objstr: when constructing str from bytes, check for existing qstr
• add config option to disable multiple inheritance
• objnamedtuple: allow to reuse namedtuple basic functionality
• objfloat: allow float() to parse anything with the buffer protocol
• modbuiltins: slightly simplify code in builtin round()
• runtime: simplify handling of containment binary operator
• opmethods: include the correct header for binary op enums
• runtime: add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN
• parsenum: improve parsing of floating point numbers
• objdict: reuse dict-view key iterator for standard dict iterator
• annotate func defs with NORETURN when their corresp decls have it
• qstr: rewrite find_qstr to make manifest that it returns a valid ptr
• gc: in gc_realloc, convert pointer sanity checks to assertions
• objgenerator: remove unreachable code for STOP_ITERATION case
• misc.h: add m_new_obj_var_with_finaliser()
• mp_call_function_*_protected(): pass-thru return value if possible
• modbuiltins: use standard arg-parsing helper func for builtin print
• mpprint: make "%p" format work properly on 64-bit systems
• mpprint: support "%lx" format on 64-bit systems
• malloc: allow to use debug logging if !MICROPY_MALLOC_USES_ALLOCATED_SIZE
• gc: add CLEAR_ON_SWEEP option to debug mis-traced objects
• emitbc, asmbase: only clear emit labels to -1 when in debug mode
• asmbase: revert removal of clearing of label offsets for native emit
• objint_longlong: check for zero division/modulo
• runtime: when tracing unary/binary ops, output op (method) name
• gc: factor out a macro to trace GC mark operations
• gc: in sweep debug output, print pointer as a pointer
• objfun: factor out macro for decoding codestate size
• objfun, vm: add comments on codestate allocation in stackless mode
• objfun: factor out macro for initializing codestate
• mkrules.mk: add "clean-frozen" target to clean frozen script/modules dir
• runtime: move mp_exc_recursion_depth to runtime and rename to raise
• introduce a Python stack for scoped allocation
• convert all uses of alloca() to use new scoped allocation API
• runtime: use the Python stack when building *arg and **kwarg state
• objexcept: use INT_FMT when printing errno value
• extend nan-boxing config to have 47-bit small integers
• objtype: implement better support for overriding native's init
• objtype: refactor object's handling of new to not create 2 objs
• builtinimport: call init for modules imported via a weak link
• objgenerator: allow to pend an exception for next execution
• map: don't include ordered-dict mutating code when not needed
• mpz: fix pow3 function so it handles the case when 3rd arg is 1
• modio: use correct config macro to enable resource_stream function
• malloc: remove unneeded code checking m_malloc return value
• nlrthumb: fix use of naked funcs, must only contain basic asm code
• nlr: clean up selection and config of NLR implementation
• nlr: factor out common NLR code to macro and generic funcs in nlr.c
• parse: split out rule name from rule struct into separate array
• parse: break rule data into separate act and arg arrays
• parse: pass rule_id to push_result_rule, instead of passing rule_t*
• parse: pass rule_id to push_result_token, instead of passing rule_t*
• parse: remove rule_t struct because it's no longer needed
• parse: compress rule pointer table to table of offsets
• parse: update debugging code to compile on 64-bit arch
• parse: fix macro evaluation by avoiding empty VA_ARGS
• mpz: simplify handling of borrow and quo adjustment in mpn_div
• mpz: in mpz_as_str_inpl, convert always-false checks to assertions
• nlr: fix nlr functions for 64bit ports built with gcc on Windows
• compile: combine compiler-opt of 2 and 3 tuple-to-tuple assignment
• objtype: check and prevent delete/store on a fixed locals map
• modbuiltins: for builtin_chr, use uint8_t instead of char for array
• vm: simplify stack sentinel values for unwind return and jump
• parsenum: fix parsing of floats that are close to subnormal
• objfloat: fix case of raising 0 to -infinity
• unicode: clean up utf8 funcs and provide non-utf8 inline versions
• modbuiltins: simplify casts from char to byte ptr in builtin ord
• objrange: implement (in)equality comparison between range objects
• objexcept: remove long-obsolete mp_const_MemoryError_obj
• nlrthumb: do not mark nlr_push as not returning anything
• pystack: use "pystack exhausted" as error msg for out of pystack mem
• objdict: disallow possible modifications to fixed dicts
• gc: reduce code size by specialising VERIFY_MARK_AND_PUSH macro
• gc: rename gc_drain_stack to gc_mark_subtree and pass it first block
• gc: make GC stack pointer a local variable
• qstr: add QSTR_TOTAL() macro to get number of qstrs
• modbuiltins: simplify and generalise dir() by probing qstrs
• repl: generalise REPL autocomplete to use qstr probing
• objstr: protect against creating bytes(n) with n negative
• objmodule: factor common code for calling init on builtin module
• builtinimport: add compile-time option to disable external imports
• modmicropython: allow to have stack_use() func without mem_info()
• objstr: remove unnecessary check for positive splits variable
• objint: use MP_OBJ_IS_STR_OR_BYTES macro instead of 2 separate ones
• objdeque: implement ucollections.deque type with fixed size
• objdeque: protect against negative maxlen in deque constructor
• objdeque: use m_new0 when allocating items to avoid need to clear
• py.mk: split list of uPy sources into core and extmod files
• py.mk: remove .. path component from list of extmod files
• use "GEN" consistently for describing files generated in the build
• compile: adjust c_assign_atom_expr() to use return instead of goto
• asm*.c: remove unnecessary check for num_locals<0 in asm entry func
• mpz: in mpz_clone, remove unused check for NULL dig
• mpstate.h: add repl_line state for MICROPY_REPL_EVENT_DRIVEN
• vm: fix case of handling raised StopIteration within yield from
• vm: simplify handling of special-case STOP_ITERATION in yield from
• formatfloat: fix rounding of %f format with edge-case FP values
• formatfloat: fix case where floats could render with a ":" character
• formatfloat: fix case where floats could render with negative digits
• objint: remove unreachable code checking for int type in format func
• misc.h: remove unused count_lead_ones() inline function
• obj.h: clean up by removing commented-out inline versions of macros
• obj.h: move declaration of mp_o...

Introduction of ports subdirectory where all ports are moved to

The main change in this release is the introduction of a "ports/" subdirectory at the top-level of the repository, and all of the ports are moved here. In the process the "stmhal" port is renamed to "stm32" to better reflect the MCU that it targets. In addition, the STM32 CMSIS and HAL sources are moved to a new submodule called "stm32lib".

The bytecode has changed in this release, compared to the previous release, and as a consequence the .mpy version number has increased to version 3. This means that scripts compiled with the previous mpy-cross must be recompiled to work with this new version.

There have also been various enhancements and optimisations, such as: check for valid UTF-8 when creating str objects, support for reverse special binary operations like __radd__, full domain checking in the math module, support for floor-division and modulo in the viper emitter, and addition of stack overflow checking when executing a regex.

The stm32 port sees improved support for F7 MCUs, addition of a new board B_L475E_IOT01A based on the STM32L475, and support for the Wiznet W5500 chipset along with improved socket behaviour.

A detailed list of changes follows.

py core:

• objstr: startswith, endswith: check arg to be a string
• nlrx86,x64: replace #define of defined() with portable macro usage
• objtype: handle NotImplemented return from binary special methods
• objtype: mp_obj_class_lookup: improve debug logging
• map: remove unused new/free functions
• make m_malloc_fail() have void return type, since it doesn't return
• modstruct: in struct.pack, stop converting if there are no args left
• modstruct: check and prevent buffer-read overflow in struct unpacking
• modstruct: check and prevent buffer-write overflow in struct packing
• nlrthumb: get working again on standard Thumb arch (ie not Thumb2)
• objfloat: fix binary ops with incompatible objects
• obj: fix comparison of float/complex NaN with itself
• objtype: implement fallback for instance inplace special methods
• objtuple: properly implement comparison with incompatible types
• objstr: add check for valid UTF-8 when making a str from bytes
• objlist: properly implement comparison with incompatible types
• runtime0.h: move relational ops to the beginning of mp_binary_op_t
• runtime0.h: move MP_BINARY_OP_DIVMOD to the end of mp_binary_op_t
• objtype: make sure mp_binary_op_method_name has full size again
• runtime0.h: put inplace arith ops in front of normal operations
• builtinhelp: simplify code slightly by extracting object type
• runtime: implement dispatch for "reverse op" special methods
• nlrx86: fix building for Android/x86
• builtinhelp: change signature of help text var from pointer to array
• runtime.h: change empty mp_warning macro so var-args are non empty
• modbuiltins: implement abs() by dispatching to MP_UNARY_OP_ABS
• {objfloat,objcomplex}: optimise MP_UNARY_OP_ABS by reusing variables
• mpconfig.h: add note that using computed gotos in VM is not C99
• objstr: strip: don't strip "\0" by default
• objexcept: prevent infinite recursion when allocating exceptions
• stream: remove unnecessary checks for NULL return from vstr_add_len
• vstr: raise a RuntimeError if fixed vstr buffer overflows
• vm: use lowercase letter at start of exception message
• persistentcode: define mp_raw_code_save_file() for any unix target
• add config option to print warnings/errors to stderr
• objfloat: support raising a negative number to a fractional power
• objset: simplify set and frozenset by separating their locals dicts
• objset: check that RHS of a binary op is a set/frozenset
• objset: include the failed key in a KeyError raised from set.remove
• objtype: change type of enum-to-qstr table to uint16_t to save space
• objstr: make empty bytes object have a null-terminating byte
• mpprint: only check for null string printing when NDEBUG not defined
• objtype: clean up unary- and binary-op enum-to-qstr mapping tables
• persistentcode: bump .mpy version number to version 3
• bc: update opcode_format_table to match the bytecode
• modmath: add full checks for math domain errors
• modmath: convert log2 macro into a function
• formatfloat: don't print the negative sign of a NaN value
• formatfloat: use standard isinf, isnan funcs instead of custom ones
• modbuiltins: use existing utf8_get_char helper in builtin ord func
• emitnative: implement floor-division and modulo for viper emitter
• objtype: use CPython compatible method name for sizeof
• objtype: fit qstrs for special methods in byte type
• objtype: define all special methods if requested
• objtype: introduce MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS

extmod:

• modubinascii: only include uzlib/tinf.h when it's really needed
• modussl_mbedtls: allow to compile with MBEDTLS_DEBUG_C disabled
• machine_pinbase: put PinBase singleton in ROM
• re1.5: upgrade to v0.8.2, adds hook for stack overflow checking
• modure: add stack overflow checking when executing a regex
• uos_dupterm: update uos.dupterm() and helper funcs to have index
• uos_dupterm: swallow any errors from dupterm closing the stream
• vfs: replace VLA in proxy func with small, static sized array
• modussl: add finaliser support for ussl objects
• modussl_mbedtls: allow to compile with unix coverage build

lib:

• add new submodule, stm32lib containing STM32 CMSIS and HAL source
• embed/abort_: use mp_raise_msg helper function
• libm: fix tanhf so that it correctly handles +/- infinity args
• libm: remove implementation of log2f, use MP_NEED_LOG2 instead
• axtls: update, support for SSL_EAGAIN return code
• berkeley-db-1.xx: update, allow to override MINCACHE, DEFPSIZE

drivers:

• memory/spiflash: change from hard-coded soft SPI to generic SPI
• display/ssd1306.py: improve performance of graphics methods
• nrf24l01: make nRF24L01 test script more portable
• display/ssd1306: implement SSD1306_I2C poweron method
• display/ssd1306: make poweron() work the same with SSD1306_SPI
• wiznet5k: improve the performance of socket ops with threading
• wiznet5k: get low-level W5500 driver working

tools:

• upip: upgrade to 1.2.2
• pyboard: use repr() when quoting data in error messages
• pyboard: update docstring for additional device support

tests:

• object_new: better messages, check user new() method
• class_new: add checks for init being called and other improvements
• class_new: add another testcase for new/init interaction
• class_inplace_op: test for inplace op fallback to normal one
• run-bench-tests: update locations of executables, now in ports/
• class_reverse_op: test for reverse arith ops special methods
• run-tests: skip class_inplace_op for minimal profile
• run-tests: fix copy-paste mistake in var name
• cpydiff: add cases for locals() discrepancies
• extmod: add test for ure regexes leading to infinite recursion
• extmod: add test for '-' in character class in regex
• run-tests: close device under test using "finally"
• net_inet: update tls test to work with CPython and incl new site

unix port:

• rename modsocket.c to modusocket.c
• modusocket: remove #if MICROPY_SOCKET_EXTRA code blocks
• enable MICROPY_PY_REVERSE_SPECIAL_METHODS

stm32 port:

• modmachine: make machine.bootloader() work when MPU is enabled
• modmachine: improve support for sleep/deepsleep on F7 MCUs
• compute PLL freq table during build instead of at run time
• modmachine: for F7 MCU, save power by reducing internal volt reg
• boards/pllvalues.py: make script work with both Python 2 and 3
• Makefile: use lib/stm32lib instead of local cmsis and hal files
• remove cmsis and hal files, they are now a submodule
• Makefile: automatically fetch stm32lib submodule if needed
• update to new STM Cube HAL library
• fix clock initialisation of L4 MCUs
• rename stmhal port directory to stm32
• remove unused usbd_msc.c file
• boards: change remaining stm32f4xx_hal_conf.h to unix line ending
• boards: change linker scripts to use "K" instead of hex byte size
• boards: fix I2C1 pin mapping on NUCLEO_F401RE/F411RE boards
• i2c: when scanning for I2C devices only do 1 probe per address
• modnwwiznet5k: release the GIL on blocking network operations
• boards: add new board B_L475E_IOT01A based on STM32L475
• make-stmconst.py: make sure mpz const data lives in ROM
• timer: make pyb.Timer() instances persistent
• mpconfigport.h: add configuration for max periphs on L4 series
• usbdev: make the USBD callback struct const so it can go in ROM
• usbdev: change static function variable to non-static
• usbdev: put all CDC state in a struct
• usbdev: put all HID state in a struct
• usbdev: simplify CDC tx/rx buffer passing
• usbdev: simplify HID tx/rx buffer passing
• usbdev/core: add state parameter to all callback functions
• usbdev: put all state for the USB device driver in a struct
• usbdev: simplify pointers to MSC state and block dev operations
• usbdev: merge all global USB device state into a single struct
• usbdev: make device descriptor callbacks take a state pointer
• usbdev: move all the USB device descriptor state into its struct
• timer: enable ARPE so that timer freq can be changed smoothly
• modnwwiznet5k: get the IP address of an established socket
• boards: fix typos in stm32f767_af.csv table
• usbd_cdc_interface: don't reset CDC output buf on initialisation
• modnwwiznet5k: implement WIZNET5K.isconnected() method
• modusocket: make getaddrinfo() work when passed an IP address
• modusocket: return OSError(-2) if getaddrinfo fails
• mpconfigport.h: add MICROPY_THREAD_YIELD() macro
• modnwwiznet5k: add support for W5500 Ethernet chip
• modnwwiznet5k: increase SPI bus speed to 42MHz
• modnwwiznet5k: implement stream ioctl for the Wiznet driver
• mphalport: improve efficiency of mp_hal_stdout_tx_strn_cooked
• make uos.dupterm() conform to specs by using extmod version

cc3200 port:

• enable micropy...

Double precision math library and support on pyboard, and improved ussl

This release brings general improvements and bug fixes to the core and
various ports, as well as documentation additions, clean-ups and better
consistency. And effort has been made to clean up the source code to
make it more consistent across the core and all ports.

There is a new tool "mpy_bin2res.py" to convert arbitrary (binary) files
to Python resources for inclusion in source code (frozen or otherwise).

The ussl module has seen improvements, including implementation of
server_hostname (for axtls) and server_side mode (for mbedtls).

There is now a double-precision float math library and stmhal has support
to build firmware with software or hardware double-precision.

A detailed list of changes follows.

py core:

• formatfloat: fix number of digits and exponent sign when rounding
• modthread: raise RuntimeError in release() if lock is not acquired
• compile: raise SyntaxError if positional args are given after /*
• objint: support "big" byte-order in int.to_bytes()
• objint: in to_bytes(), allow length arg to be any int and check sign
• compile: fix bug with break/continue in else of optimised for-range
• compile: optimise emitter label indices to save a word of heap
• builtinimport: remove unreachable code for relative imports
• objnamedtuple: simplify and remove use of alloca building namedtuple
• mpprint: remove unreachable check for neg return of mp_format_float
• binary: add missing "break" statements
• runtime: mark m_malloc_fail() as NORETURN
• objstr: remove unnecessary "sign" variable in formatting code
• vm: make "if" control flow more obvious in YIELD_FROM opcode
• modmath: check for zero division in log with 2 args
• makeversionhdr.py: update to parse new release line in docs/conf.py
• objdict: factorise dict accessor helper to reduce code size
• change mp_uint_t to size_t in builtins code
• repl: change mp_uint_t to size_t in repl helpers
• compile: combine arith and bit-shift ops into 1 compile routine
• compile: use switch-case to match token and operator
• objgenerator: allow to hash generators and generator instances
• gc: refactor assertions in gc_free function
• vm: make n_state variable local to just set-up part of VM
• asmx64: support moving a 64-bit immediate to one of top 8 registers
• modmicropython: cast stack_limit value so it prints correctly
• builtinevex: add typechecking of globals/locals args to eval/exec
• py.mk: make berkeley-db C-defs apply only to relevant source files
• mperrno: allow mperrno.h to be correctly included before other hdrs
• mpz: make mpz_is_zero() an inline function
• implement raising a big-int to a negative power
• mkrules.mk: show frozen modules sizes together with executable size
• objtuple: allow to use inplace-multiplication operator on tuples
• objstr: raise an exception for wrong type on RHS of str binary op
• modsys: initial implementation of sys.getsizeof()
• binary.c: fix bug when packing big-endian 'Q' values
• add verbose debug compile-time flag MICROPY_DEBUG_VERBOSE
• binary: change internal bytearray typecode from 0 to 1
• objstringio: prevent offset wraparound for io.BytesIO objects
• objstringio: fix regression with handling SEEK_SET
• stream: seek: Consistently handle negative offset for SEEK_SET
• mkrules.mk: use "find -path" when searching for frozen obj files
• compile: remove unused pn_colon code when compiling func params
• objcomplex: remove unnecessary assignment of variable
• formatfloat: don't post-increment variable that won't be used again
• use "static inline" for funcs that should be inline
• asmthumb: use existing macro to properly clear the D-cache

extmod:

• modussl_axtls: update for axTLS 2.1.3
• modussl_axtls: implement server_hostname arg to wrap_socket()
• move modonewire.c from esp8266 to extmod directory
• modure: if input string is bytes, return bytes results too
• modubinascii: add check for empty buffer passed to hexlify
• modussl_axtls: allow to close ssl stream multiple times
• modussl_mbedtls: support server_side mode
• modussl_mbedtls: when reading and peer wants to close, return 0
• modframebuf: fix invalid stride for odd widths in GS4_HMSB fmt
• modussl_mbedtls: make socket.close() free all TLS resources
• modframebuf: consistently use "col" as name for colour variables
• modussl_mbedtls: implement non-blocking SSL sockets
• machine_signal: fix parsing of invert arg when Pin is first arg
• modframebuf: use correct initialization for .locals_dict
• modlwip: implement setsockopt(IP_ADD_MEMBERSHIP)
• modussl_mbedtls.c: add ussl.getpeercert() method
• modubinascii: rewrite mod_binascii_a2b_base64
• modubinascii: don't post-increment variable that won't be used
• modonewire: rename public module to mp_module_onewire
• for uos.stat interpret st_size member as an unsigned int
• use "static inline" for funcs that should be inline

lib:

• axtls: upgrade to axTLS 2.1.3 + MicroPython patchset
• libm/math: remove implementations of float conversion functions
• add libm_dbl, a double-precision math library, from musl-1.1.16

drivers:

• onewire: move onewire.py, ds18x20.py from esp8266 to drivers
• onewire: enable pull-up when init'ing the 1-wire pin

tools:

• gen-cpydiff: use case description as 3rd-level heading
• pyboard: add license header
• mpy_bin2res: tools to convert binary resources to Python module
• mpy-tool.py: don't generate const_table if it's empty
• mpy-tool.py: fix missing argument in dump() function

tests:

• net_inet/test_tls_sites.py: integration test for SSL connections
• net_inet: add tests for accept and connect in nonblocking mode
• basics: add tests for for-else statement
• net_inet: move tests which don't require full Internet to net_hosted
• connect_nonblock: refactor towards real net_hosted test
• auto detect floating point capabilites of the target
• import: add a test for the builtin import function
• import: update comment now that uPy raises correct exception
• basics/namedtuple1: add test for creating with pos and kw args
• unix/extra_coverage: add test for mp_vprintf with bad fmt spec
• basics: add tests for arithmetic operators precedence
• cpydiff/modules_deque: elaborate workaround
• cpydiff/core_class_mro: move under Classes, add workaround
• cpydiff/core_arguments: move under Functions subsection
• cpydiff/core_class_supermultiple: same cause as core_class_mro
• cpydiff: improve wording, add more workarounds
• cpydiff: add case for str.ljust/rjust
• rename exec1.py to builtin_exec.py
• basics/builtin_exec: test various globals/locals args to exec()

minimal port:

• Makefile: enable gc-sections to remove unused code
• remove unused stmhal include from Makefile
• use size_t for mp_builtin_open argument

unix port:

• modtime: replace strftime() with localtime()
• mpconfigport.mk: update descriptions of readline and TLS options
• Makefile: disable assertions in the standard unix executable
• modjni: convert to mp_rom_map_elem_t
• for uos.stat interpret st_size member as an unsigned int

stmhal port:

• mpconfigport.h: remove config of PY_THREAD_GIL to use default
• make error messages more consistent across peripherals
• add initial implementation of Pin.irq() method
• add .value() method to Switch object, to mirror Pin and Signal
• move pybstdio.c to lib/utils/sys_stdio_mphal.c for common use
• add "quiet timing" enter/exit functions
• make available the _onewire module, for low-level bus control
• modules: provide sym-link to onewire.py driver
• boards/stm32f405.ld: increase FLASH_TEXT to end of 1MiB flash
• sdcard: allow a board to customise the SDIO pins
• add possibility to build with double-precision floating point
• boards: enable double-prec FP on F76x boards
• Makefile: use hardware double-prec FP for MCUs that support it
• Makefile: rename FLOAT_IMPL to MICROPY_FLOAT_IMPL to match C name
• Makefile: add CFLAGS_EXTRA to CFLAGS so cmdline can add options
• mpconfigport.h: allow MICROPY_PY_THREAD to be overridden
• boards: add configuration files for NUCLEO_F429ZI
• boards/NUCLEO_F429ZI: change USB config from HS to FS peripheral
• reduce size of ESPRUINO_PICO build so it fits in flash
• servo: make pyb.Servo(n) map to Pin('Xn') on all MCUs
• servo: don't compile servo code when it's not enabled
• use "static inline" for funcs that should be inline

cc3200 port:

• modusocket: simplify socket.makefile() function
• make non-zero socket timeout work with connect/accept/send
• modusocket: fix connect() when in non-blocking or timeout mode
• use the name MicroPython consistently in code

esp8266 port:

• Makefile: bump axTLS TLS record buffer size to 5K
• Makefile: allow FROZEN_DIR,FROZEN_MPY_DIR to be overridden
• Makefile: add LIB_SRC_C variable to qstr auto-extraction list
• make onewire module and support code usable by other ports
• modonewire: move low-level 1-wire bus code to modonewire.c
• modonewire: make timings static and remove onewire.timings func
• reinstate 1-wire scripts by sym-linking to drivers/onewire/
• move mp_hal_pin_open_drain from esp_mphal.c to machine_pin.c
• enable MICROPY_ENABLE_FINALISER
• README: make "Documentation" a top-level section
• machine_rtc: use correct arithmetic for aligning RTC mem len
• mpconfigport_512k: use terse error messages to get 512k to fit
• mpconfigport.h: make socket a weak link
• modesp: remove unused constants: STA_MODE, etc
• general: add known issue of WiFi RX buffers overflow
• use size_t for mp_builtin_open argument
• fix UART stop bit constants

zephyr port:

• Makefile: rework dependencies and "clean" target
• Makefile: revert prj.conf construction rule to the previous state
• remove long-obsolete machine_ptr_t typedef's
• Makefile: explicitly define default target as "all"
• modusocket: allow to use socketized net_context in upstream
• modusocket: socket, close: switch to native Zephyr socket calls
• modusocket: bind...