[nrfconnect] Updated nRF Connect SDK to 2.3.0 (#25432)

* [nrfconnect] Removed IPC priority workaround from overlays

Removed a workaround regarding IPC priority, which is
not needed anymore in Matter samples.

* [nrfconnect] Aligned DFU implementation to new mcumgr API

With nRF Connect SDK 2.3.0 mcumgr API changed and DFU over SMP
implementation required updating.

* [nrfconnect] Updated nRF Connect SDK to 2.3.0

Bumped recommended nRF Connect SDK version to 2.3.0
and bring in list of commits including mainly Wi-Fi
support fixes:

1. Implemented most of the Wi-Fi DiagnosticDataProvider.

Added PHY statistics:
  * multicast RX/TX
  * unicast RX/TX
  * beacons lost/TX

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

2. wifi: map supplicant WiFi version into what Matter expects

This is needed to return proper Wi-Fi version code to the Matter
controller when querying wifinetworkdiagnostic cluster.

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

3. Fix the 'disconnected' status appearing after scan is issued.

Restore the connection state which was set before requesting scan.
Note that the connection was not really dropped, only the status
reporting was broken.

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

4. Fix ram_report and rom_report

Stop filtering out gdwarf-4 when passing flags from Zephyr
to Matter's GN build system. Older pyelftools versions are
not able to parse DWARF5 format, so ram and rom report
would not be generated.

Signed-off-by: Damian Krolik <damian.krolik@nordicsemi.no>

5. wifi: fixed the connection callback initialization

This fixes the misbehavior when attempting to establish a connection
with the network that is not added to the networkcommissioning cluster.

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

6. Fixed the timeout when connecting to the WPA3 secured AP.

Set the WiFi MFP (management frame protection) as at least optional.
The current WiFi driver does not support this parameter within the
scan result, so it must the hard coded for the time being
(made it mandatory for secure associations, just in case).

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

7. Switch to statically allocated heap

* Use malloc/free replacements based on statically allocated
Zephyr's sys_heap to provide better control of RAM usage.

*  Fixed hard fault once advertising mdns records

In mDNS code there isn't a check that would verify if
memory was allocated successfully. In case heap is too small
the application will fall into hard fault due to usage of
non-allocated memory.

* Increased heap size for Wi-Fi.

Empirically it was measured that heap size needed to pass the
Wi-Fi commissioning is ~25k. To be on the safe side let's set it
to 28k.

8. Disabled NFC thread callback

NFC lib introduced new callbacks that are not needed by the
Matter samples and increase flash usage.

9. Update required MCUmgr config

Updates required MCUmgr configuration for zephyr upmerge changes.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>

10. [nrfconnect] Disable unnecessary shell features

Save more flash by disabling shell wildcard, colors, stats
and kernel commands.

Signed-off-by: Damian Krolik <damian.krolik@nordicsemi.no>

11. Increased the default net_mgmt stack size.

Once we were hit by the net_mgmt thread stack overflow
while show casing the Matter over WiFi solution in
WiFi RF congested environment. Increase the default
stack size of net_mgmt from 768 to 1k.

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

12. Use generic channel when connecting.

Currently using the specific channel number does not work
reliably in WiFi driver, so use the generic channel screening.

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

13. Rework net_mgmt WiFi event handling to offload CHIP thread.

In the case there are many networks available, the scan result
events may saturate the CHIP work queue which has lower priority
then the net_mgmt and main threads. So, collect the scan results
in the main thread and only ping the CHIP thread when the scan is done.

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

14. Enable runtime PA gain control when FEM is active

Output power at the antenna port can be controlled
automatically after setting
MPSL_FEM_NRF21540_RUNTIME_PA_GAIN_CONTROL config.

After enabling it the user can use the
CONFIG_OPENTHREAD_DEFAULT_TX_POWER config to control OpenThread
radio output power.

15. [nrfconnect] Introduce the WiFi connection recovery feature

The WiFi connection recovery feature allows device re-scanning
and re-connecting to the known WiFi network after the device's reboot
and when the known SSID has not been found during the last scan.
The connection recovery interval is doubled with
every occurrence to the defined maximum value and then its value
depends on the maximum value +- the defined random jitter.

After restoring the connection, the Connection Recovery
Interval is restored to the defined minimum value after elapsing
of the defined delay to avoid frequent reconnections to a
poor link quality network.

[nrfconnect] WiFi connection recovery refinements:
  * simplify recovery time resetting (due to connection status
    reporting limitations)
  * remove CHIP_WIFI_CONNECTION_RECOVERY_RESET_DELAY (not needed now)
  * avoid duplicated recovery timeout bumps
  * always start the recovery with kConnectionRecoveryMinIntervalMs
  * fixed some edge cases and memory leaks
  * minor cleanup
  * logging refinements
  * Kconfig description fixes after tech writer review
  * Move new Kconfig definitions to the Kconfig.features

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

16. Fixed several Wi-Fi issues

* Disabled Wi-Fi/BLE coex as it was not stable and caused issues
in CI tests
* Increased sockets poll and net mgmt stack to fix crash on the
application boot due to no space to alloc all sockets
* Added generating kDnssdInitialized event after getting Wi-Fi
connected event to re-start mDNS server

Fix the unused variable error.

This is warning propagated to an error in the CI.

Signed-off-by: Marcin Kajor <marcin.kajor@nordicsemi.no>

17. Mapped the WiFi security type from Zephyr to Matter

The Security type attribute was wrongly cast in the WiFi Manager.
Used switch-case-based mapping between two enum types
(Zephyr-Matter) to make sure that we properly cast between
two different types.

* [nrfconnect] Fixed incorrect DNSSD event name

In Wi-Fi platform, the obsolete kDnssdPlatformInitialized
event name is used for event generation.

Updated name to kDnssdInitialized.

* [nrfconnect] Fixed Wi-Fi version and security getters

The Wi-Fi platform uses obsolete EMBER_ZCL macros for Wi-Fi
version and security getters.

Replaced EMBER macros with enums from WiFiNetworkDiagnostics.

* [nrfconnect] Fixed restyler diff in DiagnosticDataProviderImplNrf

* [nrfconnect] Addressed code review comments

* Aligned default configs
* Refactored WiFiInfo struct to use SecurityTypeEnum instead of
uint8_t.

* Fix native_posix tests

---------

Co-authored-by: Damian Krolik <damian.krolik@nordicsemi.no>

.github/workflows/examples-nrfconnect.yaml

@@ -38,7 +38,7 @@ jobs:
         if: github.actor != 'restyled-io[bot]'
 
         container:
-            image: connectedhomeip/chip-build-nrf-platform:0.6.44
+            image: connectedhomeip/chip-build-nrf-platform:0.6.47
             volumes:
                 - "/tmp/bloat_reports:/tmp/bloat_reports"
 

config/nrfconnect/.nrfconnect-recommended-revision

@@ -1 +1 @@
-v2.2.0
+v2.3.0

config/nrfconnect/chip-module/CMakeLists.txt

@@ -114,6 +114,9 @@ if (CONFIG_NORDIC_SECURITY_BACKEND)
         zephyr_include_directories($<TARGET_PROPERTY:platform_cc3xx,INTERFACE_INCLUDE_DIRECTORIES>)
     endif()
     list(APPEND CHIP_CFLAGS -DMBEDTLS_CONFIG_FILE=<nrf-config.h>)
+elseif(CONFIG_MBEDTLS)
+    zephyr_include_directories($<TARGET_PROPERTY:mbedTLS,INTERFACE_INCLUDE_DIRECTORIES>)
+    zephyr_compile_definitions($<TARGET_PROPERTY:mbedTLS,INTERFACE_COMPILE_DEFINITIONS>)
 endif()
 
 if (CONFIG_NRF_802154_RADIO_DRIVER)

config/nrfconnect/chip-module/Kconfig

@@ -88,6 +88,9 @@ config CHIP_DEBUG_SYMBOLS
 	help
 	  Enables building the application with debug symbols.
 
+config CHIP_MALLOC_SYS_HEAP
+	default y if !ARCH_POSIX
+
 config CHIP_FACTORY_DATA
 	bool "Factory data provider"
 	select ZCBOR

config/nrfconnect/chip-module/Kconfig.defaults

@@ -161,6 +161,11 @@ config BT_DEVICE_NAME_GATT_WRITABLE
 config CHIP_NFC_COMMISSIONING
     default y
 
+# Disable not needed NFC callback to save flash
+config NFC_THREAD_CALLBACK
+    bool
+    default n
+
 config CHIP_OTA_REQUESTOR
     default y
 
@@ -230,6 +235,9 @@ config NET_IF_MCAST_IPV6_ADDR_COUNT
 config NET_SOCKETS_POSIX_NAMES
     default n
 
+config NET_SOCKETS_POLL_MAX
+    default 4
+
 config MBEDTLS_SSL_OUT_CONTENT_LEN
     default 900
 
@@ -247,6 +255,9 @@ config SHELL_STACK_SIZE
 config HEAP_MEM_POOL_SIZE
     default 200000
 
+config CHIP_MALLOC_SYS_HEAP_SIZE
+    default 28672 # 28 kB
+
 endif
 
 
@@ -368,6 +379,18 @@ config PSA_WANT_ALG_OFB
 
 # Disable not used shell modules
 
+config SHELL_WILDCARD
+    default n
+
+config SHELL_VT100_COLORS
+    default n
+
+config SHELL_STATS
+    default n
+
+config KERNEL_SHELL
+    default n
+
 config SENSOR_SHELL
     default n
 
@@ -389,6 +412,18 @@ config CLOCK_CONTROL_NRF_SHELL
 config FLASH_SHELL
     default n
 
+if MPSL_FEM
+
+config MPSL_FEM_NRF21540_RUNTIME_PA_GAIN_CONTROL
+	default y
+
+endif # MPSL_FEM
+
+config OPENTHREAD_DEFAULT_TX_POWER
+	default 20 if MPSL_FEM
+	default 3 if SOC_SERIES_NRF53X && !MPSL_FEM
+	default 8 if SOC_SERIES_NRF52X && !MPSL_FEM
+
 # SoC series related configuration
 
 if SOC_SERIES_NRF52X

config/nrfconnect/chip-module/Kconfig.features

@@ -98,6 +98,18 @@ config MCUMGR_SMP_BT_AUTHEN
 config MCUMGR_BUF_COUNT
 	default 6
 
+config MCUMGR_MGMT_NOTIFICATION_HOOKS
+	bool
+	default y
+
+config MCUMGR_GRP_IMG_UPLOAD_CHECK_HOOK
+	bool
+	default y
+
+config MCUMGR_SMP_COMMAND_STATUS_HOOKS
+	bool
+	default y
+
 # Increase BT MTU and RX buffer for big size DFU messages
 config BT_L2CAP_TX_MTU
 	default 498
@@ -126,4 +138,37 @@ endif # SOC_SERIES_NRF53X
 
 endif # CHIP_DFU_OVER_BT_SMP
 
+config CHIP_WIFI_CONNECTION_RECOVERY_MINIMUM_INTERVAL
+	int "Define the minimum connection recovery time interval in milliseconds"
+	depends on CHIP_WIFI
+	default 500
+	help
+	  Specifies the minimum connection recovery interval (in milliseconds).
+
+config CHIP_WIFI_CONNECTION_RECOVERY_MAXIMUM_INTERVAL
+	int "Define the maximum connection recovery time interval in milliseconds"
+	depends on CHIP_WIFI
+	default 3600000 # 1 hour
+	help
+	  Specifies the maximum connection recovery interval (in milliseconds).
+
+config CHIP_WIFI_CONNECTION_RECOVERY_MAX_RETRIES_NUMBER
+	int "Define the maximum amount of connection recovery occurrences"
+	depends on CHIP_WIFI
+	default 0
+	help
+	  Specifies the maximum number of connection recovery attempts.
+	  If set to 0, no limitation is applied and attempts
+	  to recover the connection are performed indefinitely.
+
+config CHIP_WIFI_CONNECTION_RECOVERY_JITTER
+	int "Define the connection recovery jitter in milliseconds"
+	depends on CHIP_WIFI
+	default 2000
+	help
+	  Specifies the maximum connection recovery jitter interval (in milliseconds).
+	  Once the wait time reaches the current maximum value (defined by CHIP_WIFI_CONNECTION_RECOVERY_MAXIMUM_INTERVAL),
+	  a random jitter interval is added to it to avoid periodicity. The random jitter is selected
+	  within range [-JITTER; +JITTER].
+
 endif # CHIP

config/nrfconnect/chip-module/Kconfig.hci_rpmsg.defaults

@@ -63,10 +63,6 @@ config BT_HCI_RAW_RESERVE
 config BT_BUF_CMD_TX_COUNT
     default 10
 
-# Enable support for Wi-Fi and Bluetooth LE coexistance
-config MPSL_CX
-    default y
-
 config ASSERT
     default y
 

config/nrfconnect/chip-module/make_gn_args.py

@@ -29,7 +29,11 @@
     '-fno-reorder-functions',
     '-ffunction-sections',
     '-fdata-sections',
-    '-g*',
+    '-g',
+    '-g0',
+    '-g1',
+    '-g2',
+    '-g3',
     '-O*',
     '-W*',
 ]

config/zephyr/Kconfig

@@ -394,7 +394,7 @@ config CHIP_MALLOC_SYS_HEAP_OVERRIDE
 
 config CHIP_MALLOC_SYS_HEAP_SIZE
 	int "Heap size used by memory allocator based on Zephyr sys_heap"
-	default 16384 # 16kB
+	default 12288 # 12kB
 	help
 	  Provides a value that controls how much of the device RAM is reserved for
 	  the heap used by the memory allocation functions based on sys_heap from

examples/all-clusters-app/nrfconnect/boards/nrf5340dk_nrf5340_cpuapp.overlay

@@ -14,18 +14,12 @@
  *    limitations under the License.
  */
 
-#include <zephyr/dt-bindings/ipc_service/static_vrings.h>
-
 / {
 	chosen {
 		nordic,pm-ext-flash = &mx25r64;
 	};
 };
 
-&ipc0 {
-    zephyr,priority = <0 PRIO_COOP>;
-};
-
 /* Disable unused peripherals to reduce power consumption */
 &adc {
 	status = "disabled";

examples/all-clusters-minimal-app/nrfconnect/boards/nrf5340dk_nrf5340_cpuapp.overlay

@@ -14,18 +14,12 @@
  *    limitations under the License.
  */
 
-#include <zephyr/dt-bindings/ipc_service/static_vrings.h>
-
 / {
 	chosen {
 		nordic,pm-ext-flash = &mx25r64;
 	};
 };
 
-&ipc0 {
-    zephyr,priority = <0 PRIO_COOP>;
-};
-
 /* Disable unused peripherals to reduce power consumption */
 &adc {
 	status = "disabled";

examples/light-switch-app/nrfconnect/boards/nrf5340dk_nrf5340_cpuapp.overlay

@@ -14,14 +14,8 @@
  *    limitations under the License.
  */
 
-#include <zephyr/dt-bindings/ipc_service/static_vrings.h>
-
 / {
 	chosen {
 		nordic,pm-ext-flash = &mx25r64;
 	};
 };
-
-&ipc0 {
-    zephyr,priority = <0 PRIO_COOP>;
-};

examples/lighting-app/nrfconnect/boards/nrf5340dk_nrf5340_cpuapp.overlay

@@ -14,8 +14,6 @@
  *    limitations under the License.
  */
 
-#include <zephyr/dt-bindings/ipc_service/static_vrings.h>
-
 / {
 	chosen {
 		nordic,pm-ext-flash = &mx25r64;
@@ -38,10 +36,6 @@
 	};
 };
 
-&ipc0 {
-    zephyr,priority = <0 PRIO_COOP>;
-};
-
 &pwm0 {
 	pinctrl-0 = <&pwm0_default_alt>;
 	pinctrl-1 = <&pwm0_sleep_alt>;

examples/lock-app/nrfconnect/boards/nrf5340dk_nrf5340_cpuapp.overlay

@@ -14,18 +14,12 @@
  *    limitations under the License.
  */
 
-#include <zephyr/dt-bindings/ipc_service/static_vrings.h>
-
 / {
 	chosen {
 		nordic,pm-ext-flash = &mx25r64;
 	};
 };
 
-&ipc0 {
-    zephyr,priority = <0 PRIO_COOP>;
-};
-
 /* Disable unused peripherals to reduce power consumption */
 &adc {
 	status = "disabled";

examples/platform/nrfconnect/util/DFUOverSMP.cpp

@@ -27,11 +27,9 @@
 
 #include <lib/support/logging/CHIPLogging.h>
 
-#include <img_mgmt/img_mgmt.h>
-#include <os_mgmt/os_mgmt.h>
-#include <zephyr/bluetooth/bluetooth.h>
 #include <zephyr/dfu/mcuboot.h>
-#include <zephyr/mgmt/mcumgr/smp_bt.h>
+#include <zephyr/mgmt/mcumgr/mgmt/callbacks.h>
+#include <zephyr/mgmt/mcumgr/mgmt/mgmt.h>
 
 using namespace ::chip;
 using namespace ::chip::DeviceLayer;
@@ -42,6 +40,43 @@ constexpr uint16_t kAdvertisingIntervalMin = 400;
 constexpr uint16_t kAdvertisingIntervalMax = 500;
 constexpr uint8_t kAdvertisingFlags        = BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR;
 
+namespace {
+int32_t UploadConfirmHandler(uint32_t event, int32_t rc, bool * abort_more, void * data, size_t data_size)
+{
+    const img_mgmt_upload_check & imgData = *static_cast<img_mgmt_upload_check *>(data);
+    IgnoreUnusedVariable(imgData);
+
+    ChipLogProgress(SoftwareUpdate, "DFU over SMP progress: %u/%u B of image %u", static_cast<unsigned>(imgData.req->off),
+                    static_cast<unsigned>(imgData.action->size), static_cast<unsigned>(imgData.req->image));
+
+    return MGMT_ERR_EOK;
+}
+
+int32_t CommandHandler(uint32_t event, int32_t rc, bool * abort_more, void * data, size_t data_size)
+{
+    if (event == MGMT_EVT_OP_CMD_RECV)
+    {
+        GetFlashHandler().DoAction(ExternalFlashManager::Action::WAKE_UP);
+    }
+    else if (event == MGMT_EVT_OP_CMD_DONE)
+    {
+        GetFlashHandler().DoAction(ExternalFlashManager::Action::SLEEP);
+    }
+
+    return MGMT_ERR_EOK;
+}
+
+mgmt_callback sUploadCallback = {
+    .callback = UploadConfirmHandler,
+    .event_id = MGMT_EVT_OP_IMG_MGMT_DFU_CHUNK,
+};
+
+mgmt_callback sCommandCallback = {
+    .callback = CommandHandler,
+    .event_id = (MGMT_EVT_OP_CMD_RECV | MGMT_EVT_OP_CMD_DONE),
+};
+} // namespace
+
 DFUOverSMP DFUOverSMP::sDFUOverSMP;
 
 void DFUOverSMP::Init()
@@ -68,28 +103,8 @@ void DFUOverSMP::Init()
         }
     };
 
-    os_mgmt_register_group();
-    img_mgmt_register_group();
-
-    img_mgmt_set_upload_cb([](const img_mgmt_upload_req req, const img_mgmt_upload_action action) {
-        ChipLogProgress(SoftwareUpdate, "DFU over SMP progress: %u/%u B of image %u", static_cast<unsigned>(req.off),
-                        static_cast<unsigned>(action.size), static_cast<unsigned>(req.image));
-        return 0;
-    });
-
-    mgmt_register_evt_cb([](uint8_t opcode, uint16_t group, uint8_t id, void * arg) {
-        switch (opcode)
-        {
-        case MGMT_EVT_OP_CMD_RECV:
-            GetFlashHandler().DoAction(ExternalFlashManager::Action::WAKE_UP);
-            break;
-        case MGMT_EVT_OP_CMD_DONE:
-            GetFlashHandler().DoAction(ExternalFlashManager::Action::SLEEP);
-            break;
-        default:
-            break;
-        }
-    });
+    mgmt_callback_register(&sUploadCallback);
+    mgmt_callback_register(&sCommandCallback);
 }
 
 void DFUOverSMP::ConfirmNewImage()
@@ -111,7 +126,6 @@ void DFUOverSMP::ConfirmNewImage()
 void DFUOverSMP::StartServer()
 {
     VerifyOrReturn(!mIsStarted, ChipLogProgress(SoftwareUpdate, "DFU over SMP was already started"));
-    smp_bt_register();
 
     // Synchronize access to the advertising arbiter that normally runs on the CHIP thread.
     PlatformMgr().LockChipStack();

examples/pump-app/nrfconnect/boards/nrf5340dk_nrf5340_cpuapp.overlay

@@ -14,15 +14,8 @@
  *    limitations under the License.
  */
 
-#include <zephyr/dt-bindings/ipc_service/static_vrings.h>
-
 / {
 	chosen {
 		nordic,pm-ext-flash = &mx25r64;
 	};
 };
-
-&ipc0 {
-    zephyr,priority = <0 PRIO_COOP>;
-};
-