[POC] Azure Integration in RIOT OS #20223

tanvirBsmrstu · 2024-01-03T10:08:45Z

Contribution description (3 modules, one package, one application)

Zero-touch device provisioning with Device Provisioning Service using X.509 certificate.
Cloud-to-Device and Device-to-Cloud messaging with Azure IoT Hub.
Receiving and parsing Direct method invocation from Azure IoT Hub.
Receiving and parsing Device Twins update notifications from Azure IoT Hub.

Azure IoT SDK for embedded C is integrated using a new RIOT package
"gnrc_wolfssl_tls" module uses wolfssl and gnrc to achieve TLS
"mqtts_riot_iface" module uses paho mqtt package and gnrc_wolfssl_tls module to achieve MQTTS
"az_riot_pnp_iface" module is a wrapper on top of MQTTS and azure SDK to provide a Plug-and-play flavour.
Scripts are provided to facilitate generating self signed certificates.
A demo application /examples/az_pnp_demo is provided with a reach Readme.md file.

Testing procedure

A demo application /examples/az_pnp_demo is provided with a reach Readme.md file.
Please read carefully the Readme

Current Limitations

DNS is not integrated yet, please see the readme and use IP at the mentioned place for testing.
Error handling has to improve a lot.

[I will close/delete the other pull request([POC] Azure Integration in RIOT OS #20222) so that the code review is clean and easier ]

Thanks,
Tanvir Hasan

authored-by: tanvirBsmrstu tanvir.bsmrstu@gmail.com

authored-by: tanvirBsmrstu <tanvir.bsmrstu@gmail.com.com>

benpicco

This is really useful, thank you!
Can you please split this into different commits for the different packages and the application and modules?

That should make this easier to review - static tests also already has some comments.

benpicco · 2024-01-04T12:53:15Z

examples/azure_pnp_demo/Makefile

+USEMODULE += sock_dns
+USEMODULE += auto_init_sock_dns
+# USEMODULE += gnrc_ipv6_nib_dns 
+CFLAGS += -DCONFIG_AUTO_INIT_SOCK_DNS_SERVER_ADDR=\"fd12:dead:beef::1\"


What's wrong with the default? (I think that's Google's DNS)
fd12:dead:beef::1 is not a global address, so that won't work.

Or do we need DNS64 here?

I think we do need DNS64 for now since - according to Microsoft - Azure IoT do not support IPv6 right now.

We need DNS64 here. For example, RIOT has "global.azure-devices-provisioning.net" the address of Azure DPS which supports only IPv4. Since the POC is using gnrc, we need a IPv6 version of the DPS address. I setup a nat64 on my linux machine. When I use dig global.azure-devices-provisioning.net +short AAAA @2001:4860:4860::64 it returns

id-prod-global-endpoint.trafficmanager.net. idsu-prod-dewc-1-su-az.germanywestcentral.cloudapp.azure.com. 64:ff9b::3374:91ca

Which RIOT can not (or I don't know how to) parse using sock_dns_query function. Any help or a commit here would be much more appreciated. This you can find in gnrc_wolfssl_tls.c as a commented todo option and also as warning in the Readme.

But could you also use a public server like 2001:67c:2b0::6 so users don’t have to do the setup first?

My current setup with nat64 and nib RIOT application can only access address prefixed with 64:ff9b::/96, any other public IP is not reachable since I have created the tap interface without uplink.
The packet got lost at the tapbr0. I might have missed the routing configuration, but it did not follow the default routing for some reason.
I need some help here due to my lack of expertise in networking.
Therefore, I used google's public dns64 in my linux machine. (2001:4860:4860::64)

If I use -u, for example, -u eth0, I miss the nat64 interface.

But then you can use a public DNS64/NAT64 service and don't have to set up anything yourself.

Doing so, I get the following response with multiple answers, and the last one containing the IP(highlighted), but dns_msg_parse_reply can not parse the IP from it and I get -74(BadMsg) error returned. Please check the Wireshark screenshot below.
Note: in the make file I had to do the following

USEMODULE += sock_dns USEMODULE += auto_init_sock_dns USEMODULE += gnrc_ipv6_nib_dns CFLAGS += -DCONFIG_AUTO_INIT_SOCK_DNS_SERVER_ADDR=\"2001:67c:2b0::6\" CFLAGS += -DCONFIG_DNS_MSG_LEN=256

Uh looks like this uncovers a bug in our DNS parsing code - it fails here.

Unfortunately I have no idea what that while loop is supposed to do - maybe @miri64 has an idea?

#20857 provides a fix

benpicco · 2024-01-04T12:54:11Z

examples/azure_pnp_demo/Makefile

+CFLAGS += -Wno-error=unused-value
+CFLAGS += -Wno-error=unused-variable
+
+CFLAGS += -Wno-strict-prototypes


This should not be set by an application, if the warnings are in your code you should fix them.
If they are in the package, add those flags to the package.

The warnings are in the package, I have added these flags in the package itself and forgot to clean the application.
Thanks, I will take care of this.

benpicco · 2024-01-04T12:54:48Z

examples/azure_pnp_demo/Makefile

+
+include $(RIOTBASE)/Makefile.include
+
+ifneq (,$(filter arch_esp,$(FEATURES_USED)))


You can set those unconditionally so users of atwinc15x0 also benefit.

I have started with the paho mqtt example, I kept these as it is. I think it would be nice if I remove all the wifi related setup, since this POC can only run on Native due to some limitations (how I read the certificate from memory using dynamic memory allocation) and it does not use any wifi feature. I missed cleaning this make file.

benpicco · 2024-01-04T12:55:13Z

examples/azure_pnp_demo/Makefile.board.dep

+ifeq ($(BOARD),native)
+  USEMODULE += netdev_default


Why only native?

Thanks, I will remove the checks.

benpicco · 2024-01-04T13:40:52Z

modules/gnrc_wolfssl_tls/gnrc_wolfssl_tls.c

+// #include <net/sock/tcp.h>
+#include "ztimer.h"
+#include <stdio.h>
+


There isn't anything GNRC specific here, so why the name?

GNRC is a dependency for this module and specified in the .dep file. Any better naming suggestion is really appreciated.

sock_wolfssl_tls since it uses the sock API

benpicco · 2024-01-04T13:47:30Z

examples/azure_pnp_demo/README.md

+If Azure portal setup is not done yet, please see the [Azure Portal README](./docs/Readme.md).
+
+### NAT64 and DNS64 configuration
+Here is a nice document by Professor Oliver Hahm [NAT64 Configuration](https://teaching.dahahm.de/riot/2023/09/29/RIOT_GNRC_ipv4.html)


Instead of having the user go through the trouble of setting this all up themselves I'd just direct them to a public DNS64/NAT64 service and give this as optional information if they really want to set up their own.

But then we should also add a test that checks whether the public service is still available.

Ideally we could run such a service on the community server, then we could make it the default in RIOT

OlegHahm · 2024-01-04T23:48:04Z

I guess the new modules should go into different directories in sys/net.

fabian18 · 2024-01-07T00:16:48Z

I think the mqtts module should exist in pkg/paho-mqtt/contrib and the tls module should exist in pkg/wolfssl/sock_tls.

WolfSSL already provides something for us in wolfio.h which seems to be something like your TLSContext.

#ifdef WOLFSSL_GNRC
    #include <sock_types.h>
    #include <net/gnrc.h>
    #include <net/af.h>
    #include <net/sock.h>
    #include <net/gnrc/tcp.h>
    #include <net/gnrc/udp.h>

    struct gnrc_wolfssl_ctx {
        union socket_connector {
        #ifdef MODULE_SOCK_TCP
            sock_tcp_t tcp;
        #endif
            sock_udp_t udp;
        } conn;
        WOLFSSL_CTX *ctx;
        WOLFSSL *ssl;

        int closing;
        struct _sock_tl_ep peer_addr;
    };

    typedef struct gnrc_wolfssl_ctx sock_tls_t;

    WOLFSSL_LOCAL int GNRC_ReceiveFrom(WOLFSSL* ssl, char* buf, int sz,
                                     void* ctx);
    WOLFSSL_LOCAL int GNRC_SendTo(WOLFSSL* ssl, char* buf, int sz, void* ctx);

#endif

tanvirBsmrstu · 2024-01-07T03:21:37Z

Hi @fabian18,
Thank you so much for your time and comments. There are so many things to improve. I would love to have more comments and ideas on how this work can be refactored.

I have seen the structure in the library, but it, by default, includes udp. That's why I did not use that. There might be some scenario where UDP is not needed.
Secondly, the implementation of GNRC_** functions are specific to UDP again. I thought changing in the third party code would not be a good idea, that's why I wrote my custom implementation.

Azure Integration in RIOT OS

4a88e2a

authored-by: tanvirBsmrstu <tanvir.bsmrstu@gmail.com.com>

tanvirBsmrstu requested a review from jia200x as a code owner January 3, 2024 10:08

github-actions bot added Area: network Area: Networking Area: doc Area: Documentation Area: pkg Area: External package ports Area: examples Area: Example Applications labels Jan 3, 2024

tanvirBsmrstu mentioned this pull request Jan 3, 2024

[POC] Azure Integration in RIOT OS #20222

Closed

benpicco requested a review from OlegHahm January 3, 2024 12:18

benpicco reviewed Jan 4, 2024

View reviewed changes

tanvirBsmrstu mentioned this pull request Jan 14, 2024

[POC] Azure integration part 1 (TLS) #20254

Open

benpicco mentioned this pull request Feb 7, 2024

dns_msg_parse_reply() fails for response from DNS64 service #20355

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[POC] Azure Integration in RIOT OS #20223

[POC] Azure Integration in RIOT OS #20223

tanvirBsmrstu commented Jan 3, 2024 •

edited

Loading

benpicco left a comment

benpicco Jan 4, 2024

OlegHahm Jan 4, 2024

tanvirBsmrstu Jan 4, 2024

benpicco Jan 5, 2024

tanvirBsmrstu Jan 7, 2024

tanvirBsmrstu Jan 7, 2024

benpicco Jan 7, 2024

tanvirBsmrstu Jan 8, 2024

benpicco Jan 8, 2024 •

edited

Loading

benpicco Sep 9, 2024

benpicco Jan 4, 2024

tanvirBsmrstu Jan 4, 2024

benpicco Jan 4, 2024

tanvirBsmrstu Jan 4, 2024

benpicco Jan 4, 2024

tanvirBsmrstu Jan 5, 2024

benpicco Jan 4, 2024

tanvirBsmrstu Jan 5, 2024

benpicco Jan 5, 2024

benpicco Jan 4, 2024

OlegHahm Jan 4, 2024

benpicco Jan 5, 2024

OlegHahm commented Jan 4, 2024

fabian18 commented Jan 7, 2024 •

edited

Loading

tanvirBsmrstu commented Jan 7, 2024


		include $(RIOTBASE)/Makefile.include

		ifneq (,$(filter arch_esp,$(FEATURES_USED)))

[POC] Azure Integration in RIOT OS #20223

Are you sure you want to change the base?

[POC] Azure Integration in RIOT OS #20223

Conversation

tanvirBsmrstu commented Jan 3, 2024 • edited Loading

Contribution description (3 modules, one package, one application)

Testing procedure

Current Limitations

benpicco left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benpicco Jan 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OlegHahm commented Jan 4, 2024

fabian18 commented Jan 7, 2024 • edited Loading

tanvirBsmrstu commented Jan 7, 2024

tanvirBsmrstu commented Jan 3, 2024 •

edited

Loading

benpicco Jan 8, 2024 •

edited

Loading

fabian18 commented Jan 7, 2024 •

edited

Loading