Examples broken at runtime.

[AnthonyGrondin]

I've tried to run the following examples after a clean git clone and they all fail due to various issues:

• sync_server
• sync_client
• async_client

EDIT: After further testing, the problem seems to occur on Xtensa targets. It had issues on esp32s3 but with esp32c3 everything is running without any failure.

Here's the output from running async_client:

Waiting to get IP address...
Got IP: 192.168.69.165/24
connecting...
Start tls connect


Exception occured 'InstrProhibited'
Context
PC=0x00000000       PS=0x00060a10
0x00000000 - XT_STK_PC
    at ??:??
0x00060a10 - PS_WOE
    at ??:??
A0=0x8037b2eb       A1=0x3fc9b900       A2=0x00000001       A3=0x00000000       A4=0x3fcbbfe0
0x8037b2eb - _rtc_fast_data_end
    at ??:??
0x3fc9b900 - _ZN8esp_wifi7preempt10TASK_STACK17hc849bc2d8dfdc35eE.llvm.8558853931319064652
    at ??:??
0x00000001 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x3fcbbfe0 - g_pm
    at ??:??
A5=0x3fcbbfe0       A6=0x3fcef940       A7=0x1d800160       A8=0x820a44aa       A9=0x00000000
0x3fcbbfe0 - g_pm
    at ??:??
0x3fcef940 - g_osi_funcs_p
    at ??:??
0x1d800160 - PS_WOE
    at ??:??
0x820a44aa - _rtc_fast_data_end
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
A10=0x00019000      A11=0x00000001      A12=0x3fc9b900      A13=0xeeeeeeef      A14=0x00000000
0x00019000 - RESERVE_ICACHE
    at ??:??
0x00000001 - XT_STK_PC
    at ??:??
0x3fc9b900 - _ZN8esp_wifi7preempt10TASK_STACK17hc849bc2d8dfdc35eE.llvm.8558853931319064652
    at ??:??
0xeeeeeeef - _rtc_fast_data_end
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
A15=0x00000006
0x00000006 - XT_STK_PS
    at ??:??
SAR=00000004
EXCCAUSE=0x00000014 EXCVADDR=0x00000000
0x00000014 - XT_STK_A3
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
LBEG=0x00000000     LEND=0x00000000     LCOUNT=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
THREADPTR=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
SCOMPARE1=0x00000001
0x00000001 - XT_STK_PC
    at ??:??
BR=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
ACCLO=0x00000000    ACCHI=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
M0=0x00000000       M1=0x00000000       M2=0x00000000       M3=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F64R_LO=0x00000000  F64R_HI=0x00000000  F64S=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
FCR=0x00000000      FSR=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F0=0x00000000       F1=0x00000000       F2=0x00000000       F3=0x00000000       F4=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F5=0x00000000       F6=0x00000000       F7=0x00000000       F8=0x00000000       F9=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F10=0x00000000      F11=0x00000000      F12=0x00000000      F13=0x00000000      F14=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F15=0x00000000
0x00000000 - XT_STK_PC
    at ??:??

0x4003d024
0x4003d024 - rom_rx_gain_force
    at ??:??
0x4037a49c
0x4037a49c - ppTask
    at ??:??
0x42021f84
0x42021f84 - core::sync::atomic::atomic_load
    at /home/anthony/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:3184
0x40000000
0x40000000 - ets_rom_layout_p
    at ??:??

[bjoernQ]

I cannot reproduce that exception on my side on ESP32-S3

I do see some TLS errors sometimes - but sometimes it just works

[AnthonyGrondin]

I'm on toolchain rustc 1.74.0-nightly (bd6bb31df 2023-11-15) (1.74.0.0)

Here's the steps to reproduce on my side. It's weird because it used to work at some point.

1. Clone esp-mbedtls in a new directory

git clone https://github.com/esp-rs/esp-mbedtls

2. Run sync_server on esp32s3

SSID=<SSID> PASSWORD=<SSID> cargo +esp run --example sync_server --target xtensa-esp32s3-none-elf -F esp32s3,esp-wifi/wifi-logs

3. Panics on boot

INFO - esp-wifi configuration Config { rx_queue_size: 5, tx_queue_size: 3, static_rx_buf_num: 10, dynamic_rx_buf_num: 32, static_tx_buf_num: 0, dynamic_tx_buf_num: 32, ampdu_rx_enable: 0, ampdu_tx_enable: 0, amsdu_tx_enable: 0, rx_ba_win: 6, max_burst_size: 1, country_code: "CN", country_code_operating_class: 0, mtu: 1492, heap_size: 112640, tick_rate_hz: 50, listen_interval: 3, beacon_timeout: 6, ap_beacon_timeout: 300, failure_retry_cnt: 1, scan_method: 0 }


Exception occured 'LoadProhibited'
Context
PC=0x42069cf4       PS=0x00060930
0x42069cf4 - compiler_builtins::mem::impls::c_string_length
    at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.101/src/mem/impls.rs:286
0x00060930 - PS_WOE
    at ??:??
A0=0x8206abc1       A1=0x3fcd9e40       A2=0xdeadbeaf       A3=0x000000ff       A4=0x3fc8f886
0x8206abc1 - _rtc_fast_bss_start
    at ??:??
0x3fcd9e40 - _stack_end
    at ??:??
0xdeadbeaf - _rtc_fast_bss_start
    at ??:??
0x000000ff - XT_STK_TMP
    at ??:??
0x3fc8f886 - _ZN8esp_wifi14common_adapter12RADIO_CLOCKS17haa8afa80382b6914E
    at ??:??
A5=0x00000001       A6=0x00000001       A7=0x4201bbbc       A8=0xdeadbeaf       A9=0x00000001
0x00000001 - XT_STK_PC
    at ??:??
0x00000001 - XT_STK_PC
    at ??:??
0x4201bbbc - esp_wifi::preempt::preempt::task_create
    at ~/.cargo/git/checkouts/esp-wifi-836f3b2af57fa847/5166089/esp-wifi/src/preempt/preempt_xtensa.rs:69
0xdeadbeaf - _rtc_fast_bss_start
    at ??:??
0x00000001 - XT_STK_PC
    at ??:??
A10=0x3fc8dbfc      A11=0x00000001      A12=0x00000001      A13=0x00080002      A14=0x00380000
0x3fc8dbfc - _ZN14esp_hal_common21critical_section_impl9multicore14MULTICORE_LOCK17h0e1e74c9ce5dc8b1E
    at ??:??
0x00000001 - XT_STK_PC
    at ??:??
0x00000001 - XT_STK_PC
    at ??:??
0x00080002 - PS_WOE
    at ??:??
0x00380000 - PS_WOE
    at ??:??
A15=0x3fc8daa0
0x3fc8daa0 - _ZN8esp_wifi4wifi8G_CONFIG17h435367d823eeb9a0E
    at ??:??
SAR=00000010
EXCCAUSE=0x0000001c EXCVADDR=0xdeadbeaf
0x0000001c - XT_STK_A5
    at ??:??
0xdeadbeaf - _rtc_fast_bss_start
    at ??:??
LBEG=0x3c100b51     LEND=0x00000001     LCOUNT=0x00000004
0x3c100b51 - str.2
    at ??:??
0x00000001 - XT_STK_PC
    at ??:??
0x00000004 - XT_STK_PS
    at ??:??
THREADPTR=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
SCOMPARE1=0x00000001
0x00000001 - XT_STK_PC
    at ??:??
BR=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
ACCLO=0x00000000    ACCHI=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
M0=0x00000000       M1=0x00000000       M2=0x00000000       M3=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F64R_LO=0x86a00000  F64R_HI=0x00000001  F64S=0x00000000
0x86a00000 - _rtc_fast_bss_start
    at ??:??
0x00000001 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
FCR=0x00000000      FSR=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F0=0x00000000       F1=0x00000000       F2=0x00000000       F3=0x00000000       F4=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F5=0x00000000       F6=0x00000000       F7=0x00000000       F8=0x00000000       F9=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F10=0x00000000      F11=0x00000000      F12=0x00000000      F13=0x00000000      F14=0x00000000
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
0x00000000 - XT_STK_PC
    at ??:??
F15=0x00000000
0x00000000 - XT_STK_PC
    at ??:??

0x4201a75b
0x4201a75b - core::ffi::c_str::CStr::from_ptr
    at ~/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/ffi/c_str.rs:273
0x4201a308
0x4201a308 - esp_wifi::compat::syslog::syslog
    at ??:??
0x4201ae47
0x4201ae47 - coexist_printf
    at ~/.cargo/git/checkouts/esp-wifi-836f3b2af57fa847/5166089/esp-wifi/src/common_adapter/mod.rs:252
0x42092431
0x42092431 - wdev_funcs_init
    at ??:??
0x4206db5e
0x4206db5e - esp_wifi_init_internal
    at ??:??
0x42016fad
0x42016fad - esp_wifi::wifi::wifi_init
    at ~/.cargo/git/checkouts/esp-wifi-836f3b2af57fa847/5166089/esp-wifi/src/wifi/mod.rs:624
0x4201b9e5
0x4201b9e5 - esp_wifi::initialize
    at ~/.cargo/git/checkouts/esp-wifi-836f3b2af57fa847/5166089/esp-wifi/src/lib.rs:298
0x42008520
0x42008520 - sync_server::__xtensa_lx_rt_main
    at /tmp/esp-mbedtls/examples/sync_server.rs:55
0x42057017
0x42057017 - Reset
    at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/xtensa-lx-rt-0.16.0/src/lib.rs:70
0x403788a3
0x403788a3 - ESP32Reset
    at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/esp32s3-hal-0.13.0/src/lib.rs:
    ```

[bjoernQ]

Mhhh seems like in our 1.74 build the c-variadics are broken, again. e.g. for the first string it tries to get it e.g. reads the address 0xdeadbeaf ...

Works fine with 1.73.0.1
When not enabling wifi-logs it also works on 1.74

cc @MabezDev

[MabezDev]

I'm so sorry that's completely my bad. I forgot to add the commit to 1.74, because I was looking at the patch set from 1.73.0.0, instead of 1.73.0.1 🤦. I'll fix that and get a release out soon.

[AnthonyGrondin]

Alright, I'm using rustc 1.73.0-nightly (2fba83302 2023-10-18) (1.73.0.1) to fix the issue mentionned above.

Examples run without any issue on the esp32c3.
I'm facing issues running them on the esp32s3.

Now, I'm only facing runtime issues during a connection. When I run the sync_server example like follow:

SSID=<SSID> PASSWORD=<PASS> cargo +esp run --release --example sync_server --target xtensa-esp32s3-none-elf -F esp32s3

I get the following error during a connection:

INFO - esp-wifi configuration Config { rx_queue_size: 10, tx_queue_size: 10, static_rx_buf_num: 10, dynamic_rx_buf_num: 32, static_tx_buf_num: 0, dynamic_tx_buf_num: 32, ampdu_rx_enable: 0, ampdu_tx_enable: 0, amsdu_tx_enable: 0, rx_ba_win: 6, max_burst_size: 1, country_code: "CN", country_code_operating_class: 0, mtu: 1492, heap_size: 112640, tick_rate_hz: 100, listen_interval: 3, beacon_timeout: 6, ap_beacon_timeout: 300, failure_retry_cnt: 1, scan_method: 0 }
Call wifi_connect
Wait to get connected
Wait to get an ip address
Got ip Ok(IpInfo { ip: 192.168.69.165, subnet: Subnet { gateway: 192.168.69.1, mask: Mask(24) }, dns: Some(192.168.69.16), secondary_dns: None })
We are connected!
Point your browser to https://192.168.69.165/
New connection


!! A panic occured in 'examples/sync_server.rs', at line 198, column 21

PanicInfo {
    payload: Any { .. },
    message: Some(
        MbedTlsError(-17280),
    ),
    location: Location {
        file: "examples/sync_server.rs",
        line: 198,
        col: 21,
    },
    can_unwind: true,
}

The error code is for MBEDTLS_ERR_RSA_VERIFY_FAILED. The same example works on the C3, so I don't see why it fails for the S3

[bjoernQ]

I can reproduce it. I also tried different versions of the toolchain and very few different (recent) commits of esp-mbedtls
Also, different opt-levels don't seem to make a difference (other than timing out with level 1)

Oh wait ... seems like a484bba6375d23c4e47cbf342ca18afda44482b1 works for me (but needs a local esp-wifi rev 68dc11bbb2c0efa29c4acbbf134d6f142441065e and changes in Software1 and yield_task)

Aaaaand I can also get the current main to work when copying the pre-compiled libs from a484bba6375d23c4e47cbf342ca18afda44482b1

If it's reproducible on your side then there is something wrong with how mbedtls is built for Xtensa while it's fine for RISC-V

[AnthonyGrondin]

I've tested with the pre-compiled libraries from before we refactored the build scripts to use xtasks and it works.

The issue effectively seems to be from the way we build mbedtls for Xtensa, introduced in #15

[AnthonyGrondin]

I'm going to close this because it has been fixed in #24 after recompiling using clang