stdio_ethos: infinite shell loop #17972
Comments
fjmolinas
added
the
Type: bug
|
Brainstorming for solutions: I am somewhat wondering how #17265 could have caused this... the shell and its prompt is controlled by the |
This does not seem to be the case in general. The changes in #17265 worked fine on the IoT-LAB M3 (and at least @chrysn and @benpicco do not seem to encounter this issue either). Which boards were you using for that. |
I also saw it on |
benpicco
changed the title
|
As I am unable to reliably reproduce this on boards I have at hand at the moment (or rather not at all), I think some remote debugging is required... From the output you provided, it looks to me that the shell is running into this condition: Lines 490 to 491 in 37db22e This is caused by either EOT or EOF sent by Lines 430 to 434 in 37db22e So if this bug is indeed Line 70 in 37db22e Could you maybe add a check like this: if ((tmp > 0) && ((*ptr == 0xff) || (*ptr == 0x04))) {
/* do something (e.g. toggle LED) */
}Then at least we would see if the issue is from here. |
|
Can reproduce with |
|
I have been running into this problem lately relative often. If this helps, I've seen it happening under this conditions:
In my use case I have an nrf52850dk gateway running ethos and a DSME interface. I forward all DSME traffic through ethos but at the same time I print some MAC metrics when scheduling/transmitting a frame. In order to avoid the issue, I need to set some timers between commands. I might be wrong, but I think I've seen these kind of issues even before #17265, but since most of the times ethos run on a samr21-xpro (or equivalent), we probably didn't catch it before. |
For me, just hitting the on-board reset button also just stops it from happening. Digging a bit through the code, it seems |
|
Was able to debug 😕 |
|
Now I suddenly can't reproduce anymore 😲 |
back in business... appearently |
Same here! I usually trigger a |
|
But this seems to be more of a |
|
Using picolibc also seems to work as a workaround: BUILD_IN_DOCKER=1 PROGRAMMER=jlink BOARD=feather-nrf52840 PORT=/dev/ttyUSB0 FEATURES_REQUIRED=picolibc make -C examples/gnrc_border_router flash -j term
make: Entering directory '/home/mlenders/Repositories/RIOT-OS/RIOT/examples/gnrc_border_router'
Launching build container using image "riot/riotbuild:latest".
docker run --rm --tty --user $(id -u) -v '/usr/share/zoneinfo/Europe/Berlin:/etc/localtime:ro' -v '/home/mlenders/Repositories/RIOT-OS/RIOT:/data/riotbuild/riotbase:delegated' -v '/home/mlenders/.cargo/registry:/data/riotbuild/.cargo/registry:delegated' -v '/home/mlenders/.cargo/git:/data/riotbuild/.cargo/git:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -e 'BOARD=feather-nrf52840' -e 'PROGRAMMER=jlink' -e 'DISABLE_MODULE=' -e 'DEFAULT_MODULE=' -e 'FEATURES_REQUIRED=picolibc' -e 'FEATURES_BLACKLIST=' -e 'FEATURES_OPTIONAL=' -e 'USEMODULE=auto_init_gnrc_netif gnrc_icmpv6_echo gnrc_sixlowpan_border_router_default gnrc_uhcpc netdev_default ps shell shell_commands' -e 'USEPKG=' -w '/data/riotbuild/riotbase/examples/gnrc_border_router/' 'riot/riotbuild:latest' make -j
make -C benchmark_udp
make -C ethos
make -C uhcpd
make -C sliptty
make -C zep_dispatch
make[2]: Nothing to be done for 'all'.
make[2]: Nothing to be done for 'all'.
cc -g -O3 -Wall -Wextra -I/home/mlenders/Repositories/RIOT-OS/RIOT/core/lib/include -I/home/mlenders/Repositories/RIOT-OS/RIOT/cpu/native/include -I/home/mlenders/Repositories/RIOT-OS/RIOT/drivers/include -I/home/mlenders/Repositories/RIOT-OS/RIOT/sys/include -DNDEBUG main.c topology.c zep_parser.c /home/mlenders/Repositories/RIOT-OS/RIOT/sys/net/link_layer/ieee802154/ieee802154.c /home/mlenders/Repositories/RIOT-OS/RIOT/sys/fmt/fmt.c /home/mlenders/Repositories/RIOT-OS/RIOT/sys/net/link_layer/l2util/l2util.c -o bin/zep_dispatch
make[2]: Nothing to be done for 'all'.
make[2]: Nothing to be done for 'all'.
Building application "gnrc_border_router" for "feather-nrf52840" with MCU "nrf52".
"make" -C /data/riotbuild/riotbase/boards/common/init
"make" -C /data/riotbuild/riotbase/boards/feather-nrf52840
"make" -C /data/riotbuild/riotbase/core
"make" -C /data/riotbuild/riotbase/core/lib
"make" -C /data/riotbuild/riotbase/cpu/nrf52
"make" -C /data/riotbuild/riotbase/drivers
"make" -C /data/riotbuild/riotbase/sys
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common
"make" -C /data/riotbuild/riotbase/drivers/ethos
"make" -C /data/riotbuild/riotbase/drivers/netdev
"make" -C /data/riotbuild/riotbase/cpu/nrf52/periph
"make" -C /data/riotbuild/riotbase/drivers/netdev_ieee802154_submac
"make" -C /data/riotbuild/riotbase/sys/auto_init
"make" -C /data/riotbuild/riotbase/sys/div
"make" -C /data/riotbuild/riotbase/cpu/nrf52/radio/nrf802154
"make" -C /data/riotbuild/riotbase/drivers/periph_common
"make" -C /data/riotbuild/riotbase/cpu/nrf52/vectors
"make" -C /data/riotbuild/riotbase/sys/event
"make" -C /data/riotbuild/riotbase/sys/evtimer
"make" -C /data/riotbuild/riotbase/cpu/nrf5x_common
"make" -C /data/riotbuild/riotbase/sys/fmt
"make" -C /data/riotbuild/riotbase/sys/frac
"make" -C /data/riotbuild/riotbase/sys/iolist
"make" -C /data/riotbuild/riotbase/cpu/nrf5x_common/periph
"make" -C /data/riotbuild/riotbase/sys/isrpipe
"make" -C /data/riotbuild/riotbase/sys/luid
"make" -C /data/riotbuild/riotbase/sys/malloc_thread_safe
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common/periph
"make" -C /data/riotbuild/riotbase/sys/net/application_layer/uhcp
"make" -C /data/riotbuild/riotbase/sys/net/crosslayer/inet_csum
"make" -C /data/riotbuild/riotbase/sys/net/gnrc
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/application_layer/uhcpc
"make" -C /data/riotbuild/riotbase/sys/net/link_layer/eui_provider
"make" -C /data/riotbuild/riotbase/sys/net/link_layer/ieee802154
"make" -C /data/riotbuild/riotbase/sys/net/link_layer/l2util
"make" -C /data/riotbuild/riotbase/sys/net/netif
"make" -C /data/riotbuild/riotbase/sys/net/netutils
"make" -C /data/riotbuild/riotbase/sys/net/network_layer/icmpv6
"make" -C /data/riotbuild/riotbase/sys/net/network_layer/ipv6/addr
"make" -C /data/riotbuild/riotbase/sys/net/network_layer/ipv6/hdr
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netapi
"make" -C /data/riotbuild/riotbase/sys/net/network_layer/sixlowpan
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netreg
"make" -C /data/riotbuild/riotbase/sys/net/transport_layer/udp
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/icmpv6
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/icmpv6/echo
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6
"make" -C /data/riotbuild/riotbase/sys/picolibc_syscalls_default
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6/hdr
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6/nib
"make" -C /data/riotbuild/riotbase/sys/posix/inet
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ndp
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/ctx
"make" -C /data/riotbuild/riotbase/sys/ps
"make" -C /data/riotbuild/riotbase/sys/random
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/frag
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/frag/fb
"make" -C /data/riotbuild/riotbase/sys/shell
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/ethernet
"make" -C /data/riotbuild/riotbase/sys/shell/commands
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/frag/rb
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/iphc
"make" -C /data/riotbuild/riotbase/sys/tsrb
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/sixlowpan/nd
"make" -C /data/riotbuild/riotbase/sys/ztimer
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/pkt
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/pktbuf
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/pktbuf_static
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/sock
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/hdr
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/sock/udp
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/ieee802154
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/transport_layer/udp
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/init_devs
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/pktq
text data bss dec hex filename
76096 464 22580 99140 18344 /data/riotbuild/riotbase/examples/gnrc_border_router/bin/feather-nrf52840/gnrc_border_router.elf
/home/mlenders/Repositories/RIOT-OS/RIOT/dist/tools/jlink/jlink.sh flash /home/mlenders/Repositories/RIOT-OS/RIOT/examples/gnrc_border_router/bin/feather-nrf52840/gnrc_border_router.bin
### Flashing Target ###
### Flashing at base address 0x0 with offset 0 ###
SEGGER J-Link Commander V7.66 (Compiled May 18 2022 15:57:58)
DLL version V7.66, compiled May 18 2022 15:57:43
J-Link Commander will now exit on Error
J-Link Command File read successfully.
Processing script file...
J-Link>loadbin /home/mlenders/Repositories/RIOT-OS/RIOT/examples/gnrc_border_router/bin/feather-nrf52840/gnrc_border_router.bin 0x00000000
J-Link connection not established yet but required for command.
Connecting to J-Link via USB...O.K.
Firmware: J-Link EDU Mini V1 compiled May 2 2022 09:00:59
Hardware version: V1.00
S/N: 801015296
License(s): FlashBP, GDB
USB speed mode: Full speed (12 MBit/s)
VTref=2.780V
Target connection not established yet but required for command.
Device "NRF52" selected.
Connecting to target via SWD
InitTarget() start
InitTarget() end
Found SW-DP with ID 0x2BA01477
DPIDR: 0x2BA01477
CoreSight SoC-400 or earlier
Scanning AP map to find all available APs
AP[2]: Stopped AP scan as end of AP map has been reached
AP[0]: AHB-AP (IDR: 0x24770011)
AP[1]: JTAG-AP (IDR: 0x02880000)
Iterating through AP map to find AHB-AP to use
AP[0]: Core found
AP[0]: AHB-AP ROM base: 0xE00FF000
CPUID register: 0x410FC241. Implementer code: 0x41 (ARM)
Found Cortex-M4 r0p1, Little endian.
FPUnit: 6 code (BP) slots and 2 literal slots
CoreSight components:
ROMTbl[0] @ E00FF000
[0][0]: E000E000 CID B105E00D PID 000BB00C SCS-M7
[0][1]: E0001000 CID B105E00D PID 003BB002 DWT
[0][2]: E0002000 CID B105E00D PID 002BB003 FPB
[0][3]: E0000000 CID B105E00D PID 003BB001 ITM
[0][4]: E0040000 CID B105900D PID 000BB9A1 TPIU
[0][5]: E0041000 CID B105900D PID 000BB925 ETM
Cortex-M4 identified.
Halting CPU for downloading file.
Downloading file [/home/mlenders/Repositories/RIOT-OS/RIOT/examples/gnrc_border_router/bin/feather-nrf52840/gnrc_border_router.bin]...
Comparing flash [100%] Done.
J-Link: Flash download: Bank 0 @ 0x00000000: Skipped. Contents already match
O.K.
J-Link>r
Reset delay: 0 ms
Reset type NORMAL: Resets core & peripherals via SYSRESETREQ & VECTRESET bit.
Reset: Halt core after reset via DEMCR.VC_CORERESET.
Reset: Reset device via AIRCR.SYSRESETREQ.
J-Link>g
J-Link>exit
Script processing completed.
sudo sh /home/mlenders/Repositories/RIOT-OS/RIOT/dist/tools/ethos/start_network.sh /dev/ttyUSB0 tap0 2001:db8::/64 115200
net.ipv6.conf.tap0.forwarding = 1
net.ipv6.conf.tap0.accept_ra = 0
----> ethos: sending hello.
----> ethos: activating serial pass through.
----> ethos: hello received
NETOPT_TX_END_IRQ not implemented by driver
NETOPT_TX_END_IRQ not implemented by driver
gnrc_uhcpc: Using 5 as border interface and 6 as wireless interface.
udp_sendto(): sendto(): Cannot assign requested address
main(): This is RIOT! (Version: 2022.07-devel-550-g3e531)
RIOT border router example application
All up, running the shell now
> uhcp_client(): no reply received |
|
Grr nope with that I get a flaky (literally had to type |
|
if you connect to /dev/ttyACM0 directly after it gets in the loop (instead of using ethos), does it also print endless ">"? |
If I do $ PROGRAMMER=jlink BOARD=feather-nrf52840 PORT=/dev/ttyUSB0 TERMPROG=picocom TERMFLAGS="--nolock --imap lfcrlf --baud 115200 /dev/ttyUSB0" make -C examples/gnrc_border_router -j flash termno. |
Btw: it's result already is -1 (i.e. EOF) in |
|
Interesting: the |
|
Seems to be whatever was last entered into the terminal (even after a restart of |
|
Ok, following the trace doesn't seem to make any sense (most likely due to optimization?). Going down the rabbit hole, I land here at some point according to gdb in /*
* We were reading. If there is an ungetc buffer,
* we must have been reading from that. Drop it,
* restoring the previous buffer (if any). If there
* is anything in that buffer, return.
*/
if (HASUB (fp))
{
FREEUB (ptr, fp);
if ((fp->_r = fp->_ur) != 0)
{
fp->_p = fp->_up;
return 0;
}
}resulting in int
getc (register FILE *fp)
{
int result;
struct _reent *reent = _REENT;
CHECK_INIT (reent, fp);
_newlib_flockfile_start (fp);
result = __sgetc_r (reent, fp);
_newlib_flockfile_end (fp);
return result;
}However, this doesn't make any sense, as both as a pre- and post-condition to All in all, I am pretty much at a loss what might be the problem. Given that this error happens so deep into |
Sadly not a workaround I can use with BLE border routers, since for some reason NimBLE does not work with Picolibc (at least not the border router) |
Description
Since #17265 usage of
stdio_ethoswill result in a continuously exiting shell and therefore a barrage of prompts with the shell unusable.This issue is solved if rebooting (hardware reboot) the device, but it renders automatic tests like
examples/suit_updateimpossible.Steps to reproduce the issue
Flash an applicationn using
stdio_ethos, e.g.:examples/suit_update,examples/gnrc_border_router.Expected results
Shell boots up fine.
Actual results
Versions
The text was updated successfully, but these errors were encountered: