rtl8761_bu usb bluetooth download fw command failed

[hkskoglund]

Describe the bug

Hi! I have a Trust usb stick that most of the time fails to download fw during boot on my raspberry pi 5. I have to plug/unplug it until it succeeds. From the btmon log it seems that fw loading stops after the last packet which fails to get HCI Event: Command Complete?

I have no issues with the usb stick on my laptop running fedora 40 with kernel 6.8.7-300.fc40, so maybe its related to the raspberry pi kernel?

Steps to reproduce the behaviour

dmesg --ctime --follow
Plug/unplug usb stick
Some times download fw fails

Device (s)

Raspberry Pi 5

System

Raspberry Pi reference 2024-03-15
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, f19ee211ddafcae300827f953d143de92a5c6624, stage4
2024/04/20 11:53:30
Copyright (c) 2012 Broadcom
version d1744d21 (release) (embedded)
Linux raspberrypi0 6.6.28+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.28-1+rpt1 (2024-04-22) aarch64 GNU/Linux

Logs

dmesg --ctime --follow:

[Wed May 1 18:26:14 2024] usb 1-2: new full-speed USB device number 6 using xhci-hcd
[Wed May 1 18:26:14 2024] usb 1-2: New USB device found, idVendor=0bda, idProduct=8771, bcdDevice= 2.00
[Wed May 1 18:26:14 2024] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Wed May 1 18:26:14 2024] usb 1-2: Product: Bluetooth Radio
[Wed May 1 18:26:14 2024] usb 1-2: Manufacturer: Realtek
[Wed May 1 18:26:14 2024] usb 1-2: SerialNumber: 00E04C239987
[Wed May 1 18:26:14 2024] Bluetooth: hci3: RTL: examining hci_ver=0a hci_rev=000b lmp_ver=0a lmp_subver=8761
[Wed May 1 18:26:14 2024] Bluetooth: hci3: RTL: rom_version status=0 version=1
[Wed May 1 18:26:14 2024] Bluetooth: hci3: RTL: loading rtl_bt/rtl8761bu_fw.bin
[Wed May 1 18:26:14 2024] Bluetooth: hci3: RTL: loading rtl_bt/rtl8761bu_config.bin
[Wed May 1 18:26:14 2024] Bluetooth: hci3: RTL: cfg_sz 6, total sz 30210
[Wed May 1 18:26:17 2024] Bluetooth: hci3: command 0xfc20 tx timeout
[Wed May 1 18:26:17 2024] Bluetooth: hci3: RTL: Failed to generate devcoredump
[Wed May 1 18:26:17 2024] Bluetooth: hci3: RTL: download fw command failed (-110)
[Wed May 1 18:26:18 2024] usb 1-2: USB disconnect, device number 6
[Wed May 1 18:26:18 2024] Bluetooth: hci3: RTL: RTL: Read reg16 failed (-19)
[Wed May 1 18:26:29 2024] usb 1-2: new full-speed USB device number 7 using xhci-hcd
[Wed May 1 18:26:29 2024] usb 1-2: New USB device found, idVendor=0bda, idProduct=8771, bcdDevice= 2.00
[Wed May 1 18:26:29 2024] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Wed May 1 18:26:29 2024] usb 1-2: Product: Bluetooth Radio
[Wed May 1 18:26:29 2024] usb 1-2: Manufacturer: Realtek
[Wed May 1 18:26:29 2024] usb 1-2: SerialNumber: 00E04C239987
[Wed May 1 18:26:29 2024] Bluetooth: hci3: RTL: examining hci_ver=0a hci_rev=000b lmp_ver=0a lmp_subver=8761
[Wed May 1 18:26:29 2024] Bluetooth: hci3: RTL: rom_version status=0 version=1
[Wed May 1 18:26:29 2024] Bluetooth: hci3: RTL: loading rtl_bt/rtl8761bu_fw.bin
[Wed May 1 18:26:29 2024] Bluetooth: hci3: RTL: loading rtl_bt/rtl8761bu_config.bin
[Wed May 1 18:26:29 2024] Bluetooth: hci3: RTL: cfg_sz 6, total sz 30210
[Wed May 1 18:26:30 2024] Bluetooth: hci3: RTL: fw version 0xdfc6d922
[Wed May 1 18:26:30 2024] Bluetooth: MGMT ver 1.22

tail btmon-1.log of failed fw download:

< HCI Command: Vendor (0x3f|0x0020) plen 223 #245 [hci1] 6.804458
f7 68 01 44 51 02 00 40 41 06 00 64 30 08 00 b0 .h.DQ..@A..d0...
00 66 00 59 40 0a 06 db 50 0c 06 f2 7b 10 06 8c .f.Y@...P...{...
55 12 06 0a 28 14 06 27 01 00 f0 60 00 02 90 77 U...(..'......w f8 64 06 27 00 02 01 00 00 34 00 04 27 38 00 83 .d.'.....4..'8.. 80 00 20 11 50 01 20 00 02 02 20 00 3b 3f 20 28 .. .P. ... .;? ( 03 20 20 2c 90 02 01 20 00 40 00 00 80 42 00 00 . ,... .@...B.. 01 44 00 00 00 46 00 84 00 48 00 c0 00 4a 00 00 .D...F...H...J.. 00 4c 00 00 00 4e 00 28 00 50 00 40 cc 52 00 00 .L...N.(.P.@.R.. 14 54 00 00 00 56 00 00 20 58 00 11 55 5a 00 00 .T...V.. X..UZ.. 80 5c 00 00 00 5e 00 05 28 0e 01 01 00 02 01 60 .\...^..(......
c0 02 01 22 c0 7c e1 64 00 02 01 20 00 30 06 26 ...".|.d... .0.&
67 34 06 7f c8 3a 06 d3 00 5a 03 45 00 42 02 bf g4...:...Z.E.B..
05 2e 01 00 68 02 00 40 41 08 00 b0 00 02 02 47 ....h..@A......G
99 a4 25 d5 d6 22 d9 c6 df 55 ab 23 87 00 00 ..%.."...U.#...
= Close Index: 00:00:00:00:00:00 [hci1] 8.819086

same in fw download success btmon-2.log

< HCI Command: Vendor (0x3f|0x0020) plen 223 #245 [hci3] 7.428504
f7 68 01 44 51 02 00 40 41 06 00 64 30 08 00 b0 .h.DQ..@A..d0...
00 66 00 59 40 0a 06 db 50 0c 06 f2 7b 10 06 8c .f.Y@...P...{...
55 12 06 0a 28 14 06 27 01 00 f0 60 00 02 90 77 U...(..'......w f8 64 06 27 00 02 01 00 00 34 00 04 27 38 00 83 .d.'.....4..'8.. 80 00 20 11 50 01 20 00 02 02 20 00 3b 3f 20 28 .. .P. ... .;? ( 03 20 20 2c 90 02 01 20 00 40 00 00 80 42 00 00 . ,... .@...B.. 01 44 00 00 00 46 00 84 00 48 00 c0 00 4a 00 00 .D...F...H...J.. 00 4c 00 00 00 4e 00 28 00 50 00 40 cc 52 00 00 .L...N.(.P.@.R.. 14 54 00 00 00 56 00 00 20 58 00 11 55 5a 00 00 .T...V.. X..UZ.. 80 5c 00 00 00 5e 00 05 28 0e 01 01 00 02 01 60 .\...^..(......
c0 02 01 22 c0 7c e1 64 00 02 01 20 00 30 06 26 ...".|.d... .0.&
67 34 06 7f c8 3a 06 d3 00 5a 03 45 00 42 02 bf g4...:...Z.E.B..
05 2e 01 00 68 02 00 40 41 08 00 b0 00 02 02 47 ....h..@A......G
99 a4 25 d5 d6 22 d9 c6 df 55 ab 23 87 00 00 ..%.."...U.#...

HCI Event: Command Complete (0x0e) plen 5 #246 [hci3] 7.461497
Vendor (0x3f|0x0020) ncmd 3
Status: Success (0x00)
f7 .
< HCI Command: Read Local Version... (0x04|0x0001) plen 0 #247 [hci3] 7.461506
HCI Event: Command Complete (0x0e) plen 12 #248 [hci3] 7.463502
Read Local Version Information (0x04|0x0001) ncmd 2
Status: Success (0x00)
HCI version: Bluetooth 5.1 (0x0a) - Revision 57286 (0xdfc6)
LMP version: Bluetooth 5.1 (0x0a) - Subversion 55586 (0xd922)
Manufacturer: Realtek Semiconductor Corporation (93)

Additional context

I have dtoverlay=disable-bt in /boot/firmware/config.txt and have disabled hciuart, since I dont need the internal adapter

[hkskoglund]

I've tried to trace down the source code responsible for downloading firmware, and found it in rtl_download_firmware https://github.com/torvalds/linux/blob/master/drivers/bluetooth/btrtl.c#L802

The function only tries 1 time to download a fragment before giving up.

/* Send download command */
		skb = __hci_cmd_sync(hdev, 0xfc20, frag_len + 1, dl_cmd,
				     HCI_INIT_TIMEOUT);
		if (IS_ERR(skb)) {
			rtl_dev_err(hdev, "download fw command failed (%ld)",
				    PTR_ERR(skb));
			ret = PTR_ERR(skb);
			goto out;
		}

In addition gemini AI/chatgpt gives :

gemini AI

The provided C code appears mostly correct for downloading firmware to an RTL Bluetooth controller. Here's a breakdown with some explanations and potential improvements:

Functionality:

• The code downloads firmware in fragments to the Bluetooth controller identified by hdev.
• It iterates through the firmware data (data) in chunks of RTL_FRAG_LEN and sends them using HCI commands.
• After successful download, it reads the local firmware version using another HCI command.

Strengths:

• The code uses kernel memory allocation (kmalloc) for the download command structure.
• It checks for errors during command sending and data length verification.
• It logs debugging and informational messages using appropriate functions.

Potential Improvements:

• Error Handling: Consider adding checks for dl_cmd allocation failures beyond just checking the return value. You can use a similar approach for skb allocations.
• Loop Termination: The loop condition i < frag_num might be susceptible to overflow if frag_num is very large. Consider using a loop that iterates until all data is sent (data + frag_len > data).
• Resource Management: Move the kfree(dl_cmd); outside the loop to ensure it's always freed even in error cases.
• Clarity: Adding comments within the loop to explain the purpose of setting dl_cmd->index and handling the last fragment could improve code readability.

Uncertainties:

• The code relies on undefined types like struct rtl_download_cmd and struct rtl_download_response. Without knowing their definitions, it's impossible to fully verify their usage.
• The functions rtl_dev_dbg, rtl_dev_err, rtl_dev_info, and btrtl_read_local_version are assumed to be specific to the RTL Bluetooth driver and are not part of the standard Linux kernel API.

Overall:

The code seems like a reasonable implementation for downloading firmware to an RTL Bluetooth controller. However, for a complete review, the missing type definitions and specific driver functionalities would need to be analyzed. The suggested improvements can further enhance the code's robustness and readability.

chatgpt:

The code you provided appears to be a function for downloading firmware to a Realtek Bluetooth device. Here's a quick analysis:

1. Memory Allocation: The function allocates memory for a struct rtl_download_cmd using kmalloc. This allocation is later freed using kfree.

2. Looping through Fragments: The function loops through the firmware data in fragments of size RTL_FRAG_LEN. It constructs a struct rtl_download_cmd for each fragment and sends it using __hci_cmd_sync. The loop also handles the case when the last fragment might be smaller than RTL_FRAG_LEN.

3. Error Handling: The function properly handles errors by returning appropriate error codes and freeing allocated resources (skb, dl_cmd) before returning.

4. Verification: It verifies the length of the response received after sending the download command and also reads the local version information after downloading the firmware.

5. Correctness: The code appears logically correct, assuming RTL_FRAG_LEN is defined and rtl_dev_dbg, rtl_dev_err, and rtl_dev_info are properly implemented for debugging and logging.

However, I noticed a couple of potential issues or areas for improvement:

• Data Copying: Inside the loop, memcpy(dl_cmd->data, data, frag_len); appears to copy the same data (data) into dl_cmd->data in each iteration. It seems like it should copy a portion of data corresponding to the current fragment. You might need to adjust the starting position in data for each fragment.

• Unused Variables: The skb variable is declared twice inside the loop. You can remove the declaration inside the loop since it's already declared outside.

• Error Handling in Fragment Loop: In the fragment loop, if an error occurs during sending the download command (__hci_cmd_sync), the loop will break out early with an error code, but it's not clearing resources allocated before the error occurred (e.g., previously allocated skb). You should make sure to free resources properly before exiting the function.

Overall, the code seems structurally sound, but you might need to verify the details of the firmware download process and adjust the data copying logic accordingly.

[sninaber]

hi hkskoglund,

i have the same issue with several aliexpress rtl8761bu dongles on the pi5, plug/unplug until firmware loads and then everything is fine. strange thing is that my pi4's dont have this problem at all.
same raspberry pi os, all rpi-updated to 6.6.30 to be sure. overclocking doesn't seem to matter much.
pi's are in flirc cases so external bluetooth is pretty much essential.

just thought this might be helpful in tracking down the issue.

[djholt]

Same exact issue here.

[sninaber]

the latest firmware (raspberrypi/firmware@5c83250) seems to show some improvement on the pi5, sometimes rtl_bt/rtl8761bu now even loads succesfully at boot (about 30-40% succes rate).
could this be related to this change: #6171 ?

pi4 still loads the driver without problems, also with the same rpi-update.

[djholt]

@sninaber I saw this same improvement just after my comment above. I updated firmware and also enabled usb_max_current_enable=1 at the same time, so I wasn't sure which change helped.

But, still not good reliability. As a workaround, I can repeatedly power cycle USB power using uhubctl until the hci interface comes up successfully, but this isn't ideal. Have you found any way to reload the driver/module or device from software? Or, have you found any alternative Bluetooth chip that plays nicely on Pi 5?

Thanks for the help!

[sninaber]

i have tried to modprobe, hcitool, rfkill and bluetoothctl (bluez) to no avail. i have some older bluetooth dongles that work with pi5 and some newer that don't have linux drivers yet. but i can't really recommend any for the pi5 as long as we don't know what causes the issue.
let's see if this patch ( 0a19b52 ) for the next firmware release solves the timeout.

[P33M]

Same exact issue here.

Thanks for providing a comprehensive bug report. It's very useful.

i have tried to modprobe, hcitool, rfkill and bluetoothctl (bluez) to no avail. i have some older bluetooth dongles that work with pi5 and some newer that don't have linux drivers yet. but i can't really recommend any for the pi5 as long as we don't know what causes the issue. let's see if this patch ( 0a19b52 ) for the next firmware release solves the timeout.

Why would it? You've got a USB device, and the patch there only concerns a SoC:PCIe link.

the latest firmware (raspberrypi/firmware@5c83250) seems to show some improvement on the pi5, sometimes rtl_bt/rtl8761bu now even loads succesfully at boot (about 30-40% succes rate). could this be related to this change: #6171 ?

pi4 still loads the driver without problems, also with the same rpi-update.

This is interesting. Can you iterate through rpi-update versions before and after this date and see which lead to the improvement? Commit list here: https://github.com/raspberrypi/firmware/commits/master/

[djholt]

Thanks for providing a comprehensive bug report. It's very useful.

Whoa—uncalled for. I had no differentiating information to contribute at the time, but I'm following and interested. Thanks for the support through this issue.

[djholt]

Can you iterate through rpi-update versions

Okay. The problem started occurring in e632362—looks like 6.6.16 introduced the issue. I don't experience the issue whatsoever in 6.1.77.

[sninaber]

ok that was fast i was still typing my reply.. thank for looking in to this!

[sninaber]

Okay. The problem started occurring in e632362—looks like 6.6.16 introduced the issue. I don't experience the issue whatsoever in 6.1.77.

i couldn't find this commit, did you mean revert back to raspberrypi/firmware@e7590ad to check?

Why would it? You've got a USB device, and the patch there only concerns a SoC:PCIe link.

my guess would be that on the pi5, RP1 is also connected to pcie, which among others holds the usb controller. (see latency considerations: https://datasheets.raspberrypi.com/rp1/rp1-peripherals.pdf ). so any pcie issues would affect RP1 devices.

[djholt]

I was reverting using rpi-update, so I referenced the commits from rpi-firmware instead: https://github.com/raspberrypi/rpi-firmware/commits/master/

So, for example, the issue didn't manifest after reverting all the way back to 5fc4f64 (kernel 6.1.77):

sudo rpi-update 5fc4f643d2e9c5aa972828705a902d184527ae3f

[P33M]

That's a clue that it might be an upstream issue (compounded by whatever's different on Pi 5[1]). Does a Pi 4 exhibit the same when updated to either version?

Note that running rpi-update on a Pi 4/5 will also update the bootloader, which has slightly changed XHCI behaviour over time. rpi-update won't downgrade it afterwards, so that should keep the same version for both 6.1 and 6.6.

There's certainly scope for a regression given the commit logs...

$ git log --oneline  rpi-6.1.y..rpi-6.6.y btrtl.c
9ee252868787 Bluetooth: btrtl: Ignore error return for hci_devcd_register()
bd003fb338af Bluetooth: btrtl: Load FW v2 otherwise FW v1 for RTL8852C
9e14606d8f38 Bluetooth: msft: Extended monitor tracking by address filter
6b42f04e2417 Bluetooth: btrtl: Correct the length of the HCI command for drop fw
044014ce85a1 Bluetooth: btrtl: Add Realtek devcoredump support
bb23f07cb639 Bluetooth: btrtl: Add missing MODULE_FIRMWARE declarations
7948fe1c92d9 Bluetooth: btrtl: Add the support for RTL8851B
9a24ce5e29b1 Bluetooth: btrtl: Firmware format v2 support
253cf30e8d3d Bluetooth: btrtl: check for NULL in btrtl_set_quirks()
c0123cb6c4c7 Bluetooth: btrtl: add support for the RTL8723CS
10c9e0104e3b Bluetooth: hci_h5: btrtl: Add support for RTL8821CS
fe4b71b59ed5 Bluetooth: btrtl: Add support for RTL8852BS
a479e71322ce Bluetooth: btusb: Ignore zero length of USB packets on ALT 6 for specific chip

[1] On Pi 5, the device is directly connected to a root port. On Pi 1-4, all full-speed devices are accessed though at least one hub.

[sninaber]

sudo rpi-update 5fc4f643d2e9c5aa972828705a902d184527ae3f

must admit i expected the problem te have existed on the pi5 all along, but can confirm this completely resolves the issue, so my compliments to how quickly you found this. still, looking at the upstream changes to rtl, generic or specifically rtl8761bu, it doesn't really explain why the pi5 sometimes fails to load the firmware while others (including pi4) don't. i have tried different usb hubs on the pi5 (powered and nonpowered) which doesn't make a difference. can't comment on XHCI or the binary files in e632362 where the problem started, so i'm quite useless from here on..

firmware update today raspberrypi/firmware@f31d28d improves a bit further (about 80% succes loading rtl firmware), but still fails sometimes.

[sninaber]

i thought about what @P33M said earlier about xhci and the bootloader, so i wanted to see if the otg port behaved in the same way. so i added dtoverlay=dwc2,dr_mode=host to config.txt and connected the bluetooth adapter to the otg port.

this solves the problem and recognises the adapter without problems, at boot or multiple unplug/plug. does this help in any way in finding the cause?

is the firmware for RP1 also held in eeprom? if so downgrading to kernel 6.1.77 would not change this.
@P33M: i can send you one of these adapters if that helps.