FTDI USB Serial cause kernel panic

[winstonma]

I updated my R-Pi kernel to #125.

I have a USB serial device (ID: 0403:6001) and I have a python code doing all the read/write. After a while the program stop working and generated kernel panic. The message is as follows:

Unable to handle kernel paging request at virtual address1757d280
pgd = caf14000
[1757d280] *pgd=00000000
Internal error: Oops: 5 [#1] PREEMPT

Entering kdb (current=0xcaf605c0, pid 1142) Oops: (null)
due to oops @ 0xbf0118e4

Pid: 1142, comm: python
CPU: 0 Not tainted (3.1.9+ #1225)
PC is at ftdi_process_read_urb+0x9c/0x320[ftdi_sio]
LR is at tty_port_tty_get+0x58/0x8c
pc : [] lr: []psr: 20000193
sp: caf45c60 ip: caf45c50 fp: caf45cb4
....
Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user

Since it's a picture I took a picture and type most of it.

Thanks

By the way is there a better way to capture the kernel panic event?

[popcornmix]

The panic might be in /var/log/messages after reboot. If not can you post a link to a photo of the screen showing the backtrace.

[winstonma]

Thanks for your help.

1. Screen capture
   https://www.dropbox.com/s/jpf6jahhfzkyby6/Kernel%20Panic.jpg
2. Message
   https://www.dropbox.com/s/d3lo5k8e8u8d9ft/messages
3. Older message
   https://www.dropbox.com/s/iura8nn9a6fvl3x/messages.1

[popcornmix]

Is it easily reproducible? Can you provide some code that provokes this bug on demand?

[winstonma]

It's purely random. I had a system halt after 2 days and another one just halt after 2 hours.

However I am running the system 24/7 therefore it's very likely to repeat.

I wrote a python program to do the I/O process. In the python program I use pySerial to send the read/write command. It was a separate project but after 2.7 it was integrated to python.
Here is the link:
http://pyserial.sourceforge.net/

serial_device = serial.Serial('/dev/ttyUSB0', 625000, timeout=30)
data = serial_device.read()
serial_device.write(data)

[winstonma]

I have some new error message from kern.log
http://pastebin.com/Wj7nNjwu

From message
http://pastebin.com/LJUST7Fi

In addition I got the following message:
Jun 26 08:59:13 testc kernel: [ 692.236094] SysRq : HELP : loglevel(0-9) reBoot Crash terminate-all-tasks(E) memory-full-oom-kill(F) debug(G) kill-all-tasks(I) thaw-filesystems(J) saK show-memory-usage(M) nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw Sync show-task-states(T) Unmount show-blocked-tasks(W) dump-ftrace-buffer(Z)

Also from command 'cat /proc/tty/driver/usbserial'
usbserinfo:1.0 driver:2.0
0: module:ftdi_sio name:"FTDI USB Serial Device" vendor:0403 product:6001 num_ports:1 port:1 path:usb-bcm2708_usb-1.3.5

[winstonma]

I installed system monitoring tools on my client. And found out that thread leak is the major cause of kernel panic.

[popcornmix]

The NAK holdoff fix added to kernel a few days ago may fix this. Please update and test.

[Jbravo2]

I'm running Linux raspberrypi 3.2.27+ #160 PREEMPT Mon Sep 17 23:18:42 BST 2012 armv6l GNU/Linux,
which I assume contains the fix mentioned above.

I got a FT232BM (embedded in a card reader), the device is discovered correctly, but when accessing /dev/ttyUSB0 the RPi just hangs.

When forcing usb to full speed in cmdline.txt as described here; http://www.raspberrypi.org/phpBB3/viewtopic.php?p=164633#p164633

the device works as expected, but performance of usb full speed in not acceptable.

[winstonma]

I tried to following the steps on that forum and make the changes on cmdline.txt, however when I set usb to full speed (dwc_otg.speed=1) the keyboard and ethernet is not functional.

[winstonma]

After running newer kernel a new message comes in. After prolonged run I saw the following message:
[ 114.094343] ERROR::handle_hc_chhltd_intr_dma:2113: handle_hc_chhltd_intr_dma: Channel 1, DMA Mode -- ChHltd set, but reason for halting is unknown, hcint 0x00000402, intsts 0x06600001

No logs have been written in /var/log/message. However I got similar problem with another user.
http://www.raspberrypi.org/phpBB3/viewtopic.php?p=164633#p164633

I also tried to replace the ftdi_sio driver with the FTDI official D2xx driver. Same crashes happened to me.

It seems to me that the root cause of the crash is the DWC OTG instead of the FTDI, although it trigger the crash.

[ghollingworth]

Looking at the headers it looks like the reason for halting is bit 10 is set in hcint which is a data toggle error...

This means that the device replied to an IN transaction with a data0 / 1 pid it did not expect... I don't know why this isn't handled in the chhlt interrupt would need to go look into it... Unfortunately am currently feeling ill and not up to it!

Gordon

[winstonma]

Thanks for the information.

I would like to build my own kernel. I would like to make changes on the header. Would you please locate the header files? Thanks

Also I still have a strong faith that DWC OTG is the cause of the DMA error. Not the FTDI driver/library.

[gholling]

If you can rebuilt the kernel, it would be useful if you can replace the test on line 86 with if(0)

The effect of this is to increase the FIQ interrupts back up to 8000 per second, but will keep the NAK holdoff so the serial devices won't flood the processor. But it will always trigger the interrupt, I think the problem I'm seeing is the interrupt trigger going missing (probably a race condition I haven't thought about for long enough)

Gordon

[winstonma]

Thanks for the info.

I tried to understand the kernel tree structure. And I found the usb portion:
https://github.com/raspberrypi/linux/tree/rpi-3.2.27/drivers/usb/host/dwc_otg

I downloaded and compiled the kernel. I would like to know how to change the FIQ parameter (to be more specific - which file). Thanks a lot