Low download speed MAVFTP

[urpylka]

Issue details

Hello, I download log-files through mavftp, but speed does not rise more than 36KB/s. PX4 connect via USB. Change chunk size does not give a result. If I download logs in QGC, connected through gsc_bridge, speed rise to 300KB/s. How I can accelerate download with mavftp?

MAVROS version and platform

Mavros: 0.20.1 (rosversion mavros)
ROS: Kinetic
Raspbian Stretch: Linux raspberrypi 4.9.41-v7+ #1023 SMP Tue Aug 8 16:00:15 BST 2017 armv7l GNU/Linux

Autopilot type and version

[ ] ArduPilot
[ x ] PX4

Version: 1.6.5

Node logs

pi@raspberrypi:~/px4logs_manager $ rosrun mavros mavftp download /fs/microsd/log/sess035/log001.ulg log001.ulgDownloading:   3% |##                                                              | ETA:  0:03:07  31.61 kB/s

Diagnostics

header: 
  seq: 47
  stamp: 
    secs: 1511785064
    nsecs:  49836849
  frame_id: ''
status: 
  - 
    level: 0
    name: "mavros: FCU connection"
    message: "connected"
    hardware_id: "/dev/ttyACM0"
    values: 
      - 
        key: "Received packets:"
        value: "10107"
      - 
        key: "Dropped packets:"
        value: "0"
      - 
        key: "Buffer overruns:"
        value: "0"
      - 
        key: "Parse errors:"
        value: "0"
      - 
        key: "Rx sequence number:"
        value: "244"
      - 
        key: "Tx sequence number:"
        value: "72"
      - 
        key: "Rx total bytes:"
        value: "415058"
      - 
        key: "Tx total bytes:"
        value: "7634"
      - 
        key: "Rx speed:"
        value: "35874.000000"
      - 
        key: "Tx speed:"
        value: "448.000000"
  - 
    level: 2
    name: "mavros: GPS"
    message: "No satellites"
    hardware_id: "/dev/ttyACM0"
    values: 
      - 
        key: "Satellites visible"
        value: "0"
      - 
        key: "Fix type"
        value: "0"
      - 
        key: "EPH (m)"
        value: "Unknown"
      - 
        key: "EPV (m)"
        value: "Unknown"
  - 
    level: 1
    name: "mavros: Heartbeat"
    message: "Frequency too low."
    hardware_id: "/dev/ttyACM0"
    values: 
      - 
        key: "Heartbeats since startup"
        value: "26"
      - 
        key: "Frequency (Hz)"
        value: "0.001167"
      - 
        key: "Vehicle type"
        value: "Octorotor"
      - 
        key: "Autopilot type"
        value: "PX4 Autopilot"
      - 
        key: "Mode"
        value: "MANUAL"
      - 
        key: "System status"
        value: "Standby"
  - 
    level: 0
    name: "mavros: System"
    message: "Normal"
    hardware_id: "/dev/ttyACM0"
    values: 
      - 
        key: "Sensor present"
        value: "0x00000000"
      - 
        key: "Sensor enabled"
        value: "0x00000000"
      - 
        key: "Sensor helth"
        value: "0x00000000"
      - 
        key: "CPU Load (%)"
        value: "61.3"
      - 
        key: "Drop rate (%)"
        value: "0.0"
      - 
        key: "Errors comm"
        value: "0"
      - 
        key: "Errors count #1"
        value: "0"
      - 
        key: "Errors count #2"
        value: "0"
      - 
        key: "Errors count #3"
        value: "0"
      - 
        key: "Errors count #4"
        value: "0"
  - 
    level: 0
    name: "mavros: Battery"
    message: "Normal"
    hardware_id: "/dev/ttyACM0"
    values: 
      - 
        key: "Voltage"
        value: "65.54"
      - 
        key: "Current"
        value: "-0.0"
      - 
        key: "Remaining"
        value: "-1.0"
  - 
    level: 0
    name: "mavros: Time Sync"
    message: "Normal"
    hardware_id: "/dev/ttyACM0"
    values: 
      - 
        key: "Timesyncs since startup"
        value: "110"
      - 
        key: "Frequency (Hz)"
        value: "10.004747"
      - 
        key: "Last dt (ms)"
        value: "0.024133"
      - 
        key: "Mean dt (ms)"
        value: "0.027681"
      - 
        key: "Last system time (s)"
        value: "51.068520000"
      - 
        key: "Time offset (s)"
        value: "1511785012.881291866"

Check ID

OK. I got messages from 1:1.

---
Received 4836 messages, from 1 addresses
sys:comp   list of messages
  1:1     32, 0, 147, 36, 230, 231, 105, 74, 140, 141, 77, 111, 241, 83, 245, 1, 30, 31

[vooon]

Yes, speed is low because FTP plugin only implements file like operations.
QGC uses streaming, that is why it faster.

E.g. mavros ftp to get each 230 byte chunk send request message, then wait response. All that done by 255-byte long message.
QGC ftp send request to stream file and then only wait response chunks (and maybe at the end rerequest absent chunks via file io commands).

1. Switch to mavlink v2 should speed up requests, because they often small and tail is zeroed ⇒ removed from wire format.
2. You can implement streaming in ftp plugin, it is extension of ftp-proto.
   But that require changes in plugin FSM, and dealing with temp files from mavros (or request to write to some location, problem that this is not network-transparent).

[urpylka]

I change version to v2.0, but speed does not increase

[vooon]

Hm, in theory smaller packets of v2 should give some speed up. Can you use wireshark to extract msgid 110?
Other possibility that speed is limited by ftp implementation on px4 (cannot handle read requests faster).
If so, only possible way to speed up is implementation of streaming.

[TSC21]

This is somewhat also a duplicate of #686. I'm closing it so it can be continued there instead.