Slower than expected with DietPi #7092

Just2Confuse · 2024-05-30T12:29:11Z

Just2Confuse
May 30, 2024

Hi,

im doing a standard IO Test. And, its not a showstopper ;)
sudo curl https://raw.githubusercontent.com/TheRemote/PiBenchmarks/master/Storage.sh | sudo bash

Got a USB3 cable with a old SSD. Short: Everything what is written ist "slower" then i expact. I try to understand. Tests with Dietpi done 10 times.

RaspiOS-Lite

me@Pi5:~ $ date
Thu 30 May 14:07:35 CEST 2024
jens@Pi5:~ $ sudo curl https://raw.githubusercontent.com/TheRemote/PiBenchmarks/master/Storage.sh | sudo bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Trimming and syncing drives ...
100 36197  100 36197    0     0   353k      0 --:--:-- --:--:-- --:--:--  353k
/boot/firmware: 446.6 MiB (468344832 bytes) trimmed on /dev/sda1
/: 644.4 MiB (675717120 bytes) trimmed on /dev/sda2
Board information: Manufacturer: Raspberry Pi Foundation - Model: Raspberry Pi 5 Model B Rev 1.0 - Architecture: aarch64 - OS: Debian GNU/Linux 12 (bookworm)
Fetching required components ...
Clock speeds: CPU: 2400 - Core: 910
System rootfs drive (/) has been detected as /dev/sda2 (sda2)
Starting INXI hardware identification...
System:
  Kernel: 6.6.28+rpt-rpi-2712 arch: aarch64 bits: 64 compiler: gcc v: 12.2.0 Console: pty pts/2
    Distro: Debian GNU/Linux 12 (bookworm)
Machine:
  Type: ARM System: Raspberry Pi 5 Model B Rev 1.0 details: N/A rev: c04170 serial: <filter>
CPU:
  Info: quad core model: N/A variant: cortex-a76 bits: 64 type: MCP arch: ARMv8 rev: 1 cache:
    L1: 512 KiB L2: 2 MiB L3: 2 MiB
  Speed (MHz): avg: 2400 min/max: 1500/2400 cores: 1: 2400 2: 2400 3: 2400 4: 2400 bogomips: 432
  Features: Use -f option to see features
Graphics:
  Device-1: bcm2712-hdmi0 driver: vc4_hdmi v: N/A bus-ID: N/A
  Device-2: bcm2712-hdmi1 driver: vc4_hdmi v: N/A bus-ID: N/A
  Display: server: No display server data found. Headless machine?
  API: EGL v: 1.4,1.5 drivers: swrast,v3d platforms: active: gbm,surfaceless,device
    inactive: wayland,x11
Network:
  Device-1: driver: rp1 v: N/A port: N/A bus-ID: 01:00.0
  IF-ID-1: br-0995884e75c8 state: down mac: <filter>
  IF-ID-2: br-3b73f4a08d06 state: up speed: 10000 Mbps duplex: unknown mac: <filter>
  IF-ID-3: br-6d1df66fb5af state: up speed: 10000 Mbps duplex: unknown mac: <filter>
  IF-ID-4: br-741e38e0d177 state: down mac: <filter>
  IF-ID-5: br-a0e5b8bbe29d state: down mac: <filter>
  IF-ID-6: docker0 state: up speed: 10000 Mbps duplex: unknown mac: <filter>
  IF-ID-7: end0 state: up speed: 100 Mbps duplex: full mac: <filter>
  IF-ID-8: veth10428dd state: up speed: 10000 Mbps duplex: full mac: <filter>
  IF-ID-9: veth6b7ceb6 state: up speed: 10000 Mbps duplex: full mac: <filter>
  IF-ID-10: veth6f43170 state: up speed: 10000 Mbps duplex: full mac: <filter>
  IF-ID-11: veth9bf3b19 state: up speed: 10000 Mbps duplex: full mac: <filter>
  IF-ID-12: vethd31660f state: up speed: 10000 Mbps duplex: full mac: <filter>
  IF-ID-13: vethe3d8ed3 state: up speed: 10000 Mbps duplex: full mac: <filter>
Drives:
  Local Storage: total: 119.24 GiB used: 49.8 GiB (41.8%)
  ID-1: /dev/sda vendor: Crucial model: M4-CT128M4SSD2 size: 119.24 GiB type: USB
  Message: No optical or floppy data found.
Partition:
  ID-1: / size: 116.31 GiB used: 49.74 GiB (42.8%) fs: ext4 dev: /dev/sda2
Info:
  Processes: 213 Uptime: 3m Memory: total: 4 GiB available: 3.95 GiB used: 929.6 MiB (23.0%)
  igpu: 4 MiB Init: systemd target: multi-user (3) Compilers: gcc: 12.2.0 Packages: 1028
  Client: Sudo v: 1.9.13p3 inxi: 3.3.31
Running additional hardware identification tests...
Additional hardware identification tests completed.
Starting mass storage device identification...
Running HDParm tests ...
/dev/sda2:
 Timing O_DIRECT cached reads:   696 MB in  2.00 seconds = 347.55 MB/sec
 Timing O_DIRECT disk reads: 1076 MB in  3.00 seconds = 358.44 MB/sec
HDParm: 358.44 MB/sec - HDParmCached: 347.55 MB/sec
Running dd tests ...
133120+0 records in
133120+0 records out
545259520 bytes (545 MB, 520 MiB) copied, 3.59913 s, 151 MB/s
DD Write Speed: 151 MB/s
Running fio write test ...
Running fio read test ...
FIO results - 4k RandWrite: 27489 IOPS (109959 KB/s) - 4k RandRead: 33246 IOPS (132987 KB/s)
Running iozone test ...
        Iozone: Performance Test of File I/O
                Version $Revision: 3.489 $
                Compiled for 64 bit mode.
                Build: linux
        Run began: Thu May 30 14:07:58 2024
        Auto Mode
        Include fsync in write timing
        O_DIRECT feature enabled
        File size set to 81920 kB
        Record Size 4 kB
        Command line used: iozone -a -e -I -i 0 -i 1 -i 2 -s 80M -r 4k
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
           81920       4    44603    44658    44776    44633    17653    44673
iozone test complete.
RandRead: 17653 - RandWrite: 44673 - Read: 44776 - Write: 44603
Enter a description of your storage and setup (Example: Kingston A400 SSD on Pi 4 using StarTech SATA to USB adapter)
Description: Raspi5_Bookworm_Lite
(Optional) Enter alias to use on benchmark results.  Leave blank for completely anonymous.
Alias (leave blank for Anonymous): Sunnie_Micro
Result submitted successfully and will appear live on https://pibenchmarks.com within a couple of minutes.

     Category                  Test                      Result
HDParm                    Disk Read                 358.44 MB/sec
HDParm                    Cached Disk Read          347.55 MB/sec
DD                        Disk Write                151 MB/s
FIO                       4k random read            33246 IOPS (132987 KB/s)
FIO                       4k random write           27489 IOPS (109959 KB/s)
IOZone                    4k read                   44776 KB/s
IOZone                    4k write                  44603 KB/s
IOZone                    4k random read            17653 KB/s
IOZone                    4k random write           44673 KB/s

                          Score: 11692

Compare with previous benchmark results at:
https://pibenchmarks.com/
me@Pi5:~ $

And this is exaclty the same hardware. Fresh installed DietPi with no addons. On Raspi-Lite is running a docker with ~8 Containers.

root@DietPi:~# date
Thu May 30 13:20:17 BST 2024
root@DietPi:~# sudo curl https://raw.githubusercontent.com/TheRemote/PiBenchmarks/master/Storage.sh | sudo bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 36197  100 36197    0     0   359k      0 --:--:-- --:--:-- --:--:--  360k
Trimming and syncing drives ...
/boot/firmware: 95 MiB (99588096 bytes) trimmed on /dev/sda1
/: 865.2 MiB (907272192 bytes) trimmed on /dev/sda2
Board information: Manufacturer: Raspberry Pi Foundation - Model: Raspberry Pi 5 Model B Rev 1.0 - Architecture: aarch64 - OS: Debian GNU/Linux 12 (bookworm)
Fetching required components ...
Clock speeds: CPU: 2400 - Core: 910
System rootfs drive (/) has been detected as /dev/sda2 (sda2)
Starting INXI hardware identification...
System:
  Kernel: 6.6.28+rpt-rpi-2712 arch: aarch64 bits: 64 compiler: gcc v: 12.2.0 Console: pty pts/2
    Distro: Debian GNU/Linux 12 (bookworm)
Machine:
  Type: ARM System: Raspberry Pi 5 Model B Rev 1.0 details: N/A rev: c04170 serial: <filter>
CPU:
  Info: quad core model: N/A variant: cortex-a76 bits: 64 type: MCP arch: ARMv8 rev: 1 cache:
    L1: 512 KiB L2: 2 MiB L3: 2 MiB
  Speed (MHz): avg: 2100 min/max: 1500/2400 cores: 1: 2100 2: 2100 3: 2100 4: 2100 bogomips: 432
  Features: Use -f option to see features
Graphics:
  Device-1: bcm2708-fb driver: N/A bus-ID: N/A
  Device-2: bcm2712-hdmi0 driver: N/A bus-ID: N/A
  Device-3: bcm2712-hdmi1 driver: N/A bus-ID: N/A
  Display: server: No display server data found. Headless machine? tty: 80x40
  API: N/A Message: No API data available in console. Headless machine?
Network:
  Device-1: driver: rp1 v: N/A port: N/A bus-ID: 01:00.0
  IF-ID-1: eth0 state: up speed: 1000 Mbps duplex: full mac: <filter>
Drives:
  Local Storage: total: 119.24 GiB used: 1011.6 MiB (0.8%)
  ID-1: /dev/sda vendor: Crucial model: M4-CT128M4SSD2 size: 119.24 GiB type: USB
  Message: No optical or floppy data found.
Partition:
  ID-1: / size: 117.23 GiB used: 979.6 MiB (0.8%) fs: ext4 dev: /dev/sda2
Info:
  Processes: 125 Uptime: 3m Memory: total: 4 GiB available: 3.95 GiB used: 159.9 MiB (4.0%)
  igpu: 4 MiB Init: systemd target: graphical (5) Compilers: N/A Packages: 314 Client: Sudo
  v: 1.9.13p3 inxi: 3.3.31
Running additional hardware identification tests...
Additional hardware identification tests completed.
Starting mass storage device identification...
Running HDParm tests ...
/dev/sda2:
 Timing O_DIRECT cached reads:   638 MB in  2.00 seconds = 318.73 MB/sec
 Timing O_DIRECT disk reads: 968 MB in  3.00 seconds = 322.41 MB/sec
HDParm: 322.41 MB/sec - HDParmCached: 318.73 MB/sec
Running dd tests ...
133120+0 records in
133120+0 records out
545259520 bytes (545 MB, 520 MiB) copied, 18.9546 s, 28.8 MB/s
DD Write Speed: 28.8 MB/s
Running fio write test ...
Running fio read test ...
FIO results - 4k RandWrite: 6714 IOPS (26859 KB/s) - 4k RandRead: 29767 IOPS (119069 KB/s)
Running iozone test ...
        Iozone: Performance Test of File I/O
                Version $Revision: 3.489 $
                Compiled for 64 bit mode.
                Build: linux
        Run began: Thu May 30 13:21:04 2024
        Auto Mode
        Include fsync in write timing
        O_DIRECT feature enabled
        File size set to 81920 kB
        Record Size 4 kB
        Command line used: iozone -a -e -I -i 0 -i 1 -i 2 -s 80M -r 4k
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
           81920       4    26679    26653    39973    41363    16407    26781
iozone test complete.
RandRead: 16407 - RandWrite: 26781 - Read: 39973 - Write: 26679
Enter a description of your storage and setup (Example: Kingston A400 SSD on Pi 4 using StarTech SATA to USB adapter)
Description: Raspi5_diepti_fresh
(Optional) Enter alias to use on benchmark results.  Leave blank for completely anonymous.
Alias (leave blank for Anonymous): Sunnie_Micro
Result submitted successfully and will appear live on https://pibenchmarks.com within a couple of minutes.

     Category                  Test                      Result
HDParm                    Disk Read                 322.41 MB/sec
HDParm                    Cached Disk Read          318.73 MB/sec
DD                        Disk Write                28.8 MB/s
FIO                       4k random read            29767 IOPS (119069 KB/s)
FIO                       4k random write           6714 IOPS (26859 KB/s)
IOZone                    4k read                   39973 KB/s
IOZone                    4k write                  26679 KB/s
IOZone                    4k random read            16407 KB/s
IOZone                    4k random write           26781 KB/s

                          Score: 6157

Compare with previous benchmark results at:
https://pibenchmarks.com/
root@DietPi:~#
root@DietPi:~#
root@DietPi:~#
root@DietPi:~#
root@DietPi:~#
------------------
 Benchmarks completed:                                                                                                │
                                                │  - CPU performance : Duration = 2.13 seconds (lower is faster)                                                       │
                                                │  - CPU temps       : Idle = 42 °C | Full load = 52 °C                                                                │
                                                │  - RootFS I/O      : Write = 27 MiB/s | Read = 356 MiB/s                                                             │
                                                │  - RAM I/O         : Write = 1156 MiB/s | Read = 4823 MiB/s                                                          │
                                                │                                                                                                                      │
                                                │ Compare these results online with other users, using the link below:                                                 │
                                                │  - https://dietpi.com/survey#benchmark                                               ```

Answered by MichaIng

May 30, 2024

Since firmware and kernel are the same, there is two possible reasons/differences:

APM (Advanced Power Management) value set via hdparm: Most SSDs do not support APM at all, but we had one case, where it was supported, and where the default of 127 lead to slower disk I/O. 127 is the highest possible value which still allows idle spin down of spinning disks, hence we set it. Does this not report an error, and have an effect on your benchmark?
```
hdparm -B 254 /dev/sda
# or even (disabling APM compeletely)
hdparm -B 255 /dev/sda
```
The CPU governor, where DietPi uses schedutil by default, while RPi OS, AFIAK, still ondemand. schedutil is the more modern governor, meant as successor, which is mo…

View full answer

Just2Confuse · 2024-05-30T16:37:17Z

Just2Confuse
May 30, 2024
Author

Now you can find the results Online.
https://pibenchmarks.com/search/Sunnie_Micro/

0 replies

MichaIng · 2024-05-30T21:39:50Z

MichaIng
May 30, 2024
Maintainer

Since firmware and kernel are the same, there is two possible reasons/differences:

APM (Advanced Power Management) value set via hdparm: Most SSDs do not support APM at all, but we had one case, where it was supported, and where the default of 127 lead to slower disk I/O. 127 is the highest possible value which still allows idle spin down of spinning disks, hence we set it. Does this not report an error, and have an effect on your benchmark?
```
hdparm -B 254 /dev/sda
# or even (disabling APM compeletely)
hdparm -B 255 /dev/sda
```
The CPU governor, where DietPi uses schedutil by default, while RPi OS, AFIAK, still ondemand. schedutil is the more modern governor, meant as successor, which is more efficient, making more use of intermediate p-states, and with less processing overhead. But it indeed has weaknesses regarding disk and also RAM I/O on some SBCs. I observed it especially on big.LITTLE SBCs, where during I/O benchmarks, only the small cores' frequencies were raised, while the large ones remained at lower frequencies, negatively impacting I/O speeds. ondemand worked better in this case, even without the io_is_busy flag, somehow. Hence, check back at RPi OS, which governor it has, and in case test whether ondemand on DietPi leads to better results:
```
cat /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
echo ondemand > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
```

3 replies

Just2Confuse May 31, 2024
Author

Will try that tomorrow.

Just2Confuse Jun 1, 2024
Author

Hi,
perfect. It was the APM Level. This is 254. Gouvenor did not change anything. Thank you for your knowledge!
https://pibenchmarks.com/search/Sunnie_Micro/

Description: Dietpi_hdparm_B254
(Optional) Enter alias to use on benchmark results.  Leave blank for completely anonymous.
Alias (leave blank for Anonymous): Sunnie_Micro
Result submitted successfully and will appear live on https://pibenchmarks.com within a couple of minutes.

     Category                  Test                      Result
HDParm                    Disk Read                 359.92 MB/sec
HDParm                    Cached Disk Read          347.88 MB/sec
DD                        Disk Write                181 MB/s
FIO                       4k random read            33355 IOPS (133420 KB/s)
FIO                       4k random write           29132 IOPS (116529 KB/s)
IOZone                    4k read                   44585 KB/s
IOZone                    4k write                  43362 KB/s
IOZone                    4k random read            17757 KB/s
IOZone                    4k random write           44764 KB/s

                          Score: 12166

Compare with previous benchmark results at:
https://pibenchmarks.com/

MichaIng Jun 5, 2024
Maintainer

Great, thanks for verifying this. When you just remove the apm = 127 from /etc/hdparm.conf and reboot, 254 is the default?

hdparm -B

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Slower than expected with DietPi #7092

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 3 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Slower than expected with DietPi #7092

Just2Confuse May 30, 2024

Replies: 2 comments · 3 replies

Just2Confuse May 30, 2024 Author

MichaIng May 30, 2024 Maintainer

Just2Confuse May 31, 2024 Author

Just2Confuse Jun 1, 2024 Author

MichaIng Jun 5, 2024 Maintainer

Just2Confuse
May 30, 2024

Replies: 2 comments 3 replies

Just2Confuse
May 30, 2024
Author

MichaIng
May 30, 2024
Maintainer

Just2Confuse May 31, 2024
Author

Just2Confuse Jun 1, 2024
Author

MichaIng Jun 5, 2024
Maintainer