RPI Zero 2W incorrect memory size

[gugolple]

Describe the bug
After doing an rpi-update on my raspberry pi zero 2W, the available ram reduced from ~484MB to 400MB. I do understand that the memory split will take an amount of ram (check logs section). I do not understand the reason of the change, but having less resources available in such a constrained board is not beneficial.

To reproduce
Executing the command sudo rpi-update.

Expected behaviour
Available ram for the system to be at around ~484MB.

Actual behaviour
Available ram for the system is found to be at around ~400MB.

System

$ cat /etc/rpi-issue
Raspberry Pi reference 2023-10-10
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 962bf483c8f326405794827cce8c0313fd5880a8, stage4

$ vcgencmd version
Dec  8 2023 12:42:28 
Copyright (c) 2012 Broadcom
version e02d33b3122450accf9dea471a177d3b5623f5ad (clean) (release) (start_cd)

$ uname -a
Linux raspberrypi 6.6.16-v8+ raspberrypi/firmware#1728 SMP PREEMPT Tue Feb  6 17:21:42 GMT 2024 aarch64 GNU/Linux

Logs
Reduced to only provide relevant information.

$ dmesg | egrep 'Memory|CMA|cma' --color -C3
[    0.000000] Machine model: Raspberry Pi Zero 2 W Rev 1.0
[    0.000000] Memory: 125944K/507904K available (13376K kernel code, 2208K rwdata, 4308K rodata, 4928K init, 1083K bss, 119816K reserved, 262144K cma-reserved)

$ cat /proc/meminfo
MemTotal:         409760 kB

$ vcgencmd get_mem gpu
gpu=16M

Additional context
I do not know if they are relevant, but I have attached the boot configurations just in case I have enabled a device that also is reserving ram at boot.

$ cat /boot/firmware/cmdline.txt 
console=serial0,115200 console=tty1 root=PARTUUID=e668d3db-02 rootfstype=ext4 fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles cfg80211.ieee80211_regdom=BE

$ cat /boot/firmware/config.txt | grep -v '#' | grep -v -e '^$'
dtparam=spi=on
dtparam=audio=on
camera_auto_detect=0
display_auto_detect=1
auto_initramfs=1
dtoverlay=vc4-kms-v3d,spi1-3cs
max_framebuffers=1
disable_fw_kms_setup=1
arm_64bit=1
disable_overscan=1
arm_boost=1
[cm4]
otg_mode=1
[all]
start_x=0
gpu_mem=16
enable_uart=1

Thank you.

[gugolple]

Here are the logs of executing the same commands on my Raspberry Pi Zero W. These are coherent with a device with ~500MB of ram with ~64MB separated for the GPU split.

$ dmesg | egrep 'Memory|CMA|cma' --color -C3
[    0.000000] Memory: 374380K/458752K available (8694K kernel code, 1324K rwdata, 2828K rodata, 424K init, 836K bss, 18836K reserved, 65536K cma-reserved)

$ cat /proc/meminfo
MemTotal:         440340 kB

$ vcgencmd get_mem gpu
gpu=64M

[popcornmix]

Can you identify the exact update which caused this. See:
https://github.com/raspberrypi/rpi-firmware/commits/master

If you click on each commit the end of the url contains a git hash. Run
sudo rpi-update <hash>
to revert back to that version. Report the first version with this error.

[pelwell]

Some before and after comparisons would be useful - that looks to be all after.

I note that you've opted into the 64-bit kernel (and perhaps even the 64-bit OS - it's hard to tell). This is not recommended on a Zero 2 W because it will eat into your limited RAM.

[gugolple]

Can you identify the exact update which caused this. See: https://github.com/raspberrypi/rpi-firmware/commits/master

The issue was introduced at the commit e632362b0399b4ce331aacd9386685bc60938ab7

Some before and after comparisons would be useful - that looks to be all after.

I note that you've opted into the 64-bit kernel (and perhaps even the 64-bit OS - it's hard to tell). This is not recommended on a Zero 2 W because it will eat into your limited RAM.

Free before the commit (using commit af3ab905d42829d4ce860f19a0e049c3bcdbe35f):

$ free -h
               total        used        free      shared  buff/cache   available
Mem:           466Mi       165Mi       138Mi       1.2Mi       218Mi       300Mi

Free after appling the commit e632362b0399b4ce331aacd9386685bc60938ab7:

$ free -h
               total        used        free      shared  buff/cache   available
Mem:           400Mi       154Mi        88Mi       1.2Mi       209Mi       246Mi

[gugolple]

Some before and after comparisons would be useful - that looks to be all after.

I note that you've opted into the 64-bit kernel (and perhaps even the 64-bit OS - it's hard to tell). This is not recommended on a Zero 2 W because it will eat into your limited RAM.

Indeed I am running Raspberry Pi OS 64 bit version, but I do not see how it affects the topic at hand.

[popcornmix]

Ah - so was this the kernel update rather than the firmware?
That commit did move from 6.1 kernel to 6.6.

You should be able to confirm this by copying /boot/firmware/{start*.elf,fixup*.dat} from the "good" rpi-update commit and the "bad" commit somewhere safe, and check if just swapping those files over changes free memory available.

You can check firmware and kernel versions after a reboot with:

uname -a
vcgencmd version

[pelwell]

but I do not see how it affects the topic at hand.

Do you care about how much usable RAM your Z2W has?

[pelwell]

Note that as well as having unavoidably larger page tables, lower code density, etc., there is a common 64-bit kernel that runs on Pi 3s, 4s, 5 and Z2W. This means that your Z2W has used some of its RAM to hold support for booting a Pi 5 to the point where it can access its SD card, USB, and Ethernet interfaces. kernel7.img is only for Pi 3s and Z2W, therefore it is much smaller. Note also that the 64-bit kernels are GZIP compressed - uncompressed they are roughly 25MB.

[gugolple]

Ah - so was this the kernel update rather than the firmware? That commit did move from 6.1 kernel to 6.6.

You should be able to confirm this by copying /boot/firmware/{start*.elf,fixup*.dat} from the "good" rpi-update commit and the "bad" commit somewhere safe, and check if just swapping those files over changes free memory available.

You can check firmware and kernel versions after a reboot with:

uname -a
vcgencmd version

I have run out of time for the moment, but copying either set of files from the "bad" commit to the "good" one does not appear to solve the issue. I am missing to test the other permutation.

[gugolple]

Note that as well as having unavoidably larger page tables, lower code density, etc., there is a common 64-bit kernel that runs on Pi 3s, 4s, 5 and Z2W. This means that your Z2W has used some of its RAM to hold support for booting a Pi 5 to the point where it can access its SD card, USB, and Ethernet interfaces. kernel7.img is only for Pi 3s and Z2W, therefore it is much smaller. Note also that the 64-bit kernels are GZIP compressed - uncompressed they are roughly 25MB.

Correct, that affects memory utilization, not the amount of memory available to the system, that is the issue I am suffering.

[popcornmix]

I have run out of time for the moment, but copying either set of files from the "bad" commit to the "good" one does not appear to solve the issue. I am missing to test the other permutation.

I'd expect copying firmware from "bad" to "good" would either have no effect (i.e. it's not a firmware issue) or to make memory available worsen (i.e. it is a firmware issue). Can you confirm which result you had ("solving" wasn't an expected outcome from this test).

[gugolple]

The results from the improvised experiment:

	af3ab9	e63236
start*.elf af3ab9 fixup*.dat af3ab9	Correct	Bad
start*.elf e63236 fixup*.dat af3ab9	Correct	Bad
start*.elf af3ab9 fixup*.dat e63236	Correct	Does not boot
start*.elf e63236 fixup*.dat e63236	Correct	Bad

After seeing the results, I do believe the issue is with the new 6.6 kernel and not with the firmware.
Thank you so much @popcornmix for the support given. I believe the issue should be closed here and open in the kernel repo. Would you kindly confirm?

[popcornmix]

start*.elf and fixup*.dat should always be of the same version, so not booting is reasonable.

I've transferred this issue to the linux repo as that appears to be the cause of the increased memory.

As @pelwell says, it is worth testing how much of the increased memory is due to 64-bit kernel (which now also supports pi5, which the 32-bit kernel does not).

Is your OS 32-bit or 64-bit? If it's 32-bit then booting without arm_64bit=1 would be informative.
If it's 64-bit then it's trickier to test (will need a 32-bit install).

[gugolple]

As @pelwell says, it is worth testing how much of the increased memory is due to 64-bit kernel (which now also supports pi5, which the 32-bit kernel does not).

Is your OS 32-bit or 64-bit? If it's 32-bit then booting without arm_64bit=1 would be informative. If it's 64-bit then it's trickier to test (will need a 32-bit install).

Currently I'm running "Raspberry Pi OS (64-bit) : Raspberry Pi OS Lite".

What tests and versions would be intriguing to execute? I am open for suggestions.