[BUG] OSError: [Errno 12] Cannot allocate memory

[Exper1mental]

Describe the bug
When I configure Picamera2 to load my Pi Camera Module 3 Wide settings, it throws a OSError: [Errno 12] Cannot allocate memory.

To Reproduce
Run the following code:

import io, logging, time
from threading import Condition

from picamera2 import Picamera2
from picamera2.encoders import JpegEncoder
from picamera2.outputs import FileOutput

# Int Picamera2 and default settings
picam2 = Picamera2()

# Create the video_config 
video_config = picam2.create_video_configuration(main={'size': (4608, 2592)}, sensor={'output_size': (4608, 2592), 'bit_depth': 10})
print(video_config)


####################
# Streaming Class
####################

output = None
class StreamingOutput(io.BufferedIOBase):
    def __init__(self):
        self.frame = None
        self.condition = Condition()

    def write(self, buf):
        with self.condition:
            self.frame = buf
            self.condition.notify_all()

####################
# Start/Stop Steam and Take photo
####################

def start_camera_stream():
    global picam2, output, video_config
    picam2.configure(video_config)
    output = StreamingOutput()
    picam2.start_recording(JpegEncoder(), FileOutput(output))
    metadata = picam2.capture_metadata()
    time.sleep(1)

def stop_camera_stream():
    global picam2
    picam2.stop_recording()
    time.sleep(1)

####################
# Lets get the party started
####################

if __name__ == "__main__":
    # Configure logging
    logging.basicConfig(level=logging.INFO)  # Change the level to DEBUG for more detailed logging
    print('0000 here')
    
    # Start Camera stream
    start_camera_stream()
    print('0001 here')
    
    # Stop Camera stream
    stop_camera_stream()

This is a simplified form of app.py from picamera2-WebUI-Lite v0.0.4 by @monkeymademe

Expected behaviour
The Picamera2 module to successfully be configured.

Console Output, Screenshots
Console output:

python test.py 
[7:06:05.884548075] [7854]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
[7:06:05.919087598] [7867]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[7:06:05.921804928] [7867]  INFO RPI vc4.cpp:447 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media1 and ISP device /dev/media2
[7:06:05.921900703] [7867]  INFO RPI pipeline_base.cpp:1144 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[7:06:05.925683615] [7854]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
[7:06:05.958829344] [7870]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[7:06:05.961256942] [7870]  INFO RPI vc4.cpp:447 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media1 and ISP device /dev/media2
[7:06:05.961355495] [7870]  INFO RPI pipeline_base.cpp:1144 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
{'use_case': 'video', 'transform': <libcamera.Transform 'identity'>, 'colour_space': <libcamera.ColorSpace 'Rec709'>, 'buffer_count': 6, 'queue': True, 'main': {'format': 'XBGR8888', 'size': (4608, 2592)}, 'lores': None, 'raw': {'format': 'SRGGB10_CSI2P', 'size': (4608, 2592)}, 'controls': {'NoiseReductionMode': <NoiseReductionModeEnum.Fast: 1>, 'FrameDurationLimits': (33333, 33333)}, 'sensor': {'output_size': (4608, 2592), 'bit_depth': 10}, 'display': 'main', 'encode': 'main'}
0000 here
INFO:picamera2.picamera2:Camera configuration has been adjusted!
[7:06:05.972848447] [7854]  INFO Camera camera.cpp:1183 configuring streams: (0) 4608x2592-XBGR8888 (1) 4608x2592-SBGGR10_CSI2P
[7:06:05.973335174] [7870]  INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@1/imx708@1a - Selected sensor format: 4608x2592-SBGGR10_1X10 - Selected unicam format: 4608x2592-pBAA
INFO:picamera2.picamera2:Configuration successful!
[7:06:06.358239064] [7870] ERROR V4L2 v4l2_videodevice.cpp:1241 /dev/video0[16:cap]: Unable to request 2 buffers: Cannot allocate memory
[7:06:06.358460595] [7870] ERROR RPI pipeline_base.cpp:663 Failed to allocate buffers
Traceback (most recent call last):
  File "/home/admin/Documents/2023-12-14_CameraCapture/picamera2-WebUI-Lite-0.0.4/test.py", line 61, in <module>
    start_camera_stream()
  File "/home/admin/Documents/2023-12-14_CameraCapture/picamera2-WebUI-Lite-0.0.4/test.py", line 42, in start_camera_stream
    picam2.start_recording(JpegEncoder(), FileOutput(output))
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1761, in start_recording
    self.start()
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1163, in start
    self.start_()
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1131, in start_
    self.camera.start(controls)
RuntimeError: Failed to start camera: Cannot allocate memory
INFO:picamera2.picamera2:Camera closed successfully.

Hardware:
Raspberry Pi 4B v1.5 (8GB RAM)
Raspberry Pi Camera Module 3 Wide

Additional context
OS: Raspberry Pi OS with Desktop 64-bit (Bookworm / Debian 12)

Let me know if you need additional information.

Thanks!

[davidplowman]

Hi, have you increased the amount of CMA memory available? If not, please check out page 58 of the manual.

A couple of other tips:

• Do not set or increase the amount of GPU memory (this was a thing you could try with the legacy camera stack, but is actively harmful with the modern stack).
• These are very large buffers so you may need to allocate fewer. By default I think a "video configuration" gives you 6, so maybe try adding buffer_count=4 to the create_video_configuration call.

[Exper1mental]

Surprisingly, increasing CMA memory appears to have no impact. Increased to both 512 MB and 2048 MB with no effect.

Changing buffer count appears to resolve the issue. buffer_count=5 or less works. Default, as you said, is 6. Changing CMA memory did not have an effect on this.

[davidplowman]

Slightly surprising. Maybe look at grep Cma /proc/meminfo to check how much you have (and whether the update has taken effect). Note that you can't just set the CMA to any value. 512MB might be the largest, can't remember if 640 is an option too.

[Exper1mental]

Without setting CMA memory in config.txt it is already at 512MB:

CmaTotal:         524288 kB
CmaFree:          441276 kB

I reboot anytime I change CMA memory.

If the dtoverlay -h vc4=fkms-v3d command is correct, 512 MB is the max CMA memory allocatable for the Pi 4B:

Name:   vc4-fkms-v3d

Info:   Enable Eric Anholt's DRM VC4 V3D driver on top of the dispmanx
        display stack.

Usage:  dtoverlay=vc4-fkms-v3d,<param>

Params: cma-512                 CMA is 512MB (needs 1GB)
        cma-448                 CMA is 448MB (needs 1GB)
        cma-384                 CMA is 384MB (needs 1GB)
        cma-320                 CMA is 320MB (needs 1GB)
        cma-256                 CMA is 256MB (needs 1GB)
        cma-192                 CMA is 192MB (needs 1GB)
        cma-128                 CMA is 128MB
        cma-96                  CMA is 96MB
        cma-64                  CMA is 64MB
        cma-size                CMA size in bytes, 4MB aligned
        cma-default             Use upstream's default value

What is odd is that when running the stream I am not getting anywhere close to maxxing out CMA memory, which I would've expected given reducing the buffer count fixes the issue.

Maybe it is just odd to me? I got it working so I am ok with closing this issue if there are no further thoughts on the matter.

EDIT: Ran it again and checked CMA.

Before running script:

CmaTotal:         524288 kB
CmaFree:          441276 kB

After running script:

CmaTotal:         524288 kB
CmaFree:          142320 kB

[Exper1mental]

Closing, looks like everything is behaving as intended. Thanks for the help!

[noppasin-jtwt]

what the script you have run?

[Exper1mental]

what the script you have run?

Iirc this was the script in the OP. That or the one it was based on (linked in the OP)