Native windowing for Raspberry Pi 4

[ofTheo]

We've had a lot of requests since 0.11.0 for native windowing for Rpi 4.
@jvcleave has an ofxRPI4Window addon which handles just the window ( no mouse / keyboard ) using libdrm.

I recently tried using his window / surface creation from his addon with ofAppEGLWindow.cpp and it works great.
ofAppEGLWindow works fine for Rpi 4 if you sub out the window / display calls from ofxRPI4Window.

Right now ofAppEGLWindow is a bit of a mess because of the split support for X11 and native windowing on older PI devices. I am nervous to add yet another pathway with dependencies for libdrm window creation.

If the window code from https://github.com/jvcleave/ofxRPI4Window could be used for older devices too then it would be cleaner to have the native stuff in ofAppEGLWindow swapped for the DRM code. ( To me that would be ideal )

The other option would be to have something like ofAppDRMWindow which would be similar to ofxRPI4Window but with the event stuff from ofAppEGLWindow. This would add another file to OF and duplicate a lot of code that is already in ofAppEGLWindow.

Would love some thoughts on this.
Tagging @bakercp @jvcleave

[jvcleave]

My initial thought would be to be to include ofxRPI4Window as a core addon.

It would be nice to have a more general define to be able to bypass ofSetupOpenGL that ofxRPI4Window requires.

#if !defined( SKIP_OPENGL_SETUP )
    void ofSetupOpenGL...

https://github.com/openframeworks/openFrameworks/blob/master/libs/openFrameworks/app/ofAppRunner.cpp#L31

[ofTheo]

@jvcleave thanks for the feedback!

I think we'd def want to get the ofxRPI4Window to implement the rest of the window features if we were going to use it directly. ( Mouse / keyboard and window position, shape etc. )

I kept going back and forth as most of the functionality in ofAppEGLWindow still works aside from the actual window creation.

Do you know if ofxRPI4Window would work on a RPi 3 / 3+ ?
Or is it specifically only compatible with Rpi 4?

Would be great if it could be backwards compatible with older devices too ( just from a maintenance standpoint ).

Either way we can definitely make the ofSetupOpenGL stuff easier to skip if not using the GLFW Window.

[jvcleave]

ofxRPI4Window may work on an RPI3/3+ but I am guessing it depends on what version of Raspberry Pi OS.

@patriciogonzalezvivo Do you know if the libdrm/gbm stuff works on the RPI3?

[ofTheo]

I might have to grab one just to be sure.
Looking at this post they mention getting DRM approach working with SDL on a Pi 3 with Buster. So it sounds like it might be possible.

Removing that previous link, but from what I can tell VideoCore and drm-vc4 support all the Rpi chips.
There might need to be some subtle tweaking to the setup calls, but I think it would be worth trying on a Rpi 3 or 3+

https://en.wikipedia.org/wiki/VideoCore

[okyeron]

Would love to see if this work would be applicable to issues getting shader examples to work on RasPi (see also #5056) and also some jitter at high framerates I'm seeing with ofSetBackgroundAuto with the KMS drivers.

Happy to do some testing as needed on either/both pi3b+ and pi4b

[ofTheo]

Hey okyeron
Thanks! It would be really helpful to see if @jvcleave 's addon works for you on pi3+

See:https://github.com/jvcleave/ofxRPI4Window

If it does ( which I suspect it will ), then we could think more about using this method as the main windowing approach for Rpi.

Thanks!
Theo

[ofTheo]

Also for reference here's the window code from glslViewer which supports DRM and other window creation approaches: https://github.com/patriciogonzalezvivo/glslViewer/blob/master/src/window.cpp

[patriciogonzalezvivo]

hi! Sorry for jumping on this late. I abstracted all the window creation in this repo: https://github.com/patriciogonzalezvivo/ada/blob/master/src/window.cpp

It use GLFW for MacOS, Windows, WebAssembly and X11 (linux)

For the Pi use VC libraries depending on it use legacy drivers or GBM for the new chips present only on RPi4.

I notice some last draw backs on the last one. I can't get OpenMAX/MMAL libraries to work properly on those, and the context on full alpha doesn't show the console. Also @jvcleave have notice that runs a bit slower than the legacy drivers.

[ofTheo]

Thanks @patriciogonzalezvivo !
Your window.cpp is a great reference. Looks like it does support the legacy bcm stuff too.

Also saw this note in terms of performance:
_"However, the "legacy" driver has much lower overhead because it only supports the specific OpenGL ES 2.0 calls that the underlying hardware does, and it offloads a lot of the state keeping and heavy lifting to the VideoCore 4 GPU firmware." @okyeron that could be a clue to the OpenGL ES issues you were mentioning.

[patriciogonzalezvivo]

Interesting. I thought was related on the linked libraries I was using for the legacy version https://github.com/patriciogonzalezvivo/glslViewer/blob/master/Makefile#L47-L51 that have a custom/mystical Broadcom version : )

[patriciogonzalezvivo]

Yes, supports both LEGACY and GDM (aka FAKE_KMS ) versions. Note here how the GL headers also change https://github.com/patriciogonzalezvivo/glslViewer/blob/master/src/gl/gl.h

[jvcleave]

that note is interesting as I've noticed some GLSL calls like mix() are crippling slow on the RPI