Adding support for WebXR

[shiukaheng]

Hi, this is more of an open ended issue. I'm currently working on a r3f project that I'd like to use post-processing with, but it also has WebXR enabled for VR. I wonder if anyone has tried doing this previously? I would be interested to contribute towards such a feature / make a fork, but I am not entirely clear about how WebXR modifies the rendering process. Would be great if anyone could give some input, thanks!

[gsimone]

Your best bet is first researching what's involved in PP for webXR, then making an RFC at pmndrs/postprocessing and whatever changes are necessary to pmdnrs/react-postprocessing. I'd also research the three.js repo to see if any effort exists already for their "official" EffectComposer.

[CodyJasonBennett]

Three.js had the usual PR chain die from inactivity (mrdoob/three.js#18846), but they were concerned about the performance of their implementation since EffectComposer (forward) renders passes in series without MRT and wanted an uber-shader approach. postprocessing (which react-postprocessing depends on) does that and much more, IMO it's the only one of the two suited for production regarding this overhead.

I've had some attempts at making postprocessing work in WebXR:

• https://twitter.com/Cody_J_Bennett/status/1482585611781480448

And from scratch (regarding the lifecycle and how WebXR works):

• https://twitter.com/Cody_J_Bennett/status/1515269756647002114

My initial implementation is pretty lazy to render each eye completely separately, but I have concerns with effects that rely on screen resolution (it's effectively halved) and whether to stereo project ourselves via GLSL to enable stereo effects.

[shiukaheng]

@CodyJasonBennett I suppose we can find middle ground between ease of use and making a general solution. Perhaps we could create different variants of XREffectComposer components with different implementations? Considering stuff like bloom / vignette / chromatic aberrations cover most of postprocessing's use I think your implementation would already be a great solution for now, and other implementations could be added later. Would you be able to share the code for your demo? I'm really interested to see how compatible the solution would be with common effects. Thanks for the amazing work!

[CodyJasonBennett]

Centralizing the rendering logic in the effect composer sounds like a good first step. As for code, I have a sandbox here https://codesandbox.io/s/7x7p3q. IIRC the postprocessing version on this box doesn't work with WebXR layers (Oculus Quest) or three's RGBA breaking changes so try to bump it. The code in either tweet should be comparable. Do note that for render targets or layers, the viewport size must match the frame buffer resolution -- three will take care of this for you but you'll otherwise get cryptic WebGL warnings.