Panorama feedback and beyond

[leops]

Alright so now that the "update that included Panorama" has been out for a few days already and I had to mess around a bit with it, I'd like to open a discussion about the tool and where it's going as I got a few notes and remarks that don't really fit the "issue" format (I think).

First of all, the obvious pain point: the RSA signature check is annoying. That's exactly what it is for after all, to deter cheater from messing with the display in Counter Strike, but this is not a competitive multiplayer game but a community-oriented modding platform so I think it would make sense to just disable it in the short term.

Once everything is running though obviously not everything is in place but it's already fairly impressive in the possibilities it unlock, we're gonna be able to do some really cool stuff with this. The only two main limitations I see right now are:

1. The Panorama code is highly interlocked with the native code, with the C++ side expecting the layouts to be setup in a specific way and such. It ends up becoming a bit of a spit brain situation with some business logic being implemented on the native side and some being in JS with unclear separation of concerns. Since the goal of the project is to be highly customizable I guess the way to go will be to move as much stuff as possible out of the native side into the JS and limiting most interactions between the two to only go through events.
2. In-game integration is currently lacking, while the UI can act on the engine by way of dispatching commands I don't see a way for the game content to act on the UI. It seems the most sensible way of implementing this would be by dispatching event, but I don't see any API to do this yet either in VScript or through Hammer I/O. Also I don't see any entity for placing Panorama panels on world geometries, unless vgui_screen does that in Panorama mode ?

One interesting idea that stems from decoupling the native and JS code is that from what I understand P2:CE will ship with its own Panorama-based UI, and I'm no legal expert but since this is all going to be all new code it may not fall under the Valve license, and that part of the engine might be able to be developed as an open-source community effort, taking off some of the workload from just the developers under NDA.

Finally, and this is laying some ideas here but while Panorama is certainly a step forward from VGUI it feels a bit dated for anyone experienced in web development: for instance spreading the layout, styles and code over 3 different files is really something that's just not done anymore. Since Panorama takes so much from the web I think it could be interesting to take a look at how the web platform is shaped right now, where it is going and what we can learn from that. Two libraries I'm thinking of trying to integrate are the elephant in the room, React, and the outsider, Svelte.
There isn't necessarily any work to be done to "support" JS libraries as most of the work would in fact need to be done on the libraries themselves, but modern web development workflows rely a lot in live reloading, and for that to work 2 pieces are required: the ability to load the JS code from a (generally local) HTTP server instead of a file (and to reload it from inside the runtime), and an implementation of the WebSocket protocol for that the JS bundle can connect back to the server and receive updates. Obviously none of that is very important for actually using Panorama, but is certainly nice to have for productivity.

[StefanH-AT]

P2CE's panorama ui can be open sourced but we will not use react or anything like that

[leops]

Yeah I am not advocating that P2CE uses React, locking-in all users to a specific framework is not good. But it is possible to load React into Panorama, and in fact in the last few days I did exactly that (React is really easy to embed into pretty much anything since it's really decoupled from the environment it runs in due to React Native).

Now while it does work, there are a few points that could be improved, and not only for a potential React user but for anyone using Panorama. Right now the biggest limitation of the API right now is that panels cannot exist outside of the hierarchy, they have to be created with a valid parent and I don't think there's anyway to detach them but since React progressively creates the panels in the rendering phase and inserts them into the tree in the commit phase they have to be temporarily attached to the root panel and masked until they get committed. Aside from that, not being able to load new blocks of CSS at runtime is a blocker for styled-components, and the trailing-semicolon requirements on the style attribute is non-standard with regards to HTML and can throw off the static generation of layouts (but this one is easily fixable).

Also before I forget about it: there's a side project from React called Hermes that's a custom JS virtual machine, developed for the needs of React Native, and I think those needs align with those of Panorama: good startup times, small memory footprint, and possible optimizations from compiling JavaScript ahead of time into bytecode. It could make an interesting alternative to V8 as the JS Runtime in Panorama.

[JJL772]

Replacing v8 with hermes would be a substantial amount of work for something that frankly doesn't bring much benefit.

[RealityAnomaly]

On the C++ integration side, we are currently working on implementing a hotloading C# scripting framework into the engine called ChaosScript. In the long term, we're aiming to replace both the frontend JavaScript code and the C++ backend code with C#, so the logic is transparent and editable by anyone.

[leops]

So do I have to understand we can't get started writing UIs with Panorama yet since we'd have to throw away our JS code later and rewrite that in C# ?

[JJL772]

I'm going to be honest, we need to discuss this a bit more. We will get back to you on that.

[JJL772]

Sorry for the long response time here lol, I forgot about this discussion.

We've decided to just keep JS around. It works well enough, I don't see a need to rip that all out and replace it with C#.