Create blender import/export plugin for resonite packages

[gentlecolts]

Describe the Underlying Issue

It would be extremely convenient if we had a means for seamlessly moving content to and from blender.

To Reproduce

No response

Expected Behavior

There are obvious cases where blender scenes and Resonite cannot be 1:1, however models, rigs, and lighting (minus area lights) could be, and materials could still make some best effort (at least for PBR materials), all in all producing an experience that'd be better than attempting to do the same with intermediary format

Screenshots / Video

No response

Additional Context

No response

Reporters

zangooseoo

[Frooxius]

We're very unlikely to implement something specifically like this. Resonite Packages are designed for storing already existing content in Resonite, but it's not designed as an interchange format or format to be generated outside of Resonite.

Implementing that would be a significant engineering task - we'd need to implement a large chunk of Resonite's codebase for Blender to support this.

What would be more viable though is something that'd be more similar to the Unity SDK - #6 - a Blender SDK of sorts.

It's still a big chunk of work, but on technological level, this would be a lot more feasible approach. However given the size of this task, we'd need to see significant interest in this before we'd commit to implementing it. We're more likely to prioritize Unity SDK first too.

However if we take approach with the on-network "DOM" style interoperability for the Unity SDK, this would make a Blender SDK simpler to implement, as it would be built on the same mechanism - and even if we don't implement it ourselves, it'd be much easier for the community to do so.

[gentlecolts]

Oh yeah, I for sure expect this to be a big ask, and definitely something for the long-term.

we'd need to implement a large chunk of Resonite's codebase

If it's possible to split out the relevant portions, Blender plugins are python, and python can interop with c#, and some quick googling brings up the project pythonnet, which is specifically built for this purpose without having to involve ctypes. If I'm not mistaken, and forgive my ignorance on the technical details, but the upcoming performance changes are going to lead to some amount of restructuring the project anyways right? Might be worth keeping on back of mind, especially if there's plans for other forms of file format interop.

it'd be much easier for the community to do so.

This was honestly something I kind of expected the community to be working on the moment I heard resonite package was a thing, I already know there's existing efforts to try and get some unitypackage interop, and at some point I had floated the idea with friends of having a plugin that syncs a blender scene with a live reso session. Anything that further enables community solutions for this seems like a great step forward.

[FlameSoulis]

So.... if memory is serving me well, doesn't the Resonite Package also offer a non-varient version on export? If so, then that would be a version of a package that is just the base files and whatever component data is included.

I think what is mostly being asked is "how can one best talk in a way that makes Resonite not hate you" when it comes to making an export option for another system. Overte already does this with their Blender plug-in.

[TisFoolish]

Having a plugin that does the bare minimum for items and avatars I think would be enough.

EG for an avatar it lets you pick what type of material meshes use, assign the relevant colors and textures, place where the hands, grabbers, tool tip slot, tool shelf, and camera would be, and then on import into res the import process automatically sets up everything. The camera object could behave like a Blender camera in that you can view from its position/orientation.

I realize that the resonitepackage format is not set up for something like this as it's more of a 1:1 container and likely something else would arise instead, but I think this could be a way to minimize the amount of effort needed to make a plugin for exporting from Blender to res while not having to do the usual intermediary steps.

[gentlecolts]

items and avatars

Definitely an important use case, but If I'll be completely honest the use case I had in mind was making it easier to keep vrc and resonite worlds in sync

[bdunderscore]

For keeping VRC and resonite worlds in sync specifically, exporting from Unity is still probably better. This is something I hope to experiment with soon, as part of my Modular Avatar project (the core of the exporter should be extensible to worlds as well, if someone wants to put in the legwork to polish that part up...)

[Geenz]

I feel like for that specific use case, something like #6 would be far more effective here. As at that point, users can simply use our SDK and develop tooling to synchronize between Resonite and other platforms that are Unity based. The reason I say this is because there's a lot more to scenes than just geometry, just materials, just textures, etc. There's a very big functional aspect here as well.

With the methods and mechanisms we'd implement in #6 as a direct consequence of it, we could look into a "blender SDK" at some point similar to, say, how the Sony Santa Monica team has a Maya plugin that just connects to their engine running on a PS5. But that would need to come after #6 as it lays a lot of the ground work for that.

If you just want to export scenes that are more 1:1 with what is represented in blender, then you're asking more for improvements to the import aspect of the pipeline. If you're wanting things to be easier to "round trip" then today you're asking for:

• GLTF export to be more robust
• GLTF import to be more robust
  • Such as supporting the GLTF punctual light extension, the default metallic PBR shading, unlit extensions etc.

This requires some work to our import and export tooling for assets more broadly so that we can "break up" our pipeline across multiple plugins better than what we're presently able to.

[gentlecolts]

I'm away from my pc at the moment so apologies for the more brief response, but just to get my two cents out. The main reason I'd be in favor of focusing on Blender over Unity is that Unity is a less reliable third party, being proprietary and especially given how they've acted as of late, while blender is free open source software. If I'm not mistaken, the team is also further trying to move away from Unity anyways right? Interop with Blender would give paths towards working with asset pipelines involving other engines, such as Godot or Unreal, and I imagine work done on a blender plugin would more directly translate to plugins for other similar and useful software like Maya. If interop is in fact a goal, it's seems like it'd be harder to make "seamless" connections to other engines, plural, than to focus on making an import/export framework that the community could potentially build on, unless we plan to remain anchored to unity

[Frooxius]

@FlameSoulis The variants don't really matter. The base assets themselves are also heavily Resonite specific types - this includes mesh and data model assets. Those are the problematic bits here.

@TisFoolish There's unfortunately no "bare minimum" like that. The biggest part of this would be implementing support for generating and serializing data model, which is a necessary component. Once that is done, then supporting all components is the easy part. But there's no getting around of having to implement the bigger/harder bits even to support just setting up avatars (not to mention that avatars are fairly complex)

@gentlecolts While I agree that Unity is becoming more unreliable and I like/trust Blender way more over it, the main practical purposes of Unity SDK would be that:

1. Majority of social VR creators are used to its workflow. Having them learn Blender workflow could be an up-hill battle
2. A lot of users want to port their existing content in Unity to Resonite. Blender SDK wouldn't help with that unfortunately.

I think it's also important to mention that Unity SDK doesn't actually anchor us to Unity. It lets people to author/convert content from it to Resonite, but that doesn't require we keep using Unity.

---

Like I mentioned in my initial post, best approach for something like this would be analogous to Unity SDK in #6, since that will likely be implemented by introducing some sort of "DOM" that lets manipulate the data model externally.

We'd likely implement the Unity SDK with that first (given the interest in that feature and the benefits). We could look into adding Blender SDK as well, unless community takes that on instead in the meanwhile.

[TisFoolish]

The biggest part of this would be implementing support for generating and serializing data model, which is a necessary component. Once that is done, then supporting all components is the easy part.

I don't see why you'd need to do that in blender at all for my suggestion. Everything is just metadata that the engine uses to set up necessary components, slots, and references on import.

Thinking on it what I'm suggesting isn't that far of from the VRM format, just with a bunch of extra features for res.