"Always Strong" Asset Handles / Remove asset weak handles

[andriyDev]

What problem does this solve or what need does it fill?

Today, all main user APIs for accessing assets use Handles. Handles come in two flavors: strong handles which keep the asset from being removed, and weak handles which don't keep the asset from being removed. Weak handles are just a wrapper around the raw AssetId. Note that strong handles also include an AssetId. So it seems we are duplicating effort here (if you needed a "weak handle", why can't you just store AssetId directly?)

Most assets can and should be removed when all their handles are dropped. However currently there is one exception to this: UUID assets. Handles to these assets are usually defined as const with the weak_handle! macro. Since they are defined in const, they don't correspond to a live asset. Therefore, these must be weak handles. And these assets need to be used in user APIs, so it needs to be a Handle, not just an AssetId. This has also resulted in requiring the Assets resource to contain a "dense storage" for regular assets, and a "hash storage" which is just a hashmap from the UUID to the asset. This duplicate implementation is quite unfortunate!

What solution would you like?

Remove weak handles! Stop supporting UUIDs and always use strong handles.

[andriyDev]

Some cases to consider:

1. Shaders come in two forms (which I'm gonna give fake names): kernels and libraries.
   • Kernels are passed into render pipelines so we need their concrete handle to pass in there. Today having weak handles makes this very easy since you can access the shader weak handle directly in your render pipeline without needing to pass it through. The most obvious case of this is FULLSCREEN_SHADER_HANDLE which many pipelines need to use.
   • Libraries on the other hand are indexed so they can be loaded using their #define_import_path field inside the asset. When your shader does #import blah::whatever, when the pipeline is being built, this import path is looked up in the indexed shaders and is glued to your shader. This has the convenient effect that you can import any shader by its import path. However this requires the shader to be loaded, and today we do this by assigning to a weak handle. Note these shaders practically never need to be referenced directly, we only need them to be stored so they can be glued later.
2. AssetId::DEFAULT_UUID and AssetId::INVALID_UUID provide ways to define the default and invalid values for each asset type. These can be used in Default trait implementations because they are const. Note users can also define their own versions of these for their particular use case (e.g., MY_DEFAULT_SPRITE_UUID).

[mockersf]

Today, asset handles can either be "strong" - keeping an asset alive in regular dense storage, or "weak" - just a UUID that can be filled in later (or never).

This is not the difference between strong and weak asset handles.

Strong handles have a sender that they used to check the number of user of a given asset. when that number reach 0, the asset is cleaned from memory. Cloning the strong handle increments the counter, dropping it decrement it.

Weak handles don't update that counter.

Both weak and strong handles can be in the dense or uuid storage. both can be filled now or later, or never (loading a file that will result in an asset error will give you a strong handle that will never work)

[mockersf]

This results in Assets being split into two parts, the dense storage (i.e., generational Vec), or hash storage (literally just a HashMap from UUID to asset).

The types of storage are not split by the kind of handles, but by the origin of the asset.

Dense is used for assets that are directly added (with Assets::add), UUID is used for assets that are loaded from a file

[andriyDev]

Both weak and strong handles can be in the dense or uuid storage.

This is actually totally correct! I got ahead of myself in describing the solution rather than the problem. I completely missed that technically you can get a strong handle to a UUID asset, so I'm gonna rewrite the description to reflect that. Though that seems like an accident rather than intended behaviour (maybe I'm wrong on that though).

UUID is used for assets that are loaded from a file

This is not true. Assets that are loaded from a file are given a AssetId::Index which is dense storage. The only way to get AssetId::Uuid is by making one yourself, or through weak_handle! macro. The only way to get a strong handle to a UUID asset is by using weak_handle! to get the weak handle, then using Assets::get_strong_handle to turn it into a strong handle - very round about so I am fairly confident this is not desired, just happens to work out.

We are trying to remove Handle::Weak, for which the primary use case is UUID assets. Any uses that need a weak handle to a non-UUID asset would just use AssetId directly. UUID assets are the sticking point which is why I was focusing on it. Thanks for calling that out though, I will clean it up now!

Edit: reworded the description :)

[andriyDev]

For the "shader kernel" case: I think we should just be storing this in a resource - most likely the pipeline resource that we commonly make for our render pipelines. These can clone the handle all the way into the render pipeline. It is a little more piping, but seems like the right thing to do. The one case that really sucks here is specifically FULLSCREEN_SHADER_HANDLE because it's used in many places. Pretty much every other shader kernel is a "single-use" shader kernel, or is limited to a single module. FULLSCREEN_SHADER_HANDLE is so generic and easy to reach for, that it would be a little sad to have to pipe it through.

For the "shader libraries" case: So far all the shader library cases I've seen have just used load_internal_asset! to load the shader into a weak_handle const that is then never used. We should instead just have a load_shader_library! macro that 1) Assets::adds the shader, 2) std::mem::forgets the handle. Since the asset is still loaded, it will be indexed just like today, and since we did std::mem::forget the handle will be permanently "held" so the asset will never be unloaded. We could also use embedded_asset! rather than adding the asset in manually, though I'm not sure if this will have a delay that is undesirable or not?

[andriyDev]

As for the AssetId::DEFAULT_UUID, cart has previously stated that Construct should help with this. My understanding there is rather than AssetIds being filled in a Default trait implementation, Construct will allow us to look up the handle on insert to the world. Or something to that effect! This means waiting for Construct to be implemented before being able to be rid of weak handles, but hopefully that should be coming soon. We'll have to see how that turns out.