Provide API to query backend availability at runtime

[GregoryComer]

🚀 The feature, motivation and pitch

Internally, backends implement an is_available method to determine if the backend is actually present at runtime (detecting where a supported NPU is present, for example). To my knowledge, this isn't actually exposed through the public runtime API surface.

This is an important usability feature for users who want to support multiple delegates, as users need a way to determine which hardware is actually present at runtime. It is more or less required for people to be able to use NPU delegates, unless some other solution exists to solve this.

This could be done by simply adding a method to the core runtime to enumerate available backends. Alternatively, there was an ongoing discussion and design proposal for generalized backend capabilities that may also provide this functionality (I believe @guangy10 was driving this a while back).

Edit: More clearly enumerating the problem and use case, imagine you are a user that wants to target one or more NPUs on Android, falling back to CPU if none are available. You generate a PTE for each backend. How do you know which one to actually load at runtime? Ideally, executorch should be able to just say qualcomm and xnnpack backends are available, for example, or maybe just xnnpack on a device without a supported qualcomm NPU. The user code can then choose a .pte based on this information.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @JacobSzwejbka @dbort @mergennachin @byjlw

[mergennachin]

Actually #8198 was just added yesterday

Cc @dbort @tarun292

[GregoryComer]

Actually #8198 was just added yesterday

That looks like a nice to have, though I don't think it necessarily solves the entire issue, which is that the user needs to know which PTE to load at runtime depending on the available hardware. But it seems like a nice API addition, regardless.

[mergennachin]

cc @cccclai - what are your thoughts?

[cccclai]

Yeah this sounds very helpful

[dbort]

With #8198 we have an inside-out solution, but one that requires the app to know the names of the backend in question.

• Load the program
• Get the MethodMeta for a given method
• Use the new MethodMeta::uses_backend() to see if that Method uses a specific backend
• Try looking up the BackendInterface with that name in the registry using get_backend_class -- if it returns non-null, then it's available.

But this doesn't give us a way for a general-purpose solution:

• No way to ask for all backends used by a Method
• No way to ask for all backends registered with the runtime

We could add enumeration functions to answer those questions.

And/or we could add a targeted feature like bool MethodMeta::all_required_backends_present().

I think the enumerations would be a better fit for ET, since they'd unlock arbitrary logic and could answer other questions about backend usage and availability. E.g., this entire feature request could be considered part of the "is this PTE file compatible with the runtime on device X" question. Being able to enumerate available and used backends would be useful when answering that question. We could have a helper function like bool all_required_backends_present(MethodMeta& mm) that's implemented in terms of them, if we think that this is a common question that applications will have.