Introduce the object stores in datafusion-contrib as optional features in the datafusion core #2072
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently when we utilize the datafusion core as a library, it's possible to do the remote object store registration manually in the script code. However, for a long running standalone system, like Ballista, there's no place to do the registration manually. And in general, the remote object stores supported will not be changed. Therefore, it's better to provide a mechanism to register some object stores automatically rather than manually do the registration.
Describe the solution you'd like
With #2065, the object store extensions will no longer depend on the datafusion core. And now the datafusion core can depend on them. We can set those object stores as optional features in the datafusion core and let users decide whether to introduce them at the compilation time.
And since those extensions are maintained in their own crates, there's not much maintenance effort for the datafusion.
Describe alternatives you've considered
Additional context
It's blocked by #2076