plugins: load plugins from providers

[mobuchowski]

In some cases - like when provider is included by breeze prod-image build --install-providers-from-sources plugins are not loaded, as they are not separate package with entrypoint then. This PR adds loading plugins from provider.yml. I think this is somehow similar to what @potiuk is doing with stuff like config or executors.

In some cases this would make entrypoint plugins double-load - so we check if the plugin is already loaded. We could avoid that by not generating entrypoints for plugins, but this would make them not load on earlier versions of Airflow.

[potiuk]

Oh yeah. Good point. I briefly looked at it - looks like the right direction - I think there is a bit of a problem of where the plugins are loaded from during tests as some tests expect to load plugins from "example_dags/plugins" and I think this PR removes that path somehow,

It's going to be much clearer and more straightforward after #32669 will be merged, because one of the things it does is that it unifies the unit test (local venv) / and CI environment for tesst so that it is configured in one place only and dynamically loaded via pytest fixture from here:

https://github.com/apache/airflow/pull/32669/files#diff-1748727a9777b78ae8c17034fa1c5d4a4fd38de8ce986ef13e6531b347f65588R41

So it might be a wise thing to wait after #32669 is merged, rebase and try to fix it there (happy to help/brainstorm - I got quite a bit more familiar with how unit test configuration is loaded by implementing the PRs)

[potiuk]

not generating entrypoints for plugins, but this would make them not load on earlier versions of Airflow

BTW. We can't do it even for the future - because when Airflow gets providers installed as packages, using entrypoints is the only mechanism how it can discover plugins, there are no provider.yaml files any more in packaged version of Airflow (and can't be because providers are isolated and independent from Airflow and there might me different versions of both - community and 3rd-party providers installed together with Airflow, so the only way Airflow can find out what plugins are availabe is by querying the entrypoints.