Move PhysicalPlanner to physical_planer module

[alamb]

Which issue does this PR close?

Part of #1754 (see #1754 (comment))

Rationale for this change

I am trying to extract physical_plan into its own crate to improve modularity and compile times

I am also trying to make sure the organization of the DataFusion crate is easier to understand for beginners.

What changes are included in this PR?

Move PhysicalPlanner and related structures to physical_planner module

Are these changes tested?

Existing tests

Are there any user-facing changes?

Yes, documentation is improved (the planner is no longer buried in a sub module). Also, if users used the PhysicalPlanner directly they will need to update their use path

[Ted-Jiang]

Make sense to me 👍

btw: its own crate to improve modularity and compile times
How to improve the compile times ? increase the degree of parallelism 🤔

[alamb]

How to improve the compile times ? increase the degree of parallelism 🤔

Yes that is basically the hope/plan

[viirya]

I am trying to extract physical_plan into its own crate to improve modularity and compile times

So physical_planner will be into the same crate as physical_plan? Or a different crate?

[alamb]

So physical_planner will be into the same crate as physical_plan? Or a different crate?

I think initially it will be in the datafusion (datafusion/core) crate (where it is now).

We could potentially move it to its own crate in the future too (though given it is not that much code, I don't know how valuable that would be)

The issue is that the planner instantiates a bunch of things defined in datasource;

The key issue in extracting physical_plan from datafusion/core is that datasource is ALSO in datafusion/core and I can't easily take it out (b/c TableProvider depends directly on SessionState).

Since the physical planner depends on datasource it has to stay in datafusion/core too, at least initailly.

Once I have physical_plan and datasource detangled (so datasource depends on physical_plan but not the other way round) I can finally pull out physical_plan 😅 -- at least that is the hope