[EPIC] Improve examples to make them easier to navigate

[alamb]

Is your feature request related to a problem or challenge?

The current number of examples in https://github.com/apache/datafusion/tree/main/datafusion-examples is somewhat overwhelming and hard to navigate

Describe the solution you'd like

I would like fewer consolidated examples. For example a sql.rs example rather than SQL examples being spread out across (at least) 9 different files

• parqet_sql
• parqet_sql_multiple_files
• regexp
• to_char
• to_date
• to_timestamp
• make_date
• avro_sql.rs

Subtasks

• Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs #13842
• Consolidate Example: dataframe_output.rs into dataframe.rs #13844
• Consolidate Example: simplify_udwf_expression.rs into advanced_udwf.rs #13842 #13843
• Replace println! with assert! if possible in DataFusion examples #11230
• datafusion-examples CI run is failing: final link failed: No space left on device #11210
• Automatically test doc examples in user guide #1813
• Consolidate Example: dataframe_subquery.rs into dataframe.rs #13911
• Consolidate Examples: memtable.rs and parquet_multiple_files.rs #13912
• Consolidate Examples: regexp.rs into dataframe.rs and sql_query.rs #13938
• Consolidate csv_opener.rs and json_opener.rs into a single example #13955
• Minor: consolidate ConfigExtension example into API docs #13954

Describe alternatives you've considered

No response

Additional context

Also, I am trying to add some new examples, but the CI in my PR #11088 is failing due to running out of space

Example failure: https://github.com/apache/datafusion/actions/runs/9715084012/job/26815819949

  = note: /usr/bin/ld: final link failed: No space left on device
          collect2: error: ld returned 1 exit status

I think it is because each example creates its own binary which fills up the disk

Thus I also want to reduce the number of examples so I can add new ones :)

[alamb]

While working with @lewiszlw in #11173 (comment) I think I came up with a better idea -- that is to update the user guide so they code examples are run / tested

Then we can move many/all of the examples from rust to the user guide, which will make them easier to find and navigate. 🏃

[alamb]

I played around with what we have and it turns out I think we can test examples in the user guide. I made a PR with docs of how to do it: #11178

I think with that framework we could add many of the simpler examples directly into the documentation where they would be eaiser to find. We can leave the more complex ones in datafusion-examples perhaps

Thoughts?

[efredine]

As a newcomer I'll say that this seems right to me. Examples that are right in the documentation are much more discoverable.