[CI] Fix Connection time out in Python workflow

[furqaankhan]

Did you read the Contributor Guide?

• Yes, I have read the Contributor Rules and Contributor Development Guide

Is this PR related to a JIRA ticket?

• No, this is a CI update. The PR name follows the format [CI] my subject

What changes were proposed in this PR?

• Remove fetching spark jars from archive.apache.org due to download limits https://infra.apache.org/infra-ban.html.
• Restructure the workflow to remove unnecessary code.

How was this patch tested?

• Passed CI

Did this PR include necessary documentation updates?

• No, this PR does not affect any public API so no need to change the documentation.

[furqaankhan]

@jbampton @jiayuasu Not sure why the lint is not passing. It is passing on my fork's PR: furqaankhan#2

[jbampton]

Hey @furqaankhan the requirements txt fixer hook is failing.

Basically docker/sedona-spark-jupyterlab/requirements.txt is not sorted.

I have the fix in #1654

[jiayuasu]

This value /home/runner/.local/share/virtualenvs/python-${PYTHON_VERSION}/lib/python${PYTHON_VERSION}/site-packages/pyspark is so long. Can you make it a global env?

[jiayuasu]

A quick example about global env from ChatGPT

name: Matrix CI Workflow

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest

    # Define a matrix of input values
    strategy:
      matrix:
        environment: [staging, production]
        node: [16, 18]

    # Use the matrix value inside `env`
    env:
      ENV_NAME: ${{ matrix.environment }}
      NODE_VERSION: ${{ matrix.node }}

    steps:
      - name: Check out repository
        uses: actions/checkout@v3

      - name: Print Matrix Inputs as Environment Variables
        run: |
          echo "Environment: $ENV_NAME"
          echo "Node Version: $NODE_VERSION"

      - name: Use Environment-Specific Logic
        run: |
          if [ "$ENV_NAME" == "production" ]; then
            echo "This is a production build.";
          else
            echo "This is a staging build.";
          fi

[furqaankhan]

I didn't add the whole path because it would be easier this way to use the existing virtual environment at lines 143 and 158.

[jiayuasu]

Great work!