Update Golang to 1.23 and Spark to 3.5.2

[jacobsalway]

Purpose

Upgrade Golang from 1.22.5 to 1.23 and Spark from 3.5.0 to 3.5.2, which includes both the Dockerfile and the examples.

Testing

• The example JAR is present on the 3.5.2 images and examples/spark-pi-python.yaml ran on Kind.
• There is nothing in particular I can see in the Spark 3.5.1 or 3.5.2 changelogs that would impact how the operator works (there is [SPARK-45670][CORE] SparkSubmit does not support --total-executor-cores when deploying on K8s apache/spark#43536 but this flag isn't used when constructing the spark-submit command internally)

Change Category

Indicate the type of change by marking the applicable boxes:

• Bugfix (non-breaking change which fixes an issue)
• Feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that could affect existing functionality)
• Documentation update

Checklist

Before submitting your PR, please review the following:

• I have conducted a self-review of my own code.
• I have updated documentation accordingly.
• I have added tests that prove my changes are effective or that my feature works.
• Existing unit tests pass locally with my changes.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yuchaoran2011 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[yuchaoran2011]

Do we need the minor version or can we use simply 1.23?

[jacobsalway]

Definitely not strictly necessary given Go's backward compatibility guarantees and not pinning should reduce the frequency of toolchain upgrade PRs. Updated

[ChenYi015]

I think the patch version is needed, otherwise go mod tidy will fail:

$ go mod tidy
go: downloading go1.23 (darwin/arm64)
go: download go1.23 for darwin/arm64: toolchain not available

There are some issues related to the toolchain version not being available: golang/go#62278.

[ChenYi015]

@jacobsalway Could we have two dedicated PRs, one for bumping Golang version to 1.23.0, and another for bumping Spark version to 3.5.2.

[ChenYi015]

We should merge #2149 first so that the e2e test CI can pass.

[jacobsalway]

@ChenYi015 you'll need to get an approver to add the label on the other PR since I only have review permission

[ChenYi015]

@ChenYi015 you'll need to get an approver to add the label on the other PR since I only have review permission

@jacobsalway I had approved the e2e fix PR myself.

[jacobsalway]

Didn't realise the bot allowed you to approve your own PR 🤔

Splitting this PR into #2155 and #2154

[ChenYi015]

Didn't realise the bot allowed you to approve your own PR

I checked out the community guide and found that I can approve my PR, but I cannot lgtm my PR.