feat(python-setup): resolve serverless version from pyproject.toml

[rugpanov]

Why

The environments team is introducing a [tool.databricks.environment] table in pyproject.toml that carries the serverless environment_version:

[tool.databricks.environment]
environment_version = "4"

The CLI writes this section during a serverless setup-local run. Because it is an explicit user declaration, it is a more reliable signal for the serverless version picker than the current heuristics (bundle YAML, notebook metadata, workspace default), and it lets the chosen value round-trip.

What

• Add a new pyproject scoring source, weighted above every current signal (200 vs bundle's 100), so an explicit declaration becomes the recommended default — while still flowing through the picker, so there is no silent auto-selection.
• New pyprojectServerlessVersion.ts: a pure, comment-aware line scan that reads environment_version from the canonical [tool.databricks.environment] table (no TOML dependency — matching the repo's existing pyproject.toml handling in packageManagerDetection.ts), plus a thin I/O reader over the project-root file.
• Wire the source into collectServerlessVersionObservations as an independently guarded branch, and label it pyproject.toml in the picker's provenance.

Purely additive: no behaviour change when there is no pyproject.toml or no [tool.databricks.environment] table. Extends the earlier serverless version resolution work (#2052 / #2053).

Scope note

The scan handles the canonical block form the paired CLI work writes; exotic TOML spellings (dotted-key / inline-table) are intentionally not parsed. A value outside the supported range is dropped like any other source (the CLI would reject it too).

Verification

• yarn test:unit — 858 passing, 0 failing (11 new pyproject collector tests + 3 extended scoring/observations/picker tests).
• prettier -c and eslint clean on all changed files.

This pull request and its description were written by Isaac.

[rugpanov]

🤖 Integration tests triggered for 9c0772a1 — ⏳ running.
View run

[rugpanov]

🤖 Integration tests ❌ 1 of 37 test jobs failed for 8dd4ff2d (36 passed).
View run

[rugpanov]

Integration Tests failure is an unrelated flake — not caused by this PR.

Only a single Windows e2e shard failed; all Linux shards and the rest of the Windows matrix passed. The failing suite is Bundle Init (should initialize new project / should wait for connection / should find resource explorer view), and the root cause is a dropped WebDriver connection:

Error: waitUntil condition failed: Connection closed. Code: 1006
WebDriverError: no such window: target window already closed
from unknown error: web view not found

This is a WebDriver/webview websocket flake (1006 = abnormal closure) in the test harness, not a product failure. This PR only touches the serverless-version resolution utilities under python-setup/utils/, which the Bundle Init flow does not exercise. Unit tests pass on both Linux and Windows (852+ passing; the single Windows unit failure was a separate bundled CLI version process-spawn timeout flake, which has been re-run).

Re-triggering the e2e job.

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

• PR number: 2130
• Commit SHA: 39c8752287272f0c7d9f785aa89fc2aaa4b9f62f

Checks will be approved automatically on success.