[ci] Windows Wanda image: fix conda PermissionError in microcheck windowsbuild

[ans9868]

PR: Fix windowsbuild conda PermissionError on Windows

---

Description

The wanda: windowsbuild CI check was intermittently failing during the Windows image build.

What was going wrong

After installing Python and dependencies via conda, a follow-up command:

conda update -c conda-forge ca-certificates certifi

would sometimes cause conda to update itself in-place. On Windows, this triggers a
PermissionError when conda tries to clean up its own temp files (e.g. conda.exe.c~).
The CI job then fails, even though the only goal was to refresh SSL certificates.

A previous fix (#62441) disabled some
self-update behavior via conda config --set auto_update_conda false, but the explicit
conda update step was never removed, so the risky path could still run.

Why this is hard to catch

This failure is intermittent. The same script can pass on one run and fail on another
depending on agent load, caching, and timing. When cleanup does fail, conda 26.x can hit a
secondary bug in its own error handling (conda/conda#15760),
adding noise to the logs. A green windowsbuild after #62441 doesn't mean the underlying
issue is gone.

What this PR does

• Moves ca-certificates and certifi into the same initial conda install as Python
  and requests — no separate update step needed
• Adds python -m pip install -U certifi after pip installs for a full cert refresh
• Removes the conda update -c conda-forge line entirely

Why avoid conda update on Windows?

Unlike conda install with explicit specs, conda update can upgrade conda itself and widen the blast radius to unrelated packages. Replacing a running conda.exe inside a Windows Docker container is fragile and the source of this failure.

---

Related Issues

PR / Issue	Description
ray-project/ray#61545	Original Windows cert refresh attempt
ray-project/ray#62441	Partial mitigation; this PR removes the explicit conda update that still triggered failures
conda/conda#15760	Upstream conda bug in cleanup/error path (splitext)

Observed failure: intermittent PermissionError on conda.exe.c~ in the BuildKite
microcheck for wanda: windowsbuild.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit 3597901. Configure here.}

[gemini-code-assist]

Code Review

This pull request modifies the ci/ray_ci/windows/build_base.sh script to streamline the installation of ca-certificates and certifi within the conda environment, aiming to refresh the trust store for Python/requests and avoid issues with conda update on Windows. It also adds a pip install -U certifi step. The review highlights two main concerns: first, removing conda config --set auto_update_conda false might re-introduce conda self-update issues leading to PermissionError, and it's suggested to retain this configuration. Second, the manual pip install -U certifi could bypass version pinning and cause environment inconsistencies, recommending an update in the constraints file instead.

[gemini-code-assist]

Removing conda config --set auto_update_conda false re-introduces the risk of conda attempting to update itself during the conda install command. Even when using install with explicit specs, conda may still perform a self-update if a newer version is available and the configuration allows it. To reliably prevent the PermissionError on conda.exe in Windows Docker environments, this configuration should be retained.

Suggested change

	# Include ca-certificates + certifi in this solve so the trust store is refreshed with
	# Python/requests (ray-project/ray#61545). Avoid a follow-up `conda update -c conda-forge`:
	# it can replace conda.exe in place on Windows and fail in Docker (PermissionError on
	# conda.exe.c~; see conda/conda#15760).
	# Use `conda install` (explicit specs) rather than `conda update` (broader "newest compatible" refresh/upgrade more of the environment including conda itself which can cause issues with the build).
	conda install -q -y python="${PYTHON_FULL_VERSION}" requests=2.32.3 ca-certificates certifi
	conda config --set auto_update_conda false
	# Include ca-certificates + certifi in this solve so the trust store is refreshed with
	# Python/requests (ray-project/ray#61545). Avoid a follow-up conda update -c conda-forge:
	# it can replace conda.exe in place on Windows and fail in Docker (PermissionError on
	# conda.exe.c~; see conda/conda#15760).
	# Use conda install (explicit specs) rather than conda update (broader newest compatible refresh/upgrade more of the environment including conda itself which can cause issues with the build).
	conda install -q -y python="${PYTHON_FULL_VERSION}" requests=2.32.3 ca-certificates certifi

[gemini-code-assist]

Performing a manual pip install -U certifi after a constrained installation bypasses the version pinning in python/requirements_compiled.txt. This can lead to environment inconsistencies if other packages depend on a specific version of certifi. It is generally safer to update the version in the constraints file directly.

[elliot-barn]

Merged a temp fix here: #62441

[ans9868]

Hey Elliot, I think your patch is simpler and more elegant. That said, mine may be useful longer term. Conda update is very general and can touch more than intended, while conda install is more targeted and could prevent a whole class of similar issues down the road.

One thing worth noting: windowsbuild microcheck doesn't actually trigger on PRs like this one. I tested both our changes in my other open PR (#60522, the Ax/Bayesian optimization fix), which does run the Windows build CI. Both our changes passed the windowsbuild microcheck. That PR was the original motivation here as it got blocked by unrelated microcheck failures, and the Windows conda issue was one of them.

Happy to close this if the leaner fix is preferred, or keep it open if the more explicit approach is wanted. Up to you and the team.

[aslonnie]

I think this is already fixed on master branch.

[aslonnie]

this line should not be required. using the pinned version is intentional

[ans9868]

Yes, I agree. The main thing I was trying to preserve here was using conda install instead of conda update, but I think your point is fair that the underlying issue may already be covered on master. If you all see this as redundant, I’m happy to close the PR.

[aslonnie]

I am closing this PR, especially since master is unblocked already.

• the part that pip install all 4 packages is fine, and probably even better.
• but the part that pip install -U certifi is an unintended change.

thanks for the contribution anyways.