Description
This is autogenerated. Please review and update as needed.
Describe the bug
Multiple simultaneous calls to "az artifacts universal download" fail if no previous calls have been made to "az artifacts universal download". This appears to be race condition caused by each call attempting to create a new cache file and only one succeeds while the other calls all fail.
This is 100% repro on our build agents as they use a fresh image every time.
Command Name
az artifacts universal download
Errors:
EXEC : error : [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\cloudtest\\.azure-devops\\python-sdk\\cache'
Traceback (most recent call last):
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 561, in execute
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/__init__.py", line 507, in load_arguments
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 318, in load_arguments
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/commands.py", line 104, in load_arguments
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 125, in arguments_loader
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 59, in get_op_handler
File "importlib\__init__.py", line 126, in import_module
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\dev\artifacts\universal.py", line 9, in <module>
from azext_devops.dev.common.services import resolve_instance, resolve_instance_and_project
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\dev\common\services.py", line 13, in <module>
from azext_devops.devops_sdk.connection import Connection
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\devops_sdk\connection.py", line 5, in <module>
from ._file_cache import RESOURCE_CACHE as RESOURCE_FILE_CACHE
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\devops_sdk\_file_cache.py", line 113, in <module>
DEFAULT_CACHE_DIR = get_cache_dir()
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\devops_sdk\_file_cache.py", line 108, in get_cache_dir
os.makedirs(azure_devops_cache_dir)
File "os.py", line 225, in makedirs
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\cloudtest\\.azure-devops\\python-sdk\\cache'
To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.
- Put any pre-requisite steps here...
- Have a script run multiple concurrent "az artifacts universal download" calls on a fresh machine (this is happening on our hosted build agents, so it is always a fresh machine). Only one of the calls will succeed while the others will fail attempting to create the python-sdk cache file.
Expected Behavior
There is a global lock around the initial creation of the cache and when multiple concurrent calls are made they wait for the creation of the cache to complete before proceeding.
Environment Summary
Windows-10-10.0.22000-SP0
Python 3.10.8
Installer: MSI
azure-cli 2.42.0
Extensions:
azure-devops 0.25.0
Dependencies:
msal 1.20.0
azure-mgmt-resource 21.1.0b1
Additional Context
"az artifacts universal download" commands are embedded in some projects to bring down build dependencies and we cannot control whether these calls are made simultaneously or not as the build is parallelized.
This repros on 100% of our builds because we are using hosted build agents which are always a fresh image.