Intermittent test failure: NI-SCOPE returns -50550 timeout error

[bkeryan]

Bug Report

This test run failed with error "-1074108967: A synchronization object was not acquired within the time limit. The operation could not be completed as specified.\n\nStatus Code: -50550" during an NI-SCOPE test. https://github.com/ni/measurement-plugin-python/actions/runs/10626770324/job/29458939886?pr=858

This occurred during the Python 3.11 run. The scope tests passed for the other Python versions. Some of the switch tests failed for 3.10 due to #859

Repro or Code Sample

PR build

Expected Behavior

The tests pass.

Current Behavior

================================== FAILURES ===================================
__________ test___single_session___measure___creates_single_session ___________

pin_map_context = pin_map_id: "C:\\actions-runner\\1\\_work\\measurement-plugin-python\\measurement-plugin-python\\packages\\service\\tests\\assets\\acceptance\\session_management\\2Scope2Pin2Group1Site.pinmap"
sites: 0

stub_v2 = <ni_measurement_plugin_sdk_service._internal.stubs.ni.measurementlink.measurement.v2.measurement_service_pb2_grpc.MeasurementServiceStub object at 0x0000029E8CED9BD0>

    def test___single_session___measure___creates_single_session(
        pin_map_context: PinMapContext,
        stub_v2: MeasurementServiceStub,
    ) -> None:
        configurations = Configurations(pin_names=["Pin1"], multi_session=False)
    
>       outputs = _measure(stub_v2, pin_map_context, configurations)

tests\acceptance\test_niscope_measurement.py:40: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests\acceptance\test_niscope_measurement.py:67: in _measure
    responses = list(response_iterator)
.tox\py311-all-extras\Lib\site-packages\grpc\_channel.py:543: in __next__
    return self._next()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNKNOWN
	details = "Exception iterating re...be completed as specified.\n\nStatus Code: -50550", grpc_status:2, created_time:"2024-08-30T05:19:53.8653652+00:00"}"
>

    def _next(self) -> Any:
        with self._state.condition:
            if self._state.code is None:
                event_handler = _event_handler(
                    self._state, self._response_deserializer
                )
                self._state.due.add(cygrpc.OperationType.receive_message)
                operating = self._call.operate(
                    (cygrpc.ReceiveMessageOperation(_EMPTY_FLAGS),),
                    event_handler,
                )
                if not operating:
                    self._state.due.remove(cygrpc.OperationType.receive_message)
            elif self._state.code is grpc.StatusCode.OK:
                raise StopIteration()
            else:
                raise self
    
            def _response_ready():
                return self._state.response is not None or (
                    cygrpc.OperationType.receive_message not in self._state.due
                    and self._state.code is not None
                )
    
            _common.wait(self._state.condition.wait, _response_ready)
            if self._state.response is not None:
                response = self._state.response
                self._state.response = None
                return response
            elif cygrpc.OperationType.receive_message not in self._state.due:
                if self._state.code is grpc.StatusCode.OK:
                    raise StopIteration()
                elif self._state.code is not None:
>                   raise self
E                   grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
E                   	status = StatusCode.UNKNOWN
E                   	details = "Exception iterating responses: -1074108967: A synchronization object was not acquired within the time limit. The operation could not be completed as specified.
E                   
E                   Status Code: -50550"
E                   	debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"Exception iterating responses: -1074108967: A synchronization object was not acquired within the time limit. The operation could not be completed as specified.\n\nStatus Code: -50550", grpc_status:2, created_time:"2024-08-30T05:19:53.8653652+00:00"}"
E                   >

.tox\py311-all-extras\Lib\site-packages\grpc\_channel.py:969: _MultiThreadedRendezvous
------------------------------ Captured log call ------------------------------
ERROR    grpc._server:_server.py:668 Exception iterating responses: -1074108967: A synchronization object was not acquired within the time limit. The operation could not be completed as specified.

Status Code: -50550
Traceback (most recent call last):
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\.tox\py311-all-extras\Lib\site-packages\grpc\_server.py", line 654, in _take_response_from_response_iterator
    return next(response_iterator), True
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\ni_measurement_plugin_sdk_service\_internal\grpc_servicer.py", line 371, in Measure
    return_value = self._measure_function(**mapping_by_variable_name)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\tests\utilities\measurements\niscope_measurement\__init__.py", line 50, in measure
    with reservation.initialize_niscope_session() as session_info:
  File "C:\Program Files\Python311\Lib\contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\ni_measurement_plugin_sdk_service\session_management\_reservation.py", line 729, in _initialize_session_core
    with closing_function(session_constructor(session_info)) as session:
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\ni_measurement_plugin_sdk_service\_drivers\_niscope.py", line 57, in __call__
    return niscope.Session(
           ^^^^^^^^^^^^^^^^
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\.tox\py311-all-extras\Lib\site-packages\niscope\session.py", line 4038, in __init__
    self._interpreter.set_session_handle(self._init_with_options(resource_name, id_query, reset_device, options))
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\.tox\py311-all-extras\Lib\site-packages\niscope\session.py", line 5067, in _init_with_options
    vi = self._interpreter.init_with_options(resource_name, id_query, reset_device, option_string)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\.tox\py311-all-extras\Lib\site-packages\niscope\_grpc_stub_interpreter.py", line 347, in init_with_options
    response = self._invoke(
               ^^^^^^^^^^^^^
  File "C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\.tox\py311-all-extras\Lib\site-packages\niscope\_grpc_stub_interpreter.py", line 67, in _invoke
    raise errors.DriverError(error_code, error_message)
niscope.errors.DriverError: -1074108967: A synchronization object was not acquired within the time limit. The operation could not be completed as specified.

Status Code: -50550
- generated xml file: C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\service\test_results\nims-i-0d1440414f4a8ccc5-1-py311-all-extras.xml -
=========================== short test summary info ===========================
FAILED tests/acceptance/test_niscope_measurement.py::test___single_session___measure___creates_single_session
================== 1 failed, 633 passed in 185.76s (0:03:05) ==================

Possible Solution

None

Context

#858

Your Environment

• OS & Device: Windows
• ni-measurement-plugin-sdk-service version: latest main
• MeasurementLink version: whatever is on the self-hosted runner
• Python version: 3.11

AB#2843581

[danestull]

This was not reproducible