[lldb][test][Windows] Fix TestReverseContinueBreakpoints.py and TestReverseContinueWatchpoints.py (history boundary)

[slydiman]

The following tests failed on Windows x64 (python 3.12.3)
Note these tests passed on the lldb-aarch64-windows buildbot.

lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py 2 of 8:

======================================================================
FAIL: test_reverse_continue_skip_breakpoint (TestReverseContinueBreakpoints.TestReverseContinueBreakpoints.test_reverse_continue_skip_breakpoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py", line 67, in test_reverse_continue_skip_breakpoint
    self.reverse_continue_skip_breakpoint_internal(async_mode=False)
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py", line 87, in reverse_continue_skip_breakpoint_internal
    self.expect(
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2512, in expect
    self.fail(log_msg)
AssertionError: Ran command:
"thread list"

Got output:
Process 51296 stopped
* thread #1: tid = 0x0910, 0x00007ff758e21070 a.out`trigger_breakpoint at main.c:9:35, name = 'a.out', stop reason = breakpoint 3.1

Expecting sub string: "stopped" (was found)
Expecting sub string: "stop reason = history boundary" (was not found)
Process should be stopped due to history boundary
Config=x86_64-D:\build-x64\bin\clang.exe
======================================================================
FAIL: test_reverse_continue_skip_breakpoint_async (TestReverseContinueBreakpoints.TestReverseContinueBreakpoints.test_reverse_continue_skip_breakpoint_async)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py", line 71, in test_reverse_continue_skip_breakpoint_async
    self.reverse_continue_skip_breakpoint_internal(async_mode=True)
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py", line 87, in reverse_continue_skip_breakpoint_internal
    self.expect(
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2512, in expect
    self.fail(log_msg)
AssertionError: Ran command:
"thread list"

Got output:
Process 9592 stopped
* thread #1: tid = 0xc3c0, 0x00007ff75bc21070 a.out`trigger_breakpoint at main.c:9:35, name = 'a.out', stop reason = breakpoint 3.1

Expecting sub string: "stopped" (was found)
Expecting sub string: "stop reason = history boundary" (was not found)
Process should be stopped due to history boundary
Config=x86_64-D:\build-x64\bin\clang.exe
----------------------------------------------------------------------

lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py 2 of 4:

======================================================================
FAIL: test_reverse_continue_skip_watchpoint (TestReverseContinueWatchpoints.TestReverseContinueWatchpoints.test_reverse_continue_skip_watchpoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\decorators.py", line 148, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py", line 68, in test_reverse_continue_skip_watchpoint
    self.reverse_continue_skip_watchpoint_internal(async_mode=False)
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py", line 94, in reverse_continue_skip_watchpoint_internal
    self.expect(
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2512, in expect
    self.fail(log_msg)
AssertionError: Ran command:
"thread list"

Got output:
Process 52276 stopped
* thread #1: tid = 0xcbd0, 0x00007ff7b18e1067 a.out`trigger_watchpoint at main.c:7:55, name = 'a.out', stop reason = watchpoint 1

Expecting sub string: "stopped" (was found)
Expecting sub string: "stop reason = history boundary" (was not found)
Process should be stopped due to history boundary
Config=x86_64-D:\build-x64\bin\clang.exe
======================================================================
FAIL: test_reverse_continue_skip_watchpoint_async (TestReverseContinueWatchpoints.TestReverseContinueWatchpoints.test_reverse_continue_skip_watchpoint_async)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\decorators.py", line 148, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py", line 74, in test_reverse_continue_skip_watchpoint_async
    self.reverse_continue_skip_watchpoint_internal(async_mode=True)
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py", line 94, in reverse_continue_skip_watchpoint_internal
    self.expect(
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2512, in expect
    self.fail(log_msg)
AssertionError: Ran command:
"thread list"

Got output:
Process 49504 stopped
* thread #1: tid = 0xb608, 0x00007ff704241067 a.out`trigger_watchpoint at main.c:7:55, name = 'a.out', stop reason = watchpoint 1

Expecting sub string: "stopped" (was found)
Expecting sub string: "stop reason = history boundary" (was not found)
Process should be stopped due to history boundary
Config=x86_64-D:\build-x64\bin\clang.exe
----------------------------------------------------------------------

Note cmake is configured with -DLLVM_LIT_ARGS="-avj 2"

[llvmbot]

@llvm/issue-subscribers-lldb

Author: Dmitry Vasilyev (slydiman)

The following tests failed on Windows x64 (python 3.12.3)

lldb\test\API\functionalities\reverse-execution -p TestReverseContinueBreakpoints.py 2 of 8:

======================================================================
FAIL: test_reverse_continue_skip_breakpoint (TestReverseContinueBreakpoints.TestReverseContinueBreakpoints.test_reverse_continue_skip_breakpoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py", line 67, in test_reverse_continue_skip_breakpoint
    self.reverse_continue_skip_breakpoint_internal(async_mode=False)
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py", line 87, in reverse_continue_skip_breakpoint_internal
    self.expect(
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2512, in expect
    self.fail(log_msg)
AssertionError: Ran command:
"thread list"

Got output:
Process 51296 stopped
* thread #<!-- -->1: tid = 0x0910, 0x00007ff758e21070 a.out`trigger_breakpoint at main.c:9:35, name = 'a.out', stop reason = breakpoint 3.1

Expecting sub string: "stopped" (was found)
Expecting sub string: "stop reason = history boundary" (was not found)
Process should be stopped due to history boundary
Config=x86_64-D:\build-x64\bin\clang.exe
======================================================================
FAIL: test_reverse_continue_skip_breakpoint_async (TestReverseContinueBreakpoints.TestReverseContinueBreakpoints.test_reverse_continue_skip_breakpoint_async)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py", line 71, in test_reverse_continue_skip_breakpoint_async
    self.reverse_continue_skip_breakpoint_internal(async_mode=True)
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueBreakpoints.py", line 87, in reverse_continue_skip_breakpoint_internal
    self.expect(
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2512, in expect
    self.fail(log_msg)
AssertionError: Ran command:
"thread list"

Got output:
Process 9592 stopped
* thread #<!-- -->1: tid = 0xc3c0, 0x00007ff75bc21070 a.out`trigger_breakpoint at main.c:9:35, name = 'a.out', stop reason = breakpoint 3.1

Expecting sub string: "stopped" (was found)
Expecting sub string: "stop reason = history boundary" (was not found)
Process should be stopped due to history boundary
Config=x86_64-D:\build-x64\bin\clang.exe
----------------------------------------------------------------------

lldb\test\API\functionalities\reverse-execution -p TestReverseContinueWatchpoints.py 2 of 4:

======================================================================
FAIL: test_reverse_continue_skip_watchpoint (TestReverseContinueWatchpoints.TestReverseContinueWatchpoints.test_reverse_continue_skip_watchpoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\decorators.py", line 148, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py", line 68, in test_reverse_continue_skip_watchpoint
    self.reverse_continue_skip_watchpoint_internal(async_mode=False)
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py", line 94, in reverse_continue_skip_watchpoint_internal
    self.expect(
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2512, in expect
    self.fail(log_msg)
AssertionError: Ran command:
"thread list"

Got output:
Process 52276 stopped
* thread #<!-- -->1: tid = 0xcbd0, 0x00007ff7b18e1067 a.out`trigger_watchpoint at main.c:7:55, name = 'a.out', stop reason = watchpoint 1

Expecting sub string: "stopped" (was found)
Expecting sub string: "stop reason = history boundary" (was not found)
Process should be stopped due to history boundary
Config=x86_64-D:\build-x64\bin\clang.exe
======================================================================
FAIL: test_reverse_continue_skip_watchpoint_async (TestReverseContinueWatchpoints.TestReverseContinueWatchpoints.test_reverse_continue_skip_watchpoint_async)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\decorators.py", line 148, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py", line 74, in test_reverse_continue_skip_watchpoint_async
    self.reverse_continue_skip_watchpoint_internal(async_mode=True)
  File "D:\llvm-project\lldb\test\API\functionalities\reverse-execution\TestReverseContinueWatchpoints.py", line 94, in reverse_continue_skip_watchpoint_internal
    self.expect(
  File "D:\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2512, in expect
    self.fail(log_msg)
AssertionError: Ran command:
"thread list"

Got output:
Process 49504 stopped
* thread #<!-- -->1: tid = 0xb608, 0x00007ff704241067 a.out`trigger_watchpoint at main.c:7:55, name = 'a.out', stop reason = watchpoint 1

Expecting sub string: "stopped" (was found)
Expecting sub string: "stop reason = history boundary" (was not found)
Process should be stopped due to history boundary
Config=x86_64-D:\build-x64\bin\clang.exe
----------------------------------------------------------------------

[slydiman]

@labath
Ping

[labath]

I'm not sure what you expect me to do. I committed this test but @rocallahan is the author. There's also no buildbot for this configuration, which is why we didn't see the problem in the initial back and forth.

There's probably something about about the lldb-server output on this platform that confuses the gdb packet forwarder in the test. (it's interesting that it passes on arm though). I might be able to take a look next time I'm in windows, but I don't consider myself responsible for fixing this. If you want to help, you can get a trace of the gdb packet communication, and maybe I'll spot something.

[rocallahan]

As a last resort we can just disable the tests on Windows x64, right?

[DavidSpickett]

If you want to debug, comparing packet logs from X86 and AArch64 Windows might help. I can get ones from AArch64, they will be different to an extent anyway, but it's worth trying.

Also you can check whether the stopped location is different from the expected stop reason = history boundary location, to see if it's mislabeling the stop, or stopping somewhere else entirely.

[labath]

Here's the log I gathered on windows (from the test_reverse_continue_skip_breakpoint_async test). I enabled all logs so its rather long.

fail.txt

The problem is that lldb fails to evaluate the breakpoint condition, and the reason for that is that the expression function ($__lldb_arg) gets a bad pointer as an argument ($__lldb_arg is (void*)1). I don't know what causes that, but I assume it's either something to do with memory allocation or register writing/reading.