[Bug]: Recording playback too fast for web

[flyguy712]

Describe the bug

I expected my recording to play back using python -m openadapt.replay NaiveReplayStrategy to successfully follow the steps I took during the recording. What happened was when I initiated the reply, the playback was so fast that it didn't give the webpage(s) sufficient time to load. This caused the workflow to break (e.g. the playback continued to click and type, but the UI was not in sync so the subsequent activity wasn't completed successfully).

Logs from recording: recordingLogs.txt

Logs from replay: replayLogs.txt

To Reproduce

I use macOS Ventura 13.1, Apple M1, 8 GB memory. The recording was created in a new/empty shell. Steps to reproduce:

1. Create a recording that interacts with web
2. Type google.flights.com into the URL box
3. Hit enter
4. Click the destination box
5. Type the destination code (i.e. MSP)
6. Click explore button
7. End the recording
8. After the recording finishes processing, play it back using python -m openadapt.replay NaiveReplayStrategy
9. Bug, the recording moves too fast and the page doesn't have sufficient time to load

Internet speed: Speedtest Results Export-2023_07_25.csv

[abrichr]

Thank you @flyguy712 !

This will be much easier to debug once #362 is merged, but for now I think we have to assume that the NaiveReplayStrategy won't work for this use case.

Can you please try the StatefulReplayStrategy? Note that the playback will take much longer (we are currently working on improving this).

[flyguy712]

@abrichr it doesn't appear that is an option (it isn't an available strategy).

This is the command that I used: python -m openadapt.replay StatefulReplayStrategy

This is where I found the command: ‎openadapt/strategies/stateful.py

Here are the replay logs (starting with the first ERROR):

2023-07-29 12:10:40.340 | ERROR    | fire.core:_CallAndUpdateTrace:681 - An error has been caught in function '_CallAndUpdateTrace', process 'MainProcess' (38278), thread 'MainThread' (8632384128):
Traceback (most recent call last):

  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
           |         |     -> {'__name__': '__main__', '__doc__': None, '__package__': 'openadapt', '__loader__': <_frozen_importlib_external.SourceFileLoa...
           |         -> <code object <module> at 0x1025c9f20, file "/Users/taylorhorsager/OpenAdapt/openadapt/replay.py", line 1>
           -> <function _run_code at 0x1025cd6c0>
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
         |     -> {'__name__': '__main__', '__doc__': None, '__package__': 'openadapt', '__loader__': <_frozen_importlib_external.SourceFileLoa...
         -> <code object <module> at 0x1025c9f20, file "/Users/taylorhorsager/OpenAdapt/openadapt/replay.py", line 1>

  File "/Users/taylorhorsager/OpenAdapt/openadapt/replay.py", line 54, in <module>
    fire.Fire(replay)
    |    |    -> <function replay at 0x154d9b250>
    |    -> <function Fire at 0x103221cf0>
    -> <module 'fire' from '/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-...

  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      |     |          |     |                 |        -> 'replay.py'
                      |     |          |     |                 -> {}
                      |     |          |     -> Namespace(verbose=False, interactive=False, separator='-', completion=None, help=False, trace=False)
                      |     |          -> ['StatefulReplayStrategy']
                      |     -> <function replay at 0x154d9b250>
                      -> <function _Fire at 0x103487250>
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
    |                           -> <function _CallAndUpdateTrace at 0x103487370>
    -> <function replay at 0x154d9b250>
> File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                |   |          -> {}
                |   -> ['StatefulReplayStrategy', None]
                -> <function replay at 0x154d9b250>

  File "/Users/taylorhorsager/OpenAdapt/openadapt/replay.py", line 37, in replay
    raise ValueError(f"Invalid {strategy_name=}; {available_strategies=}")

ValueError: Invalid strategy_name='StatefulReplayStrategy'; available_strategies='NaiveReplayStrategy, DemoReplayStrategy'

[abrichr]

Can you please try running ‘git pull’ then try again?

…

On Sat, Jul 29, 2023 at 2:13 PM Taylor Horsager ***@***.***> wrote: @abrichr <https://github.com/abrichr> it doesn't appear that is an option (it isn't an available strategy). This is the command that I used: python -m openadapt.replay StatefulReplayStrategy Here are the logs: 2023-07-29 12:10:40.340 | ERROR | fire.core:_CallAndUpdateTrace:681 - An error has been caught in function '_CallAndUpdateTrace', process 'MainProcess' (38278), thread 'MainThread' (8632384128): Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, | | -> {'__name__': '__main__', '__doc__': None, '__package__': 'openadapt', '__loader__': <_frozen_importlib_external.SourceFileLoa... | -> <code object <module> at 0x1025c9f20, file "/Users/taylorhorsager/OpenAdapt/openadapt/replay.py", line 1> -> <function _run_code at 0x1025cd6c0> File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) | -> {'__name__': '__main__', '__doc__': None, '__package__': 'openadapt', '__loader__': <_frozen_importlib_external.SourceFileLoa... -> <code object <module> at 0x1025c9f20, file "/Users/taylorhorsager/OpenAdapt/openadapt/replay.py", line 1> File "/Users/taylorhorsager/OpenAdapt/openadapt/replay.py", line 54, in <module> fire.Fire(replay) | | -> <function replay at 0x154d9b250> | -> <function Fire at 0x103221cf0> -> <module 'fire' from '/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-... File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) | | | | | -> 'replay.py' | | | | -> {} | | | -> Namespace(verbose=False, interactive=False, separator='-', completion=None, help=False, trace=False) | | -> ['StatefulReplayStrategy'] | -> <function replay at 0x154d9b250> -> <function _Fire at 0x103487250> File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/fire/core.py", line 466, in _Fire component, remaining_args = _CallAndUpdateTrace( | -> <function _CallAndUpdateTrace at 0x103487370> -> <function replay at 0x154d9b250> > File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) | | -> {} | -> ['StatefulReplayStrategy', None] -> <function replay at 0x154d9b250> File "/Users/taylorhorsager/OpenAdapt/openadapt/replay.py", line 37, in replay raise ValueError(f"Invalid {strategy_name=}; {available_strategies=}") ValueError: Invalid strategy_name='StatefulReplayStrategy'; available_strategies='NaiveReplayStrategy, DemoReplayStrategy' — Reply to this email directly, view it on GitHub <#434 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF5DV66ANNBZ3EHXXG53MLXSVHDLANCNFSM6AAAAAA2X66GKQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[flyguy712]

Unfortunately I tried that just before doing this test...

(openadapt-py3.10) taylorhorsager@Taylors-MacBook-Air OpenAdapt % git pull
Already up to date.

[flyguy712]

Looks like there were some possibly relevant logs in the recording:

2023-07-29 12:24:08.551 | INFO     | __mp_main__:write_events:319 - event_type='window' starting
Process Process-1:
Traceback (most recent call last):
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: table screenshot has no column named png_diff_data

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/taylorhorsager/OpenAdapt/openadapt/record.py", line 124, in wrapper_logging
    result = func(*args, **kwargs)
  File "/Users/taylorhorsager/OpenAdapt/openadapt/record.py", line 342, in write_events
    write_fn(recording_timestamp, event, perf_q)
  File "/Users/taylorhorsager/OpenAdapt/openadapt/record.py", line 275, in write_screen_event
    crud.insert_screenshot(recording_timestamp, event.timestamp, event_data)
  File "/Users/taylorhorsager/OpenAdapt/openadapt/crud.py", line 105, in insert_screenshot
    _insert(event_data, Screenshot, screenshots)
  File "/Users/taylorhorsager/OpenAdapt/openadapt/crud.py", line 64, in _insert
    result = db.execute(sa.insert(table), to_insert)
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1714, in execute
    result = conn._execute_20(statement, params or {}, execution_options)
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
    ret = self._execute_context(
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
    raise exception
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table screenshot has no column named png_diff_data
[SQL: INSERT INTO screenshot (id, recording_timestamp, timestamp, png_data, png_diff_data, png_diff_mask_data) VALUES (?, ?, ?, ?, ?, ?)]
[parameters: (None, 1690655045.8700988, 1692257177.6690187, <memory at 0x163438c40>, None, None)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
2023-07-29 12:24:33.463 | INFO     | __main__:read_window_events:554 - _window_data={'title': 'Google Chrome [Bug]: Recording playback too fast for web · Issue #434 · OpenAdaptAI/OpenAdapt', 'left': 1021, 'top': 178, 'width': 1439, 'height': 1005, 'window_id': 86111}
2023-07-29 12:24:36.041 | INFO     | __main__:read_window_events:554 - _window_data={'title': 'Google Chrome Google Flights - Find Cheap Flight Options & Track Prices', 'left': 1021, 'top': 178, 'width': 1439, 'height': 1005, 'window_id': 86111}
2023-07-29 12:24:38.798 | INFO     | __main__:read_window_events:554 - _window_data={'title': 'Google Chrome', 'left': 1020, 'top': 1162, 'width': 479, 'height': 22, 'window_id': 87042}
2023-07-29 12:24:41.454 | INFO     | __main__:read_window_events:554 - _window_data={'title': 'Google Chrome Google Flights - Find Cheap Flight Options & Track Prices', 'left': 1021, 'top': 178, 'width': 1439, 'height': 1005, 'window_id': 86111}
2023-07-29 12:24:44.172 | INFO     | __main__:read_window_events:554 - _window_data={'title': 'Terminal OpenAdapt — poetry shell ▸ Python — 80×24', 'left': 333, 'top': 105, 'width': 570, 'height': 371, 'window_id': 87039}
^C2023-07-29 12:24:44.603 | INFO     | __mp_main__:write_events:348 - event_type='window' done
2023-07-29 12:24:44.603 | INFO     | __mp_main__:wrapper_logging:126 -  <- Leave: write_events(None)
2023-07-29 12:24:44.686 | INFO     | __mp_main__:write_events:348 - event_type='action' done
2023-07-29 12:24:44.687 | INFO     | __mp_main__:wrapper_logging:126 -  <- Leave: write_events(None)
2023-07-29 12:24:44.691 | INFO     | __main__:read_screen_events:518 - Done
2023-07-29 12:24:44.702 | INFO     | __main__:process_events:239 - Done
2023-07-29 12:24:44.702 | INFO     | __main__:wrapper_logging:126 -  <- Leave: process_events(None)
2023-07-29 12:24:44.919 | INFO     | __main__:wrapper_logging:126 -  <- Leave: read_window_events(None)
2023-07-29 12:24:44.998 | INFO     | __main__:log_memory_usage:70 - source='File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/mss/darwin.py", line 226'
2023-07-29 12:24:44.999 | INFO     | __main__:log_memory_usage:71 - 	new_KiB=10425610.075195312 total_KiB=10425610.075195312 new_blocks=362 total_blocks=362
2023-07-29 12:24:45.000 | INFO     | __main__:log_memory_usage:70 - source='File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/reduction.py", line 51'
2023-07-29 12:24:45.000 | INFO     | __main__:log_memory_usage:71 - 	new_KiB=64819.890625 total_KiB=64820.1787109375 new_blocks=8 total_blocks=14
2023-07-29 12:24:45.002 | INFO     | __main__:log_memory_usage:70 - source='File "/Users/taylorhorsager/OpenAdapt/openadapt/window/_macos.py", line 213'
2023-07-29 12:24:45.002 | INFO     | __main__:log_memory_usage:71 - 	new_KiB=10215.5654296875 total_KiB=10215.5654296875 new_blocks=64404 total_blocks=64404
2023-07-29 12:24:51.418 | INFO     | __main__:log_memory_usage:74 - trace_str=
                      types |   # objects |   total size
=========================== | =========== | ============
                  bytearray |         180 |      9.89 GB
                _io.BytesIO |           4 |     63.30 MB
                      tuple |      235567 |     16.11 MB
                        str |       34740 |     11.64 MB
                       dict |       27400 |      7.00 MB
                       list |       20050 |      3.64 MB
                        int |        8908 |    244.23 KB
  tracemalloc.StatisticDiff |        2132 |    149.91 KB
      tracemalloc.Traceback |        2132 |     99.94 KB
          collections.deque |         150 |     92.95 KB
                       type |          53 |     67.01 KB
                        set |         174 |     49.95 KB
                       code |         191 |     47.86 KB
                    weakref |         497 |     34.95 KB
                  frozenset |          87 |     24.10 KB
2023-07-29 12:24:51.420 | INFO     | __main__:record:965 - joining...
2023-07-29 12:24:51.420 | INFO     | __mp_main__:performance_stats_writer:597 - Performance stats writer done
2023-07-29 12:24:51.421 | INFO     | __mp_main__:wrapper_logging:126 -  <- Leave: performance_stats_writer(None)
2023-07-29 12:24:51.425 | INFO     | __mp_main__:memory_writer:644 - Memory writer done
2023-07-29 12:24:52.098 | INFO     | openadapt.utils:plot_performance:706 - fpath='performance/performance-1690655045.8700988.png'
2023-07-29 12:24:52.511 | INFO     | __main__:record:980 - Saved recording_timestamp=1690655045.8700988
2023-07-29 12:24:52.513 | INFO     | __main__:wrapper_logging:126 -  <- Leave: record(None)
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/queues.py", line 251, in _feed
    send_bytes(obj)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 205, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 410, in _send_bytes
    self._send(buf)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 373, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
(openadapt-py3.10) taylorhorsager@Taylors-MacBook-Air OpenAdapt % /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 40 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

[abrichr]

@flyguy712 please sync with @0dm and @KrishPatel13 to resolve this issue

[abrichr]

@flyguy712 can you please set SLEEP = True in naive.py and try again with NaiveReplayStrategy?

https://github.com/OpenAdaptAI/OpenAdapt/blob/main/openadapt/strategies/naive.py#L12C11-L12C11

[flyguy712]

@abrichr I've set SLEEP = True in naive.py and tried again with NaiveReplayStrategy but unfortunately that didn't work. I've attached the terminal logs.

Logs: test.txt

@0dm and @KrishPatel13 please let me know which steps you would like me to take next to help with troubleshooting.

[KrishPatel13]

Thank you @flyguy712

I am just quoting the first error that was encountered from the above transcript:

_execute_context
        self.dialect.do_execute(
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
        cursor.execute(statement, parameters)
sqlite3.OperationalError: table screenshot has no column named png_diff_data

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
        self.run()
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 108, in run
        self._target(*self._args, **self._kwargs)
    File "/Users/taylorhorsager/OpenAdapt/openadapt/record.py", line 124, in wrapper_logging
        result = func(*args, **kwargs)
    File "/Users/taylorhorsager/OpenAdapt/openadapt/record.py", line 342, in write_events
        write_fn(recording_timestamp, event, perf_q)
    File "/Users/taylorhorsager/OpenAdapt/openadapt/record.py", line 275, in write_screen_event
        crud.insert_screenshot(recording_timestamp, event.timestamp, event_data)
    File "/Users/taylorhorsager/OpenAdapt/openadapt/crud.py", line 105, in insert_screenshot
        _insert(event_data, Screenshot, screenshots)
    File "/Users/taylorhorsager/OpenAdapt/openadapt/crud.py", line 64, in _insert
        result = db.execute(sa.insert(table), to_insert)
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1714, in execute
        result = conn._execute_20(statement, params or {}, execution_options)
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
        return meth(self, args_10style, kwargs_10style, execution_options)
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection
        return connection._execute_clauseelement(
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
        ret = self._execute_context(
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
        self._handle_dbapi_exception(
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
        util.raise_(
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
        raise exception
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
        self.dialect.do_execute(
    File "/Users/taylorhorsager/Library/Caches/pypoetry/virtualenvs/openadapt-IciE2gK5-py3.10/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
        cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table screenshot has no column named png_diff_data
[SQL: INSERT INTO screenshot (id, recording_timestamp, timestamp, png_data, png_diff_data, png_diff_mask_data) VALUES (?, ?, ?, ?, ?, ?)]

This error pops up when you try to run record, visualize, playback commands on an empty recorded (.db) file.

@flyguy712 Lets us begin with a fresh approach. That is, Record a small recording again and then try to replay it using NaiveReplayStrategy.

Follow this steps:

openadapt.scripts.reset_db
python -m openadapt.record "testing out openadapt"
<Let us know if the you were able to successfully record it>
python -m openadapt.replay NaiveReplayStrategy

[KrishPatel13]

Please let us know if you need any help. :-)

[KrishPatel13]

Link for the recording of the bug:

1. Windows: https://mldsai.slack.com/archives/C0563RXRWLX/p1692201992597899
2. Mac: https://mldsai.slack.com/archives/C056JK5JQ8L/p1692200839553839 (Credit: @flyguy712 )

Edit: this is not a bug, see thread for details
Thread: https://mldsai.slack.com/archives/C056JK5JQ8L/p1692209186278869?thread_ts=1692209186.278869&cid=C056JK5JQ8L

[abrichr]

@flyguy712 can you please confirm whether you were able to resolve the issue you were experiencing?