Skip to content

Regression Database

Jump to bottom
Stephen Illingworth edited this page Jun 11, 2024 · 2 revisions

During development of the Gopher2600 it was very important that changes to the emulation code did not cause regression bugs. That is, the changes should not cause previously tested elements of the emulation to fail. This was very important to the development of the TIA in particular, where small timing changes can have dramatic influence on the console's output.

To help prevent regression bugs from arising, the REGRESS mode was added to the emulator. This can only be accessed from the command line and has no GUI output.

The regression database that is created and maintained by the REGRESS mode is stored in the emulator Configuration Directory.

Usage

Operation of the regression database is available via the REGRESS mode. The general pattern for this mode is

gopher2600 regress <sub-mode>

<sub-mode> can be one of either run, add, list, delete, redux or cleanup.

Adding

To quickly add a ROM to the database:

> gopher2600 regress add roms/Pitfall.bin

By default, this adds a video digest of the first 10 frames of the named ROM. We can alter the number of frames, and also other parameters with regress add mode flags. For example, to run for 100 frames instead of 10:

> gopher2600 regress add -frames 100 roms/Pitfall.bin

The database also supports the adding of playback files. When the test is run, the playback file is run as normal and success measured. To add a playback to the test database specify the playback file instead of a ROM:

> gopher2600 regress add recording_Pitfall_20200201_093658

Other options for the add sub-mode can be seen with gopher2600 regress add -help.

Listing

To list all previously added tests use the list sub-mode:

> gopher2600 regress list
> 000 [video] player_switching [AUTO] frames=10  [NUSIZ]
> 001 [video] NUSIZTest [AUTO] frames=10  [NUSIZ]
> 002 [video] testSize2Copies_A [AUTO] frames=10  [NUSIZ]
> 003 [video] testSize2Copies_B [AUTO] frames=10  [NUSIZ]
> 004 [video] player8 [AUTO] frames=10  [NUSIZ]
> 005 [video] player16 [AUTO] frames=10  [NUSIZ]
> 006 [video] player32 [AUTO] frames=10  [NUSIZ]
> 007 [video] barber [AUTO] frames=10  [NUSIZ]
> 008 [video] test1.bas [AUTO] frames=10  [TIMER]
> 009 [video] test2.bas [AUTO] frames=10  [TIMER]
> 010 [video] test3.bas [AUTO] frames=10  [TIMER]
> 011 [video] test4.bas [AUTO] frames=10  [TIMER]
> Total: 12

Running

To run all tests, use the run sub-mode:

> gopher2600 regress run

To run specific tests, list the test numbers (as seen in the list command result) on the command line. For example:

> gopher2600 regress run 1 3 5

Deleting

Delete tests with the delete sub-mode. For example:

> gopher2600 regress delete 3

For safety reasons, only one entry can be deleted at a time. Moreover, you will be asked to confirm the deletion before proceeding. The -yes flag can be used to skip the confirmation stage.

Redux

The redux sub-mode will rerun every supported entry type and add the new result in place of the old result.

Note that only some regression entry types are supported. The playback type in particular is not yet supported and will be skipped during redux.

Redux is a very dangerous operation and should only be used if you are absolutely sure that the new results will be valid. Because of the danger involved you will be asked twice to confirm that this is what you want to do. The -yes flag can be used to skip the confirmation stage.

Cleanup

The cleanup sub-mode will delete any orphaned scripts from the database. You will be asked to confirm the deletion before proceeding. The -yes flag can be used to skip the confirmation stage.

Redux of Playback Files

As mentioned above, it is not possible to perform a redux on playback file entries. However, it is possible to re-record playback files such that errors are ignored and a new playback file created. This new playback file can then be used to replace the existing playback file.

The following bash script shows a possible way to do this. But note that this is extremely dangerous. Do not use if you don't know what you're doing. The script is provided for reference:

#!/bin/bash
recording="playback_redux"`

for f in .gopher2600/regression/scripts/playback_*
do
    go run gopher2600.go -fpscap=false -record -playbackIgnoreDigest=true -playbackFilename="$recording" "$f"
    if [[ -f "$recording" ]]
    then
        mv "$recording" "$f"
    fi
done
Clone this wiki locally