Don't automatically cancel test if device interface is disconnected during test

[duncangreene]

Describe the bug
Some devices (Crestron AM-3200 as an example) during power up may up/down/up their NIC during bootstrap.

Expected behavior
Testrun should not automatically cancel the test. It should be acceptable for a device to up/down/up it's NIC during boot up.

Environment (please provide the following information about your setup):

• Version 2.1

[jhughesoti]

This is an expected behavior (#376) developed to guarantee network and device stability during testing. The cancellation of the test only happens during the monitor period where the device is expected to be in a stable state from a networking perspective.

If the device set their port up and down multiple times during its boot but never attained an IP address, it would not progress to the monitor state and would stay in the startup state. Once a device receives an IP address, Testrun will move to monitor state where this will be detected.

It may be worth investigating why this specific device is becoming stable enough to receive a new DHCP address but still restarting its network services.

[duncangreene]

We have now witnessed this behaviour on a DUT from a completely separate manufacturer.

It may be worth investigating why this specific device is becoming stable enough to receive a new DHCP address but still restarting its network services.

I'm not sure we'll get an expedient answer on this one due to the size of the manufacturer. I'm also not sure necessarily that it's such a bad thing if the port cycles up/down/up during boot/post an initial DHCP DORA. Port up/down behaviours aren't atypical. The STP listening/learning emulation process explained in #933 is sort of an example of this.

Maybe 'port consistently up from start' should be a test in it's own right under the connection module if it's of interest? I'm not sure that it should 'break' the functionality of Testrun as it does currently on these particular devices?

[jboddey]

Perhaps we want to make this behavior to toggle from system.json?

[jboddey]

A new property has been added to system.json and will be released in v2.2.