Mocha's retries implementation has unexpected consenquences #3274
Description
When using mocha's retries functionality, such as with Scenario().retries(2), on failure, mocha does a shallow clone of the test, copying just (some of) the properties it knows about, and then eventually replaces the original test object in the suite with the clone. This leads to surprises such as:
- Documented CodeceptJS-specific test properties such as
optsnot being present on the test object in hooks on retries - Mochawesome contexts added using MochawesomeHelper not appearing in the output, because they are added to the context for the original object, not the context for the cloned object.
This appears to be the cause of some of the issues reported in #1299, #1809, and #3028.
A cloned retry can be distinguished from an original by a truthy return from test.retriedTest(). In that case, the return value is the original test object. It might be possible for CodeceptJS to add a hook early in the lifecycle of a retried test that clones the CodeceptJS-specific properties. Otherwise, this quirk of mocha should be documented in the section on retries and in the hooks documentation so that users aren't surprised.
Likewise, MochawesomeHelper should be adapted to add context to the correct test, and maybe even (optionally?) copy the context over from previous runs.