Migrate the `./sway-lib-std/tests` into the E2E `stdlib` directory and get them all working

[mitchmindtree]

Follow-up to #1052.

Previously under the sway-lib-std repo the tests within the tests directory weren't actually run under CI. Now that we've moved them into the sway repo, it would be nice if we could migrate these tests into the existing E2E test harness, I'd imagine under should_pass/stdlib might make sense.

@nfurfaro would you be interested in having a go at this? I haven't yet looked closely at how the sway E2E harness and the sway-lib-std harnesses differ just yet, but perhaps you're already a little more familiar and might already have an idea on how to go about this?

[nfurfaro]

@mitchmindtree I could work on this, yes.
I think it's worth discussing the best approach, as the 2 test harnesses are quite a bit different.
FWIW, I was actually thinking about migrating the remaining stdlib tests out of the e2e harness and into the stdlib one (there aren't too many left as I've already ported some).

It would be good to have @sezna or @otrho chime in here as well, but I quite like the stdlib harness for working with contracts and the SDK. The e2e harness is not currently adequate for testing contracts (unless it has changed since I last worked with it) , but is probably a better choice for testing the core language.

Could we keep the harnesses separate, and have CI run them both?

[mitchmindtree]

Could we keep the tests separate, and have CI run them both?

I think I'd be fine with either approach as long as it's all tested via CI.

That said, if the E2E test suite is lacking w.r.t. contract testing, maybe that's something that could be improved there? I believe there are a few contract tests in there though, so it might have indeed evolved a little since you last worked with it.

Also of note: the tests were recently split into should_pass and should_fail, each with sub-directories for the different kinds of tests (e.g. stdlib, language, forc, test_contracts, etc), so if we do have the std lib tests separate it would be a bit of an outlier.

[otrho]

There's #685 and #723 (and others probably) which are both asking these questions. Contract tests definitely need improving and if the lib harness can address that it'd be great.

[nfurfaro]

@mitchmindtree I'm wondering if we can start by getting the tests under sway-lib-std/tests running under CI as a first step (and to have our safety net in place ASAP), and then continue the discussion about the finer points of the different harnesses and test organization?

Alternately, we could consider bringing the sway-lib-std tests into the e2e test group, but continue to run them with 2 distinct harnesses .
The reason I suggest this is that the SDK has been purpose-built for things like testing contracts, and to try to replicate some of that functionality in the e2e harness is perhaps just duplicated work.
Note that atm, this should be doable as the e2e tests only run if added manually to mod.rs, so we could just not add the std-lib tests and the e2e harness won't even know about them. This is however somewhat of a brittle assumption in light of #723 .

[mitchmindtree]

I'm wondering if we can start by getting the tests under sway-lib-std/tests running under CI as a first step

Yeah I think I might have underestimated how involved this merger-of-tests might be. I'm happy to add a new CI job to run the sway-lib-std tests independently in the meantime. The README added in #1097 should help with this - I'll open a PR shortly!

[nfurfaro]

Just bikeshedding some ideas here...
What if we:

• Move all std-lib tests into the sway/test repo
• Sort all tests primarily by which harness they're using (original or sdk-harness)
• Continue to run the tests which are currently in the test dir with the original harness
• Continue to run the stdlib tests with the sdk-harness
• Migrate the existing stdlib tests in the test dir from the original harness to the sdk-harness

Moving forward, the original harness can be improved with regards to contracts, or tests that need to use contracts can opt to use the SDK harness. An alternate approach would be to just use the sdk-harness for everything, but that seems a bit heavy when you just need to run a tiny sway script as in many of the core language tests.

Example structure:

sway/test on  furnic/migrate-std-tests [$✘?] is 📦 v0.0.0 via 🦀 v1.60.0
✦ ❯ tree -L 3
.
├── Cargo.toml
└── src
    ├── e2e_vm_tests
    │   ├── harness.rs
    │   ├── mod.rs
    │   └── test_programs
    ├── main.rs
    └── sdk-harness
        ├── Cargo.lock
        ├── Cargo.toml
        ├── README.md
        ├── target
        ├── test_artifacts
        └── test_projects

7 directories, 7 files

cc @mitchmindtree @sezna @otrho