adapter: Refactor environmentd test

[ParkMyCar]

This PR only touches test code.

In this PR we refactor the environmentd (and balancerd) tests to be "async by default". There are a couple of key changes:

1. All helpers in test_utils.rs are refactored to be async. They no longer take a runtime as an argument, or create a runtime themselves.
2. The test Config has been refactored to TestHarness, which is a struct that handles test configuration and starting the enviromentd server. There are two modes a TestHarness can be created in, either async fn start() or fn start_blocking().
   • async fn start() returns a TestServer, where all of the helper methods return tokio_postgres types, and thus are async.
   • fn start_blocking() returns a TestServerWithRuntime, where all of the helper methods return postgres types, and thus are synchronous/blocking.

The goal with the TestHarness change was to make migrating existing synchronous tests easy. The first part of that is the split between sync and async types, the other part is code refactoring. For example a sync test might look like:

let server = TestHarness::default().start_blocking();

to refactor it to the async variant all you need to do is:

let server = TestHarness::default().start().await;

3. The async version, i.e. TestServer, has a single fn connect(...) method which returns a ConnectBuilder. Instead of multiple different methods to configure how we want to connect to the running environmentd server, it's now bundled into this builder. The goal here was to reduce boilerplate and make it just as easy to use an async client as it is the sync client, e.g. handling notices with a tokio_postgres client isn't very straight forward.

For example, if you just want to run some queries against the test server:

let client = server.connect().await.unwrap();

or if you want to configure the TLS used:

let client = server.connect().with_tls(...).await.unwrap();

and much more:

let (client, conn_handle) = server.connect()
    .user("parker")
    .password("1234")
    .dbname("test")
    .notice_callback(move |notice| notice_tx.send(notice))
    .with_tls(...)
    .with_handle()
    .await
    .unwrap();

Motivation

Making changes to our environmentd tests was painful because our tests and test utils were split between async and synchronous. For example, making changes to fix issues like https://github.com/MaterializeInc/database-issues/issues/6773 would end up being more complicated than necessary because some tests would rely on both async and sync test helpers.

Tips for reviewer

Removing white space would be useful, and this PR is split into 3 commits which can be reviewed independently:

1. Changes to test_utils.rs to make all helpers async, and introduce the split between TestServer and TestServerWithRuntime, as well as introduce ConnectBuilder.
2. Refactoring a number of tests to be async, i.e. use #[mz_ore::test(tokio::test)]. Any tests that now relied on an async helper, or was already spawning futures on a runtime, were refactored.
3. Refactoring all remaining tests to use the new let server = TestHarness::default().start_blocking() API. There are no functional changes in this commit, just code movement.

Checklist

• This PR has adequate test coverage / QA involvement has been duly considered.
• This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
• If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
• If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
• This PR includes the following user-facing behavior changes:
  • N/a, only changes testing code

[jkosh44]

LGTM, couple of small nits/questions

[maddyblue]

What happens if you leave out flavor = "multi_thread", worker_threads = 1?

[ParkMyCar]

For some reason the test hangs, everything runs but we're blocked on cleanup. I haven't figured out exactly what's happening yet, but will dig in further when I have a free cycle or two

[maddyblue]

This is going to conflict with #23176. Who's gonna merge first?!

[ParkMyCar]

IMO #23176 unblocks more important workflows, so you can merge first!

[maddyblue]

This is pretty ugly. After both these PRs merge we should do a similar refactor to the frontegg server so it's async-first also.

[ParkMyCar]

Totally agree, once you merge #23176 I'll rebase and have a think about how to make this cleaner

[ParkMyCar]

After rebasing on top of #23176 I moved the task::spawn into the FronteggMockServer itself, so we no longer need these spawn calls!