Some additional work that came out of #178897, we need to have some assurance that Synthetics will not catastrophically fail on Serverless when dependencies change. This is a visible risk that is illustrated in the linked issue for that PR.

As such, we should port at least a subset of the Synthetics API tests to the Serverless section of x-pack testing. This will give us at least assurance that our Kibana server is functioning nominally as more changes go in.

I've converted this task to a meta ticket with sub-issues stubbed out. I'll update each issue with links to the code in question, and add a guide to the meta for adapting tests that rely on features like roles and the ES security API that aren't available in Serverless. I'll also include links to the testing docs for Kibana's Serverless test suite.

Prior art

You can see the initial implementation as part of #179861.

Key differences

When interacting via API endpoints, many of the Stateful tests utilize supertestWithAuth and make requests via users/roles they create using the security service:

          const username = 'admin';
          const roleName = `synthetics_admin-${privilege}`;
          const password = `${username}-password`;
          try {
            await security.role.create(roleName, {
              kibana: [
                {
                  feature: {
                    uptime: ['all'],
                  },
                  spaces: ['*'],
                },
              ],
              elasticsearch: {
                cluster: [privilege],
                indices: getServiceApiKeyPrivileges(false).indices,
              },
            });

In Serverless, there's a SAML simulator you can use to provide the necessary auth via supertestWithoutAuth:

note: at this time it's recommended to avoid using system_indices_superuser because this does not exist in MKI and will cause the test to fail and be skipped.

// `role` can be 'system_indices_superuser', 'admin', 'editor', 'viewer'
supertestWithoutAuth
  .put(SYNTHETICS_API_URLS.SYNTHETICS_ENABLEMENT)
  // internal request header is different from the typical kbn-xsrf header
  // implementation details are available in the PR linked above
  .set(internalRequestHeader)
  .set(await svlUserManager.getApiCredentialsForRole(role))
  .expect(expectedStatus);

Smells to look for are declarations like:

// custom user creation
const username = 'admin';
const roleName = `synthetics_admin-${privilege}`;

// calls to create custom roles
await security.role.create(roleName, ...)

The Serverless alternative is to choose the appropriate SAML role to utilize to replicate the spirit of the original test you're porting.

Guidance

You can learn more about writing and manipulating serverless tests in general and API tests specifically by reviewing the README for the test_serverless section of the Kibana repo.