feat: add conformance test for authorization server metadata by Michito-Okai · Pull Request #170 · modelcontextprotocol/conformance

Michito-Okai · 2026-02-25T11:38:13Z

Motivation and Context

Described in #169

How Has This Been Tested?

npm test (Test Files 6 passed, Tests 85 passed)
npm test -- src/scenarios/authorization-server/authorization-server-metadata.test.ts (Test Files 1 passed, Tests 1 passed)
npm run start -- authorization --url http://localhost:8080/.well-known/oauth-authorization-server/realms/mp
--url URL of the authorization server metadata endpoint
npm run start -- authorization --url http://localhost:8080/.well-known/oauth-authorization-server/realms/invalid
--url invalid URL
npm run start -- list

Breaking Changes

add authorization server mode (-- authorization)
add url option for authorization server mode (--url )
add a conformance test for authorization server metadata

Types of changes

New feature (non-breaking change which adds functionality)

Checklist

I have read the MCP Documentation
My code follows the repository's style guidelines
New and existing tests pass locally
I have added appropriate error handling
I have added or updated documentation as needed

tnorimat

@Michito-Okai Hello, I added some review comments. Could you check them?

tnorimat · 2026-02-26T05:07:38Z

src/scenarios/authorization-server/authorization-server-metadata.ts

+  implements ClientScenarioForAuthorizationServer
+{
+  name = 'authorization-server-metadata-endpoint';
+  specVersions: SpecVersion[] = ['2025-03-26', '2025-06-18', '2025-11-25'];


To avoid duplication, instead of defining spec version here, could you import SpecVersion of types.ts like

import { ClientScenarioForAuthorizationServer, ConformanceCheck, SpecVersion } from '../../types';

?

I fixed. Could you check this?

tnorimat · 2026-02-26T05:13:13Z

src/scenarios/index.ts

+export function getClientScenarioForAuthorizationServer(
+  name: string
+): ClientScenarioForAuthorizationServer | undefined {
+  return clientScenariosForAuthorizationServer.get(name);


getClientScenarioForAuthorizationServer declares its return type as ClientScenarioForAuthorizationServer but that type isn't imported. Could you add it to the import from '../types' on line 1 ?

I fixed. Could you check this?

tnorimat · 2026-02-26T05:18:03Z

src/index.ts

+    versionScenarios = listClientScenariosForSpec(version);
+  } else if (command === 'authorization') {
+    versionScenarios =
+      listClientScenariosForAuthorizationServerForSpec(version);


The function is called inside filterScenariosBySpecVersion but was never imported from './scenarios'. It's exported from index.ts but missing from the import block at line18-34 of index.ts. Could you import that ? like

import { listScenarios, listClientScenarios, listActiveClientScenarios, listPendingClientScenarios, listAuthScenarios, listMetadataScenarios, listCoreScenarios, listExtensionScenarios, listBackcompatScenarios, listScenariosForSpec, listClientScenariosForSpec, getScenarioSpecVersions, listClientScenariosForAuthorizationServer, listClientScenariosForAuthorizationServerForSpec, ALL_SPEC_VERSIONS } from './scenarios';

I fixed. Could you check this?

tnorimat · 2026-02-26T05:20:33Z

src/index.ts

+        authorizationServerScenarios = filterScenariosBySpecVersion(
+          authorizationServerScenarios,
+          specVersionFilter,
+          'client'


authorization instead of client ?

I fixed. Could you check this?

tnorimat · 2026-02-26T05:23:16Z

src/index.ts

+          console.error(`  ${err.path.join('.')}: ${err.message}`);
+        });
+        console.error('\nAvailable authorization server scenarios:');
+        listClientScenarios().forEach((s) => console.error(`  - ${s}`));


listClientScenariosForAuthorizationServer instead of listClientScenarios ?

I fixed. Could you check this?

tnorimat · 2026-02-26T05:29:03Z

src/index.ts

+        console.log('');
+      }
+      console.log(
+        'Authorization server scenarios (test against a authorization server):'


an authorization server instead of a authorization server ?

I fixed. Could you check this?

tnorimat · 2026-02-26T05:33:42Z

src/types.ts

  run(serverUrl: string): Promise<ConformanceCheck[]>;
 }
+
+export interface ClientScenarioForAuthorizationServer {


Both interfaces ClientScenario and ClientScenarioForAuthorizationServer have the exact same shape. Consider using a type alias (type ClientScenarioForAuthorizationServer = ClientScenario) or just reusing ClientScenario directly?.

The shape of ClientScenarioForAuthorizationServer is likely to change with enhancements, so I defined ClientScenarioForAuthorizationServer.

tnorimat · 2026-02-26T05:35:55Z

src/scenarios/authorization-server/authorization-server-metadata.ts

+    let errorMessage: string | undefined;
+    let details: any;
+
+    const fail = (msg: string) => {


The fail is never called.
All validation methods throw instead, and the catch block sets status and errorMessage directly.

For example,

} catch (error) { status = 'FAILURE'; errorMessage = error instanceof Error ? error.message : String(error); }

I fixed. Could you check this?

tnorimat · 2026-02-26T05:42:04Z

src/scenarios/authorization-server/authorization-server-metadata.ts

+      );
+    }
+
+    const expectedIssuer = serverUrl.replace(


It only removes the first occurrence. If the URL were e.g. https://example.com/.well-known/oauth-authorization-server/tenant, the replace would produce https://example.com//tenant (note the path portion is kept). Per RFC 8414 §3, the issuer for a path-aware metadata URL /.well-known/oauth-authorization-server/<path> should be https://example.com/<path>. The current approach works correctly for the root case but may be incorrect for path-based issuers.

For example,

const url = new URL(serverUrl); const wellKnownPrefix = '/.well-known/oauth-authorization-server'; const suffix = url.pathname.startsWith(wellKnownPrefix + '/') ? url.pathname.slice(wellKnownPrefix.length) : ''; const expectedIssuer = `${url.origin}${suffix}`;

I think that if the URL were e.g. https://example.com/.well-known/oauth-authorization-server/tenant, the replace would produce not https://example.com//tenant but https://example.com/tenant. Therefore I think that the current approach works correctly for path-based issuers.

tnorimat · 2026-02-26T05:44:29Z

src/index.ts

-    if (options.server || (!options.client && !options.server)) {
+    if (
+      options.server ||
+      (!options.client && !options.server && !options.authorization)


The server command supports running a single scenario (--scenario), suite selection (suite), expected-failures baselines, and verbose output (--verbose). The new authorization command has none of these, making it less flexible. This may be intentional for an initial implementation but is worth noting. It is up to you to consider this point by this PR.

The scenario, suite, expected-failures and verbose options are outside the scope of this PR. I am considering to add these options in a future enhancement. How does that sound?

tnorimat reviewed Feb 26, 2026

View reviewed changes

feat: add conformance test for authorization server metadata

eacc9e7

Michito-Okai force-pushed the authorization-server-metadata branch from 9617458 to eacc9e7 Compare February 27, 2026 02:37

Conversation

Michito-Okai commented Feb 25, 2026

Motivation and Context

How Has This Been Tested?

Breaking Changes

Types of changes

Checklist

Uh oh!

tnorimat left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants