vscode extension test run modifies package.json after 1.67.0

[hyangah]

Our presubmit tests started to fail consistently since our test setup picked up code 1.67.0 yesterday.
golang/vscode-go#2230

Our test is basically doing

• first, run extension tests written with runTests from @vscode/test-electron
• after the extension tests succeed, run a check on the consistency between our package.json file and documentation/settings info.

After running the extension tests with 1.67.0, it looks like the second step that involves reading package.json sees a differently formatted package.json. Actually, it's not only a differently formatted package.json, but it includes a new "__metadata" property (id, publisherId, ...) at the end.

The test passes if I make the extension tests to use 1.66.2.

We can work around by arranging our presubmit test order or rewriting our package.json test logic to handle this.
But, we want to understand what is going on behind the scene and why this package.json modification is necessary.

Thanks!

[connor4312]

@sandy081 this looks suspicious...

vscode/src/vs/platform/extensionManagement/common/extensionsScannerService.ts

Lines 229 to 241 in e09e27e

	async updateMetadata(extensionLocation: URI, metaData: Partial<Metadata>): Promise<void> {
	const manifestLocation = joinPath(extensionLocation, 'package.json');
	const content = (await this.fileService.readFile(manifestLocation)).value.toString();
	const manifest: IScannedExtensionManifest = JSON.parse(content);
	// unset if false
	metaData.isMachineScoped = metaData.isMachineScoped || undefined;
	metaData.isBuiltin = metaData.isBuiltin || undefined;
	metaData.installedTimestamp = metaData.installedTimestamp || undefined;
	manifest.__metadata = { ...manifest.__metadata, ...metaData };
	await this.fileService.writeFile(joinPath(extensionLocation, 'package.json'), VSBuffer.fromString(JSON.stringify(manifest, null, '\t')));
	}

[sandy081]

This is designed and expected. It is the metadata VS Code stores per extension while installing that can be used later for other features like updating etc.,

So I would recommend your extension to handle this.

[andyleejordan]

@sandy081 this has started breaking our extension's tests without warning, and moreover, the way it writes to package.json it's modifying the whole file (looks like whitespace changes).

[andyleejordan]

@hyangah how have you gone about working around this breaking change? I'm trying to get a release out myself and am suddenly blocked 😭

[andyleejordan]

Whelp, something else went on that broke my tests, but definitely noticed this too and will need to be careful about not committing the reformatted package.json.

[sandy081]

@andschwa Sorry about the caused breakage. I am not sure what is your test doing, but I think it's not a good idea to depend on the content of package.json, because VS Code uses it to write metadata about the extension. We do not remove any existing data but add more data.

[symmb]

What's the recommended way of dealing with the __metadata property when testing an extension with --extensionDevelopmentPath set to a version-controlled source directory? Should the changes be committed or do extension authors have to take care of reverting it (which probably means parsing package.json and clearing the property).