Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/ASF_SOURCE_RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ tag and staging directory for the bytes selected by the Release Manager.

## Build and test an unsigned candidate

Normal PR and main CI create and validate an unsigned archive of the checked-out commit on every run, including documentation and asset changes. This install-free check catches source inventory and dependency-boundary drift before a Release Manager dispatches the full candidate workflow. It does not sign, stage or publish an RC. The reviewed external website build dependency boundary is recorded in [`docs/code-origin-audit.md`](../docs/code-origin-audit.md#website-build-dependency-boundary).

Run **Prepare ASF source candidate** from `main`, supplying the exact version.
The workflow:

Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,10 @@ jobs:
- name: Check ASF source headers
run: npm run check:asf-headers

# Every tracked input can change the source archive, including docs and images.
- name: Verify repository source archive
run: node scripts/asf-source-release.mjs create --revision HEAD --version "$(node -p 'require("./package.json").version')"

# Locale hygiene ratchet (node built-ins only, so it runs install-free): a new UI locale must only ever mean a new
# UiCatalog key, so locale-literal branches, silent locale defaults,
# and payload sniffing may shrink but never grow.
Expand Down
10 changes: 10 additions & 0 deletions docs/code-origin-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,9 @@ non-text image must match one of these paths; executable and archive magic is
rejected even if a path is listed here.

- `.github/assets/*.png`: the README hero images rendered from the website, recorded above.
- `.github/assets/workhub-turn-admission/receipts-*.png`: light/dark Storybook screenshots of Maka's coordination receipts, contributed as visual review evidence in pull request #4993 (commit `a41d3ac4a`).
- `website/src/assets/incubator.png`: unmodified Apache Incubator logo from `https://www.apache.org/logos/res/incubator/default.png`, used as ASF trademark branding; its origin is also recorded in `website/README.md`. SHA-256: `e7ec2b8078606a77c36c3b9e295409c80dc28266118e32c61548f01cd59c0f0d`.
- `website/src/assets/social.*.png`: social previews rendered from Maka's own website by `website/scripts/readme-hero.mjs`, using the same copy, styles and licensed fonts as the README heroes.
- `apps/desktop/assets/icon.png`: the AI-generated application mark recorded above.
- `apps/desktop/assets/app-icons/*.png`: `mono.png` is the contributor-submitted grayscale derivative of the application mark from pull request #3431; the remaining variants are reproducibly rendered from the Apache-licensed geometry and palette in `scripts/generate-app-icons.py` and byte-checked by `scripts/generate-app-icons.test.mjs`.
- `apps/desktop/build/*.png`: contributor-submitted DMG artwork from pull request #3817; that contribution records Codex as review and verification assistance, not as the source of the artwork.
Expand All @@ -257,6 +260,13 @@ rejected even if a path is listed here.
- `packages/storage/src/__tests__/foreign-session-store.test.ts`: Apache-licensed storage fixture containing literal bidi and bell characters to verify durable imported-title sanitization.
- `packages/storage/src/__tests__/mcp-config-store.test.ts`: Apache-licensed validation fixture containing a literal control character in a rejected MCP tool name.
- `packages/storage/test-fixtures/v0.1.6-operational-state/runtime.sqlite`: migration fixture created through Maka's public storage APIs at tag `v0.1.6`; its exact origin and SHA-256 are recorded in the adjacent `README.md`.
- `patches/@xterm+xterm+6.0.0.patch`: MIT-licensed xterm.js patch attributed in root `LICENSE`; upstream context includes literal terminal control characters whose bytes must be preserved for patch application.

### Website build dependency boundary

The private `website` workspace uses Astro's optional sharp image processor while building the static website. The lockfile records external `@img/sharp-libvips-*`, `@img/sharp-win32-*` and `@img/sharp-wasm32` packages containing LGPL-3.0-or-later libvips; their code and binaries are not included in the source archive or the static website output. Install them through the root `npm ci`, then build with `npm --workspace @maka/website run build`.

[ASF Category X policy](https://www.apache.org/legal/resolved.html#prohibited) allows external build tools when they do not affect the product's licensing, but prohibits distributing their source or binaries. A lockfile describes dependency resolution, not the contents of the source archive. The source verifier checks the manifests and payloads actually included; it does not approve external dependencies for runtime use or redistribution. After installation, the candidate workflow checks Desktop and CLI dependency notices through `scripts/generate-third-party-notices.mjs` and the existing shipped dependency closure, including renderer dependencies even when declared as development dependencies. External-tool usage and provenance still require release review.

## Bootstrap generative tooling

Expand Down
7 changes: 5 additions & 2 deletions packages/core/src/__tests__/attachments.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,10 @@ describe('resolveAttachmentMimeType (content-first precedence)', () => {

test('keeps a non-image document claim so real document kinds still resolve', () => {
const docx = 'application/vnd.openxmlformats-officedocument.wordprocessingml.document';
assert.equal(resolveAttachmentMimeType(Buffer.from('PK'), docx, 'notes.docx'), docx);
assert.equal(resolveAttachmentMimeType(Buffer.from('PK'), undefined, 'notes.docx'), docx);
assert.equal(resolveAttachmentMimeType(Buffer.from('PK\x03\x04'), docx, 'notes.docx'), docx);
assert.equal(
resolveAttachmentMimeType(Buffer.from('PK\x03\x04'), undefined, 'notes.docx'),
docx,
);
});
});
187 changes: 10 additions & 177 deletions scripts/asf-source-release.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -71,17 +71,13 @@ const knownNonCategoryXLicenses = new Set([
'WTFPL OR ISC',
]);
const categoryXLicensePattern = /(?:^|[^A-Za-z])(?:A?GPL|LGPL)-?\d/i;
const packageDependencyFields = [
'dependencies',
'devDependencies',
'optionalDependencies',
'peerDependencies',
];
const bundledDependencyFields = ['bundleDependencies', 'bundledDependencies'];

const textSourceExtensions = new Set([
'.astro',
'.cjs',
'.css',
'.csv',
'.diff',
'.html',
'.js',
'.json',
Expand Down Expand Up @@ -120,6 +116,7 @@ const textSourceBasenames = new Set([
'LICENSE',
'NOTICE',
'network-policy',
'pre-commit',
]);
const maxCommandBuffer = 64 * 1024 * 1024;
const rejectedGpgStatuses = new Set([
Expand Down Expand Up @@ -571,165 +568,24 @@ function validateArchiveContents(archivePath, identity, entries) {
source: basename(archivePath),
version: identity.version,
});
validateNodePackageInputs(candidateRoot, identity, entries);
validateSourcePackageLicenses(candidateRoot, identity, entries);
validateNonTextInputs(candidateRoot, identity, entries);
} finally {
rmSync(temporaryRoot, { force: true, recursive: true });
}
}

function validateNodePackageInputs(candidateRoot, identity, entries) {
const files = entries.filter((entry) => entry && !entry.endsWith('/'));
const rootPrefix = `${identity.rootDirectory}/`;
const readEntry = (entry) =>
readFileSync(join(candidateRoot, entry.slice(rootPrefix.length)), 'utf8');
const noticeLicenses = new Map();
const generatedNpmNotice = `${rootPrefix}apps/desktop/resources/licenses/npm/THIRD_PARTY_NOTICES.txt`;
if (files.includes(generatedNpmNotice)) {
const entry = generatedNpmNotice;
for (const block of readEntry(entry).split(/\n={20,}\n/u)) {
const packageKey = /^Package: (.+)$/mu.exec(block)?.[1];
const license = /^Selected license: (.+)$/mu.exec(block)?.[1];
if (!packageKey || !license) continue;
const previous = noticeLicenses.get(packageKey);
if (previous && previous !== license) {
throw new Error(`Conflicting license inventory for ${packageKey}: ${previous}, ${license}`);
}
noticeLicenses.set(packageKey, license);
}
}

const lockEntries = files.filter((name) =>
/\/(?:package-lock|npm-shrinkwrap)\.json$/u.test(name),
);
const parsedLocks = new Map();
const readLock = (entry) => {
if (parsedLocks.has(entry)) return parsedLocks.get(entry);
const lock = parseArchiveJson(readEntry(entry), entry);
if (lock.lockfileVersion !== 3 || Object.hasOwn(lock, 'dependencies')) {
throw new Error(
`Unsupported npm lockfile version ${lock.lockfileVersion} or schema: ${entry}`,
);
}
if (!lock.packages || typeof lock.packages !== 'object' || Array.isArray(lock.packages)) {
throw new Error(`Cannot safely classify package lockfile without packages: ${entry}`);
}
parsedLocks.set(entry, lock);
return lock;
};

const manifests = new Map(
files
.filter((name) => name.endsWith('/package.json'))
.map((entry) => [entry, parseArchiveJson(readEntry(entry), entry)]),
);
const rootLockEntry = `${rootPrefix}package-lock.json`;
for (const [entry, manifest] of manifests) {
function validateSourcePackageLicenses(candidateRoot, identity, entries) {
for (const entry of entries.filter((name) => name.endsWith('/package.json'))) {
const path = join(candidateRoot, entry.slice(identity.rootDirectory.length + 1));
const manifest = parseArchiveJson(readFileSync(path, 'utf8'), entry);
if (manifest.license) validateReleaseLicense(manifest.license, entry);
else if (manifest.private !== true) {
throw new Error(`Cannot safely classify package manifest without a license: ${entry}`);
}
const dependencyNames = declaredPackageDependencies(manifest, entry);
if (dependencyNames.length > 0) {
const directory = dirname(entry);
const lockEntry = [
`${directory}/npm-shrinkwrap.json`,
`${directory}/package-lock.json`,
rootLockEntry,
].find((name) => lockEntries.includes(name));
if (!lockEntry) {
throw new Error(`Cannot safely classify package manifest ${entry} without lock provenance`);
}
const lock = readLock(lockEntry);
for (const name of dependencyNames) {
if (
!Object.keys(lock.packages).some(
(lockPath) =>
lockPath === `node_modules/${name}` || lockPath.endsWith(`/node_modules/${name}`),
)
) {
throw new Error(
`Cannot safely classify ${name} declared by ${entry} without matching lock provenance`,
);
}
}
}
}

for (const entry of files.filter((name) => /\/(?:pnpm-lock\.yaml|yarn\.lock)$/u.test(name))) {
throw new Error(`Cannot safely classify unsupported package lockfile: ${entry}`);
}
for (const entry of lockEntries) {
const lock = readLock(entry);
for (const [lockPath, dependency] of Object.entries(lock.packages)) {
if (!lockPath) continue;
if (dependency?.link === true) {
const name = lockPath.slice(lockPath.lastIndexOf('node_modules/') + 13);
const targetManifestEntry = join(dirname(entry), dependency.resolved ?? '', 'package.json');
const targetManifest = manifests.get(targetManifestEntry);
const targetLockPath = dependency.resolved;
const targetLock = targetLockPath ? lock.packages[targetLockPath] : undefined;
if (
!targetManifest ||
targetManifest.name !== name ||
!targetLock ||
targetLock.name !== targetManifest.name ||
targetLock.version !== targetManifest.version
) {
throw new Error(`Cannot safely classify workspace link ${name} in ${entry}`);
}
continue;
}
if (!lockPath.includes('node_modules/')) {
const manifestEntry = join(dirname(entry), lockPath, 'package.json');
const manifest = manifests.get(manifestEntry);
if (!manifest || manifest.name !== dependency.name) {
throw new Error(`Cannot safely classify workspace package ${lockPath} in ${entry}`);
}
if (dependency.version !== manifest.version) {
throw new Error(`Workspace version mismatch for ${lockPath} in ${entry}`);
}
if (dependency.license) {
validateReleaseLicense(dependency.license, `${lockPath} in ${entry}`);
}
continue;
}
const name = dependency.name ?? lockPath.slice(lockPath.lastIndexOf('node_modules/') + 13);
const version = dependency.version;
const packageKey = version ? `${name}@${version}` : undefined;
const license =
(packageKey ? noticeLicenses.get(packageKey) : undefined) ?? dependency.license;
if (!license) {
throw new Error(
`Cannot safely classify ${name}${version ? `@${version}` : ''} in ${entry}`,
);
}
validateReleaseLicense(license, `${name}${version ? `@${version}` : ''} in ${entry}`);
}
}
}

function declaredPackageDependencies(manifest, entry) {
const names = new Set();
for (const field of packageDependencyFields) {
const dependencies = manifest[field];
if (dependencies === undefined) continue;
if (!dependencies || typeof dependencies !== 'object' || Array.isArray(dependencies)) {
throw new Error(`Invalid ${field} in package manifest: ${entry}`);
}
for (const name of Object.keys(dependencies)) names.add(name);
}
for (const field of bundledDependencyFields) {
const dependencies = manifest[field];
if (dependencies === undefined) continue;
if (!Array.isArray(dependencies) || dependencies.some((name) => typeof name !== 'string')) {
throw new Error(`Invalid ${field} in package manifest: ${entry}`);
}
for (const name of dependencies) names.add(name);
}
return [...names];
}

function parseArchiveJson(contents, entry) {
try {
return JSON.parse(contents);
Expand Down Expand Up @@ -779,16 +635,13 @@ function validateNonTextInputs(candidateRoot, identity, entries) {
} catch {
// Invalid UTF-8 continues to the candidate-owned provenance inventory.
}
const imageFormat = sourceImageFormat(contents);
const relativePath = entry.slice(`${identity.rootDirectory}/`.length);
if (
provenancePatterns.some((pattern) => sourceInventoryPatternMatches(pattern, relativePath))
) {
continue;
}
throw new Error(
`Cannot safely classify non-text release input ${entry}${imageFormat ? ` (${imageFormat})` : ''}`,
);
throw new Error(`Cannot safely classify non-text release input ${entry}`);
}
}

Expand Down Expand Up @@ -829,26 +682,6 @@ function compiledArtifactFormat(contents) {
return undefined;
}

function sourceImageFormat(contents) {
const prefix = contents.subarray(0, 12);
if (prefix.subarray(0, 8).equals(Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]))) return 'PNG';
if (prefix.subarray(0, 3).equals(Buffer.from([0xff, 0xd8, 0xff]))) return 'JPEG';
if (
prefix
.subarray(0, 6)
.toString('ascii')
.match(/^GIF8[79]a$/u)
)
return 'GIF';
if (
prefix.subarray(0, 4).toString('ascii') === 'RIFF' &&
prefix.subarray(8, 12).toString('ascii') === 'WEBP'
) {
return 'WebP';
}
return undefined;
}

function sourceNonTextProvenancePatterns(provenance) {
const section = provenance
.split('### Source archive non-text inventory\n', 2)[1]
Expand Down
Loading