Skip to content

Proof of concept: Remove the requirement to check out the gutenberg repository and running the full build script#11019

Draft
desrosj wants to merge 35 commits intoWordPress:trunkfrom
desrosj:try/remove-gutenberg-git-checkout-and-build
Draft

Proof of concept: Remove the requirement to check out the gutenberg repository and running the full build script#11019
desrosj wants to merge 35 commits intoWordPress:trunkfrom
desrosj:try/remove-gutenberg-git-checkout-and-build

Conversation

@desrosj
Copy link
Member

@desrosj desrosj commented Feb 23, 2026

This is a work in progress.

This PR attempts to remove the need to checkout the WordPress/gutenberg Git repository locally by instead downloading a copy of the built plugin from the GitHub Container Registry for the respective pinned hash value (see WordPress/gutenberg#75844 for that part of this approach).

This PR also changes usage of ref to sha to avoid confusion. Ref is typically a human-readable branch path. A commit hash or SHA value is the full length, unique hash for a commit.

Trac ticket: Core-64393.

Use of AI Tools

I used Claude Code to make some of the adjustments to the Gruntfile.js file and tools/gutenberg folder.


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

@desrosj desrosj self-assigned this Feb 23, 2026
With this new approach, a pre-built zip will be downloaded instead.
This is no longer necessary because a Git repository is no longer used.
This switches to downloading a prebuilt zip file uploaded to the GitHub Container Registry for a given commit instead.

This eliminates the need to run any Gutenberg build scripts within `wordpress-develop`.
A REF is typically a human-readable path to a branch where as a SHA is a hash representing an individual commit. Since the latter is what's being used here, this aims to avoid confusion.
@desrosj desrosj changed the title Proof of concept: Stop checking out gutenberg repository and running two build scripts Proof of concept: Remove the requirement to check out the gutenberg repository and running the full build script Feb 24, 2026
@desrosj
Copy link
Member Author

desrosj commented Feb 24, 2026

There is one remaining issue to figure out: how to map the icons package correctly from the built plugin.

I've removed the chunk of code responsible for mapping those files to their respective locations within wordpress-develop and the build process succeeds.

@desrosj
Copy link
Member Author

desrosj commented Feb 24, 2026

It seems that the icons are not currently included in the Gutenberg plugin, only in wordpress-develop.

@mcsf I see you introduced this in 5b7a3ad. Could you share a bit more context as to why these are not yet included in the Gutenberg plugin? I don't see them in the latest plugin version, but I may be missing them.

Copy link
Member

@dmsnell dmsnell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a step up @desrosj and should avoid a lot of the delay when checking out the repo. 🙇‍♂️

does the zip contain the proper built versions of the artifacts or does it contain source copies? that is, are there any build steps that are occurring on the Gutenberg side that occur before that ZIP is available?

it would definitely be helpful to know in the docs, at least in download-gutenberg.js because I think it’s really not clear if we are downloading the Gutenberg source or some output. if output, I would love to know on the wordpress-develop side what our expectations are of that artifact and why .layers[0].digest is special.

Perhaps there is a command we could include which comparably would generate the right files were we given the Gutenberg source.


is it right that this still leaves the icon disparity in place?

fs.readFileSync( packageJsonPath, 'utf8' )
);
sha = packageJson.gutenberg?.sha;
ghcrRepo = packageJson.gutenberg?.ghcrRepo;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we short on bytes? the gh part is obvious to me as a reader, but I cannot figure out what cr means, and since this only appears to be documented inside this file, I’m not sure it would be clear from the package.json file, especially since it looks like a standard GitHub repo name.

perhaps something githubRepo would suffice? it doesn’t seem like the package.json value has anything specific to do with the Github Container Registry

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am new to this as well, so I found that GHCR stands for the GitHub Container Registry

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a note within the inlind docblock to explain this at the first occurrence of ghcrRepo. Does that address your concerns here @dmsnell?

I also changed the name of the package to gutenberg-wp-develop-build, making the full reference WordPress/gutenberg/gutenberg-wp-develop-build. It's more specific to the purpose, and hopefully won't lead to any confusion.

Within the GHCR, packages are published to a specific repository. While they are listed on a separate screen, you need to use the Org/Repository/package path to reference them.

const installedHash = fs.readFileSync( hashFilePath, 'utf8' ).trim();
if ( installedHash !== sha ) {
throw new Error(
`SHA mismatch: expected ${ sha } but found ${ installedHash }. Run \`npm run grunt gutenberg-download -- --force\` to download the correct version.`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at first I was going to ask about making this check before downloading, but I see it exists after downloading.

this is probably better because it informs someone that their files are out of date without eagerly downloading a new copy and replacing the old ones.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I also want to create a file in the root of the repository to store a hash of the entire gutenberg directory immediately after unzipping. Then the user can also be made aware of instances where the files in the gutenberg directory were modified, which may be unexpected.

const packageJsonPath = path.join( rootDir, 'package.json' );

/**
* Execute a command, streaming stdio directly so progress is visible.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really streaming? It seems the stdout variable is populated with streaming, but the promise only resolves with the captured stdout when the child process is closed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, how does that look?

@mcsf
Copy link
Contributor

mcsf commented Feb 24, 2026

It seems that the icons are not currently included in the Gutenberg plugin, only in wordpress-develop.

@mcsf I see you introduced this in 5b7a3ad. Could you share a bit more context as to why these are not yet included in the Gutenberg plugin? I don't see them in the latest plugin version, but I may be missing them.

Oof, I'm so glad you pinged me! Fix at WordPress/gutenberg#75866

desrosj and others added 4 commits February 24, 2026 09:11
The Gutenberg repository has been updated to include the icon assets in the plugin build. See github.com/WordPress/gutenberg/pull/75866.
- Add missing hard stop at the end of inline comment.
- Make use if `fetch()` instead of `exec( 'curl' )`.

Co-authored-by: Weston Ruter <westonruter@gmail.com>
@github-actions
Copy link

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
    it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

throw new Error( `Failed to download blob: ${ response.status } ${ response.statusText }` );
}
const buffer = await response.arrayBuffer();
fs.writeFileSync( zipPath, Buffer.from( buffer ) );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at this point it’s probably possible to remove all of the _Sync versions and stream the output directly into the file.

not essential, but it does feel like we’re possibly going out of our way to do more work to avoid a simpler default.

await fs.writeFile( zipPath, response.body );

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not deeply knowledgeable here, but Claude told me that this wouldn't work as you've written "since fs.promises.writeFile doesn't accept a ReadableStream — stream.pipeline is the correct API for this." Does this seem right now?

// Extract the zip into ./gutenberg.
console.log( `\n📦 Extracting ${ zipName } into ./gutenberg...` );
try {
await exec( 'unzip', [ '-q', zipPath, '-d', gutenbergDir ] );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we use zlib.Unzip we should be able to eliminate the exec() wrapper altogether and not depend on the runtime system having unzip in the PATH

of course, if we have a .gz file available from the container registry (which seems like it would exist) then we also have zlib.gunzipSync()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as an additional bit of polish, which is not quite as impactful as removing the exec() abstraction and runtime-dependence, we can also pipe directly from the fetch() into zlib. this would eliminate the temporary files and reduce the memory load. I hesitate to leave this comment because it truly is polish and not part of the basic requirements of this proof-of-concept.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file only exists as a zip currently. If we want that format to be available, we would have to change the workflow generating and pushing the compressed file to the registry.

@desrosj
Copy link
Member Author

desrosj commented Feb 28, 2026

I still need to go through and polish this, but the PR for the Gutenberg repository is ready. That should be merged first anyway, and these changes can't be committed until after the first hash update following that PR being merged. So I'm hoping to clean this up on Monday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants