Site Editor Benchmark: add custom WordPress environment support#2614
Open
Site Editor Benchmark: add custom WordPress environment support#2614
Conversation
Implements automated benchmarking across a 10-environment matrix to measure site editor performance impact of having 10 plugins installed. Includes orchestration script that: - Sets up isolated Studio, Playground CLI, and Playground Web environments - Supports bare, multi-worker, and plugin variants - Installs 10 popular plugins via reusable blueprint - Runs site-editor-benchmark test against each environment - Outputs comparison table showing performance metrics across all variants Files: - metrics/tests/site-editor-benchmark.test.ts: Playwright benchmark test (from PR #2368) - scripts/benchmark-site-editor/benchmark.ts: Main orchestration script with environment setup - scripts/benchmark-site-editor/plugins-blueprint.json: Blueprint with 10 plugins - scripts/benchmark-site-editor/{package.json,tsconfig.json,README.md}: Script configuration
…udio-benchmark-local-sites-for-comparison
Replace WP-CLI binary approach with HTTP-based REST API for cross-platform simplicity and proper activation hook firing. Also improves login flow timeout handling for Local sites and ensures teardown cleanup even if plugin install fails. - install-plugins-local.ts: Use cookie-based auth + nonce extraction + POST /wp-json/wp/v2/plugins - benchmark.ts: Wrap plugin install in try/catch, fix variable ordering, improve teardown guarantees - measure-site-editor.ts: Use Promise.all for click+navigation to prevent timeout on redirects
…port Remove the Local GraphQL client and site lifecycle management (~500 lines). Instead, accept any running WordPress site via repeatable --custom flags with URL and credentials. Plugin installation uses the standard WP REST API and gracefully skips already-installed plugins.
…hmark-local-sites-for-comparison
…hmark-local-sites-for-comparison
Collaborator
📊 Performance Test ResultsComparing e828ddc vs trunk site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
Related to STU-1305
Follow up to #2600
Proposed Changes
Adds the ability to benchmark any running WordPress site, not just Studio and Playground environments. This PR:
--customflag. A running WordPress site can be added to the benchmark run via--custom=<name>,<url>[,<user>,<password>]. The flag is repeatable, so multiple custom sites can be compared in a single run (eg. bare vs with plugins). Username defaults to admin, password to password.--install-pluginsflag. This installs the plugins in theplugins-blueprint.jsonfile using the standard WordPress REST API (cookie auth + nonce).--install-pluginsinstalls plugins for all custom environments, or--install-plugins=<name1>,<name2>targets specific ones./wp-admin/explicitly.Examples
Testing Instructions
cd tools/benchmark-site-editor npm install npm run benchmark -- --skip-studio --skip-playground-cli --skip-playground-web \ --custom=my-site,http://localhost:10003--install-pluginsand verify plugins are installed--customflags to verify multiple sites worknpm run benchmark -- --only=studioPre-merge Checklist