Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Site Migration: Add A/B test check before showing certain UI/UX updates #97572

Open
wants to merge 4 commits into
base: trunk
Choose a base branch
from

Conversation

sixhours
Copy link
Contributor

@sixhours sixhours commented Dec 17, 2024

Proposed Changes

  • We have a variation of the new migration flow working behind a feature flag migration-flow/experiment.
  • This swaps out the changes behind the feature flag for an upcoming A/B test assignment -- users assigned to the treatment group will see the UI/UX changes.

Why are these changes being made?

  • p9Jlb4-fiH-p2 A/B testing these changes

Testing Instructions

  • TBD until the test has been set up in ExPlat

Pre-merge Checklist

  • Has the general commit checklist been followed? (PCYsg-hS-p2)
  • Have you written new tests for your changes?
  • Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
  • Have you checked for TypeScript, React or other console errors?
  • Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
  • Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
    • For UI changes, have we tested the change in various languages (for example, ES, PT, FR, or DE)? The length of text and words vary significantly between languages.
  • For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

@sixhours sixhours added the [Feature] Site Migration Features related to site migrations to WPcom label Dec 17, 2024
@sixhours sixhours requested a review from a team December 17, 2024 21:49
@sixhours sixhours self-assigned this Dec 17, 2024
@matticbot matticbot added [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. labels Dec 17, 2024
@matticbot
Copy link
Contributor

matticbot commented Dec 17, 2024

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

App Entrypoints (~1739 bytes removed 📉 [gzipped])

name                   parsed_size           gzip_size
entry-stepper              -2133 B  (-0.1%)     -383 B  (-0.1%)
entry-main                  -758 B  (-0.0%)    -1530 B  (-0.3%)
entry-login                 -758 B  (-0.0%)    -1543 B  (-0.3%)
entry-subscriptions         -729 B  (-0.0%)     -214 B  (-0.0%)
entry-domains-landing       -729 B  (-0.1%)     -214 B  (-0.1%)
entry-browsehappy           -729 B  (-0.3%)     -214 B  (-0.3%)

Common code that is always downloaded and parsed every time the app is loaded, no matter which route is used.

Sections (~3356 bytes removed 📉 [gzipped])

name                                parsed_size           gzip_size
update-design-flow                       -962 B  (-0.1%)     -667 B  (-0.2%)
design-first-flow                        -896 B  (-2.2%)      -64 B  (-0.7%)
onboarding-flow                          -858 B  (-0.9%)     -109 B  (-0.9%)
with-theme-assembler-flow                -766 B  (-0.4%)     -100 B  (-0.2%)
update-options-flow                      -766 B  (-1.0%)     -100 B  (-1.5%)
trial-wooexpress-flow                    -766 B  (-1.0%)      -94 B  (-1.2%)
tailored-ecommerce-flow                  -766 B  (-0.9%)      -99 B  (-1.2%)
site-setup-wg                            -766 B  (-0.5%)     -107 B  (-0.3%)
site-setup-flow                          -766 B  (-0.5%)     -107 B  (-0.3%)
site-migration-flow                      -766 B  (-0.6%)     -104 B  (-0.5%)
readymade-template-flow                  -766 B  (-0.3%)      -96 B  (-0.1%)
migration-signup                         -766 B  (-0.9%)     -103 B  (-0.9%)
migration-flow                           -766 B  (-0.7%)     -104 B  (-0.6%)
import-flow                              -766 B  (-0.8%)     -100 B  (-0.7%)
hosted-site-migration-flow               -766 B  (-0.6%)     -104 B  (-0.5%)
free-post-setup-flow                     -766 B  (-1.0%)      -98 B  (-1.5%)
entrepreneur-flow                        -766 B  (-0.3%)      -98 B  (-0.1%)
assembler-first-flow                     -766 B  (-0.3%)      -97 B  (-0.2%)
ai-assembler-flow                        -766 B  (-0.6%)     -103 B  (-0.5%)
hundred-year-plan                        -740 B  (-1.5%)      -42 B  (-0.3%)
start-writing-flow                       -696 B  (-1.8%)      -57 B  (-0.6%)
newsletter-flow                          -481 B  (-1.4%)      -87 B  (-1.0%)
newsletter-post-setup-flow               +337 B  (+0.1%)      +62 B  (+0.1%)
link-in-bio-post-setup-flow              +337 B  (+0.1%)      +62 B  (+0.1%)
write-flow                               -289 B  (-0.0%)    -1564 B  (-0.5%)
build-flow                               -289 B  (-0.0%)    -1568 B  (-0.5%)
link-in-bio-tld-flow                     -251 B  (-0.0%)    -2327 B  (-0.4%)
stepper-user-step                        -196 B  (-0.1%)     +445 B  (+0.6%)
copy-site-flow                           -196 B  (-0.0%)     -689 B  (-0.2%)
reblogging-flow                          -173 B  (-2.3%)      -31 B  (-1.6%)
new-hosted-site-flow                     -173 B  (-1.5%)      -34 B  (-1.2%)
hundred-year-domain                      -108 B  (-0.2%)      -29 B  (-0.2%)
new-hosted-site-flow-user-included        -65 B  (-0.9%)       -7 B  (-0.3%)
connect-domain                            -65 B  (-0.2%)       -7 B  (-0.1%)

Sections contain code specific for a given set of routes. Is downloaded and parsed only when a particular route is navigated to.

Async-loaded Components (~820 bytes removed 📉 [gzipped])

name                                        parsed_size           gzip_size
async-load-calypso-my-sites-checkout-modal       -196 B  (-0.0%)     -775 B  (-0.2%)
async-load-signup-steps-design-picker            -108 B  (-0.1%)      -45 B  (-0.2%)

React components that are loaded lazily, when a certain part of UI is displayed for the first time.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

@sixhours sixhours changed the title Site Migration: Enable A/B test on feature flag Site Migration: Add A/B test check before showing certain UI/UX updates Dec 19, 2024
@matticbot
Copy link
Contributor

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

  • notifications
  • wpcom-block-editor

To test WordPress.com changes, run install-plugin.sh $pluginSlug add/ab-test-migration-experiment on your sandbox.

@sixhours sixhours requested a review from a team as a code owner December 19, 2024 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Feature] Site Migration Features related to site migrations to WPcom [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants