-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix bundle size diff message #16576
Fix bundle size diff message #16576
Conversation
The bundle size diff message is using the wrong point of comparision, leading to misleading results on feature branches that have been merged with `develop` since they were created. When this feature was introduced, we went back and forth a few times on what we should be comparing the branch with to get an accurate bundle size comparison. The first attempt used `develop` as the point of comparison, but that didn't work because it was a moving target, and because it didn't reflect the changes made on this branch. As bundle increases or decreases were added on `develop`, they would alter the diff on each feature PR. Then we chose to use the fork-point of the branch, the commit of `develop` that the branch forked off of. This works for feature branches that don't merge in `develop`. But the minute `develop` gets merged in, then unrelated changes on `develop` affect the measurement. The _most recent_ commit from `develop` on the current branch is a better comparison. Any difference between this commit and the feature branch in terms of bundle size would be attributable to the feature branch changes. This is what `merge-base` gives us.
Builds ready [cdd47cd]
Page Load Metrics (2242 ± 130 ms)
Bundle size diffs
|
The solution on It's the correct solution to the wrong problem. The diagram demonstrates the change being made here:
Previously we'd be getting B, the fork point for the current branch. But when comparing B to HEAD, we're also counting the After this PR, we'll be comparing F to HEAD. At first I thought this would be skipping the effects of commits H and I, J, etc. but that's not really the case. They still are a part of the changeset in HEAD, they're there and accounted for. By comparing F, we can be confident that everything between there and HEAD is a result of this branch. |
Builds ready [eb81646]
Page Load Metrics (2173 ± 98 ms)
Bundle size diffs
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice 🙌
The bundle size diff message is using the wrong point of comparision, leading to misleading results on feature branches that have been merged with
develop
since they were created.When this feature was introduced, we went back and forth a few times on what we should be comparing the branch with to get an accurate bundle size comparison.
The first attempt used
develop
as the point of comparison, but that didn't work because it was a moving target, and because it didn't reflect the changes made on this branch. As bundle increases or decreases were added ondevelop
, they would alter the diff on each feature PR.Then we chose to use the fork-point of the branch, the commit of
develop
that the branch forked off of. This works for feature branches that don't merge indevelop
. But the minutedevelop
gets merged in, then unrelated changes ondevelop
affect the measurement.The most recent commit from
develop
on the current branch is a better comparison. Any difference between this commit and the feature branch in terms of bundle size would be attributable to the feature branch changes. This is whatmerge-base
gives us.Manual Testing Steps
The diff message should show zero changes.
It's hard to test the branch cases directly because this always uses
develop
as the point of comparison, so we'll have to just watch future PRs to ensure the results make sense.Pre-merge author checklist
Pre-merge reviewer checklist
If further QA is required (e.g. new feature, complex testing steps, large refactor), add the
Extension QA Board
label.In this case, a QA Engineer approval will be be required.