Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bundle size diff message #16576

Merged
merged 2 commits into from
Nov 23, 2022
Merged

Fix bundle size diff message #16576

merged 2 commits into from
Nov 23, 2022

Conversation

Gudahtt
Copy link
Member

@Gudahtt Gudahtt commented Nov 17, 2022

The bundle size diff message is using the wrong point of comparision, leading to misleading results on feature branches that have been merged with develop since they were created.

When this feature was introduced, we went back and forth a few times on what we should be comparing the branch with to get an accurate bundle size comparison.

The first attempt used develop as the point of comparison, but that didn't work because it was a moving target, and because it didn't reflect the changes made on this branch. As bundle increases or decreases were added on develop, they would alter the diff on each feature PR.

Then we chose to use the fork-point of the branch, the commit of develop that the branch forked off of. This works for feature branches that don't merge in develop. But the minute develop gets merged in, then unrelated changes on develop affect the measurement.

The most recent commit from develop on the current branch is a better comparison. Any difference between this commit and the feature branch in terms of bundle size would be attributable to the feature branch changes. This is what merge-base gives us.

Manual Testing Steps

The diff message should show zero changes.

It's hard to test the branch cases directly because this always uses develop as the point of comparison, so we'll have to just watch future PRs to ensure the results make sense.

Pre-merge author checklist

  • I've clearly explained:
    • What problem this PR is solving
    • How this problem was solved
    • How reviewers can test my changes
  • Sufficient automated test coverage has been added

Pre-merge reviewer checklist

  • Manual testing (e.g. pull and build branch, run in browser, test code being changed)
  • PR is linked to the appropriate GitHub issue
  • IF this PR fixes a bug in the release milestone, add this PR to the release milestone

If further QA is required (e.g. new feature, complex testing steps, large refactor), add the Extension QA Board label.

In this case, a QA Engineer approval will be be required.

The bundle size diff message is using the wrong point of comparision,
leading to misleading results on feature branches that have been
merged with `develop` since they were created.

When this feature was introduced, we went back and forth a few times on
what we should be comparing the branch with to get an accurate bundle
size comparison.

The first attempt used `develop` as the point of comparison, but that
didn't work because it was a moving target, and because it didn't
reflect the changes made on this branch. As bundle increases or
decreases were added on `develop`, they would alter the diff on each
feature PR.

Then we chose to use the fork-point of the branch, the commit of
`develop` that the branch forked off of. This works for feature
branches that don't merge in `develop`. But the minute `develop` gets
merged in, then unrelated changes on `develop` affect the measurement.

The _most recent_ commit from `develop` on the current branch is a
better comparison. Any difference between this commit and the feature
branch in terms of bundle size would be attributable to the feature
branch changes. This is what `merge-base` gives us.
@Gudahtt Gudahtt marked this pull request as ready for review November 17, 2022 23:49
@Gudahtt Gudahtt requested review from a team and kumavis as code owners November 17, 2022 23:49
@Gudahtt Gudahtt requested a review from ryanml November 17, 2022 23:49
@metamaskbot
Copy link
Collaborator

Builds ready [cdd47cd]
Page Load Metrics (2242 ± 130 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint97156122188
domContentLoaded176927732224281135
load183927942242271130
domInteractive176927722224281135
Bundle size diffs
  • background: 0 bytes
  • ui: 0 bytes
  • common: 0 bytes

@Gudahtt
Copy link
Member Author

Gudahtt commented Nov 18, 2022

The solution on develop came from this StackOverflow post: https://stackoverflow.com/questions/12516169/git-find-the-oldest-commit-of-mine-which-does-not-exist-in-theirs/12517430#12517430

It's the correct solution to the wrong problem. The diagram demonstrates the change being made here:

[other-branch]     H--I--J   
                  /       \
[develop]        A--B--C--M1--D--E--M2--F--G
                    \      \             \
[current-branch]     H---I--M3---J---K---M4--M--HEAD

Previously we'd be getting B, the fork point for the current branch. But when comparing B to HEAD, we're also counting the develop commits B, C, etc, through to F as being caused by the current branch, which isn't accurate.

After this PR, we'll be comparing F to HEAD.

At first I thought this would be skipping the effects of commits H and I, J, etc. but that's not really the case. They still are a part of the changeset in HEAD, they're there and accounted for. By comparing F, we can be confident that everything between there and HEAD is a result of this branch.

@metamaskbot
Copy link
Collaborator

Builds ready [eb81646]
Page Load Metrics (2173 ± 98 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint923801236029
domContentLoaded17402485215420297
load17402485217320398
domInteractive17402485215420297
Bundle size diffs
  • background: 0 bytes
  • ui: 0 bytes
  • common: 0 bytes

Copy link
Contributor

@digiwand digiwand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 🙌

@Gudahtt Gudahtt merged commit 65f2f17 into develop Nov 23, 2022
@Gudahtt Gudahtt deleted the fix-bundle-size-diff-message branch November 23, 2022 17:05
@github-actions github-actions bot locked and limited conversation to collaborators Nov 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants