improve performance of pr script in dev/stats #32735
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Michael Robinson <merobi@gmail.com>
Signed-off-by: Michael Robinson <merobi@gmail.com>
Signed-off-by: Michael Robinson <merobi@gmail.com>
… dev/protm-script
Signed-off-by: Michael Robinson <merobi@gmail.com>
Signed-off-by: Michael Robinson <merobi@gmail.com>
Signed-off-by: Michael Robinson <merobi@gmail.com>
Signed-off-by: Michael Robinson <merobi@gmail.com>
… dev/protm-script
Signed-off-by: Michael Robinson <merobi@gmail.com>
mobuchowski
approved these changes
Jul 21, 2023
Contributor
mobuchowski
left a comment
mobuchowski
left a comment
There was a problem hiding this comment.
LGTM, just need to add the newline at the end of file :)
Lee-W
approved these changes
Jul 21, 2023
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
Signed-off-by: Michael Robinson <merobi@gmail.com>
… dev/protm-script
Signed-off-by: Michael Robinson <merobi@gmail.com>
Signed-off-by: Michael Robinson <merobi@gmail.com>
… dev/protm-script
Signed-off-by: Michael Robinson <merobi@gmail.com>
… dev/protm-script
Signed-off-by: Michael Robinson <merobi@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the
get_contributor_statsscript in dev/stats, the endpoint used to get PRs from the GitHub API, which does not accept dates or date ranges, causes the script to error out with a rate-limit exception before it can finish executing.With this change, the script will get PRs from the API by first getting commits, which allows for date specification. Together with other changes, this will significantly decrease the number of calls required and allow the script to run to completion.
All-told, the changes here reduce the number of requests by around 50%, cutting well over 1000 requests when executing the script for this month.
Also included:
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.