Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validation to log-regression benchmark #436

Conversation

lovisek
Copy link
Contributor

@lovisek lovisek commented Feb 10, 2024

Validation currently supports only binary classification which returns a single intercept value. If multinomial logistic regression is needed, the validation needs to be updated to support multiple intercept values.

The validation checks logistic regression model's coefficients, intercept, and the number of classes the model can classify. For the coefficients, it compares the sum and sum of squares of all coefficient values with the expected values, and then validates the coefficient count. Regarding the intercept, its value is compared with the expected value, and then the validation checks that the model contains only one intercept.

The validation does not change the code that is being measured and should not really impact benchmark performance. As a quick check, I calculated an average duration from the last 5 repetitions of a single run of the benchmark before and after validation:

The variant with validation appears approx. 1% slower, but this is just from a single run. Given that the benchmark code did not change, I don't think this indicates an actual change, but if necessary, I can do the comparison for multiple runs.

@lbulej lbulej marked this pull request as ready for review February 11, 2024 11:37
Copy link
Collaborator

@farquet farquet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Thanks for the contribution.

@lbulej lbulej merged commit b68f55e into renaissance-benchmarks:master Mar 8, 2024
13 checks passed
@lovisek lovisek deleted the topic/log-regression-validation branch April 11, 2024 18:56
@lovisek lovisek restored the topic/log-regression-validation branch April 30, 2024 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants