Skip to content

[MNT] Enable RDSTRegressor and RISTRegressor Tests After Ubuntu CI Verification #2599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

SalmanDeveloperz
Copy link

Description

This PR removes the exclusions for the RDSTRegressor and RISTRegressor tests (check_regressor_against_expected_results), allowing them to run on CI.

Changes Implemented

  • Removed the test exclusions for:
    • RDSTRegressor
    • RISTRegressor
  • Verified successful test execution locally using:
    pytest aeon/testing/estimator_checking/ -v
    

Since #2486 has been merged, these tests should now pass consistently on Ubuntu in CI.

Reference Issues/PRs:

Fixes #2458
Depends on #2486

Additional Notes:
Please see the attached ss below for local test results.
Screenshot from 2025-03-08 22-40-38

@aeon-actions-bot aeon-actions-bot bot added enhancement New feature, improvement request or other non-bug code enhancement testing Testing related issue or pull request labels Mar 8, 2025
@aeon-actions-bot
Copy link
Contributor

Thank you for contributing to aeon

I have added the following labels to this PR based on the title: [ $\color{#FEF1BE}{\textsf{enhancement}}$ ].
I have added the following labels to this PR based on the changes made: [ $\color{#2C2F20}{\textsf{testing}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

  • Run pre-commit checks for all files
  • Run mypy typecheck tests
  • Run all pytest tests and configurations
  • Run all notebook example tests
  • Run numba-disabled codecov tests
  • Stop automatic pre-commit fixes (always disabled for drafts)
  • Disable numba cache loading
  • Push an empty commit to re-run CI checks

@SalmanDeveloperz
Copy link
Author

SalmanDeveloperz commented Mar 8, 2025

Hi @baraline , @MatthewMiddlehurst,
I noticed that GitHub Actions workflow is failing due to a missing merge base, Likely because of a shallow fetch (fetch-depth: 1). Would it be possible to update the workflow to use fetch-depth: 0 in actions/checkout to fetch the full history?

Alternatively, if there’s a recommended way to resolve this issue on my end (such as rebasing or modifying the PR structure), please let me know.
Thanks!

@baraline
Copy link
Member

baraline commented Mar 9, 2025

I would guess the pre commit issue is coming from your end, other contributors don't have this issue, so I don't know, did you follow the instructions in the documenation ?
The other failures are due to the tests not passing, so the bug for these two estiamtors is still there

@SalmanDeveloperz
Copy link
Author

Hi,
Thank you for your response!
I followed the instructions in the documentation, but I am still encountering the pre-commit issue. I will recheck my setup to ensure everything is correctly configured. If you have any specific debugging steps or common pitfalls that I should look into, please let me know.

Regarding the test failures, I understand that the bug still exists. Would you like me to investigate further, or should I wait for additional guidance? Let me know how I can proceed. Thanks!

@baraline
Copy link
Member

I don't think we had this issue before, if you find the source we could had it to the documentation if it's relevant.

Concerning the original issue, feel free to investigate, we don't have much guidance to offer on it as we couldn't identify the issue ourselves.

@SalmanDeveloperz
Copy link
Author

SalmanDeveloperz commented Mar 13, 2025

Hey,
While investigating issue, I ran pre-commit run --all-files and noticed that Black reformatted a few files. To ensure consistency, I also executed:

black .
flake8 .

These reformatted 5 files, while the rest remained unchanged. Before committing, I wanted to confirm whether applying these formatting changes aligns with the project's contribution guidelines?
Please let me know if this is the preferred approach. Below is the evidence ss
Best Regards
Screenshot from 2025-03-13 20-15-37

@MatthewMiddlehurst
Copy link
Member

I wouldn't worry about the pre-commit issue for now, best focus on the failing tests.

@MatthewMiddlehurst MatthewMiddlehurst changed the title [ENH] Enable RDSTRegressor and RISTRegressor Tests After Ubuntu CI Verification [MNT] Enable RDSTRegressor and RISTRegressor Tests After Ubuntu CI Verification Mar 15, 2025
@MatthewMiddlehurst MatthewMiddlehurst added maintenance Continuous integration, unit testing & package distribution and removed enhancement New feature, improvement request or other non-bug code enhancement labels Mar 15, 2025
@MatthewMiddlehurst
Copy link
Member

Is this still active @SalmanDeveloperz?

@SalmanDeveloperz
Copy link
Author

Is this still active @SalmanDeveloperz?

Hi @MatthewMiddlehurst, I sincerely apologize for a long delay. Actually I was admitted in the hospital for a week and as soon as I discharged,my Mid term exams at University started. Now my exams are over and I will actively work on this PR.
Thanks for patience and your support :)
Best Regards

@SalmanDeveloperz
Copy link
Author

Hi,
I tested the PR thoroughly using the following command:

pytest aeon/testing/tests/test_all_estimators.py -k "Catch22Regressor" --tb=short -v

Test for Catch22Regressor on the CardanoSentiment dataset is failing due to slight numerical differences between the actual and expected results. As an example:

ACTUAL : [0.25, 0.14, 0.30, 0.13, 0.35, 0.19, 0.18, 0.13, 0.16, 0.38]
DESIRED: [0.27, 0.17, 0.32, 0.13, 0.33, 0.19, 0.18, 0.13, 0.17, 0.37]

There is minor difference (max absolute diff ~0.03), and probably caused by outdated dependencies or slight behavior shifts in older tools and packages. All other tests passed successfully. Check the ss below:
Screenshot from 2025-04-05 09-55-01

Please let me know if is it okay to regenerate the expected results or if any specific changes are needed from my end.

Side Note: While working on this, I also noticed a few parts of the setup and workflows that might be benefit from an update. Might be worth considering improvements there to keep things clean and current.
Thanks

@MatthewMiddlehurst
Copy link
Member

That is not the target of this issue nor is is currently failing on CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Continuous integration, unit testing & package distribution testing Testing related issue or pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ENH] estimators failing check_regressor_against_expected_results
3 participants