Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SP-1612 Add tests to ensure the Scheduler Snapshot Dashboard is working properly in LFA mode #111

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

emanehab99
Copy link
Collaborator

@emanehab99 emanehab99 commented Oct 2, 2024

This PR adds 2 tests for the LFA mode in addition to the playwright tests. One test basically makes sure the dashboard is rendering as expected, whether it's an LFA environment or not.
If it's an LFA environment, the other test checks that at least one snapshot is retrieved and that it is read successfully.
I used the environment variables needed for accessing the S3 buckets to check if it's a LFA environment or not.

The tests are async because the efdclient functions are async. Hence, the pytest-asyncio dependency is added to test-requirements.txt. A related config is added to pyproject.yaml as well.

The async tests had to be moved to a separate file. Apparantly the way pytest runs async tests aren't compatible with how the sync unittest tests run. When added to the main file, the tests always pass, regardless of what happens in there.

@emanehab99
Copy link
Collaborator Author

The tests should pass once the LFA fix is merged

@emanehab99
Copy link
Collaborator Author

@ehneilsen Thanks for merging the fix into this PR. If you're happy with the changes, can you please approve it?

@ehneilsen
Copy link
Collaborator

ehneilsen commented Oct 7, 2024

@ehneilsen Thanks for merging the fix into this PR. If you're happy with the changes, can you please approve it?

I'd like to see the tests pass first, but am having trouble getting it to. The issue isn't with this code itself, I think, but rather that I need to find a version of rubin_scheduler to run it with that will both work with this schedview and also the pickles in the LFA. What version of rubin_scheduler are you using when you are running it?
I think these issues will calm down a bit after the latest round of breaking changes to rubin_scheduler and their incorporation into the version the observatory uses, at which point the pickles in the LFA will start to be compatible with contemporary versions of schedview.

@emanehab99
Copy link
Collaborator Author

I use the latest version of rubin-scheduler and the tests don't pass in my dev environment at USDF (and they shouldn't as long as the snapshot files are not compatible with rubin-scheduler).
My intention from creating this PR was to make sure the dashboard renders successfully in LFA mode using the traditional tests (which runs in the CI) in addition to the playwright tests, which originally failed due to the missing function but they don't run in the CI.
Anyway, no problem. I was just hoping to merge this PR soon to avoid leaving it for long time then struggling with rebasing and changing environment to be able to run it again.

Copy link
Collaborator

@ehneilsen ehneilsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just hoping to merge this PR soon to avoid leaving it for long time then struggling with rebasing and changing environment to be able to run it again.

Go ahead an rebase it now, and then merge. Yes, the LFA stuff will still be broken, but not more broken than it already is, or for a different reason, so I think merging is okay.

@emanehab99
Copy link
Collaborator Author

Thanks Eric. One test is failing though. The dashboard also fails when trying to make the basis functions dataframe. It's the same error raised in the failing test

2024-10-17 11:08:36,025: Starting to make scheduler summary dataframe.
  File "/home/eman/CodeProjects/schedview/schedview/app/scheduler_dashboard/unrestricted_scheduler_snapshot_dashboard.py", line 642, in make_scheduler_summary_df
    self._reward_df = self._scheduler.make_reward_df(self._conditions)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eman/anaconda3/envs/schedview/lib/python3.11/site-packages/rubin_scheduler/scheduler/schedulers/core_scheduler.py", line 550, in make_reward_df
    survey_df = survey.make_reward_df(conditions, accum=accum)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eman/anaconda3/envs/schedview/lib/python3.11/site-packages/rubin_scheduler/scheduler/surveys/base_survey.py", line 334, in make_reward_df
    bf_reward = basis_function(conditions)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eman/anaconda3/envs/schedview/lib/python3.11/site-packages/rubin_scheduler/scheduler/basis_functions/basis_functions.py", line 163, in __call__
    self.value = self._calc_value(conditions, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eman/anaconda3/envs/schedview/lib/python3.11/site-packages/rubin_scheduler/scheduler/basis_functions/mask_basis_funcs.py", line 313, in _calc_value
    if self.altaz_limit_pad > 0:
       ^^^^^^^^^^^^^^^^^^^^
2024-10-17 11:08:36,299: Scheduler summary dataframe unable to be updated: 
Traceback (most recent call last):
  File "/home/eman/anaconda3/envs/schedview/lib/python3.11/site-packages/rubin_scheduler/scheduler/basis_functions/mask_basis_funcs.py", line 313, in _calc_value
    if self.altaz_limit_pad > 0:
       ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'AltAzShadowMaskBasisFunction' object has no attribute 'altaz_limit_pad' 

@ehneilsen
Copy link
Collaborator

AttributeError: 'AltAzShadowMaskBasisFunction' object has no attribute 'altaz_limit_pad'

Yes, there was another late change in rubin_scheduler which required another update to the sample data in schedview. Please rebase again and give it another try.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants