Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression test failures #598

Open
14 tasks
karllark opened this issue Jun 30, 2020 · 4 comments
Open
14 tasks

Regression test failures #598

karllark opened this issue Jun 30, 2020 · 4 comments

Comments

@karllark
Copy link
Member

karllark commented Jun 30, 2020

We have updated things in the BEAST that are causing some of the regression tests to fail. The cached files used by these tests need updating. Most of the standard cached files (i.e., generated by running the phat_small example) have already been updated. The remaining regression tests that need updated files or revising so that they are independent of the cached files are:

Updates for cached files/data needed (would also be useful to add code to the appropriate beast-example subdir to recreate these when needed)

  • test_simobs
  • test_compare_spec_type_inFOV
  • test_compare_spec_type_notFOV
  • test_star_type_probability_no_Av
  • test_calc_depth_from_completeness
  • test_create_physicsmodel_with_subgrid
  • test_create_obsmodel_no_subgrid
  • test_create_obsmodel_with_subgrid
  • test_ast_pick_models
  • test_read_lnp_data
  • test_read_noise_data
  • test_create_obsmodel_with_subgrid
  • test_ast_pick_models
  • test_star_type_probability_all_params

We need to document how to generate the cached test files so that we can maintain these regression tests.

@lea-hagen
Copy link
Member

I'm pretty sure the underlying reason for the failures of these is the number of PDF bins used in the fitting. Going from 50 to 100 bins changes the 1D/2D PDFs (obviously) as well as the stats. I keep getting mixed up with how many bins the tests should be using, so it would help to clarify that.

@karllark
Copy link
Member Author

I don't think it is the number of bins, but that the SED file has changed due to removing the LogL < -9. points from the isochrones.

@lea-hagen
Copy link
Member

Oh yeah, that too!

@karllark
Copy link
Member Author

These tests are marked to be skipped in PR #596. When each test is fixed, the skip notation should be removed. Skipping the tests allows for the coverage report to be generated which is useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants