Use asdf files for testing compare_spec_type and star_type_probability

[lea-hagen]

The generate_files_for_tests code in BEAST-Fitting/beast-examples#16 creates asdf files for testing the compare_spec_type and star_type_probability tools. This updates test_regresscheck to use those files. @karllark, could you use generate_files_for_tests to update the cached metal_small files (assuming, of course, that it doesn't need any changes first)? Then I'll be able to check if test_regresscheck is working as expected.

This PR also adds hdu=1 to the stats file read-in for compare_spec_type.

[karllark]

@lea-hagen : it would be quicker if you were to generate the needed new files and then just tell me where they are on central storage? Saves me from having to clone your branch, etc.

[lea-hagen]

No problem, I'll send you a message with the path!

[karllark]

Codestyle failures need to be addressed in addition to the files.

[lea-hagen]

Even after uploading new files during the hack day last week, there were still some failing tests. I think I tracked down the issue, so along with new code in BEAST-Fitting/beast-examples#19, this will hopefully work.

[karllark]

Still failing. Hopefully you can find a solution fairly soon. If not, maybe tag some of these tests as skip?

Need this PR to be merged before merging PR #678.

[lea-hagen]

Yesterday I narrowed it down to an issue with folder teardown when doing tests for code in tools.run. I'm fine solving that in a different PR, so I'll skip the relevant tests and push the branch shortly.

[lea-hagen]

@karllark do you know why the noisemodel test is failing? I double checked that all of the input files and settings match, so maybe I just need to rebase and regenerate files again??

[karllark]

No idea. Try a rebase. We have been working on the noise model recently, so I could see one of the recent PRs changing the noisemodel. Depends on how out-of-date your branch is.

[karllark]

Not that I can see what PR would have caused this looking at the network graph.

[lea-hagen]

Yeah, I couldn't figure anything out about noisemodel changes either. I'll rebase and see if the files change.

[lea-hagen]

I rebased and ran
(a) generate_files_from_tests from start to finish as normal
(b) the contents test_toothpick_noisemodel, using the cached files (settings, SED grid, ASTs)

The resulting noise models are identical. So the best I can figure is it's the Mac vs Linux issue (e.g., #169).

Edit - I created the cached files on a Linux virtual server (only accessed from my Mac). So I have no idea.

[karllark]

So, not a Mac versus Linux issue. Sigh.

[lea-hagen]

If you run generate_files_for_tests on your system, does it match the current cached files?

[karllark]

Testing things on my laptop now. Hopefully that will provide some clue.

[karllark]

I have done a number of tests. The tests were failing on my machine until I regenerated the files on my computer and pushed them to the cache directory. Now the tests are passing on my computer. Re-running the github action tests now to see if the online tests pass.

[karllark]

Tests pass online! Yeah!

The only thing I can think of is that there is some package/setup on your computer that is causing the problem. FYI, I am using python 3.8. One thought, did you recently update your system/conda/...?

[karllark]

As the tests pass, this the PR ready for review/merging? If so, please mark as ready to review.

[lea-hagen]

I did rebuild my environment recently (in the past ~month?). I didn't have python 3.8 so tox was grumpy. Super weird that we're getting different results, but I'm glad it's sort of figured out! Thanks for taking the time to poke at it.

I'll mark as ready for review.

[karllark]

Looks great.

[karllark]

Will merge. Would be good to update/add to #598 with what is needed/has been fixed with this PR.