Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change assertEqual to assertAlmostEqual in tests #412

Merged
merged 1 commit into from
Apr 13, 2017

Conversation

huanzhang12
Copy link
Contributor

We should use assertAlmostEqual instead of assertEqual for comparing floating point numbers in tests. Otherwise sometimes there will be some tiny differences between the results and the tests fail.
Similarly, we should use np.testing.assert_almost_equal to replace assertListEqual.
I searched assertEqual and assertListEqual in all tests and it seems this is the last one we should replace.

@guolinke
Copy link
Collaborator

guolinke commented Apr 13, 2017

@huanzhang12
This is used for checking the consistency of model file, so it should be exact same.

refer to #31 (comment)

@huanzhang12
Copy link
Contributor Author

@guolinke I see. So what actually happens when the test fails?
I get this on my laptop, compiled without GPU support:

----------------------------------------------------------------------
running test_basic.py
([('training', u'auc', 0.982663878622836, True)], [('valid_1', u'auc', 0.98148148148148151, True)])
([('training', u'auc', 0.99521007586072752, True)], [('valid_1', u'auc', 0.98290598290598286, True)])
([('training', u'auc', 0.99632043052583807, True)], [('valid_1', u'auc', 0.98433048433048431, True)])
F
======================================================================
FAIL: test (__main__.TestBasic)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_basic.py", line 50, in test
    self.assertEqual(*preds)
AssertionError: 0.028836746890177237 != 0.028836746890177251

----------------------------------------------------------------------
Ran 1 test in 0.065s

FAILED (failures=1)

@guolinke
Copy link
Collaborator

@huanzhang12
not very sure, maybe the sumup order by multi-threading.

https://github.com/Microsoft/LightGBM/blob/master/src/boosting/gbdt.cpp#L904-L907

you can try to remove the omp in these predict function and check it again ?

@huanzhang12
Copy link
Contributor Author

@guolinke It is indeed the multi-threading problem in gbdt.cpp. After removing the pragma the test can pass on the two machines previously failed the test.

So what should we do here? I think we can either do assertAlmostEqual with more digits (like 15 digits), or setting num_threads=1 for this test. Which way do you prefer? Or do you have any better suggestions on this issue?

@guolinke
Copy link
Collaborator

I prefer set num_threads=1 in test_basic.py.

@huanzhang12
Copy link
Contributor Author

huanzhang12 commented Apr 13, 2017

OK, I have amended my commit to use one thread on this test.
Also I added a comment before the assertEqual test in case someone else raises the same question.

@guolinke guolinke merged commit a5f11d4 into microsoft:master Apr 13, 2017
@huanzhang12 huanzhang12 deleted the fix-equal-tests branch April 13, 2017 19:56
@lock lock bot locked as resolved and limited conversation to collaborators Mar 12, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants