Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🏗🐛 Prevent most async throwing of errors during tests #16916

Merged
merged 3 commits into from
Jul 25, 2018
Merged

🏗🐛 Prevent most async throwing of errors during tests #16916

merged 3 commits into from
Jul 25, 2018

Conversation

rsimha
Copy link
Contributor

@rsimha rsimha commented Jul 19, 2018

This PR does the following:

  1. Stubs out rethrowAsync in src/log.js during tests with rethrowAsyncForTests
  2. Creates a mechanism with which tests can opt in to the default async throwing behavior (for example, test-log.js)

With this. the vast majority of async throw problems we're seeing should be eliminated.

Partial fix for #14406
Partial fix for #14360
Fixes #15658

@rsimha rsimha self-assigned this Jul 19, 2018
Copy link
Contributor

@lannka lannka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised (in a good way) that only 19 tests start to fail.

You can skip them for now and assign owners to fix.

@rsimha
Copy link
Contributor Author

rsimha commented Jul 19, 2018

No, it's 19 failures out of 400 tests. Most of the 8000 tests didn't run. This isn't ready yet.

src/log.js Outdated
* @visibleForTesting
* @param {...*} var_args
*/
export function rethrowAsyncForTests(var_args) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we allow rest/spread in the code base right? lets just use that instead of var_args

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use var_args in several other spots in log.js, so I wanted to be consistent here. Also, there's a lint rule against using the spread operator.

"amphtml-internal/no-spread": 2,

If it's possible to use rest/spread here, I think it makes sense to switch this and all other instances in the file in a separate PR.

src/log.js Outdated
* @visibleForTesting
* @param {...*} var_args
*/
export function rethrowAsyncForTests(var_args) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move this method to init_tests? or any where under test folders.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant to do this once I got everything to work. Done.

allowConsoleError(() => { expect(() => {
clock.tick(1);
validateData({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess you can remove all the clock ticks above in this test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor Author

@rsimha rsimha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lannka PTAL.

src/log.js Outdated
* @visibleForTesting
* @param {...*} var_args
*/
export function rethrowAsyncForTests(var_args) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant to do this once I got everything to work. Done.

allowConsoleError(() => { expect(() => {
clock.tick(1);
validateData({
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@codecov-io
Copy link

Codecov Report

Merging #16916 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #16916      +/-   ##
==========================================
- Coverage   77.89%   77.88%   -0.02%     
==========================================
  Files         562      562              
  Lines       41142    41142              
==========================================
- Hits        32048    32042       -6     
- Misses       9094     9100       +6
Flag Coverage Δ
#integration_tests 36.18% <ø> (+0.02%) ⬆️
#unit_tests 76.93% <ø> (-0.03%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dfb7c7e...e3e4b37. Read the comment docs.

@rsimha rsimha merged commit 078a2a9 into ampproject:master Jul 25, 2018
@rsimha rsimha deleted the 2018-07-19-AsyncThrow branch July 25, 2018 21:50
@@ -381,7 +381,8 @@ describe.configure().skipSafari().run('XHR', function() {
});
});

it('should do simple JSON fetch', () => {
// TODO(#16916): Make this test work with synchronous throws.
it.skip('should do simple JSON fetch', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no explanation why but after this, it silently fails rest of the tests in the file.

What should have been an execution of 100 tests altogether, are now just 9 tests but passing

results via locally running gulp test --files=test/functional/test-xhr.js --watch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you hit the debug button in the karma window, do you see any errors when the tests are re run?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@prateekbh I was able to repro your problem. One of your tests is throwing an error in beforeEach(). Mocha considers this a catastrophic error and bails. Let me know if you need help reproing it.

Copy link
Member

@prateekbh prateekbh Jul 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well i can use some help on how to find whats the issue, I am creating a PR which just touched xhr-impl.js and hence I reproduced this error.
Not really sure about the setup and test cases here in functional

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mocha silently skips tests when an async error is thrown
6 participants