Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tests] increase timeout to allow plugin views to be prepared #8151

Merged
merged 2 commits into from
Jul 20, 2020

Conversation

akosyakov
Copy link
Member

What it does

How to test

  • CI should be green

Review checklist

Reminder for reviewers

as well fixes some exceptions caused by disposed widgets

Signed-off-by: Anton Kosyakov <anton.kosyakov@typefox.io>
@akosyakov akosyakov added ci issues related to CI / tests test issues related to unit and api tests labels Jul 8, 2020
@@ -404,6 +404,9 @@ export class TabBarToolbarRegistry implements FrontendApplicationContribution {
* By default returns with all items where the command is enabled and `item.isVisible` is `true`.
*/
visibleItems(widget: Widget): Array<TabBarToolbarItem | ReactTabBarToolbarItem> {
if (widget.isDisposed) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fo instance in the side bar, the title is still visible, but associated widget is disposed since it is collapsed, with bad timing it can throw in own contributions

if (layout instanceof ViewContainerLayout) {
return layout;
}
throw new Error('view container is disposed');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if a widget is disposed then layout is null

@@ -1178,6 +1182,10 @@ export class ViewContainerLayout extends SplitLayout {

// The update function is called on every animation frame until the predefined duration has elapsed.
const updateFunc = (time: number) => {
if (!this.parent) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if a widget gets closed during the animation it will throw in MessageLoop.sendMessage

@akosyakov akosyakov requested a review from kittaakos July 8, 2020 09:20
Copy link
Member

@vince-fugnitto vince-fugnitto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, the tests are still failing locally (which is the case since fdd7c41).
I verified by running the suite 3 times using npx run test @theia/example-browser.

I am running on Linux (Ubuntu 18.04), but it is also reproducible on Gitpod (which uses Ubuntu 20.04 LTS) when executing the tests.

@akosyakov
Copy link
Member Author

akosyakov commented Jul 8, 2020

@vince-fugnitto Could it be that your machine is slow and preparing plugin views takes longer? Gitpod can hit noisy neighbourhood problem then everything is slow. Try to increase timeout and see whether issue is gone or share the output of failed timeouts.

@vince-fugnitto
Copy link
Member

@vince-fugnitto Could it be that your machine is slow and preparing plugin views takes longer? Gitpod can hit noisy neighbourhood problem then everything is slow. Try to increase timeout and see whether issue is gone or share the output of failed timeouts.

I increased the timeout for both to 14000 and it still fails locally. I noticed the following errors:

Error
root INFO   Views
root INFO     ✓ should toggle Explorer (3073ms)
root INFO     ✓ should toggle Source Control (230ms)
root INFO     ✓ should toggle Outline (206ms)
root INFO     ✓ should toggle Problems (43ms)
root INFO   893 passing (2m)
root INFO   9 failing
root INFO   1) TypeScript
       highligh semantic (write) occurences:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/evinfug/workspaces/theia-work/examples/api-tests/src/typescript.spec.js:640:16)

root INFO   2) TypeScript
       editor.action.goToImplementation:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/evinfug/workspaces/theia-work/examples/api-tests/src/typescript.spec.js:653:16)

root INFO   3) TypeScript
       editor.action.goToTypeDefinition:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/evinfug/workspaces/theia-work/examples/api-tests/src/typescript.spec.js:672:16)

root INFO   4) TypeScript
       references-view.find:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/evinfug/workspaces/theia-work/examples/api-tests/src/typescript.spec.js:834:20)

root INFO   5) TypeScript
       references-view.findImplementations:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/evinfug/workspaces/theia-work/examples/api-tests/src/typescript.spec.js:834:20)

root INFO   6) TypeScript
       editor.action.revealDefinition
         within an editor:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/evinfug/workspaces/theia-work/examples/api-tests/src/typescript.spec.js:283:24)

root INFO   7) TypeScript
       editor.action.revealDefinition
         within an editor preview:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/evinfug/workspaces/theia-work/examples/api-tests/src/typescript.spec.js:283:24)

root INFO   8) TypeScript
       editor.action.peekDefinition
         within an editor:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/evinfug/workspaces/theia-work/examples/api-tests/src/typescript.spec.js:354:24)

root INFO   9) TypeScript
       editor.action.peekDefinition
         within an editor preview:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container

@akosyakov
Copy link
Member Author

akosyakov commented Jul 8, 2020

Your errors are very different, not about timeouts. How do you run test? yarn test or yarn test:debug. In the second case you should make sure to keep the page in focus. Could you try from the clean setup?

@vince-fugnitto
Copy link
Member

Your errors are very different, not about timeouts. How do you run test? yarn test or yarn test:debug. In the second case you should make sure to keep the page in focus. Could you try from the clean setup?

I either do:

npx run test @theia/example-browser

or

(cd exampes/browser && yarn test)

Both of them fail similarly, is there a specific way I should run the tests?

@akosyakov
Copy link
Member Author

akosyakov commented Jul 8, 2020

No, but I've never seen such errors on travis or in Gitpod. I'm not sure how to reproduce it. I wonder maybe you have something specific locally.

@akosyakov
Copy link
Member Author

I ran several times in Gitpod and got once:

editor.action.peekDefinition
root INFO       2) within an editor

will check tomorrow what is wrong whether, but rather than that I could not get other errors

I've restarted Travis again to see whether it will fail.

@akosyakov
Copy link
Member Author

Maybe someone else could try to run tests as well. I could not reproduce timeouts anymore on travis or my machine :(

@vince-fugnitto
Copy link
Member

Maybe someone else could try to run tests as well. I could not reproduce timeouts anymore on travis or my machine :(

@marcdumais-work I believe you mentioned the tests fail consistently as well for you?

Copy link
Member

@vince-fugnitto vince-fugnitto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I performed the following and the tests passed locally for me:

  • updated views timeout to 15000
  • updated menus timeout to 15000
  • removed any pre-existing configurations rm -rf ~/.theia

@marcdumais-work
Copy link
Contributor

Maybe someone else could try to run tests as well. I could not reproduce timeouts anymore on travis or my machine :(

@marcdumais-work I believe you mentioned the tests fail consistently as well for you?

On master, yes. I have not tried this PR. Let me give it a quick try locally and I'll report back.

@marcdumais-work
Copy link
Contributor

I performed the following and the tests passed locally for me:

  • updated views timeout to 15000
  • updated menus timeout to 15000
  • removed any pre-existing configurations rm -rf ~/.theia

@vince-fugnitto IIUC you had to step both timeouts from 7500ms to 15000ms for tests to pass consistently?

@vince-fugnitto
Copy link
Member

I performed the following and the tests passed locally for me:

  • updated views timeout to 15000
  • updated menus timeout to 15000
  • removed any pre-existing configurations rm -rf ~/.theia

@vince-fugnitto IIUC you had to step both timeouts from 7500ms to 15000ms for tests to pass consistently?

Yes, else they'd fail on a timeout for the explorer mainly.

@lmcbout
Copy link
Contributor

lmcbout commented Jul 9, 2020

I tried it on my computer and the result is:
yarn run test

Succeed 2
Fail    2

Fail 1:


root INFO   901 passing (2m)
root INFO   1 failing
root INFO   1) TypeScript
       editor.action.peekDefinition
         within an editor:
     Error: Timeout of 30000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

Fail 2:


root INFO   901 passing (2m)
root INFO   1 failing
root INFO   1) Views
       should toggle Explorer:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

@marcdumais-work
Copy link
Contributor

I tested this a bit. I still got the occasional failure with both timeouts set at 30000 but it passes most of the time. With the timeout at 7500 I get similar results as @lmcbout - about 50% success.

@marcdumais-work
Copy link
Contributor

@akosyakov in these tests, is an actual browser used? Is data stored in local storage cleared between runs?

@akosyakov
Copy link
Member Author

akosyakov commented Jul 10, 2020

should toggle Explorer:
Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

How do you get that? Timeout is 7500ms, not 2000ms. Are you sure you are testing against this PR?

in these tests, is an actual browser used? Is data stored in local storage cleared between runs?

If you do yarn test then it should be a clean browser each time.

@RomanNikitenko
Copy link
Contributor

RomanNikitenko commented Jul 10, 2020

Result of tests on my machine(Fedora 31-64-bit/32GiB/Intel® Core™ i7-6820HQ CPU @ 2.70GHz × 8):

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
[rnikitenko /home/rnikitenko/projects/forks/theia/examples/browser]$ root INFO [nsfw-watcher: 17405] Stopped watching: /home/rnikitenko/projects/forks/theia/examples/browser/webpack.config.js
root INFO   Views
root INFO     14) should toggle Explorer
root INFO     ✓ should toggle Source Control (180ms)
root INFO     ✓ should toggle Outline (166ms)
root INFO     ✓ should toggle Problems (44ms)
root INFO   888 passing (2m)
root INFO   14 failing
root INFO   1) Find and Replace
       Replace in the active editor:
     Error: view container is disposed
      at ViewContainer.get (bundle.js:128727:19)
      at ViewContainer.../../packages/core/lib/browser/view-container.js.ViewContainer.getParts (bundle.js:128716:21)
      at ViewContainer.../../packages/core/lib/browser/view-container.js.ViewContainer.getTrackableWidgets (bundle.js:128914:21)
      at ApplicationShell.../../packages/core/lib/browser/shell/application-shell.js.ApplicationShell.track (bundle.js:118249:47)
      at ApplicationShell.<anonymous> (bundle.js:118052:26)
      at step (bundle.js:117314:23)
      at Object.next (bundle.js:117295:53)
      at http://127.0.0.1:3000/bundle.js:117289:71
      at new Promise (<anonymous>)
      at ../../packages/core/lib/browser/shell/application-shell.js.__awaiter (bundle.js:117285:12)

root INFO   2) Menus
       should toggle 'Explorer' view:
     Error: Timeout of 7500ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
  

root INFO   3) Menus
       reveal more context menu in the explorer view container toolbar:
     Error: view container is disposed
      at ViewContainer.get (bundle.js:128727:19)
      at ViewContainer.../../packages/core/lib/browser/view-container.js.ViewContainer.getParts (bundle.js:128716:21)
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/menus.spec.js:88:42)

root INFO   4) TypeScript
       editor.action.triggerSuggest:
     Error: Timeout of 30000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
  

root INFO   5) TypeScript
       highligh semantic (write) occurences:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:640:16)

root INFO   6) TypeScript
       editor.action.goToImplementation:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:653:16)

root INFO   7) TypeScript
       editor.action.goToTypeDefinition:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:672:16)

root INFO   8) TypeScript
       references-view.find:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:834:20)

root INFO   9) TypeScript
       references-view.findImplementations:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:834:20)

root INFO   10) TypeScript
       editor.action.revealDefinition
         within an editor:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:283:24)

root INFO   11) TypeScript
       editor.action.revealDefinition
         within an editor preview:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:283:24)

root INFO   12) TypeScript
       editor.action.peekDefinition
         within an editor:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:354:24)

root INFO   13) TypeScript
       editor.action.peekDefinition
         within an editor preview:

      AssertionError: expected 'foo' to equal 'container'
      + expected - actual

      -foo
      +container
      
      at Context.<anonymous> (/home/rnikitenko/projects/forks/theia/examples/api-tests/src/typescript.spec.js:354:24)

root ERROR Uncaught Exception:  Error: Connection got disposed.
root ERROR Error: Connection got disposed.
    at Object.dispose (/home/rnikitenko/projects/forks/theia/node_modules/vscode-jsonrpc/lib/main.js:904:25)
    at connection.onClose (/home/rnikitenko/projects/forks/theia/node_modules/vscode-ws-jsonrpc/lib/socket/connection.js:14:41)
    at CallbackList.invoke (/home/rnikitenko/projects/forks/theia/node_modules/vscode-jsonrpc/lib/events.js:62:39)
    at Emitter.fire (/home/rnikitenko/projects/forks/theia/node_modules/vscode-jsonrpc/lib/events.js:121:36)
    at closeHandler (/home/rnikitenko/projects/forks/theia/node_modules/vscode-jsonrpc/lib/main.js:240:26)
    at CallbackList.invoke (/home/rnikitenko/projects/forks/theia/node_modules/vscode-jsonrpc/lib/events.js:62:39)
    at Emitter.fire (/home/rnikitenko/projects/forks/theia/node_modules/vscode-jsonrpc/lib/events.js:121:36)
    at WebSocketMessageReader.fireClose (/home/rnikitenko/projects/forks/theia/node_modules/vscode-jsonrpc/lib/messageReader.js:111:27)
    at WebSocketMessageReader.fireClose (/home/rnikitenko/projects/forks/theia/node_modules/vscode-ws-jsonrpc/lib/socket/reader.js:67:19)
    at WebSocketMessageReader.socket.onClose (/home/rnikitenko/projects/forks/theia/node_modules/vscode-ws-jsonrpc/lib/socket/reader.js:24:18)
root ERROR Uncaught Exception:  Error: Connection got disposed.

btw: after tests execution I have modified webpack.config.js:

modified

@akosyakov
Copy link
Member Author

I suggest we merge this PR to fix travis. One has to dive in the rest of errors. It does not seem that Theia behaves the same for all operating systems and machines. I will remove fix marker from the commit message to indicate it.

@akosyakov
Copy link
Member Author

Do you know why in your case the tests use typescript 3.9.2? Do you have it globally installed?

It is one which should be installed: https://github.com/eclipse-theia/theia/blob/master/package.json#L39 built-ins should be configured to use the workspace version:

"typescript.tsdk": "node_modules/typescript/lib",

I suspect something in your workspace storage under user settings telling buil-in extension to ignore the project settings.

@vince-fugnitto
Copy link
Member

Do you know why in your case the tests use typescript 3.9.2? Do you have it globally installed?

It is one which should be installed: https://github.com/eclipse-theia/theia/blob/master/package.json#L39 built-ins should be configured to use the workspace version:

"typescript.tsdk": "node_modules/typescript/lib",

I suspect something in your workspace storage under user settings telling buil-in extension to ignore the project settings.

@akosyakov do you perhaps have the preference value set at the user level (reason for why it is picked up).
Based on the description of the preference, when the preference is set at the user level it will replace the builtin version from the extension:

Screenshot from 2020-07-13 10-27-47

I suspect it may be the reason that different users experience different results when running the suite.
When I did set the preference at the user level as well the tests were much more stable.

@marcdumais-work
Copy link
Contributor

marcdumais-work commented Jul 13, 2020

do you perhaps have the preference value set at the user level (reason for why it is picked up).

I found the typescript version can also be set globally in another way. The built-in typescript-language-features creates a memento when the user selects which typescript to use (VS Code's or Workspace's) through the Select TypeScript Version command (can be triggered from typescript version on status bar).

That memento is saved in user storage, under workspace-storage. e.g. /home/user/.theia/workspace-storage. File name: workspace-state.json (there may be more than one, for other purposes). Here's what the right one looks like:

{"vscode.typescript-language-features":{"typescript.useWorkspaceTsdk":true}}

@marcdumais-work
Copy link
Contributor

I found the typescript version can also be set globally in another way.

In fact I am not totally sure this setting is global. The memento are in subdirectories that have hash-like names, and might be the hashed workspace, for all I know. But in any case I think the setting at least sticks for the workspace it was set.

@marcdumais-work
Copy link
Contributor

@akosyakov Maybe stupid question, but is there an unstated assumption that one should explicitly select "Use Workspace Version" for TypeScript, at least once? Because once I do this, the setting sticks, but I can't seem to trigger the workspace TypeScript being used otherwise.

Or are the workspace configs supposed to be enough for this to happen automatically?

P.S.. on Gitpod too, TypeScript 3.8.3 is used for me ATM, when I launch the example application built from this PR.

@akosyakov
Copy link
Member Author

akosyakov commented Jul 14, 2020

do you perhaps have the preference value set at the user level (reason for why it is picked up).

@vince-fugnitto I wonder how does it work on Travis.

Maybe stupid question, but is there an unstated assumption that one should explicitly select "Use Workspace Version" for TypeScript, at least once? Because once I do this, the setting sticks, but I can't seem to trigger the workspace TypeScript being used otherwise.

I will need to read the code of the extension again. I am not sure that what is stored in the memento actually affecting the resolution and does not only affect how the quick pick is decorated.

@akosyakov
Copy link
Member Author

akosyakov commented Jul 14, 2020

I'm wrong tests are running from examples/browser folder, so 3.8.3 is the proper version. Deleting .theia folder from a user home helps, but it is not typescript settings in the workspace storage causing issue, something else.

@akosyakov
Copy link
Member Author

@marcdumais-work @vince-fugnitto @RomanNikitenko I've pushed another commit which ensures that a new temp directory is used for user data on each new run of yarn test. Could you try again please?

@akosyakov akosyakov force-pushed the akosyakov/some-browser-tests-do-8093 branch from 064319c to fd1bb85 Compare July 14, 2020 12:34
@marcdumais-work
Copy link
Contributor

@akosyakov One little thing I noticed: since these tests open folder examples/browser as workspace, a .theia folder is created there that may contain state. It's invisible to git status because we exclude this in root .gitignore.

@akosyakov
Copy link
Member Author

It's invisible to git status because we exclude this in root .gitignore.

Is there a reason why you ask? Is it for testing scm support?

@lmcbout
Copy link
Contributor

lmcbout commented Jul 14, 2020

I ran the test using commit fd1bb85
Tested on UBUNTU 18.04

Tested 5 times
Test Pass 4
Test Fail : 1

root INFO 901 passing (2m)
root INFO 1 failing
root INFO 1) TypeScript
editor.action.peekDefinition
within an editor:
Error: Timeout of 30000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

@akosyakov
Copy link
Member Author

@lmcbout thanks for testing, I see that this test is failing from time to time too, but could we address it with another PR. I think this PR already has enough improvements.

Copy link
Contributor

@lmcbout lmcbout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine with me, it is already a good improvement

@RomanNikitenko
Copy link
Contributor

RomanNikitenko commented Jul 14, 2020

Tested after update - sometimes tests are failing.
peekError

It's OK to me to merge the PR as is: with these changes tests are more stable on my machine.

@marcdumais-work
Copy link
Contributor

Is there a reason why you ask? Is it for testing scm support?

No, I was just thinking this state left behind by the suite is a potential source of entropy, that could make the tests behave differently the second time run in an environment and beyond.

@marcdumais-work
Copy link
Contributor

I noticed another potential source of entropy, when running the test suite locally: the test suite uses default port 3000 for its Theia backend(s). Theia developers can be expected to use that port a lot locally, and sometimes may have one or more forgotten browser tabs, patiently waiting for a Theia app to start serving that port. What if one or more of these is connected at the same time as the test's driver?

I tried to test this locally, and my "extra" frontend ends-up waiting for user input:

image

@marcdumais-work
Copy link
Contributor

I noticed another potential source of entropy, when running the test suite locally: the test suite uses default port 3000 for its Theia backend(s). Theia developers can be expected to use that port a lot locally, and sometimes may have one or more forgotten browser tabs, patiently waiting for a Theia app to start serving that port. What if one or more of these is connected at the same time as the test's driver?

I tried to test this locally, and my "extra" frontend ends-up waiting for user input:

image

BTW this is made much less bad now, that the tests do not re-use the user-storage folder (~/.theia). Before that, ~/.theia/recent-workspace.json would make is so that a newly started frontend would re-use the same workspace as the tests, increasing much the risks of conflicts. 👍

@marcdumais-work
Copy link
Contributor

I've pushed another commit which ensures that a new temp directory is used for user data on each new run of yarn test. Could you try again please?

Running locally, the tests now pass a great majority of the time, maybe ~90%. Much improved - thanks @akosyakov !

@akosyakov
Copy link
Member Author

@marcdumais-work You are right there can be other reasons for entropy, but i think we can address them as they come up.

I tried to test this locally, and my "extra" frontend ends-up waiting for user input:

It should be resolved by #7908 It handles user triggered file changes and external differently, so it won't block anymore another window.

Signed-off-by: Anton Kosyakov <anton.kosyakov@typefox.io>
@akosyakov akosyakov force-pushed the akosyakov/some-browser-tests-do-8093 branch from fd1bb85 to c585d6a Compare July 15, 2020 08:26
Copy link
Member

@vince-fugnitto vince-fugnitto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified the latest changes and the api integration tests pass consistently 👍

Copy link
Contributor

@marcdumais-work marcdumais-work left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - tests now pass locally much more frequently than before. As suggested, let's start with this and we can further improve as we go.

@marcdumais-work
Copy link
Contributor

One little thing I noticed: since these tests open folder examples/browser as workspace, a .theia folder is created there that may contain state. It's invisible to git status because we exclude this in root .gitignore.

I tried to look-for a correlation between keeping or deleting that workspace .theia folder and test failures, but I could not find any.

@akosyakov akosyakov merged commit e02ac48 into master Jul 20, 2020
@akosyakov akosyakov deleted the akosyakov/some-browser-tests-do-8093 branch July 20, 2020 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci issues related to CI / tests test issues related to unit and api tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Some browser tests do not succeed anymore
5 participants