Flaky test: RunsWithIterationEnvVariable

[ryanbrandenburg]

This test fails occasionally with the following error:

System.TimeoutException : The operation at /_/test/dotnet-watch.FunctionalTests/AwaitableProcess.cs:83 timed out after reaching the limit of 30000ms.
   at Microsoft.AspNetCore.Testing.TaskExtensions.TimeoutAfter[T](Task`1 task, TimeSpan timeout, String filePath, Int32 lineNumber)
   at Microsoft.DotNet.Watcher.Tools.FunctionalTests.AwaitableProcess.GetOutputLineAsync(String message, TimeSpan timeout) in /_/test/dotnet-watch.FunctionalTests/AwaitableProcess.cs:line 83
   at Microsoft.DotNet.Watcher.Tools.FunctionalTests.DotNetWatcherTests.RunsWithIterationEnvVariable() in /_/test/dotnet-watch.FunctionalTests/DotNetWatcherTests.cs:line 42
--- End of stack trace from previous location where exception was thrown ---
------- Stdout: -------
The temporary test folder is /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde
Copying project /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/TestProjects/KitchenSink to /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink
Restoring msbuild project in KitchenSink
Project directory: '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink'
Started process 44313: /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/.dotnet/dotnet restore
  Restoring packages for /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/KitchenSink.csproj...
  Generating MSBuild file /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/.net/obj/KitchenSink.csproj.nuget.g.props.
  Generating MSBuild file /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/.net/obj/KitchenSink.csproj.nuget.g.targets.
  Restore completed in 446.21 ms for /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/KitchenSink.csproj.
Process exited 44313
Process exited 44313 with code 0
Building KitchenSink
Project directory: '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink'
Started process 44343: /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/.dotnet/dotnet build
Microsoft (R) Build Engine version 15.7.179.6572 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
  Restore completed in 76.01 ms for /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/KitchenSink.csproj.
  KitchenSink -> /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/.net/bin/Release/netcoreapp2.2/KitchenSink.dll
Build succeeded.
    0 Warning(s)
    0 Error(s)
Time Elapsed 00:00:03.35
Process exited 44343
Process exited 44343 with code 0
7/30/2018 3:57:52 PM: process start: '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/.dotnet/dotnet /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tool/dotnet-watch.dll run --'
Waiting for output line [msg == 'Started']. Will wait for 120 sec.
7/30/2018 3:57:52 PM: post: 'watch : Running MSBuild target 'GenerateWatchList' on '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/KitchenSink.csproj''
7/30/2018 3:57:52 PM: recv: 'watch : Running MSBuild target 'GenerateWatchList' on '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/KitchenSink.csproj''
7/30/2018 3:57:52 PM: post: 'watch : Started '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/.dotnet/dotnet' with process id 44377'
7/30/2018 3:57:52 PM: recv: 'watch : Started '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/.dotnet/dotnet' with process id 44377'
7/30/2018 3:57:54 PM: post: 'watch : Process id 44377 ran for 1760ms'
7/30/2018 3:57:54 PM: recv: 'watch : Process id 44377 ran for 1760ms'
7/30/2018 3:57:54 PM: post: 'watch : Watching 2 file(s) for changes'
7/30/2018 3:57:54 PM: recv: 'watch : Watching 2 file(s) for changes'
7/30/2018 3:57:54 PM: post: 'watch : Started '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/.dotnet/dotnet' with process id 44398'
7/30/2018 3:57:54 PM: recv: 'watch : Started '/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/.dotnet/dotnet' with process id 44398'
7/30/2018 3:57:54 PM: post: 'watch : Running dotnet with the following arguments: run --'
7/30/2018 3:57:54 PM: recv: 'watch : Running dotnet with the following arguments: run --'
7/30/2018 3:57:54 PM: post: 'watch : Started'
7/30/2018 3:57:54 PM: recv: 'watch : Started'
7/30/2018 3:57:54 PM: post: 'Telemetry is: Disabled'
7/30/2018 3:57:54 PM: recv: 'Telemetry is: Disabled'
7/30/2018 3:57:58 PM: post: 'Running dotnet exec "/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/.net/bin/Release/netcoreapp2.2/KitchenSink.dll"'
7/30/2018 3:57:58 PM: recv: 'Running dotnet exec "/Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/DotNetTools/bin/dotnet-watch.FunctionalTests/Release/netcoreapp2.2/tmp/ie5ss3zj.qde/KitchenSink/.net/bin/Release/netcoreapp2.2/KitchenSink.dll"'
7/30/2018 3:57:58 PM: post: 'Process ID: 44434'
7/30/2018 3:57:58 PM: recv: 'Process 
...

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.

CC @Eilon,@muratg,@mkArtakMSFT

This issue was made automatically. If there is a problem contact @ryanbrandenburg.

[ryanbrandenburg]

Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

• Is this actually a flaky test?

  • No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
  • Yes, proceed below...

• Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)

  • File a bug against the product (TODO: Link to other bug)
  • Is it possible to change the test to avoid the flakiness?
    • Yes? Go to the "Change the test!" section.
    • No?
      • Disable the test (TODO: Link to PR/commit)
      • Wait for other bug to be resolved
      • Wait for us to get build that has the fix
      • Re-enable our test (TODO: Link to PR/commit)
      • Close this bug

• Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)

  • Is there is a way to change our test to avoid this flakiness?
    • Yes? Change the test!
      • Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
      • Run the test 100 times locally as a sanity check.
      • Close this bug
    • No?
      • Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

[ryanbrandenburg]

RunsWithIterationEnvVariable failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

[ryanbrandenburg]

RunsWithIterationEnvVariable failed again.

This comment was made automatically. If there is a problem contact @ryanbrandenburg.

[natemcmaster]

Duplicate of #467

This should have been resolved already in #469. Those changes have not yet made it into a Universe build.