Skip to content

Flaky Tests #2279

Closed as not planned
Closed as not planned

Description

The following tests are flaky and affecting our NetCore builds in Linux

tests

  • ResumesTransmissionSenderAfterPauseDurationWhenTransmissionGetServerError
  • ResumesTransmissionSenderAfterPauseDurationWhenTransmissionGetServerUnavailable
  • ResumesTransmissionSenderAfterPauseDurationWhenTransmissionTimesOut
  • ProactivelySampledInTelemetryCapturedWhenProactiveSamplingRateIsLowerThanTarget
  • SendingLogicMarkedAsInternalSdkOperation
  • ResumesTransmissionSenderAfterPauseDurationForBadGateway
  • WaitsUntilTelemetryBufferIsSafeToModify
  • FlushesBufferWhenNumberOfTelemetryItemsReachesMax

these tests from the Web project consistently fail on NetFramework

  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleEnvironmentReturnsCorrectDeploymentId
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleInstanceReturnAnInstanceIdThatMatchesAnAzureInstanceId
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleEnvironmentReturnsTheCurrentRoleInstanceWhichIsNotNull
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.ServiceRuntimeProducesARoleEnvironmentObject
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleInstanceReturnsARollWhichIsNotNull
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleReturnsCorrectName
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleEnvironmentReturnsCorrectAvailabilityState
  • Microsoft.ApplicationInsights.WindowsServer.Implementation.ServiceRuntimeTests.RoleEnvironmentIsAvailableReturnsFalseIfServiceRuntimeDoesntExit

test class

  • ServerTelemetryChannelE2ETests

other

#1671

[xUnit.net 00:00:05.4325976]       actual.Duration '00:00:00.0387438' is not less than timer.Elpased '00:00:00.0154093'
[xUnit.net 00:00:05.4326745]       Expected: True
[xUnit.net 00:00:05.4327267]       Actual:   False
##[error][xUnit.net 00:00:05.4309248]     WebApi.FunctionalTests.FunctionalTest.ExceptionTelemetryWebApiTests.TestBasicRequestPropertiesAfterRequestingControllerThatThrows [FAIL]
[xUnit.net 00:00:05.4337861]       Stack Trace:
[xUnit.net 00:00:05.4351258]         E:\A\_work\20\s\NETCORE\test\FunctionalTestUtils\TelemetryTestsBase.cs(64,0): at FunctionalTestUtils.TelemetryTestsBase.ValidateBasicRequest(InProcessServer server, String requestPath, RequestTelemetry expected)
[xUnit.net 00:00:05.4354944]         E:\A\_work\20\s\NETCORE\test\WebApi.FunctionalTests\FunctionalTest\ExceptionTelemetryWebApiTests.cs(33,0): at WebApi.FunctionalTests.FunctionalTest.ExceptionTelemetryWebApiTests.TestBasicRequestPropertiesAfterRequestingControllerThatThrows()
[xUnit.net 00:00:05.7104819]   Finished:    WebApi.FunctionalTests.Tests

#1037

2018-12-28T22:39:32.2777167Z Starting test execution, please wait...
2018-12-28T22:40:23.4984741Z Failed   Microsoft.ApplicationInsights.WindowsServer.Channel.AdaptiveSamplingTelemetryProcessorTest.SamplingPercentageAdjustsAccordingToConstantHighProductionRate
2018-12-28T22:40:23.4985603Z Error Message:
2018-12-28T22:40:23.4985771Z  Assert.IsTrue failed. 
2018-12-28T22:40:23.4987663Z Stack Trace:
2018-12-28T22:40:23.4988582Z    at Microsoft.ApplicationInsights.WindowsServer.Channel.AdaptiveSamplingTelemetryProcessorTest.SamplingPercentageAdjustsAccordingToConstantHighProductionRate() in /mnt/a/1/s/Test/ServerTelemetryChannel.Test/Shared.Tests/AdaptiveSamplingTelemetryProcessorTest.cs:line 128
2018-12-28T22:40:23.4989225Z 
2018-12-28T22:40:23.4990530Z Standard Output Messages:
2018-12-28T22:40:23.4990780Z  
2018-12-28T22:40:23.4990970Z  
2018-12-28T22:40:23.4991151Z  Debug Trace:
2018-12-28T22:40:23.4992222Z  [Sampling% evaluation] 2018-12-28T22:39:59.4728938+00:00, Eps: 20, Current %: 100, New %: 25, Changed: False
2018-12-28T22:40:23.4992867Z  [Sampling% evaluation] 2018-12-28T22:40:00.4723852+00:00, Eps: 20.5, Current %: 100, New %: 20, Changed: False
2018-12-28T22:40:23.4993395Z  [Sampling% evaluation] 2018-12-28T22:40:01.4727203+00:00, Eps: 20.375, Current %: 100, New %: 20, Changed: True
2018-12-28T22:40:23.4993927Z  [Sampling% evaluation] 2018-12-28T22:40:02.4739779+00:00, Eps: 4, Current %: 20, New %: 25, Changed: False
2018-12-28T22:40:23.4994406Z  [Sampling% evaluation] 2018-12-28T22:40:03.4702612+00:00, Eps: 3.75, Current %: 20, New %: 25, Changed: False
2018-12-28T22:40:23.4994919Z  [Sampling% evaluation] 2018-12-28T22:40:04.4701655+00:00, Eps: 4.3125, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4995400Z  [Sampling% evaluation] 2018-12-28T22:40:05.4708253+00:00, Eps: 4.734375, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4995927Z  [Sampling% evaluation] 2018-12-28T22:40:06.4749983+00:00, Eps: 4.55078125, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4996416Z  [Sampling% evaluation] 2018-12-28T22:40:07.4757375+00:00, Eps: 4.9130859375, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4996954Z  [Sampling% evaluation] 2018-12-28T22:40:08.4715758+00:00, Eps: 4.434814453125, Current %: 20, New %: 20, Changed: False
2018-12-28T22:40:23.4997771Z  [Sampling% evaluation] 2018-12-28T22:40:09.4718122+00:00, Eps: 3.82611083984375, Current %: 20, New %: 25, Changed: True
2018-12-28T22:40:23.4998604Z  [Sampling% evaluation] 2018-12-28T22:40:10.4719975+00:00, Eps: 1, Current %: 25, New %: 100, Changed: False
2018-12-28T22:40:23.4999198Z  [Sampling% evaluation] 2018-12-28T22:40:11.4754562+00:00, Eps: 1.25, Current %: 25, New %: 100, Changed: True
2018-12-28T22:40:23.4999748Z  [Sampling% evaluation] 2018-12-28T22:40:12.4760677+00:00, Eps: 20, Current %: 100, New %: 25, Changed: False
2018-12-28T22:40:23.5000204Z  [Sampling% evaluation] 2018-12-28T22:40:13.4687556+00:00, Eps: 20, Current %: 100, New %: 25, Changed: False
2018-12-28T22:40:23.5000736Z  [Sampling% evaluation] 2018-12-28T22:40:14.4733533+00:00, Eps: 20, Current %: 100, New %: 25, Changed: True
2018-12-28T22:40:23.5001716Z  [Sampling% evaluation] 2018-12-28T22:40:15.4698518+00:00, Eps: 7, Current %: 25, New %: 16.6666666666667, Changed: False
2018-12-28T22:40:23.5002287Z  [Sampling% evaluation] 2018-12-28T22:40:16.4694474+00:00, Eps: 6, Current %: 25, New %: 20, Changed: False
2018-12-28T22:40:23.5002762Z  [Sampling% evaluation] 2018-12-28T22:40:17.4740224+00:00, Eps: 7, Current %: 25, New %: 16.6666666666667, Changed: True
2018-12-28T22:40:23.5003310Z  [Sampling% evaluation] 2018-12-28T22:40:18.4702060+00:00, Eps: 6, Current %: 16.6666666666667, New %: 12.5, Changed: False
2018-12-28T22:40:23.5003797Z  [Sampling% evaluation] 2018-12-28T22:40:19.4708565+00:00, Eps: 5, Current %: 16.6666666666667, New %: 16.6666666666667, Changed: False
2018-12-28T22:40:23.5004364Z  [Sampling% evaluation] 2018-12-28T22:40:20.4709985+00:00, Eps: 4.5, Current %: 16.6666666666667, New %: 16.6666666666667, Changed: False
2018-12-28T22:40:23.5004852Z  [Sampling% evaluation] 2018-12-28T22:40:21.4702893+00:00, Eps: 4.375, Current %: 16.6666666666667, New %: 16.6666666666667, Changed: False
2018-12-28T22:40:23.5005423Z  [Sampling% evaluation] 2018-12-28T22:40:22.4703054+00:00, Eps: 4.03125, Current %: 16.6666666666667, New %: 20, Changed: True
2018-12-28T22:40:23.5005872Z  [Sampling% evaluation] 2018-12-28T22:40:23.4702887+00:00, Eps: 4, Current %: 20, New %: 25, Changed: False
2018-12-28T22:40:23.5006317Z  'Ideal' telemetry item count: 125
2018-12-28T22:40:23.5006481Z  Expected range: from 63 to 187
2018-12-28T22:40:23.5006673Z  Actual telemetry item count: 201 (160.8% of ideal)
2018-12-28T22:40:23.5006781Z 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions