Skip to content

Introduced a basic logger to the integration tests framework #2293

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Mar 19, 2020
Merged

Introduced a basic logger to the integration tests framework #2293

merged 3 commits into from
Mar 19, 2020

Conversation

pracucci
Copy link
Contributor

What this PR does:
We have some flaky integration tests in query_frontend_test.go. I can't understand the root cause but I've the feeling it's a timing issue. In this PR I'm adding a very simple logger to prefix each log with the time, so that we can have a better idea where the time is being spent when running in CI.

An example of the tests output is:

=== RUN   TestQueryFrontendWithChunksStorageViaConfigFile
10:00:47 Starting consul
10:00:48 consul: ==> Starting Consul agent...
10:00:48 consul: ==> Consul agent running!
10:00:48 consul: Version: 'v0.9.4'
10:00:48 consul: Node ID: 'b67d8cc8-029c-d764-dc65-a7205203b3cd'
10:00:48 consul: Node name: 'consul'

Which issue(s) this PR fixes:
N/A

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@pracucci
Introduced a basic logger to the integration tests framework
9832720 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci pracucci force-pushed the add-logger-to-integration-tests branch from b86b16b to 9832720 Compare March 19, 2020 09:06
@pracucci
Fixed linter
c608924 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Increased integration tests timeout in CI
329b4a9 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Copy link
Contributor Author

And thanks to the timing displayed in logs I've also found and fixed the root cause, which is trivial. We currently have a 300s timeout for the integration tests, but due to the new tests we're adding, the whole suite sometimes takes more than 5 minutes to run in CI and thus it breaks due to the max execution time reached.

I've increased the timeout to 10 minutes (commit). In the future we may experiment with parallel integration tests execution.

@pracucci pracucci merged commit 305b16c into cortexproject:master Mar 19, 2020
@pracucci pracucci deleted the add-logger-to-integration-tests branch March 19, 2020 09:46
pracucci added a commit to grafana/cortex that referenced this pull request Mar 19, 2020
@pracucci
Squashed commit of the following:
5ee1e6e 
commit 8e9c601
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 10:44:34 2020 +0100

    Avoid a superfluous querier integration test

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit 9691dc6
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 09:26:30 2020 +0100

    Separated metrics helpers unit tests using go blocks

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit d32afe8
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 09:23:34 2020 +0100

    Reworded util.StringsContain() comment and params

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit a5000a8
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 09:21:54 2020 +0100

    Update docs/operations/dns-service-discovery.md

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

    Co-Authored-By: Peter Štibraný <pstibrany@gmail.com>

commit 1babd96
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 09:21:37 2020 +0100

    Update integration/e2e/service.go

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

    Co-Authored-By: Peter Štibraný <pstibrany@gmail.com>

commit c1ed809
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 09:01:57 2020 +0100

    Added a timeout to the 'docker inspect' command to avoid the command being stuck

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit 271fb8d
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 08:52:09 2020 +0100

    Updated doc

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit a922d32
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 08:43:20 2020 +0100

    Fixed unit tests

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit 50398ab
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 08:40:40 2020 +0100

    Fixed doc

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit f077389
Author: Marco Pracucci <marco@pracucci.com>
Date:   Wed Mar 18 18:57:52 2020 +0100

    Documented blocks storage index cache backends

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit 330e449
Author: Marco Pracucci <marco@pracucci.com>
Date:   Wed Mar 18 18:14:12 2020 +0100

    Updated CHANGELOG

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit 0233519
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 12 17:40:21 2020 +0100

    Added memcached support to the blocks storage index cache

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit 305b16c
Author: Marco Pracucci <marco@pracucci.com>
Date:   Thu Mar 19 10:46:43 2020 +0100

    Introduced a basic logger to the integration tests framework (cortexproject#2293)

    * Introduced a basic logger to the integration tests framework

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

    * Fixed linter

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

    * Increased integration tests timeout in CI

    Signed-off-by: Marco Pracucci <marco@pracucci.com>

commit 51f7047
Author: Jay Batra <jaybatra73@gmail.com>
Date:   Thu Mar 19 13:19:39 2020 +0530

    Removes all the occurences of omitempty (cortexproject#2209)

    * Removes all the occurences of omitempty(cortexproject#2209)

    This PR removes all the occurences of omitempty
    from structs.

    Signed-off by Jay Batra <jaybatra73@gmail.com>

    Signed-off-by: Jay Batra <jaybatra73@gmail.com>

    * Update CHANGELOG.md

    Co-authored-by: Marco Pracucci <marco@pracucci.com>

commit bf59f9f
Author: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
Date:   Wed Mar 18 17:00:11 2020 +0530

    support for cassandra in integration tests (cortexproject#2275)

    * support for cassandra in integration tests

    Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

    * changes suggested from PR review

    Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

    * using ConcreteService.Endpoint for getting http endpoint for readiness probe

    Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

commit 87fa252
Author: Rajat Vig <rajatvig@users.noreply.github.com>
Date:   Wed Mar 18 08:49:45 2020 +0000

    Fix the import for test for bucket_client (cortexproject#2285)

    Signed-off-by: Rajat Vig <rvig@etsy.com>
simonswine pushed a commit to grafana/e2e that referenced this pull request Jan 13, 2022
@pracucci
Introduced a basic logger to the integration tests framework (cortexp…
038a525 
…roject/cortex#2293)

* Introduced a basic logger to the integration tests framework

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Fixed linter

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Increased integration tests timeout in CI

Signed-off-by: Marco Pracucci <marco@pracucci.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pstibrany pstibrany pstibrany approved these changes

@gouthamve gouthamve gouthamve approved these changes

Assignees
No one assigned
Labels
size/M
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pracucci @pstibrany @gouthamve