Toolset cleanup & remove TE database dependencies #3574

NateBrady23 · 2018-04-15T19:23:29Z

We no longer have any tests that use local dependencies. Everything pulls from a dockerhub dependency.

Remove __build_dependencies
Remove __gather_dependencies
Remove build arguments (TFB_DATABASE, MAX_CONCURRENCY)
Move wrk and database up and remove toolset/setup/docker and toolset/setup
Remove the techempower database dependencies and build the database images locally. Right now this will only minimally affect travis runs especially since the 3 database images are pulling from ubuntu:16.04 instead of the official database images (some of the official images had some quirks I haven't worked out yet.) For continuous benchmarking, this will have virtually no time impact.
- We'll be able to see some of the time differences between the toolset cleanup commit and the build database dockerfiles locally commit
Yes I hoorayed my own PR
Grouped more tests together with TESTLANG from Group more languages for travis #3575 bringing down the total time impact of building the database tests locally

Edit: JavaScript, which is using all 3 databases, went from 13:52 to 15:10. And this is with using the ubuntu:16.04 base and doing apt updates. I'd say this is a win, especially when we can pull from the official bases for the database.

NateBrady23 · 2018-04-16T02:11:11Z

@msmith-techempower FYI, aspnet-mono has been failing but it's not being reported by travis as failed.

…hmarks into toolset

msmith-techempower

Looking good.

msmith-techempower · 2018-04-16T16:15:46Z

toolset/utils/docker_helper.py

    for the given tests.
    '''
-    docker_buildargs = {
-        'MAX_CONCURRENCY': str(max(benchmarker_config.concurrency_levels)),


Is this no longer needed because we're just leveraging nproc now?

msmith-techempower · 2018-04-16T16:16:13Z

toolset/utils/docker_helper.py

                            tag="techempower/tfb.test.%s" %
                            test_docker_file.replace(".dockerfile", ""),
-                            buildargs=docker_buildargs,
                            forcerm=True):


Do we want forcerm anymore?

msmith-techempower · 2018-04-16T16:19:02Z

FYI, aspnet-mono has been failing but it's not being reported by travis as failed.

@michaelhixson and I broke Travis on Friday by decoupling failure state of the toolset from failure state of any test.

This should probably be fixed by switching on the running mode as to whether that holds - verify should return failure for the suite if any test fails in any way, but benchmark should only fail if there is an error thrown (the toolset should return non-zero only if it could not successfully run; if it benchmarked everything and some tests failed, it should return 0).

msmith-techempower · 2018-04-16T16:47:15Z

toolset/run-tests.py

-        benchmarker.run()
+        all_passed = benchmarker.run()
+        if config.mode == "verify":
+            return 0 if all_passed else 1


Why not just return all_passed?

msmith-techempower · 2018-04-16T16:48:12Z

toolset/benchmark/benchmarker.py

        # Get a list of all known  tests that we can run.
        all_tests = gather_remaining_tests(self.config, self.results)

+        all_passed = True


Ugh, change this to something like any_failed and reverse the binary logic so that you can just bubble the return value up. That line of return 0 if all_passed else 1 is so gross.

…ing [ci lang-only Scala]

msmith-techempower · 2018-04-16T21:53:05Z

finch: [info] Packaging /finch/target/scala-2.12/techempower-benchmarks-finch-assembly-0.19.0.jar ...
finch: [info] Done packaging.
...
finch: Error: Unable to access jarfile target/scala-2.12/techempower-benchmarks-finch-assembly-1.0.jar

These are all different for each test, too >_<

michaelhixson · 2018-04-16T22:48:27Z

Regarding the scala travis failures: I think the CMD lines of those dockerfiles need to be updated to align with the version number changes in this commit 2bb5629

NateBrady23 · 2018-04-16T23:32:47Z

I'll open a separate PR for the scala tests

NateBrady23 added 2 commits April 15, 2018 12:15

Toolset cleanup

3f884f5

build database dockerfiles locally

8bc0e63

NateBrady23 changed the title ~~Toolset cleanup~~ Toolset cleanup & Build remove TE database dependencies Apr 15, 2018

NateBrady23 changed the title ~~Toolset cleanup & Build remove TE database dependencies~~ Toolset cleanup & remove TE database dependencies Apr 15, 2018

Merge branch 'master' of https://github.com/techempower/frameworkbenc…

a24977a

…hmarks into toolset

msmith-techempower approved these changes Apr 16, 2018

View reviewed changes

fix return code during mode verify

da8bf7d

msmith-techempower requested changes Apr 16, 2018

View reviewed changes

NateBrady23 added 3 commits April 16, 2018 09:51

fix return code during mode verify

fac3eac

stop being dumb

6f6f564

fixed race on checking running containers

bf2e790

msmith-techempower approved these changes Apr 16, 2018

View reviewed changes

fix possible container referenced before assignment and global shadow…

23e7214

…ing [ci lang-only Scala]

NateBrady23 merged commit a4a52cf into TechEmpower:master Apr 16, 2018

NateBrady23 deleted the toolset branch April 18, 2018 19:22

msmith-techempower mentioned this pull request Apr 23, 2018

[ci skip] Avoid waiting on a container that died #3631

Merged

joshka mentioned this pull request Mar 23, 2025

Don't remove docker layers (set forcerm = false) #9718

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Toolset cleanup & remove TE database dependencies #3574

Toolset cleanup & remove TE database dependencies #3574

Uh oh!

NateBrady23 commented Apr 15, 2018 •

edited

Loading

Uh oh!

NateBrady23 commented Apr 16, 2018 •

edited

Loading

Uh oh!

msmith-techempower left a comment

Uh oh!

msmith-techempower Apr 16, 2018

Uh oh!

msmith-techempower Apr 16, 2018

Uh oh!

msmith-techempower commented Apr 16, 2018

Uh oh!

msmith-techempower Apr 16, 2018

Uh oh!

msmith-techempower Apr 16, 2018

Uh oh!

msmith-techempower commented Apr 16, 2018 •

edited

Loading

Uh oh!

michaelhixson commented Apr 16, 2018

Uh oh!

NateBrady23 commented Apr 16, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Toolset cleanup & remove TE database dependencies #3574

Toolset cleanup & remove TE database dependencies #3574

Uh oh!

Conversation

NateBrady23 commented Apr 15, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NateBrady23 commented Apr 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

msmith-techempower left a comment

Choose a reason for hiding this comment

Uh oh!

msmith-techempower Apr 16, 2018

Choose a reason for hiding this comment

Uh oh!

msmith-techempower Apr 16, 2018

Choose a reason for hiding this comment

Uh oh!

msmith-techempower commented Apr 16, 2018

Uh oh!

msmith-techempower Apr 16, 2018

Choose a reason for hiding this comment

Uh oh!

msmith-techempower Apr 16, 2018

Choose a reason for hiding this comment

Uh oh!

msmith-techempower commented Apr 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michaelhixson commented Apr 16, 2018

Uh oh!

NateBrady23 commented Apr 16, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

NateBrady23 commented Apr 15, 2018 •

edited

Loading

NateBrady23 commented Apr 16, 2018 •

edited

Loading

msmith-techempower commented Apr 16, 2018 •

edited

Loading