Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run tests in parallel #234

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open

Run tests in parallel #234

wants to merge 21 commits into from

Commits on Nov 22, 2023

  1. Add data structure foundation.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    32e8eed View commit details
    Browse the repository at this point in the history

  2. Work from pairing session with Arne.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    e0f0300 View commit details
    Browse the repository at this point in the history

  3. Add semi-functional version (with :parallel) as the flag.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    f128717 View commit details
    Browse the repository at this point in the history

  4. Add some debugging code for now.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    3382198 View commit details
    Browse the repository at this point in the history

  5. Add workarounds for now.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    6a35e46 View commit details
    Browse the repository at this point in the history

  6. Replace workarounds by locking require.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    a5d2f7f View commit details
    Browse the repository at this point in the history

  7. Allow parallelization to happen at any level.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    20e2812 View commit details
    Browse the repository at this point in the history

  8. Clean up.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    cb09129 View commit details
    Browse the repository at this point in the history

  9. Clean up.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    503bfe7 View commit details
    Browse the repository at this point in the history

  10. Add benchmarking.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    544a3d3 View commit details
    Browse the repository at this point in the history

  11. Add warning.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    141683e View commit details
    Browse the repository at this point in the history

  12. Add commandline flag.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    cc983c1 View commit details
    Browse the repository at this point in the history

  13. Clean up.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    e6382b8 View commit details
    Browse the repository at this point in the history

  14. Add docs and CHANGELOG entry.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    8e07ed6 View commit details
    Browse the repository at this point in the history

  15. Fix dash.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    fd5bfe9 View commit details
    Browse the repository at this point in the history

  16. Tweak wording further.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    dc52eb1 View commit details
    Browse the repository at this point in the history

  17. Add option to filter out by test-suite.

    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    0136e65 View commit details
    Browse the repository at this point in the history

  18. Add thread info. 

    Add some information about the thread each test ran on. May want to move
this to something not specific to parallelization, otherwise
non-parallel tests won't have any thread info.
    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    78f2312 View commit details
    Browse the repository at this point in the history

  19. Very WIP plugin for profiling parallel execution. 

    Internal refers to whether there were delays between running tests in
the same thread. It doesn't measure waiting for I/O or many kinds of
contention.

External refers to how much of the overall time the thread spent.

Why do I think this is useful? Well, if you've parallelized your test
suite and you end up with a thread that runs from time 0 to time 100 and
another that runs from time 1 to time 10, the burden is heavily on the
first thread, suggsting the load is not well-balanced. Both may have
good internal utilization, but the second has poor external
globalization.

Part of the reason this is WIP is that there may be better (ideally
standard) terms for "internal" and "external".

There are some calculation issues and hacks:
* Some testables are missing the ::start and ::end for some reason. I
  don't think this should happen?
* I don't think this version counts the number of threads accurately.
    @alysbrooks
    alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    356d22c View commit details
    Browse the repository at this point in the history

  20. Post-rebase fixups

    @plexus @alysbrooks
    plexus authored and alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    fbfd541 View commit details
    Browse the repository at this point in the history

  21. Polish up parallelization: clearer config, docs, vestigial code 

    Clean up the changes around parallelization, make opting-in from test types
explicit, allow config on testable and metadata level, simplify future handling.
    @plexus @alysbrooks
    plexus authored and alysbrooks committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    615ea82 View commit details
    Browse the repository at this point in the history