Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add dynamic scheduling operation mode #271

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Commits on Nov 24, 2019

  1. Add dynamic scheduling operation mode

    This commit adds an opt-in scheduler option for dynamic scheduling.
    Instead of partitioning the test list up-front based on historical
    timing data this commit lets each worker ask for the next test
    dynamically.  This is built using python's multiprocess module to
    launch new workers instead of shelling out to call python via
    subprocess.
    
    This hopefully will provide a better worker balance since we will keep
    each worker occupied until there are no more tests to be run. Instead
    of trying to pack fill each work optimially up front. Additionally this
    should hopefully improve the pdb story for users who use pdb with tests.
    Since instead of spawning subprocesses calling python to invoke the
    subunit runner and reading the subunit stream from stdout and instead
    uses multiprocessing to fork workers and uses pipes to pass the subunit
    streams between workers.
    mtreinish committed Nov 24, 2019
    Configuration menu
    Copy the full SHA
    068c507 View commit details
    Browse the repository at this point in the history
  2. Update stestr/commands/run.py

    Co-Authored-By: Adam Spiers <github@adamspiers.org>
    mtreinish and aspiers committed Nov 24, 2019
    Configuration menu
    Copy the full SHA
    59189e6 View commit details
    Browse the repository at this point in the history
  3. Fix failing tests

    This commit fixes the failing tests by catching a couple of missing
    things from the update. The biggest fix was that for the --no-discover
    case we still use a subprocess and because of that we need to tell
    output.ReturnCodeToSubunit to that the input is not dynamic (and
    therefore a Popen object) so it can handle that properly. The other
    major change is that the return code tests are updated so that the
    stdout and stderr from the subprocess calls are always decoded in the
    non-subunit test cases. This was done primarily for ease of debugging,
    but it also enabled the removal of several decode() calls when the
    output is parsed.
    mtreinish committed Nov 24, 2019
    Configuration menu
    Copy the full SHA
    4196953 View commit details
    Browse the repository at this point in the history
  4. Clean up test_return_code tests slightly

    This is a refinement on the previous commit to reduce unecessary changes
    to the functional tests in the test_return_codes module. Mainly always
    decoding the output from the subprocess for testing broken things
    unexpectedly when a bytes object was expected.
    mtreinish committed Nov 24, 2019
    Configuration menu
    Copy the full SHA
    e68b506 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2020

  1. Merge remote-tracking branch 'origin/master' into dynamic-schedule

    Conflicts:
    	stestr/commands/run.py
    	stestr/output.py
    	stestr/test_processor.py
    	stestr/tests/test_return_codes.py
    mtreinish committed Aug 9, 2020
    Configuration menu
    Copy the full SHA
    e47ad6e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e8989ca View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2020

  1. Configuration menu
    Copy the full SHA
    64d8a0b View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2021

  1. Configuration menu
    Copy the full SHA
    a474f3f View commit details
    Browse the repository at this point in the history
  2. Fix pep8

    mtreinish committed Apr 15, 2021
    Configuration menu
    Copy the full SHA
    4fcb27a View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2021

  1. Configuration menu
    Copy the full SHA
    1fa2454 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2023

  1. Configuration menu
    Copy the full SHA
    6055ca8 View commit details
    Browse the repository at this point in the history
  2. Remove check for Python < 3.5

    I originally developed this feature when we still supported older
    python versions in stestr. The dynamic scheduling feature depends on
    functionality added in Python 3.5. Since then the WIP feature branch sat
    stale for years since that time we've bumped the minimum version of
    Python supported to 3.7 so the runtime check for older python versions
    is no longer needed.
    mtreinish committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    e23f8ed View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2024

  1. Configuration menu
    Copy the full SHA
    f7900e7 View commit details
    Browse the repository at this point in the history