bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

pablogsal · 2018-06-15T22:59:37Z

https://bugs.python.org/issue33873

… an extra time than specified

pablogsal · 2018-06-15T23:43:52Z

I have added the checks for the --huntrleaks/-R in Regrtest._main to avoid entering Regrtest.run_test and do the sys.exit() there but I am not sure what is the best location. Also, I used a new errorcode (2) but again, not sure if we want a custom error code for that.

pablogsal · 2018-06-15T23:45:54Z

~/github/cpython bpo33873 ✔ 8s
❯ ./python -m test -R 0:0 test_os -m test_access
Invalid values for the --huntrleaks/-R parameters
                                                                                                                                               
~/github/cpython bpo33873 ✔
❯ ./python -m test -R 0:1 test_os -m test_access
Invalid values for the --huntrleaks/-R parameters
                                                                                                                                               
~/github/cpython bpo33873 ✔
❯ ./python -m test -R 1:0 test_os -m test_access
Invalid values for the --huntrleaks/-R parameters
                                                                                                                                               
~/github/cpython bpo33873 ✔
❯  ./python -m test -R 0:-2 test_list
Invalid values for the --huntrleaks/-R parameters

vstinner · 2018-06-16T00:22:45Z

Lib/test/libregrtest/main.py

+        if self.ns.huntrleaks:
+            warmup, repetitions, _ = self.ns.huntrleaks
+            if warmup < 1 or repetitions < 1:
+                print("Invalid values for the --huntrleaks/-R parameters",


You must mention the minimum number of warmups and minimum number of repetitions (1 and 1).

vstinner

IMHO a NEWS entry would be worth it here. You can use the name "regrtest" to name the CPython test runner libregrtest.

vstinner · 2018-06-20T10:37:23Z

Lib/test/libregrtest/main.py

+            if warmup < 1 or repetitions < 1:
+                msg = ("Invalid values for the --huntrleaks/-R parameters. The "
+                       "number of warmups and repetitions must be at least 1 "
+                       "each (1:1).")


Nitpick: maybe write "(-R 1:1)" rather than "(1:1)"?

vstinner

LGTM.

Since you are now a core dev, would you like to merge it yourself? :-)

vstinner · 2018-06-20T10:39:13Z

"VSTS: docs — docs_20180616.05 failed"

I don't understand the failure. It says "Initialize phase: Failed" but I don't see any failure in logs. I suggest to ignore this error.

vstinner · 2018-06-26T11:47:35Z

@pablogsal: The change LGTM, do you want to merge it? It would help me to work on my PR #7827.

pablogsal · 2018-06-26T14:17:51Z

Merged! ✨

miss-islington · 2018-06-26T21:05:43Z

Thanks @pablogsal for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

miss-islington · 2018-06-26T21:05:43Z

Thanks @pablogsal for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6.
🐍🍒⛏🤖

bedevere-bot · 2018-06-26T21:07:03Z

GH-7933 is a backport of this pull request to the 3.7 branch.

…rameters (pythonGH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

bedevere-bot · 2018-06-26T21:07:54Z

GH-7934 is a backport of this pull request to the 3.6 branch.

…rameters (pythonGH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

* bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686) * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration() (cherry picked from commit 4ffe9c2) * bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736) regrtest: Add warning when using less than 3 warmup runs like -R 1:3. (cherry picked from commit cac4fef) * bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730)

* bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686) * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration() (cherry picked from commit 4ffe9c2) * bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736) regrtest: Add warning when using less than 3 warmup runs like -R 1:3. (cherry picked from commit cac4fef) * bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730) (cherry picked from commit d1f9481)

the-knights-who-say-ni added the CLA signed label Jun 15, 2018

bedevere-bot added the awaiting review label Jun 15, 2018

Fix bug in Lib/test/libregrtest/runtest.py that makes running tests…

0f0ea7f

… an extra time than specified

pablogsal force-pushed the bpo33873 branch from c56c94a to 0f0ea7f Compare June 15, 2018 23:08

Add check for invalid --huntrleaks/-R parameters

bc1c874

pablogsal force-pushed the bpo33873 branch from aee5345 to bc1c874 Compare June 15, 2018 23:44

pablogsal changed the title ~~bpo-33873: Fix bug in runtest.py that makes running tests an extra time~~ bpo-33873: Fix bug in runtest.py and add checks for invalid -R parameters Jun 15, 2018

vstinner reviewed Jun 16, 2018

View reviewed changes

pablogsal added 2 commits June 16, 2018 01:32

fixup! Add check for invalid --huntrleaks/-R parameters

7510359

Add News entry

7a5ffc3

pablogsal force-pushed the bpo33873 branch from 8a0b6c0 to 7a5ffc3 Compare June 16, 2018 00:39

vstinner reviewed Jun 20, 2018

View reviewed changes

vstinner approved these changes Jun 20, 2018

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting review labels Jun 20, 2018

pablogsal merged commit 58ed730 into python:master Jun 26, 2018

bedevere-bot removed the awaiting merge label Jun 26, 2018

pablogsal deleted the bpo33873 branch June 26, 2018 14:17

vstinner added needs backport to 3.6 labels Jun 26, 2018

bedevere-bot removed the needs backport to 3.7 label Jun 26, 2018

bedevere-bot removed the needs backport to 3.6 label Jun 26, 2018

This was referenced Jun 26, 2018

bpo-33873: Backport regrtest from master to 3.7 #7935

Merged

[2.7] bpo-33873: Backport regrtest from master #7936

Merged

vstinner mentioned this pull request Jun 26, 2018

[3.6] bpo-33873: Backport regrtest from master to 3.6 (GH-7935) #7937

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

Uh oh!

pablogsal commented Jun 15, 2018 •

edited by bedevere-bot

Loading

Uh oh!

pablogsal commented Jun 15, 2018

Uh oh!

pablogsal commented Jun 15, 2018

Uh oh!

vstinner Jun 16, 2018

Uh oh!

vstinner left a comment

Uh oh!

vstinner Jun 20, 2018

Uh oh!

vstinner left a comment

Uh oh!

vstinner commented Jun 20, 2018

Uh oh!

vstinner commented Jun 26, 2018

Uh oh!

pablogsal commented Jun 26, 2018

Uh oh!

miss-islington commented Jun 26, 2018

Uh oh!

miss-islington commented Jun 26, 2018

Uh oh!

bedevere-bot commented Jun 26, 2018

Uh oh!

bedevere-bot commented Jun 26, 2018

Uh oh!

Uh oh!

Uh oh!

bpo-33873: Fix bug in runtest.py and add checks for invalid -R parameters #7735

bpo-33873: Fix bug in runtest.py and add checks for invalid -R parameters #7735

Uh oh!

Conversation

pablogsal commented Jun 15, 2018 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pablogsal commented Jun 15, 2018

Uh oh!

pablogsal commented Jun 15, 2018

Uh oh!

vstinner Jun 16, 2018

Choose a reason for hiding this comment

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

vstinner Jun 20, 2018

Choose a reason for hiding this comment

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

vstinner commented Jun 20, 2018

Uh oh!

vstinner commented Jun 26, 2018

Uh oh!

pablogsal commented Jun 26, 2018

Uh oh!

miss-islington commented Jun 26, 2018

Uh oh!

miss-islington commented Jun 26, 2018

Uh oh!

bedevere-bot commented Jun 26, 2018

Uh oh!

bedevere-bot commented Jun 26, 2018

Uh oh!

Uh oh!

bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

pablogsal commented Jun 15, 2018 •

edited by bedevere-bot

Loading