make regen-all doesn't work in subfolder: No module named Parser.pgen

[hroncok]

BPO	36733
Nosy	@vstinner, @hroncok, @pablogsal
PRs	bpo-36733: Fix PYTHONPATH for make regen-add #12969

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-04-26.11:10:06.909>
created_at = <Date 2019-04-26.10:17:42.615>
labels = ['build', '3.8', '3.9']
title = "make regen-all doesn't work in subfolder: No module named Parser.pgen"
updated_at = <Date 2019-04-26.11:10:58.280>
user = 'https://github.com/hroncok'

bugs.python.org fields:

activity = <Date 2019-04-26.11:10:58.280>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-04-26.11:10:06.909>
closer = 'vstinner'
components = ['Build']
creation = <Date 2019-04-26.10:17:42.615>
creator = 'hroncok'
dependencies = []
files = []
hgrepos = []
issue_num = 36733
keywords = ['patch']
message_count = 5.0
messages = ['340902', '340903', '340904', '340909', '340910']
nosy_count = 3.0
nosy_names = ['vstinner', 'hroncok', 'pablogsal']
pr_nums = ['12969']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue36733'
versions = ['Python 3.8', 'Python 3.9']

Linked PRs

• gh-81094: Refer to PEP 318 in compound_statements.rst #113588
• gh-80914: Adding Period on Line 1365 #113597

[hroncok]

When I attempt to build CPython from a subfolder (as we do in Fedora) make regen-all dies with:

python3.8 -m Parser.pgen ../../Grammar/Grammar \
../../Grammar/Tokens \
../../Include/graminit.h.new \
../../Python/graminit.c.new
/usr/bin/python3.8: No module named Parser.pgen

To reproduce, run:

  $ rm -rf build && mkdir -p build/mybuild && (cd build/mybuild && ../../configure && make regen-all)

This is probably a regression, as it works in the 3.7 branch.

Setting PYTHON_FOR_REGEN="python3" (python3.7 on my system) doesn't make a difference:

python3 -m Parser.pgen ../../Grammar/Grammar \
../../Grammar/Tokens \
../../Include/graminit.h.new \
../../Python/graminit.c.new
/usr/bin/python3: No module named Parser.pgen

build/mybuild/Parser exists but it is empty directory.

Setting PYTHON_FOR_REGEN="PYTHOINPATH=<path_to_sources> python3" workarounds the issue.

On branch 3.7 (3076a3e), build/mybuild/Parser is also empty, but the make-regen populates it.

I'll bisect.

[vstinner]

regen-keyword is recent: bpo-36143.

regen-grammar has been modified to use Parser.pgen in bpo-35808.

[hroncok]

1f24a71

[vstinner]

New changeset 06d04e7 by Victor Stinner in branch 'master':
bpo-36733: Fix PYTHONPATH for make regen-add (GH-12969)
06d04e7

[vstinner]

Thanks for your bug report Miro Hrončok. The fix will be part of the next Python 3.8 alpha release:

"3.8.0 alpha 4: Monday, 2019-04-29"
https://www.python.org/dev/peps/pep-0569/