Skip to content

bpo-45866: pegen strips directory of "generated from" header #29777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 26, 2021
Merged

bpo-45866: pegen strips directory of "generated from" header #29777

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Misc/NEWS.d/next/Build/2021-11-25-13-53-36.bpo-45866.ZH1W8N.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
``make regen-all`` now produces the same output when run from a directory
other than the source tree: when building Python out of the source tree.
pegen now strips directory of the "generated by pygen from <FILENAME" header
Patch by Victor Stinner.
2 changes: 1 addition & 1 deletion Parser/parser.c
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// @generated by pegen from ./Grammar/python.gram
// @generated by pegen from python.gram
#include "pegen.h"

#if defined(Py_DEBUG) && defined(Py_BUILD_CORE)
Expand Down
4 changes: 3 additions & 1 deletion Tools/peg_generator/pegen/c_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import ast
import os.path
import re
from dataclasses import dataclass, field
from enum import Enum
Expand Down Expand Up @@ -416,7 +417,8 @@ def out_of_memory_goto(self, expr: str, goto_target: str) -> None:

def generate(self, filename: str) -> None:
self.collect_rules()
self.print(f"// @generated by pegen from {filename}")
basename = os.path.basename(filename)
Copy link
Member

@arhadthedev arhadthedev Nov 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use pathlib instead of os.path?

from pathlib import Path
# [...]
        basename = Path(filename).name

Copy link
Member

@tiran tiran Nov 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pathlib has a much more complex import tree and relies on more modules. os.path has the benefit that it works with a minimal interpreter at an early stage of the bootstrapping process.

Copy link
Member Author

@vstinner vstinner Nov 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, pathlib can be used, but I prefer os.path :-) For this specific change, I don't think that pathlib is much better.

self.print(f"// @generated by pegen from {basename}")
header = self.grammar.metas.get("header", EXTENSION_PREFIX)
if header:
self.print(header.rstrip("\n"))
Expand Down
2 changes: 1 addition & 1 deletion Tools/peg_generator/pegen/grammar_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env python3.8
# @generated by pegen from ./Tools/peg_generator/pegen/metagrammar.gram
# @generated by pegen from metagrammar.gram

import ast
import sys
Expand Down
4 changes: 3 additions & 1 deletion Tools/peg_generator/pegen/python_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import os.path
import token
from typing import IO, Any, Dict, Optional, Sequence, Set, Text, Tuple

Expand Down Expand Up @@ -212,7 +213,8 @@ def generate(self, filename: str) -> None:
self.collect_rules()
header = self.grammar.metas.get("header", MODULE_PREFIX)
if header is not None:
self.print(header.rstrip("\n").format(filename=filename))
basename = os.path.basename(filename)
Copy link
Member

@arhadthedev arhadthedev Nov 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

self.print(header.rstrip("\n").format(filename=basename))
subheader = self.grammar.metas.get("subheader", "")
if subheader:
self.print(subheader)
Expand Down