Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-93691: Compiler's code-gen passes location around instead of holding it on the global compiler state #98001

Merged
merged 13 commits into from
Oct 17, 2022

Conversation

iritkatriel
Copy link
Member

@iritkatriel iritkatriel commented Oct 6, 2022

This implements #93691 (comment).

The line numbers are almost identical to those we calculated before (see the individual commits if you're interested in the process I followed). There were a few place (around annotations, for instance) where the line number was not set so it remained whatever it was before. I didn't bother going through hoops to make this backwards compatible because it's obviously wrong. I just take the line number from the current AST node.

In a few places (like pattern matching) I pass around a pointer to the location which can be updated deep in the recursion. This is not how it should be, but it's how it was until now (the global was updated in the recursion and impacted further nodes). So I used pointers to emulate this and get similar results, but my intention is to get rid of these pointers in the future, and just take locations from the relevant, current, AST node.

@iritkatriel iritkatriel added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 6, 2022
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit c7764ce 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 6, 2022
@JelleZijlstra
Copy link
Member

Going to retrigger buildbots after merging the refleak fix.

@JelleZijlstra JelleZijlstra added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 7, 2022
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @JelleZijlstra for commit f8d061b 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 7, 2022
@markshannon
Copy link
Member

I notice that some function take a location, and others take a pointer to a location. Why?
A typedef might help readability as well.

typedef struct location { ... } Location;

@iritkatriel
Copy link
Member Author

I notice that some function take a location, and others take a pointer to a location. Why?

See above (#98001 (comment)).

A typedef might help readability as well.

typedef struct location { ... } Location;

Yeah could do.

@iritkatriel iritkatriel self-assigned this Oct 8, 2022
Copy link
Member

@markshannon markshannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvement.

A few comments, but no major issues.

Python/compile.c Outdated
static location
last_location_in_body(asdl_stmt_seq *stmts)
{
for (int i = asdl_seq_LEN(stmts) - 1; i >= 0; i++) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIndows warning. Use Py_ssize_t instead of int.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this? There are a few other for loops with int variable in this file, I don't know if they need to change too.

Python/compile.c Outdated Show resolved Hide resolved
Python/compile.c Outdated Show resolved Hide resolved
Python/compile.c Show resolved Hide resolved
@@ -4015,8 +4066,10 @@ compiler_visit_stmt(struct compiler *c, stmt_ty s)
n++;
}
}
ADDOP_I(c, RAISE_VARARGS, (int)n);
location loc = LOC(s);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a few cases of this pattern:

    location loc = LOC(s);
    ADDOP...(..., loc, ...);

Any reason not to shorten this to:

    ADDOP...(..., LOC(S), ...);

?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll land this and do this and other cleanup in followup PRs. (There are still the SET_LOC and UNSET_LOC to remove - they do nothing now).

Thanks for the review!

@bedevere-bot
Copy link

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

And if you don't make the requested changes, you will be put in the comfy chair!

Copy link
Member

@markshannon markshannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@iritkatriel iritkatriel added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 17, 2022
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit 5418bb5 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 17, 2022
@iritkatriel iritkatriel merged commit 6da1a2e into python:main Oct 17, 2022
carljm added a commit to carljm/cpython that referenced this pull request Oct 17, 2022
* main: (31 commits)
  pythongh-95913: Move subinterpreter exper removal to 3.11 WhatsNew (pythonGH-98345)
  pythongh-95914: Add What's New item describing PEP 670 changes (python#98315)
  Remove unused arrange_output_buffer function from zlibmodule.c. (pythonGH-98358)
  pythongh-98174: Handle EPROTOTYPE under macOS in test_sendfile_fallback_close_peer_in_the_middle_of_receiving (python#98316)
  pythonGH-98327: Reduce scope of catch_warnings() in _make_subprocess_transport (python#98333)
  pythongh-93691: Compiler's code-gen passes location around instead of holding it on the global compiler state (pythonGH-98001)
  pythongh-97669: Create Tools/build/ directory (python#97963)
  pythongh-95534: Improve gzip reading speed by 10% (python#97664)
  pythongh-95913: Forward-port int/str security change to 3.11 What's New in main (python#98344)
  pythonGH-91415: Mention alphabetical sort ordering in the Sorting HOWTO (pythonGH-98336)
  pythongh-97930: Merge with importlib_resources 5.9 (pythonGH-97929)
  pythongh-85525: Remove extra row in doc (python#98337)
  pythongh-85299: Add note warning about entry point guard for asyncio example (python#93457)
  pythongh-97527: IDLE - fix buggy macosx patch (python#98313)
  pythongh-98307: Add docstring and documentation for SysLogHandler.createSocket (pythonGH-98319)
  pythongh-94808: Cover `PyFunction_GetCode`, `PyFunction_GetGlobals`, `PyFunction_GetModule` (python#98158)
  pythonGH-94597: Deprecate child watcher getters and setters (python#98215)
  pythongh-98254: Include stdlib module names in error messages for NameErrors (python#98255)
  Improve speed. Reduce auxiliary memory to 16.6% of the main array. (pythonGH-98294)
  [doc] Update logging cookbook with an example of custom handling of levels. (pythonGH-98290)
  ...
@iritkatriel iritkatriel deleted the linenos branch October 18, 2022 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants