Skip to content

gh-133367: Add missing options to ast CLI #133369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
May 5, 2025
34 changes: 28 additions & 6 deletions Doc/library/ast.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
:mod:`!ast` --- Abstract Syntax Trees
:mod:`!ast` --- Abstract syntax trees
=====================================

.. module:: ast
Expand Down Expand Up @@ -29,7 +29,7 @@

.. _abstract-grammar:

Abstract Grammar
Abstract grammar
----------------

The abstract grammar is currently defined as follows:
Expand Down Expand Up @@ -2152,14 +2152,14 @@
body of an :class:`AsyncFunctionDef`.

.. note::
When a string is parsed by :func:`ast.parse`, operator nodes (subclasses

Check warning on line 2155 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs / Docs

py:class reference target not found: ast.operator [ref.class]

Check warning on line 2155 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs / Docs

py:class reference target not found: ast.unaryop [ref.class]

Check warning on line 2155 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs / Docs

py:class reference target not found: ast.cmpop [ref.class]

Check warning on line 2155 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs / Docs

py:class reference target not found: ast.boolop [ref.class]

Check warning on line 2155 in Doc/library/ast.rst

View workflow job for this annotation

GitHub Actions / Docs / Docs

py:class reference target not found: ast.expr_context [ref.class]
of :class:`ast.operator`, :class:`ast.unaryop`, :class:`ast.cmpop`,
:class:`ast.boolop` and :class:`ast.expr_context`) on the returned tree
will be singletons. Changes to one will be reflected in all other
occurrences of the same value (e.g. :class:`ast.Add`).
occurrences of the same value (for example, :class:`ast.Add`).


:mod:`ast` Helpers
:mod:`ast` helpers
------------------

Apart from the node classes, the :mod:`ast` module defines these utility functions
Expand Down Expand Up @@ -2484,7 +2484,7 @@

.. _ast-compiler-flags:

Compiler Flags
Compiler flags
--------------

The following flags may be passed to :func:`compile` in order to change
Expand Down Expand Up @@ -2533,7 +2533,7 @@

.. _ast-cli:

Command-Line Usage
Command-line usage
------------------

.. versionadded:: 3.9
Expand Down Expand Up @@ -2572,6 +2572,28 @@

Indentation of nodes in AST (number of spaces).

.. option:: --feature-version <version>

Python version in the format 3.x (for example, 3.10). Defaults to the
current version of the interpreter.

.. versionadded:: next

.. option:: -O <level>
--optimize <level>

Optimization level for parser. Defaults to no optimization.

.. versionadded:: next

.. option:: --show-empty

Show empty lists and fields that are ``None``. Defaults to not showing empty
objects.

.. versionadded:: next


If :file:`infile` is specified its contents are parsed to AST and dumped
to stdout. Otherwise, the content is read from stdin.

Expand Down
5 changes: 5 additions & 0 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -870,6 +870,11 @@ ast
that the root node type is appropriate.
(Contributed by Irit Katriel in :gh:`130139`.)

* Add new ``--feature-version``, ``--optimize``, ``--show-empty`` options to
command-line interface.
(Contributed by Semyon Moroz in :gh:`133367`.)


bdb
---

Expand Down
27 changes: 25 additions & 2 deletions Lib/ast.py
Original file line number Diff line number Diff line change
Expand Up @@ -643,6 +643,15 @@ def main(args=None):
'column offsets')
parser.add_argument('-i', '--indent', type=int, default=3,
help='indentation of nodes (number of spaces)')
parser.add_argument('--feature-version',
type=str, default=None, metavar='VERSION',
help='Python version in the format 3.x '
'(for example, 3.10)')
parser.add_argument('-O', '--optimize',
type=int, default=-1, metavar='LEVEL',
help='optimization level for parser (default -1)')
parser.add_argument('--show-empty', default=False, action='store_true',
help='show empty lists and fields in dump output')
args = parser.parse_args(args)

if args.infile == '-':
Expand All @@ -652,8 +661,22 @@ def main(args=None):
name = args.infile
with open(args.infile, 'rb') as infile:
source = infile.read()
tree = parse(source, name, args.mode, type_comments=args.no_type_comments)
print(dump(tree, include_attributes=args.include_attributes, indent=args.indent))

# Process feature_version
feature_version = None
if args.feature_version:
try:
major, minor = map(int, args.feature_version.split('.', 1))
except ValueError:
parser.error('Invalid format for --feature-version; '
'expected format 3.x (for example, 3.10)')

feature_version = (major, minor)

tree = parse(source, name, args.mode, type_comments=args.no_type_comments,
feature_version=feature_version, optimize=args.optimize)
print(dump(tree, include_attributes=args.include_attributes,
indent=args.indent, show_empty=args.show_empty))

if __name__ == '__main__':
main()
95 changes: 94 additions & 1 deletion Lib/test/test_ast/test_ast.py
Original file line number Diff line number Diff line change
Expand Up @@ -3272,6 +3272,9 @@ def test_invocation(self):
('--no-type-comments', '--no-type-comments'),
('-a', '--include-attributes'),
('-i=4', '--indent=4'),
('--feature-version=3.13', '--feature-version=3.13'),
('-O=-1', '--optimize=-1'),
('--show-empty', '--show-empty'),
)
self.set_source('''
print(1, 2, 3)
Expand Down Expand Up @@ -3389,7 +3392,7 @@ def test_include_attributes_flag(self):
self.check_output(source, expect, flag)

def test_indent_flag(self):
# test 'python -m ast -i/--indent'
# test 'python -m ast -i/--indent 0'
source = 'pass'
expect = '''
Module(
Expand All @@ -3400,6 +3403,96 @@ def test_indent_flag(self):
with self.subTest(flag=flag):
self.check_output(source, expect, flag)

def test_feature_version_flag(self):
# test 'python -m ast --feature-version 3.9/3.10'
source = '''
match x:
case 1:
pass
'''
expect = '''
Module(
body=[
Match(
subject=Name(id='x', ctx=Load()),
cases=[
match_case(
pattern=MatchValue(
value=Constant(value=1)),
body=[
Pass()])])])
'''
self.check_output(source, expect, '--feature-version=3.10')
with self.assertRaises(SyntaxError):
self.invoke_ast('--feature-version=3.9')

def test_no_optimize_flag(self):
# test 'python -m ast -O/--optimize -1/0'
source = '''
match a:
case 1+2j:
pass
'''
expect = '''
Module(
body=[
Match(
subject=Name(id='a', ctx=Load()),
cases=[
match_case(
pattern=MatchValue(
value=BinOp(
left=Constant(value=1),
op=Add(),
right=Constant(value=2j))),
body=[
Pass()])])])
'''
for flag in ('-O=-1', '--optimize=-1', '-O=0', '--optimize=0'):
with self.subTest(flag=flag):
self.check_output(source, expect, flag)

def test_optimize_flag(self):
# test 'python -m ast -O/--optimize 1/2'
source = '''
match a:
case 1+2j:
pass
'''
expect = '''
Module(
body=[
Match(
subject=Name(id='a', ctx=Load()),
cases=[
match_case(
pattern=MatchValue(
value=Constant(value=(1+2j))),
body=[
Pass()])])])
'''
for flag in ('-O=1', '--optimize=1', '-O=2', '--optimize=2'):
with self.subTest(flag=flag):
self.check_output(source, expect, flag)

def test_show_empty_flag(self):
# test 'python -m ast --show-empty'
source = 'print(1, 2, 3)'
expect = '''
Module(
body=[
Expr(
value=Call(
func=Name(id='print', ctx=Load()),
args=[
Constant(value=1),
Constant(value=2),
Constant(value=3)],
keywords=[]))],
type_ignores=[])
'''
self.check_output(source, expect, '--show-empty')


class ASTOptimiziationTests(unittest.TestCase):
def wrap_expr(self, expr):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add the ``--feature-version``, ``--optimize``, and ``--show-empty`` options
to the :mod:`ast` command-line interface. Patch by Semyon Moroz.
Loading