Allow json.tool to have identical infile and outfile

[michael-kuhlmann]

BPO	33927
Nosy	@rhettinger, @ezio-melotti, @matrixise, @pablogsal, @remilapeyre, @michael-kuhlmann, @CharString
PRs	bpo-33927: Add support for same infile and outfile to json.tool #7865 bpo-33927: Pass stdin and stdout are default arguments to argpars infile/outfile #11992 bpo-33927: Handle pyton -m json.tool --json-lines infile infile #29478

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-06-21.10:45:40.446>
labels = ['type-bug', 'library', '3.11']
title = 'Allow json.tool to have identical infile and outfile'
updated_at = <Date 2021-11-08.22:09:34.365>
user = 'https://github.com/michael-kuhlmann'

bugs.python.org fields:

activity = <Date 2021-11-08.22:09:34.365>
actor = 'CharString'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-06-21.10:45:40.446>
creator = 'kuhlmann'
dependencies = []
files = []
hgrepos = []
issue_num = 33927
keywords = ['patch']
message_count = 10.0
messages = ['320151', '320152', '320173', '320302', '320326', '320328', '320329', '320368', '320729', '390853']
nosy_count = 7.0
nosy_names = ['rhettinger', 'ezio.melotti', 'matrixise', 'pablogsal', 'remi.lapeyre', 'kuhlmann', 'CharString']
pr_nums = ['7865', '11992', '29478']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33927'
versions = ['Python 3.11']

[michael-kuhlmann]

It would be nice to have same infile and outfile for json.tool to replace json files with their pretty-printed version.

Currently, if I try this I get an error:
$ python3 -m json.tool example.json example.json
Expecting value: line 1 column 1 (char 0)

[matrixise]

yep, or you could use sponge

cat example.json | python3 -m json.tool | sponge example.json

a small workaround ;-)

[remilapeyre]

Hi Michael, looking at the current code of json.tool, there is no reason for it not to be able to do this, I will a patch to do this tonight.

[remilapeyre]

Hi, I proposed a path in #7865, I'm not sure if I can apply the label skip news or if only a reviewer can.

[pablogsal]

The current status of json.tool also leaks a file descriptor if you use the same filename or an invalid one (needs debug build to receive this error message):

$ ./python -m json.tool invalid_file.dat nofile.dat
Expecting value: line 1 column 1 (char 0)
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='nofile.dat' mode='w' encoding='UTF-8'>

$ ./python -m json.tool valid.dat valid.dat
Expecting value: line 1 column 1 (char 0)
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='valid.dat' mode='w' encoding='UTF-8'>

[pablogsal]

@rémi can you include a NEWS entry?

Also, indicate that your patch prevents a file descriptor to be leaked in the cases indicated in my last message.

[pablogsal]

Maybe we should include a test that checks that if you provide an invalid file the file descriptors are not leaked (it can be in a different PR, maybe).

[remilapeyre]

Hi Pablo, while this patch should fix both problems, I'm not sure how to write a regression test for this, assert_python_ok in https://github.com/python/cpython/pull/7865/files#diff-7d4645839a05727ebdf39226fd56e29cR97 forks the interpreter so I'm not sure I can check for unclosed fd during the test.

Should I call https://github.com/remilapeyre/cpython/blob/04b2ade751b318460c1f0f9566676ef519358328/Lib/json/tool.py#L18 directly and mock io.open and os.close?

[remilapeyre]

Hi Pablo, I added two tests to confirm that file descriptors do not link anymore.

The tests are rather ugly but I'm not sure if it's possible to do better.

Is this patch ok for you?