Deprecate ob_shash in BytesObject

[methane]

BPO	46864
Nosy	@tiran, @methane, @serhiy-storchaka, @animalize, @brandtbucher
PRs	bpo-46864: Deprecate PyBytesObject.ob_shash. #31598 bpo-46864: Suppress deprecation warnings for ob_shash. #32042 bpo-46864: Suppress even more ob_shash deprecation warnings (GH-32176) #32176
Files	remove-bytes-hash.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-25.02:15:16.155>
created_at = <Date 2022-02-26.08:49:05.075>
labels = ['interpreter-core', 'build', '3.11']
title = 'Deprecate ob_shash in BytesObject'
updated_at = <Date 2022-03-30.06:35:35.207>
user = 'https://github.com/methane'

bugs.python.org fields:

activity = <Date 2022-03-30.06:35:35.207>
actor = 'christian.heimes'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-25.02:15:16.155>
closer = 'methane'
components = ['Interpreter Core']
creation = <Date 2022-02-26.08:49:05.075>
creator = 'methane'
dependencies = []
files = ['50649']
hgrepos = []
issue_num = 46864
keywords = ['patch']
message_count = 21.0
messages = ['414083', '414096', '414134', '414136', '414605', '415684', '415743', '415744', '415747', '415748', '415749', '415854', '415855', '415857', '415865', '415923', '415925', '415927', '415958', '415984', '416325']
nosy_count = 5.0
nosy_names = ['christian.heimes', 'methane', 'serhiy.storchaka', 'malin', 'brandtbucher']
pr_nums = ['31598', '32042', '32176']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue46864'
versions = ['Python 3.11']

[methane]

Code objects have more and more bytes attributes for now.
To reduce the RAM by code, I want to remove ob_shash (cached hash value) from bytes object.

Sets and dicts have own hash cache.
Unless checking same bytes object against dicts/sets many times, this don't cause big performance loss.

[serhiy-storchaka]

I think it is a legacy of Python 2. Attributes and variable names are Unicode strings in Python 3, so the main reason of this optimization is no longer relevant.

But some programs can still work with encoded bytes instead of strings. In particular os.environ and os.environb are implemented as dict of bytes on non-Windows.

[methane]

But some programs can still work with encoded bytes instead of strings. In particular os.environ and os.environb are implemented as dict of bytes on non-Windows.

This change doesn't affect to os.environ.

os.environ[key] does key.encode(sys.getfilesystemencoding(), "surrogateescape") internally. So the encoded key doesn't have cached hash.
On the other hand, dict (self._data) has own hash cache. So it don't use hash cached in the bytes objects.

On the other hand, this change will affect os.environb[key] if key is used repeatedly.

[methane]

When removed shash:

## small key
$ ./python -m pyperf timeit --compare-to ../cpython/python -s 'd={b"foo":1, b"bar":2, b"buzz":3}' -- 'b"key" in d'
/home/inada-n/work/python/cpython/python: ..................... 23.2 ns +- 1.7 ns
/home/inada-n/work/python/remove-bytes-hash/python: ..................... 40.0 ns +- 1.5 ns

Mean +- std dev: [/home/inada-n/work/python/cpython/python] 23.2 ns +- 1.7 ns -> [/home/inada-n/work/python/remove-bytes-hash/python] 40.0 ns +- 1.5 ns: 1.73x slower

## large key
$ ./python -m pyperf timeit --compare-to ../cpython/python -s 'd={b"foo":1, b"bar":2, b"buzz":3};k=b"key"*100' -- 'k in d'
/home/inada-n/work/python/cpython/python: ..................... 22.3 ns +- 1.2 ns
/home/inada-n/work/python/remove-bytes-hash/python: ..................... 108 ns +- 2 ns

Mean +- std dev: [/home/inada-n/work/python/cpython/python] 22.3 ns +- 1.2 ns -> [/home/inada-n/work/python/remove-bytes-hash/python] 108 ns +- 2 ns: 4.84x slower

I will reconsider the removal before remove the cache.
We changed code object too often. If Python 3.13 don't use so much bytes objects, we don't need to remove the hash to save some RAM.

[methane]

New changeset 2d8b764 by Inada Naoki in branch 'main':
bpo-46864: Deprecate PyBytesObject.ob_shash. (GH-31598)
2d8b764

[tiran]

I'm getting tons of deprecation warnings from deepfreeze script. Please add _Py_COMP_DIAG_IGNORE_DEPR_DECL to Tools/scripts/deepfreeze.py.