Storing LOAD_CONST value in inline cache?

[Fidget-Spinner]

Currently LOAD_CONST is pretty fast, it indexes into an offset of a pointer to a tuple, then pushes a pointer.

        TARGET(LOAD_CONST) {
            PREDICTED(LOAD_CONST);
            PyObject *value = GETITEM(consts, oparg);
            Py_INCREF(value);
            PUSH(value);
            DISPATCH();
        }

A random thought I had: what if we stored all the constants as CACHE following a LOAD_CONST. So it would look like:

Bytecode:
LOAD_CONST
CACHE
CACHE
CACHE
CACHE

ceval.c:
        TARGET(LOAD_CONST) {
            PREDICTED(LOAD_CONST);
            _PyConstCache* cache = (_PyConstCache*)next_instr;
            PyObject* value = read_obj(cache);
            Py_INCREF(value);
            PUSH(value);
            DISPATCH();
        }

In theory, this could be even faster right? Accessing it would just mean incrementing the instruction pointer. This avoids the first pointer indirection we currently have. (Memory locality should also help it, since the co_consts tuple is somewhere else on the heap). The downside is that it's 4 non-dependent memory reads which may all be in cpu cache.

This shouldn't take any additional memory at all, because we can get rid of co_consts. co_consts can instead be a lazily generated tuple that we create on demand.

[brandtbucher]

I tried this a couple of months ago, but I ended up abandoning the project as "not worth it". If I remember correctly, the cost of the (relatively large) inline caches more than offset any tiny gains made from removing the indirection.

[gvanrossum]

Also currently co_consts has opportunities for sharing the entire co_const tuple which this would seem to lose.

[Fidget-Spinner]

Wow, thanks for sharing. I'd like to close this then but seems like discussions can't be closed :(.

[brandtbucher]

4 non-dependent memory reads

Also, just a small note: we're confident that all of the cache read_* and write_* functions are inlined as a single instruction on the major platforms and compilers. So only one read!