Improve startup time.

[markshannon]

Before we attempt this, we need to know where the time is spent.
@ericsnowcurrently Do you have any profiling information from running python -S -c "print('Hi')", or a similar script?

It takes about 10 ms on my machine for 3.9 which seems a long time to print "Hi".
For comparison, 2.7 takes about 7ms.

I suspect there is a lot of incidental stuff going on that we can eliminate.

The tasks that have to be performed to run python -S -c "print('Hi')" are:

• Load executable from disk (it should be in memory cache, but it still needs work by the O/S).
• Build config object from command line and environment variables
• Create a new interpreter
• Create a new thread
• Compile (including parsing) print(Hi")
• Execute print(Hi")
• Dispose of thread
• Dispose of interpreter

None of those tasks should take long, so what is slow?

[gvanrossum]

We also need to o initialize some extension modules, sys and built ons.

[markshannon]

If creating and/or destroying the new interpreter is what takes the time, then one possible way to speed it up is to freeze the object graph of a newly created interpreter.

That would work roughly as follows:

1. Modify CPython to walk the object graph, immediately after interpreter creation, dumping it to a text file.
2. Convert that text file into a declarative spec of the initial object graph.
3. Check in that spec, we will want to modify it later, and it will probably need fixing up by hand.
4. Discard the modified CPython, we shouldn't need it anymore.
5. Make a tool that generates two pieces of C code:
   • A static structure containing the whole object graph, with offsets not pointers
   • A table containing offsets of those offsets

We can now create the entire object graph for the interpreter by:

1. Copying the static structure into the newly allocated memory for the interpreter.
2. Traversing the offsets of offsets, converting them into pointers.

The above assumes that interpreters are fully independent.
Since they are not, at least not yet, we need to break the object graph into the part belonging to the runtime, and that belonging to each interpreter. For the runtime (static objects) there is no need for offsets, just create the graph with pointers. For the interpreter, the data structure needs to mark offsets so we can tell which are pointers into the runtime, and which are pointers into the interpreter.

[markshannon]

I've just remembered python3 -v.

Running python3 -v -S -c "print('Hi')" prints

import _frozen_importlib # frozen
import _imp # builtin
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
# installing zipimport hook
import 'time' # <class '_frozen_importlib.BuiltinImporter'>
import 'zipimport' # <class '_frozen_importlib.FrozenImporter'>
# installed zipimport hook
# /home/mark/repos/cpython/Lib/encodings/__pycache__/__init__.cpython-310.pyc matches /home/mark/repos/cpython/Lib/encodings/__init__.py
# code object from '/home/mark/repos/cpython/Lib/encodings/__pycache__/__init__.cpython-310.pyc'
# /home/mark/repos/cpython/Lib/__pycache__/codecs.cpython-310.pyc matches /home/mark/repos/cpython/Lib/codecs.py
# code object from '/home/mark/repos/cpython/Lib/__pycache__/codecs.cpython-310.pyc'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x7f6fbcf2d4f0>
# /home/mark/repos/cpython/Lib/encodings/__pycache__/aliases.cpython-310.pyc matches /home/mark/repos/cpython/Lib/encodings/aliases.py
# code object from '/home/mark/repos/cpython/Lib/encodings/__pycache__/aliases.cpython-310.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x7f6fbcec6d00>
import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x7f6fbcf2d340>
# /home/mark/repos/cpython/Lib/encodings/__pycache__/utf_8.cpython-310.pyc matches /home/mark/repos/cpython/Lib/encodings/utf_8.py
# code object from '/home/mark/repos/cpython/Lib/encodings/__pycache__/utf_8.cpython-310.pyc'
import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x7f6fbcf2d370>
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
# /home/mark/repos/cpython/Lib/__pycache__/io.cpython-310.pyc matches /home/mark/repos/cpython/Lib/io.py
# code object from '/home/mark/repos/cpython/Lib/__pycache__/io.cpython-310.pyc'
# /home/mark/repos/cpython/Lib/__pycache__/abc.cpython-310.pyc matches /home/mark/repos/cpython/Lib/abc.py
# code object from '/home/mark/repos/cpython/Lib/__pycache__/abc.cpython-310.pyc'
import '_abc' # <class '_frozen_importlib.BuiltinImporter'>
import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f6fbcec6f70>
import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x7f6fbcec6e50>

[ericsnowcurrently]

Before we attempt this, we need to know where the time is spent.
@ericsnowcurrently Do you have any profiling information from running python -S -c "print('Hi')", or a similar script?

I don't have any such info presently. I know there were investigations in the past that involved such profiling but I expect any resulting profiling data is outdated at this point.

Regardless, it shouldn't take much effort to get at least some basic data. Furthermore, we can already get at least some insight by running ./python -v -S -c "print('Hi')".

[gvanrossum]

Regarding Mark's strategy, can you explain what we would have to do when we need to change the object graph? IIUR currently we change some of the modules that are frozen (_frozen_importlib and friends), and then regenerate the frozen bytecode (for which we have tools). I am worried that your item (3) makes this process more complicated.

Otherwise, it sounds like a decent strategy (many data structures are already static webs of pointers, the new thing is that we allow static webs of PyObject pointers).

And yes, we need to work on profiling startup.

[ericsnowcurrently]

FTR, facebook/instagram has been using a related strategy, which they described at the language summit a couple years back.

[methane]

I've just remembered python3 -v.

Please don't forget -X importtime (or PYTHONPROFILEIMPORTTIME=1 python) too.

[gvanrossum]

Mark has a nascent proposal

[fweimer]

Mark has a nascent proposal

This is only beneficial if the size of mapped files exceeds a certain threshold, which is fairly large (probably larger than 100 KiB by now). Below that, copying is faster. With small files, I expect a run-time penalty even after loading because the less dense packing of information (distributed across a larger number of pages) results in increased TLB misses. And by default, Linux will only allow 65,530 separate mappings per process.

[gvanrossum]

My intuition is that it will be hard to show that mmap is faster than copying here, so I would personally be happy if the first version just read (i.e., copied) the file into memory.

[markshannon]

The point of the proposal is to avoid the overhead of unmarshalling, whether the file is copied or mapped isn't so important.

The important thing is that the pyc format is position independent (no pointers), is immutable, and requires minimal decoding before being usable.
This is important, because it means we can merge the pyc files of common modules into a single file, or embed them in the interpreter.

[gvanrossum]

This is important, because it means we can merge the pyc files of common modules into a single file, or embed them in the interpreter.

Oh, that's really neat!

[gvanrossum]

I just discovered -X importtime. Running 3.10 with this, -S, and '-c pass' on my Windows laptop, shows:

import time: self [us] | cumulative | imported package
import time:       169 |        169 |   _io
import time:        38 |         38 |   marshal
import time:       135 |        135 |   nt
import time:        66 |         66 |   winreg
import time:       708 |       1114 | _frozen_importlib_external
import time:       593 |        593 |   time
import time:       294 |        886 | zipimport
import time:       124 |        124 |     _codecs
import time:      1108 |       1231 |   codecs
import time:      1105 |       1105 |   encodings.aliases
import time:      2101 |       4436 | encodings
import time:       702 |        702 | encodings.utf_8
import time:      1028 |       1028 | encodings.cp1252
import time:       106 |        106 | _signal
import time:        52 |         52 |     _abc
import time:      1504 |       1555 |   abc
import time:      1311 |       2866 | io

The biggest cumulative time (4.4 ms) is the encodings package, plus another 1.7 ms for two specific encodings. Maybe we can somehow avoid importing the full encodings package, and only import the encoding we really need?

The next biggest cost is io (2.8 ms), which is largely due to importing abc. Again, a possible target for specific hacks. (In both cases I suspect @vstinner has already thought about this.)

Of course, without -S, we spend a ton of time (25-30 ms!) importing site, which pulls in the kitchen sink. A possible hack targeting this could be to write a configuration file somewhere that lists the outcome of all the work done by site, plus a list of directories and their mtimes that should be statted so that the full thing only has to run if any of those directories have changed. Or something like that. (Maybe a hash of certain directories and files would be needed -- I imagine even that is faster than importing os and contextlib.)

Here's the raw data for that:

import time: self [us] | cumulative | imported package
import time:       223 |        223 |   _io
import time:        62 |         62 |   marshal
import time:       162 |        162 |   nt
import time:        73 |         73 |   winreg
import time:       960 |       1477 | _frozen_importlib_external
import time:       636 |        636 |   time
import time:       299 |        935 | zipimport
import time:       141 |        141 |     _codecs
import time:      1381 |       1522 |   codecs
import time:      1226 |       1226 |   encodings.aliases
import time:      2273 |       5020 | encodings
import time:       655 |        655 | encodings.utf_8
import time:       603 |        603 | encodings.cp1252
import time:        50 |         50 | _signal
import time:        45 |         45 |     _abc
import time:       762 |        806 |   abc
import time:       750 |       1556 | io
import time:        69 |         69 |       _stat
import time:       684 |        753 |     stat
import time:      1259 |       1259 |     _collections_abc
import time:       893 |        893 |       genericpath
import time:      1332 |       2225 |     ntpath
import time:      1284 |       5519 |   os
import time:       947 |        947 |   _sitebuiltins
import time:      1381 |       1381 |   types
import time:       695 |        695 |       warnings
import time:       852 |       1546 |     importlib
import time:       558 |        558 |     importlib._abc
import time:       140 |        140 |         itertools
import time:       585 |        585 |         keyword
import time:        74 |         74 |           _operator
import time:       754 |        827 |         operator
import time:       643 |        643 |         reprlib
import time:       540 |        540 |         _collections
import time:      1422 |       4154 |       collections
import time:        69 |         69 |         _functools
import time:      1073 |       1142 |       functools
import time:      1040 |       6335 |     contextlib
import time:       972 |       9410 |   importlib.util
import time:       755 |        755 |   importlib.machinery
import time:       715 |        715 |   sitecustomize
import time:       383 |        383 |   usercustomize
import time:      6707 |      25813 | site