Please consider splitting up traitlets.py

[NeilGirdhar]

traitlets.py is 2152 lines today. It might be easier to work with if it were broken down into multiple files, one for each section. We can still import all the relevant things in the __init__.py (which should only import the relevant things — not *). We can protect your change history by creating the new files using git copy.

[SylvainCorlay]

I guess we could separate things in multiple files, but we should probably wait for the main changes planned with #48 to be completed and the API to stabilize before we decide how to do this.

[minrk]

Importing * is still probably the right thing to do, and relevance can be governed by __all__ in the files.

[minrk]

@jasongrout after looking into it, I think it's largely our habit of liberally splitting files as proposed here that contributes most to the slow startup time. I'm actually investigating merging files elsewhere in our code.

I never thought I'd say this, but I wish Python had javascript-like module concatenation...

[jasongrout]

Isn't that what zip modules are for? We can import from zip files.

[jasongrout]

That's what the zipsafe option is about in setup.py, for example.

[jasongrout]

Lazy imports could possible help too (see also the comments on zip modules): https://www.rfk.id.au/blog/entry/frozen-app-starting-faster/

[minrk]

I'm not sure how much lazy imports can help IPython, since we use ~all of the files we import by the time objects are instantiated. I think we would have to refactor lots of code to delay instantiation. I doubt lazy imports would help much at all.

[SylvainCorlay]

ok, I was not especially thinking of complexity, but rather in terms of logical separation of features in different files.

[NeilGirdhar]

Please don't merge the files. One reason for logical separation is that it makes things easy to understand. At the very least, the TraitType subclasses should each be in their own file, or at least in a separate file from the other class definitions.

Also, I don't understand this concern over startup time. On my laptop, traitlets is imported in half a microsecond.

[minrk]

Please don't merge the files. One reason for logical separation is that it makes things easy to understand.

100% agreed, and that's why we habitually split files throughout IPython and Jupyter. traitlets.py is a rare exception, and I'm actually in support of splitting it up. What gives me pause is recent comments about IPython startup time. When a collection of projects have this same pattern, it adds up, and you get things like IPython taking ~1s to start up, most of which time is spent importing.

[NeilGirdhar]

@minrk It's true. My IPython takes a while to start up. I just use the notebook now and never close it :)

[rmorshea]

I think we can close this for now.