system image build time regression (after new optimizer and iteration protocol)

[JeffBezanson]

For me this takes about 3m10s on v0.6, and 5m30s on master. One theory is that this is due to more inlining and thus generating longer functions (perhaps in just a few pathological cases), but we don't really know yet.

[StefanKarpinski]

This took long enough that I thought it was hung, so yeah, this has definitely regressed. On the other hand, now that #265 is fixed, sysimg build time only really matters to people who build Julia themselves often, so I'm not sure how critical this is.

[timholy]

Inlining is a good theory, but:

• d6a0bbdb2dd9694f032bf04eb008756625d33e5c (just before merging Decide inline-worthiness based on a more nuanced cost model #22210): 3m17.835s
• After the merger: 3m41.295s

Explains some but not a very large fraction of the change.

[maleadt]

I'm currently measuring release sysimg build times on an old but idle system of mine, and it just got past df63db6 (the large increase on the chart below):

Should take ~10 days until it reaches release-0.5.

EDIT: here is a better chart, with commit msg on hover (clicking redirects to the GitHub commit page).

[StefanKarpinski]

We should be tracking and charting this and other performance metrics automatically all the time.

[maleadt]

The script reached v0.5.0, with some clear regressions and improvements visible.

Is this a useful thing to have? If so, I can deploy this in a more persistent manner (ie. keeping track of master). Or will @KristofferC's codespeed cover this?

Also, I should have measured real+user+sys & rss+vsz instead of just wall seconds; would that be useful to regenerate the dataset for?

[djsegal]

Is there a public-facing dashboard for this kind of stuff?

[ViralBShah]

We have now added the new Package Manager and still have the old one, the new broadcasting stuff and many other additions. I assume the goal here is not to reduce sysimage build time as much as improve the compiler speed that everyone can benefit from.

Do we need more specific issues instead?

[JeffBezanson]

This is a specific issue --- it's an example case where compiler performance regressed. In fact when the new iteration protocol and optimizer were merged, there was another significant regression in this case. So it's not just caused by adding more code to the system image.

[ViralBShah]

Ok, I'll see if I can quantify that for discussion here and update the title.