Combine dataclass exec calls

[ssweber]

Feature or enhancement

Proposal:

As mentioned here, dataclass definition can possibly be sped up by combining the multiple exec calls into one.

I have two versions, and I found that this results in either a 1.1-1.22 1.26, or 1.37 speedup versus main (old time / new time)

version 1 combines the exec calls by passing a list of name/txt/locals from each function definition.
version 2 simplifies the final exec call to avoid __create_fn__, but I'm not experienced to know if that is safe. (see below for open pr)

@ericvsmith Let me know if this is a suitable change, and if so, which version we should open a pr based on.

Thank you!

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

https://discuss.python.org/t/improving-dataclasses-startup-performance/15442/18

Linked PRs

• gh-109870: Combine exec calls in dataclass #110186
• gh-109870: Dataclasses: batch up exec calls #110851

[ssweber]

I measured the difference with this benchmark which includes order=True.

Since we are combining exec calls, the speed up over main will be less if order=False

[ssweber]

This could be seen as complementary to Caching dataclass, but this proposal shouldn’t use more memory than the status quo. (we arnt storing a cache in the class)

[ssweber]

I updated my numbers. I had missed _hash_add, and my simplified exec without the __create_fn__ call was naive and didn't work 😅

The new numbers 1.1-1.18 speedup, depending on how many methods are generated (eg, order=True, frozen=True, etc).

I didn't take the time to refactor _hash_add to be included with the other functions, so its exec is still separate. If it was refactored to be included, we could once again bump those numbers up a bit.

[ericvsmith]

@ssweber : As I said elsewhere, I think this is a good idea. Thanks for the work on it! I plan on looking at this during the core dev sprint next week.

[ssweber]

Glad I can help in some way! I cleaned up and created a pull request. I tried to not change to much, mainly just to prove the modest speedup.

[ericvsmith]

I have a PR that's almost ready for this. I got about a 15% speedup for creating the most basic of dataclasses, which adds 3 methods: __init__, __repr__, and __eq__. I expect the speedup to be faster if adding comparison functions, etc.

[ssweber]

It's like 1.22 when you include order/frozen! 😄 love the new FuncBuilder class.