bpo-32403: Faster date and datetime constructors #4993

pganssle · 2017-12-23T20:02:37Z

This fixes bpo-32403 and bpo-32404. The first one is a performance enhancement, the second is a bugfix.

Using this script to benchmark the alternate constructors, I see significant improvements in speed across the board (note that since the subclasses don't take the fast path, they are a reasonable proxy for the speed from before this patch). You can see the results on my laptop (using the debug build, not the profiling optimized build) below:

Running with N = 10000, k=20
                   |           datetime           |       DateTimeSubclass       
=================================================================================
constructor        |  1416.5  [1463.9 (±71) ns]   |  1494.8  [1569.2 (±119) ns]  
fromordinal        |  1081.5  [1117.2 (±24) ns]   |  2180.7  [2357.3 (±279) ns]  
fromisoformat      |   928.8   [962.5 (±34) ns]   |  2834.8  [3179.7 (±281) ns]  
fromtimestamp      |  1504.3  [1600.9 (±63) ns]   |  3625.1  [3869.6 (±280) ns]  
utcfromtimestamp   |  1178.6  [1281.2 (±172) ns]  |  3242.6  [3445.0 (±205) ns]  
combine            |  1060.1  [1111.0 (±73) ns]   |  2808.7  [3013.0 (±236) ns]  
now                |  1219.3  [1325.0 (±216) ns]  |  3625.5  [3892.7 (±300) ns]  
today              |  4868.5  [5261.0 (±565) ns]  |  7189.5  [7663.0 (±436) ns]  
=================================================================================
                   |             date             |         DateSubclass         
=================================================================================
constructor        |  974.5   [1010.1 (±39) ns]   |  1114.7  [1298.8 (±229) ns]  
fromordinal        |  1059.7  [1109.4 (±59) ns]   |  1923.1  [2258.8 (±375) ns]  
fromisoformat      |  817.5   [905.1 (±123) ns]   |  1734.1  [2160.8 (±542) ns]  
fromtimestamp      |  1285.4  [1484.7 (±378) ns]  |  2363.6  [3038.6 (±783) ns]  
today              |  4551.7  [4976.6 (±584) ns]  |  5828.9  [6883.9 (±934) ns]  
=================================================================================

For comparision, here is the same script run against master:

Running with N = 10000, k=20
                   |           datetime           |       DateTimeSubclass       
=================================================================================
constructor        |  1335.9  [1350.3 (±24) ns]   |  1406.0  [1426.9 (±15) ns]   
fromordinal        |  1905.0  [1948.3 (±42) ns]   |  2011.5  [2055.4 (±32) ns]   
fromisoformat      |   918.1   [929.3 (±6) ns]    |  2660.8  [2707.7 (±25) ns]   
fromtimestamp      |  1570.1  [1611.8 (±28) ns]   |  1732.1  [1766.3 (±30) ns]   
utcfromtimestamp   |  1202.7  [1234.5 (±18) ns]   |  1334.9  [1371.7 (±26) ns]   
combine            |  2584.9  [2626.3 (±22) ns]   |  2722.8  [2759.8 (±25) ns]   
now                |   1317.2  [1338.2 (±9) ns]   |  1509.7  [1563.7 (±24) ns]   
today              |  5027.1  [5096.9 (±71) ns]   |  5237.6  [5334.9 (±73) ns]   
=================================================================================
                   |             date             |         DateSubclass         
=================================================================================
constructor        |   912.8   [928.8 (±15) ns]   |   996.4   [1003.9 (±7) ns]   
fromordinal        |  1662.9  [1706.4 (±40) ns]   |  1773.6  [1820.7 (±41) ns]   
fromisoformat      |   818.5   [829.0 (±9) ns]    |  1630.0  [1743.3 (±289) ns]  
fromtimestamp      |  2155.2  [2213.4 (±41) ns]   |  2321.9  [2486.9 (±590) ns]  
today              |  5502.0  [5624.6 (±76) ns]   |  5679.8  [5845.4 (±445) ns]  
=================================================================================

https://bugs.python.org/issue32403

pganssle · 2017-12-24T17:51:19Z

Modules/_datetimemodule.c

+    // We have "fast path" constructors for two subclasses: date and datetime
+    if ( (PyTypeObject *)cls == & PyDateTime_DateType ) {
+        result = new_date_ex(year, month, day, (PyTypeObject *)cls);
+    } else if ((PyTypeObject *)cls == & PyDateTime_DateTimeType ) {


This is the part I'm not sure about. It feels weird to put special-case logic for a specific subclass in the base class logic, but it makes the methods that datetime inherits and doesn't re-implement itself (datetime.today and datetime.toordinal) considerably faster when I do this.

Another option might be to do a token implementation of all the date methods specifically calling the new_datetime_subclass_ex, or explicitly calling date.themethod and then using datetime.combine on the result.

I don't see a problem here.

abalkin · 2018-01-09T18:06:33Z

Modules/_datetimemodule.c

+new_date_subclass_ex(int year, int month, int day, PyObject *cls) {
+    PyObject *result;
+    // We have "fast path" constructors for two subclasses: date and datetime
+    if ( (PyTypeObject *)cls == & PyDateTime_DateType ) {


Please try to follow the code style used in the rest of the file. We don't leave space between & and the variable or inside the parentheses.

abalkin · 2018-01-09T18:07:02Z

Modules/_datetimemodule.c

+    // We have "fast path" constructors for two subclasses: date and datetime
+    if ( (PyTypeObject *)cls == & PyDateTime_DateType ) {
+        result = new_date_ex(year, month, day, (PyTypeObject *)cls);
+    } else if ((PyTypeObject *)cls == & PyDateTime_DateTimeType ) {


I don't see a problem here.

bedevere-bot · 2018-01-12T18:02:35Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

pganssle · 2018-01-13T21:45:37Z

@abalkin I believe I fixed the style issues.

the-knights-who-say-ni added the CLA signed label Dec 23, 2017

bedevere-bot added the awaiting review label Dec 23, 2017

pganssle commented Dec 24, 2017

View reviewed changes

abalkin requested changes Jan 12, 2018

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting review labels Jan 12, 2018

abalkin self-assigned this Jan 12, 2018

pganssle force-pushed the faster_datetime_constructors branch from 6c6115c to 92ac0c4 Compare January 13, 2018 17:36

pganssle added 6 commits January 13, 2018 12:37

Add tests for date subclass alternate constructors

5147013

Switch over alternate date constructors to fast path

5879867

Switch datetime constructors to fastpath, fix bpo-32404

2e47897

Add fast path for datetime in date subclass constructor

a64e967

Set fold in constructor in datetime.combine

62b52f6

Add news entries.

b438cbe

pganssle force-pushed the faster_datetime_constructors branch from 92ac0c4 to b438cbe Compare January 13, 2018 17:39

abalkin approved these changes Jan 16, 2018

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting changes labels Jan 16, 2018

abalkin merged commit 9f1b7b9 into python:master Jan 16, 2018

bedevere-bot removed the awaiting merge label Jan 16, 2018

pganssle mentioned this pull request Feb 22, 2018

bpo-10381, bpo-32403: What's new entries for changes to datetime #5814

Merged

pganssle deleted the faster_datetime_constructors branch October 25, 2018 14:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-32403: Faster date and datetime constructors #4993

bpo-32403: Faster date and datetime constructors #4993

pganssle commented Dec 23, 2017 •

edited

Loading

pganssle Dec 24, 2017

abalkin Jan 9, 2018

abalkin Jan 9, 2018

abalkin Jan 9, 2018

bedevere-bot commented Jan 12, 2018

pganssle commented Jan 13, 2018

bpo-32403: Faster date and datetime constructors #4993

bpo-32403: Faster date and datetime constructors #4993

Conversation

pganssle commented Dec 23, 2017 • edited Loading

pganssle Dec 24, 2017

Choose a reason for hiding this comment

abalkin Jan 9, 2018

Choose a reason for hiding this comment

abalkin Jan 9, 2018

Choose a reason for hiding this comment

abalkin Jan 9, 2018

Choose a reason for hiding this comment

bedevere-bot commented Jan 12, 2018

pganssle commented Jan 13, 2018

pganssle commented Dec 23, 2017 •

edited

Loading