Cast datetime and timedelta to signed 64-bit int #127
Conversation
The `NaT` type is represented as `-0`. As a result, casting to an unsigned integral fails and throws an error. However casting to a signed integral type does not have this problem and proceeds without issues.
Make sure that we are able to encode NaT values without issues by introducing `-0`s in our test data.
|
Hi @jakirkham, this looks good, but there is an issue to do with how the backwards compatibility tests are setup, I'll try to explain. Basically the list of arrays in the test module for each codec can be added to, but existing arrays in that list shouldn't be changed. If they are changed, then they will be out of sync with the fixture data. This won't show up in the tests, as previous test arrays get saved to files, so the backwards compatibility tests run off the previously saved data. In other words, if you change any of the arrays in the list, then the corresponding fixture data should also be deleted and regenerated. You can find the data to delete because the order in which the arrays appear in the list is used to generate a file name. E.g., within the test_zlib module, the first array in the list gets saved as fixture/zlib/array.00.npy, and the corresponding encoded data from the first codec in the codecs list gets saved as fixture/zlib/codec.00/encoded.00.dat. I.e., if you changed the first array in the list, then you should delete Alternatively and probably better in this case, leave all the existing arrays as-is, and append some new arrays to the list, in which case some new fixture data will get generated on the first test run. Hope that makes sense! |
This restores the old datetime/timedelta tests in their same position in the parameterized compression tests. Should ensure their fixtures stay the same. Also should make sure we haven't accidentally broken backwards compatibility by including the original tests. The new tests with NaT values are included afterwards. This will make sure that new fixture data is generated for them.
Commits the fixture data automatically generated by pytest for the new datetime/timedelta NaT tests.
|
Thanks @alimanfoo. IIUC you would like the tests added as a new set in the parameterized group. If so that makes sense to me, sorry I didn't think about it earlier. Have added the |
| @@ -47,6 +47,10 @@ | |||
| np.random.randint(0, 2**60, size=1000, dtype='u8').view('m8[ns]'), | |||
| np.random.randint(0, 2**25, size=1000, dtype='u8').view('M8[m]'), | |||
| np.random.randint(0, 2**25, size=1000, dtype='u8').view('m8[m]'), | |||
| (-np.random.randint(-2**30, 2**30, size=1000, dtype='i8')).view('M8[ns]'), | |||
There was a problem hiding this comment.
Out of interest, if this is random numbers between -2**30 and 2**30, why do we need to take the negative of the array? Is it not enough to specify the limits within randint?
There was a problem hiding this comment.
You're right. This wasn't correct.
Basically NaT in float64 is the same as -0.0. However int64 lacks a -0 equivalent, but -2**63 maps to the same value, which is what we want to include.
PR ( #131 ) fixes this.
Related to issue ( zarr-developers/zarr-python#342 )
The
NaTtype is represented as-0. As a result, casting to an unsigned integral fails and throws an error. However casting to a signed integral type does not have this problem and proceeds without issues.TODO:
tox -e py37passes locallytox -e py27passes locallytox -e docspasses locally