ENH: support writing datetime columns in write_dataframe

[theroggy]

Reading datetime columns seems to work, but when they are written again the following error is thrown: Exception has occurred: NotImplementedError field type is not supported datetime64[ns].

I just saw it is mentioned as a todo in the readme, but I suppose it is useful to have an issue for it as well?

[theroggy]

Some extra info...

I just noticed that geopandas.to_file() doesn't support writing pandas datetime64 columns either, or more precisely: not to shapefiles.

The problem of not being able to write datetime64's is more obvious in pyogrio because pyorgio.read_dataframe() reads it to a datetime64 in de GeoDataFrame, and is then unable to write it again.
In geopandas.read_file() a datetime column is read to an "object" column in the GeoDataFrame instead of in a datetime64, and so it doesn't have problems writing it to file again.

Edit: apparently shapefile as a file format simply doesn't support datetime columns. There is only a date type. So, if you want to store a datetime that includes time info, it needs to be stored as a string, or you need to somehow split it in two columns (date versus time part).
Source: https://desktop.arcgis.com/en/arcmap/latest/manage-data/shapefiles/geoprocessing-considerations-for-shapefile-output.htm#GUID-A10ADA3B-0988-4AB1-9EBA-AD704F77B4A2

[jorisvandenbossche]

I just noticed that geopandas.to_file() doesn't support writing pandas datetime64 columns either, or more precisely: not to shapefiles.

I think that's a specific limitation of Shapefiles.

For example, writing to GeoPackage and reading back in seems to work:

In [2]: gdf = geopandas.GeoDataFrame({'datetime': [pd.Timestamp("2012-01-01 09:00:00")]}, geometry=geopandas.points_from_xy([0], [0]))

In [3]: gdf
Out[3]: 
             datetime                 geometry
0 2012-01-01 09:00:00  POINT (0.00000 0.00000)

In [4]: gdf.dtypes
Out[4]: 
datetime    datetime64[ns]
geometry          geometry
dtype: object

In [5]: gdf.to_file("test_datetime.gpkg")

In [6]: geopandas.read_file("test_datetime.gpkg")
Out[6]: 
             datetime                 geometry
0 2012-01-01 09:00:00  POINT (0.00000 0.00000)

In [7]: geopandas.read_file("test_datetime.gpkg").dtypes
Out[7]: 
datetime    datetime64[ns]
geometry          geometry
dtype: object

Writing to Shapefile gives a driver specific error:

In [8]: gdf.to_file("test_datetime.shp")
...

~/miniconda3/envs/geo-dev/lib/python3.9/site-packages/fiona/collection.py in _check_schema_driver_support(self)
    428                     raise DriverSupportError("GDAL 1.x GPKG driver does not support datetime fields")
    429                 else:
--> 430                     raise DriverSupportError("{driver} does not support {field_type} "
    431                                              "fields".format(driver=self.driver,
    432                                                              field_type=field_type))

DriverSupportError: ESRI Shapefile does not support datetime fields

(now, I am not super familiar with ESRI Shapefile, this seems to be hardcoded in Fiona)

[theroggy]

jorisvandenbossche: now, I am not super familiar with ESRI Shapefile

Good idea, keep it that way, it's a horrible file format ;-).

[davidnfox]

Using the geopandas syntax I can successfully write datetime64[ns] fields, but using the pyogrio syntax I get an error:

Using joris's test code to build the geodataframe ( I see the same behavior testing FlatGeobuf):

pyogrio.write_dataframe(gdf, "test.gpkg", DRIVER="GPKG")

NotImplementedError Traceback (most recent call last)
Input In [10], in <cell line: 1>()
----> 1 pyogrio.write_dataframe(gdf, "test.gpkg", DRIVER="GPKG")

......
File pyogrio_io.pyx:1147, in pyogrio._io.ogr_write()

File pyogrio_io.pyx:997, in pyogrio._io.infer_field_types()

NotImplementedError: field type is not supported datetime64[ns] (field index: 0)