Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.mean_horizontal() does not work with pl.datetime (but .mean() does) #17237

Open
2 tasks done
Elvynzs opened this issue Jun 27, 2024 · 1 comment
Open
2 tasks done

.mean_horizontal() does not work with pl.datetime (but .mean() does) #17237

Elvynzs opened this issue Jun 27, 2024 · 1 comment
Labels
A-panic Area: code that results in panic exceptions bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars

Comments

@Elvynzs
Copy link

Elvynzs commented Jun 27, 2024

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
import datetime

df = pl.DataFrame({"t1":pl.datetime_range(datetime.datetime(2000,1,1),datetime.datetime(2001,1,1), eager=True),
                  "t2":pl.datetime_range(datetime.datetime(2000,1,1),datetime.datetime(2001,1,1), eager=True) + datetime.timedelta(seconds=3600)})
df.mean() # OK !
df.mean_horizontal() # NOK !

Log output

thread 'polars-2' panicked at crates/polars-core/src/frame/mod.rs:2585:26:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Error message :

PanicException                            Traceback (most recent call last)
Cell In[2], line 7
      4 df = pl.DataFrame({"t1":pl.datetime_range(datetime.datetime(2000,1,1),datetime.datetime(2001,1,1), eager=True),
      5                   "t2":pl.datetime_range(datetime.datetime(2000,1,1),datetime.datetime(2001,1,1), eager=True) + datetime.timedelta(seconds=3600)})
      6 df.mean() # OK !
----> 7 df.mean_horizontal() # NOK !

File C:\...\polars\dataframe\frame.py:9131, in DataFrame.mean_horizontal(self, ignore_nulls)
   9099 def mean_horizontal(self, *, ignore_nulls: bool = True) -> Series:
   9100     """
   9101     Take the mean of all values horizontally across columns.
   9102 
   (...)
   9129     ]
   9130     """
-> 9131     return wrap_s(self._df.mean_horizontal(ignore_nulls)).alias("mean")

PanicException: called `Option::unwrap()` on a `None` value

Issue description

I did not find this exact issue being reported, but it is releated to #13599

Expected behavior

I expected .mean_horizontal to produce similar result to .mean for datetime objects.

Installed versions

--------Version info---------
Polars:               0.20.31
Index type:           UInt32
Platform:             Windows-10-10.0.22000-SP0
Python:               3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:40:50) [MSC v.1937 64 bit (AMD64)]

----Optional dependencies----
adbc_driver_manager:  <not installed>
cloudpickle:          <not installed>
connectorx:           <not installed>
deltalake:            <not installed>
fastexcel:            <not installed>
fsspec:               <not installed>
gevent:               <not installed>
hvplot:               <not installed>
matplotlib:           3.8.4
nest_asyncio:         1.6.0
numpy:                1.26.4
openpyxl:             3.1.2
pandas:               2.2.2
pyarrow:              15.0.2
pydantic:             <not installed>
pyiceberg:            <not installed>
pyxlsb:               <not installed>
sqlalchemy:           <not installed>
torch:                <not installed>
xlsx2csv:             <not installed>
xlsxwriter:           <not installed>
@Elvynzs Elvynzs added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Jun 27, 2024
@stinodego stinodego added the A-panic Area: code that results in panic exceptions label Jun 27, 2024
@DougieHauser
Copy link

Seems like mean_horizontal is using sum_horizontal, which doesn't support addition operations on dates

polars.exceptions.InvalidOperationError: add operation not supported for dtypes `datetime[μs]` and `datetime[μs]

It's possible to add this support by casting the columns to &DataType::Float64 (and back again after applying the mean).
This would make sense when applying mean_horizontal on dates, but not for sum_horizontal since all timestamps start at 1970, and results would seem wrong (for example, adding (2000, 1, 1) + (2000, 1, 1) would result in (2030, 1, 1))

I would be happy to try doing a first time contribution if I get a clear understanding of expected behavior. ℹ️ 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-panic Area: code that results in panic exceptions bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

3 participants