KeyError '60min' when fetching data

[hjaarnio]

parse_prices tries to access series dict with soup_series.index.freqstr

series = {
        '15T': [],
        '30T': [],
        '60T': []
    }
    for soup in _extract_timeseries(xml_text):
        soup_series = _parse_timeseries_generic(soup, 'price.amount')
        series[soup_series.index.freqstr].append(soup_series)

The string gets set to '60min', and not '60T', which results in a KeyError. The index is set in _parse_datetimeindex:

delta = _resolution_to_timedelta(res_text=soup.find('resolution').text)
index = pd.date_range(start=start, end=end, freq=delta, inclusive='left')

Here's the stacktrace for completeness:

    prices = client.query_day_ahead_prices(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/entsoe/decorators.py", line 122, in year_wrapper
    frame = func(*args, start=_start, end=_end, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/entsoe/entsoe.py", line 1186, in query_day_ahead_prices
    series = parse_prices(text)[resolution]
             ^^^^^^^^^^^^^^^^^^
  File "site-packages/entsoe/parsers.py", line 38, in parse_prices
    series[soup_series.index.freqstr].append(soup_series)
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '60min'

[fboerman]

Hi! That is very strange. Could you please provide a snippet which call you tried to make and make sure you are running the latest version? I do not provide support for older versions.

…

-------- Original Message --------

On 25 Jan 2024, 16:49, Henrik Aarnio wrote: parse_prices tries to access series dict with soup_series.index.freqstr series = { '15T': [], '30T': [], '60T': [] } for soup in _extract_timeseries(xml_text): soup_series = _parse_timeseries_generic(soup, 'price.amount') series[soup_series.index.freqstr].append(soup_series) The string gets set to '60min', and not '60T', which results in a KeyError. The index is set in _parse_datetimeindex: delta = _resolution_to_timedelta(res_text=soup.find('resolution').text) index = pd.date_range(start=start, end=end, freq=delta, inclusive='left') Here's the stacktrace for completeness: prices = client.query_day_ahead_prices( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "site-packages/entsoe/decorators.py", line 122, in year_wrapper frame = func(*args, start=_start, end=_end, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "site-packages/entsoe/entsoe.py", line 1186, in query_day_ahead_prices series = parse_prices(text)[resolution] ^^^^^^^^^^^^^^^^^^ File "site-packages/entsoe/parsers.py", line 38, in parse_prices series[soup_series.index.freqstr].append(soup_series) ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^ KeyError: '60min' — Reply to this email directly, [view it on GitHub](#295), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ABEC63ROBIT5FTDH7HKSQIDYQJ5JJAVCNFSM6AAAAABCKX7RZSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEYDANRXGU2TOMI). You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[hjaarnio]

Here is a minimal example that replicates it. It happens on all versions newer than 0.5.7 (since the dict lookup was introduced in 0.5.8)

from pandas import Timestamp
from entsoe import EntsoePandasClient
import datetime
import pytz

now = datetime.datetime.now().astimezone(pytz.utc)

beginning = now.replace(hour=0, minute=0, second=0, microsecond=0)
end = beginning + datetime.timedelta(days=1)

client = EntsoePandasClient(api_key=apikey)
prices = client.query_day_ahead_prices("FI", start=Timestamp(beginning), end=Timestamp(end))

Traceback with version 0.6.2:

Traceback (most recent call last):
  File "test_script.py", line 14, in <module>
    prices = client.query_day_ahead_prices("FI", start=Timestamp(beginning), end=Timestamp(end))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv/lib/python3.12/site-packages/entsoe/decorators.py", line 122, in year_wrapper
    frame = func(*args, start=_start, end=_end, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv/lib/python3.12/site-packages/entsoe/entsoe.py", line 1186, in query_day_ahead_prices
    series = parse_prices(text)[resolution]
             ^^^^^^^^^^^^^^^^^^
  File "venv/lib/python3.12/site-packages/entsoe/parsers.py", line 38, in parse_prices
    series[soup_series.index.freqstr].append(soup_series)
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '60min'

[hjaarnio]

Okay, found the relevant information: This happens on pandas==2.2.0 (released a week ago), but not on the previous pandas==2.1.4

[fboerman]

hi @hjaarnio that is indeed the key issue here. thanks for pointing that out that really helps. I would like to stay compatible with older pandas versions but that seems kinda difficult without completely rewriting the parse prices function. I have there for decided to require pandas >=2.2.0 for now. thanks for reporting and investigating!
I will release a new version today which fixes this.