Skip to content

Conversation

@joscha
Copy link
Contributor

@joscha joscha commented May 13, 2025

Tell us what you do here

  • implementing verified source (please link a relevant issue labeled as verified source)
  • fixing a bug (please link a relevant bug report)
  • improving, documenting, or customizing an existing source (please link an issue or describe below)
  • anything else (please link an issue or describe below)

Short description

Wed, 12 Jan 2022 13:08:52 GMT is a valid RFC 2822 date, however pendulum runs into problems with it, due to the GMT only portion:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/pendulum/parser.py", line 36, in parse
    return _parse(text, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/pendulum/parser.py", line 54, in _parse
    return pendulum.datetime(
           ^^^^^^^^^^^^^^^^^^
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/pendulum/__init__.py", line 140, in datetime
    return DateTime.create(
           ^^^^^^^^^^^^^^^^
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/pendulum/datetime.py", line 103, in create
    tz = pendulum._safe_timezone(tz)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/pendulum/__init__.py", line 109, in _safe_timezone
    elif obj.tzname(None) == "UTC":
         ^^^^^^^^^^^^^^^^
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/dateutil/tz/tz.py", line 238, in tzname
    return self._tznames[self._isdst(dt)]
                         ^^^^^^^^^^^^^^^
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/dateutil/tz/tz.py", line 291, in _isdst
    dstval = self._naive_is_dst(dt)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/dateutil/tz/tz.py", line 259, in _naive_is_dst
    timestamp = _datetime_to_timestamp(dt)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joscha/dev/dagster-gmail-extractor/.devenv/state/venv/lib/python3.12/site-packages/dateutil/tz/tz.py", line 1814, in _datetime_to_timestamp
    return (dt.replace(tzinfo=None) - EPOCH).total_seconds()
            ^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'replace'

The fix is to use email.utils.parsedate_to_datetime() to parse RFC 2822-style email date headers. It’s designed for exactly this case. We then upgrade the object to a pendulum instance after.

Copy link
Contributor

@rudolfix rudolfix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! thanks!

@rudolfix rudolfix added ci from fork Allows to run tests from PR coming from fork and removed ci from fork Allows to run tests from PR coming from fork labels May 19, 2025
@rudolfix rudolfix merged commit 3e27a86 into dlt-hub:master May 19, 2025
12 of 13 checks passed
@joscha joscha deleted the patch-3 branch May 19, 2025 20:23
evidy-bot added a commit to evidycz/verified-sources that referenced this pull request Jun 2, 2025
* fix: tests import exception from correct location (dlt-hub#613)

* exception moved

* bump dltpure dependency version

* Fix wrong initialization command on Stripe documentation (dlt-hub#618)

* docs(inbox): GMail concurrency (dlt-hub#615)

* fix(inbox): senders filter is optional (dlt-hub#614)

* fixes forks test action

* fix(inbox): RFC 2822 dates parsing (dlt-hub#616)

* update pipedrive source with projects and tasks (dlt-hub#612)

* docs(inbox): remove mention of unused dependencies (dlt-hub#617)

* change contribution guide (dlt-hub#620)

* update contributing docs

* bumps uv lock

* fixes some tests

* Update CONTRIBUTING.md

Co-authored-by: anuunchin <88698977+anuunchin@users.noreply.github.com>

---------

Co-authored-by: anuunchin <88698977+anuunchin@users.noreply.github.com>

---------

Co-authored-by: djudjuu <julius@dlthub.com>
Co-authored-by: Pedro <pdfrod@users.noreply.github.com>
Co-authored-by: Joscha Feth <joscha@feth.com>
Co-authored-by: Marcin Rudolf <rudolfix@rudolfix.org>
Co-authored-by: diwu-sf <di.wu@shadowfaxdata.com>
Co-authored-by: anuunchin <88698977+anuunchin@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci from fork Allows to run tests from PR coming from fork

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants