Skip to content

Conversation

@Chiwendaiyue
Copy link
Contributor

This PR improves the robustness of DST handling in resample operations involving timezone-aware DatetimeIndex, especially for timezones that contain DST gaps such as Africa/Cairo. #62601
What this PR does

  1. Fixes NonExistentTimeError triggered by internally generated bin edges
    During resampling, pandas internally constructs bin edges using date_range.
    For certain DST transitions (e.g., Africa/Cairo 2024-04-26), these generated edges may fall into nonexistent local times, producing NonExistentTimeError even when user data does not include such timestamps.

This PR resolves this by:

  • Integrating the UTC fallback path directly into _get_time_bins
  • Automatically retrying range generation in UTC when encountering a nonexistent time
  1. Adds a comprehensive new DST test suite
    A new file pandas/tests/resample/test_dst_handling.py is added.
    It includes:
  • Africa/Cairo DST transition tests
  • Before-DST and after-DST boundary conditions
  • Tests for nonexistent times & shifted times
  • Cross-timezone DST behavior (London, New York, Cairo)
  • Edge cases: NA values, microseconds, single-point, empty index
  • Parametrized frequency tests (2h, 6h, 12h, D)
  • Tests matching original issue reports

These tests are API-driven and avoid relying on internal implementation.

No performance tests are added to pytest; those belong to asv.

@jbrockmendel
Copy link
Member

Did a human write this?

@jbrockmendel jbrockmendel added the AI Slop Suspected of being AI-generated, which is not welcome. label Nov 10, 2025
@Chiwendaiyue
Copy link
Contributor Author

@jbrockmendel Thanks for the feedback. I'm ashamed and you are right.
But I’ve rewritten the code. Please let me know if further adjustments are needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Slop Suspected of being AI-generated, which is not welcome.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants