Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python] pin pandas-stubs>=2 during pre-commit mypy hook #2854

Merged
merged 2 commits into from
Aug 8, 2024

Conversation

ryan-williams
Copy link
Member

also simplify some if TYPE_CHECKING blocks

Issue and/or context: #2839, #2853

The unpinned pandas-stubs dep in .pre-commit-config.yaml allowed existing local environments (and old cached setup-python environments, which in some cases were being incorrectly used in our GHAs, per actions/setup-python#919) to run the pre-commit mypy hook using pandas-stubs<2, where pd.Series is the type, and pd.Series[Any] requires a # type: ignore[misc] annotation.

With pandas-stubs>=2 (which new environments have been picking up), pd.Series[Any] is the correct type, and the # type: ignore[misc] annotation was flagged as "unused" (example).

Pinning pandas-stubs>=2, and using the corresponding pd.{Series,Index}[Any] annotations when TYPE_CHECKING, should resolve this conflict, in both new and existing environments (in existing environments, pre-commit will recognize the new pandas-stubs>=2 pin, and create a new venv that incorporates it).

also simplify some `if TYPE_CHECKING` blocks

fixes #2839
Copy link

codecov bot commented Aug 8, 2024

Codecov Report

Attention: Patch coverage is 71.42857% with 2 lines in your changes missing coverage. Please review.

Project coverage is 90.19%. Comparing base (274c6f4) to head (d6e68f8).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2854      +/-   ##
==========================================
+ Coverage   90.01%   90.19%   +0.17%     
==========================================
  Files          37       37              
  Lines        3957     3957              
==========================================
+ Hits         3562     3569       +7     
+ Misses        395      388       -7     
Flag Coverage Δ
python 90.19% <71.42%> (+0.17%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
python_api 90.19% <71.42%> (+0.17%) ⬆️
libtiledbsoma ∅ <ø> (∅)

@ryan-williams ryan-williams marked this pull request as ready for review August 8, 2024 15:11
Copy link
Member

@johnkerl johnkerl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

@ryan-williams ryan-williams merged commit 3fae283 into main Aug 8, 2024
13 checks passed
@ryan-williams ryan-williams deleted the rw/mypy-fix branch August 8, 2024 17:22
github-actions bot pushed a commit that referenced this pull request Aug 8, 2024
* pin `pandas-stubs>=2` during pre-commit `mypy` hook

also simplify some `if TYPE_CHECKING` blocks

fixes #2839

* add r-ci.yml exclusions
johnkerl pushed a commit that referenced this pull request Aug 8, 2024
…2855)

* pin `pandas-stubs>=2` during pre-commit `mypy` hook

also simplify some `if TYPE_CHECKING` blocks

fixes #2839

* add r-ci.yml exclusions

Co-authored-by: Ryan Williams <ryan.williams@tiledb.com>
ryan-williams added a commit that referenced this pull request Aug 8, 2024
- '!.pre-commit-config.yaml'
- '!apis/python/**'
- '!docs/**'
- '!LICENSE''
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo here, fix in #2856

ryan-williams added a commit that referenced this pull request Aug 8, 2024
* CI: use `pip` directly (instead of `python -m pip`)

* CI: checkout@v4, setup-python@v5 cache@v4

* disable setup-python cache for lint/pre-commit

See actions/setup-python#919; setup-python cache likely doesn't help much here, as we only `pip install pre-commit`, and cache `~/.cache/pre-commit` separately using `actions/cache`.

* bump to codecov-action@v4

rm coverage/uploader pin added in #2827, per codecov/uploader#1673 (comment)

* `s/MacOS/macOS/

* fix typo from #2854
mojaveazure added a commit that referenced this pull request Aug 9, 2024
* [python] pin `pandas-stubs>=2` during pre-commit `mypy` hook (#2854)

* pin `pandas-stubs>=2` during pre-commit `mypy` hook

also simplify some `if TYPE_CHECKING` blocks

fixes #2839

* add r-ci.yml exclusions

* `actions{cache,checkout,setup-python}` upgrades (#2856)

* CI: use `pip` directly (instead of `python -m pip`)

* CI: checkout@v4, setup-python@v5 cache@v4

* disable setup-python cache for lint/pre-commit

See actions/setup-python#919; setup-python cache likely doesn't help much here, as we only `pip install pre-commit`, and cache `~/.cache/pre-commit` separately using `actions/cache`.

* bump to codecov-action@v4

rm coverage/uploader pin added in #2827, per codecov/uploader#1673 (comment)

* `s/MacOS/macOS/

* fix typo from #2854

* [python] Offer better guidance on attribute names with `.` (#2864)

* [python] Move `_update_column` into pybind11 (#2862)

* [r] Expose `tiledb_timestamp` parameter to `$reopen()`
Allow `$reopen()` to reopen at a particular timestamp; by default, the
timestamp is set to `NULL` to reopen at the curren time. This is needed
for compatibility between libtiledbsoma's timestamp handling and
resume-mode

[SC-52694](https://app.shortcut.com/tiledb-inc/story/52694/allow-reopen-to-take-a-timestamp-for-reopening)

---------

Co-authored-by: Ryan Williams <ryan.williams@tiledb.com>
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
Co-authored-by: nguyenv <vivian@tiledb.com>
eddelbuettel pushed a commit that referenced this pull request Aug 9, 2024
* [python] pin `pandas-stubs>=2` during pre-commit `mypy` hook (#2854)

* pin `pandas-stubs>=2` during pre-commit `mypy` hook

also simplify some `if TYPE_CHECKING` blocks

fixes #2839

* add r-ci.yml exclusions

* `actions{cache,checkout,setup-python}` upgrades (#2856)

* CI: use `pip` directly (instead of `python -m pip`)

* CI: checkout@v4, setup-python@v5 cache@v4

* disable setup-python cache for lint/pre-commit

See actions/setup-python#919; setup-python cache likely doesn't help much here, as we only `pip install pre-commit`, and cache `~/.cache/pre-commit` separately using `actions/cache`.

* bump to codecov-action@v4

rm coverage/uploader pin added in #2827, per codecov/uploader#1673 (comment)

* `s/MacOS/macOS/

* fix typo from #2854

* [python] Offer better guidance on attribute names with `.` (#2864)

* [python] Move `_update_column` into pybind11 (#2862)

* [r] Expose `tiledb_timestamp` parameter to `$reopen()`
Allow `$reopen()` to reopen at a particular timestamp; by default, the
timestamp is set to `NULL` to reopen at the curren time. This is needed
for compatibility between libtiledbsoma's timestamp handling and
resume-mode

[SC-52694](https://app.shortcut.com/tiledb-inc/story/52694/allow-reopen-to-take-a-timestamp-for-reopening)

---------

Co-authored-by: Ryan Williams <ryan.williams@tiledb.com>
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
Co-authored-by: nguyenv <vivian@tiledb.com>
eddelbuettel pushed a commit that referenced this pull request Aug 19, 2024
* [python] pin `pandas-stubs>=2` during pre-commit `mypy` hook (#2854)

* pin `pandas-stubs>=2` during pre-commit `mypy` hook

also simplify some `if TYPE_CHECKING` blocks

fixes #2839

* add r-ci.yml exclusions

* `actions{cache,checkout,setup-python}` upgrades (#2856)

* CI: use `pip` directly (instead of `python -m pip`)

* CI: checkout@v4, setup-python@v5 cache@v4

* disable setup-python cache for lint/pre-commit

See actions/setup-python#919; setup-python cache likely doesn't help much here, as we only `pip install pre-commit`, and cache `~/.cache/pre-commit` separately using `actions/cache`.

* bump to codecov-action@v4

rm coverage/uploader pin added in #2827, per codecov/uploader#1673 (comment)

* `s/MacOS/macOS/

* fix typo from #2854

* [python] Offer better guidance on attribute names with `.` (#2864)

* [python] Move `_update_column` into pybind11 (#2862)

* [r] Expose `tiledb_timestamp` parameter to `$reopen()`
Allow `$reopen()` to reopen at a particular timestamp; by default, the
timestamp is set to `NULL` to reopen at the curren time. This is needed
for compatibility between libtiledbsoma's timestamp handling and
resume-mode

[SC-52694](https://app.shortcut.com/tiledb-inc/story/52694/allow-reopen-to-take-a-timestamp-for-reopening)

---------

Co-authored-by: Ryan Williams <ryan.williams@tiledb.com>
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
Co-authored-by: nguyenv <vivian@tiledb.com>
eddelbuettel pushed a commit that referenced this pull request Aug 22, 2024
* [python] pin `pandas-stubs>=2` during pre-commit `mypy` hook (#2854)

* pin `pandas-stubs>=2` during pre-commit `mypy` hook

also simplify some `if TYPE_CHECKING` blocks

fixes #2839

* add r-ci.yml exclusions

* `actions{cache,checkout,setup-python}` upgrades (#2856)

* CI: use `pip` directly (instead of `python -m pip`)

* CI: checkout@v4, setup-python@v5 cache@v4

* disable setup-python cache for lint/pre-commit

See actions/setup-python#919; setup-python cache likely doesn't help much here, as we only `pip install pre-commit`, and cache `~/.cache/pre-commit` separately using `actions/cache`.

* bump to codecov-action@v4

rm coverage/uploader pin added in #2827, per codecov/uploader#1673 (comment)

* `s/MacOS/macOS/

* fix typo from #2854

* [python] Offer better guidance on attribute names with `.` (#2864)

* [python] Move `_update_column` into pybind11 (#2862)

* [r] Expose `tiledb_timestamp` parameter to `$reopen()`
Allow `$reopen()` to reopen at a particular timestamp; by default, the
timestamp is set to `NULL` to reopen at the curren time. This is needed
for compatibility between libtiledbsoma's timestamp handling and
resume-mode

[SC-52694](https://app.shortcut.com/tiledb-inc/story/52694/allow-reopen-to-take-a-timestamp-for-reopening)

---------

Co-authored-by: Ryan Williams <ryan.williams@tiledb.com>
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
Co-authored-by: nguyenv <vivian@tiledb.com>
mojaveazure added a commit that referenced this pull request Aug 26, 2024
* Timestamp range support for dataframe

* Extended support

* (First half) new timestamp test for SOMADataFrame

* Second half of new timestamp passing all but three reopen tests

* [r] Expose `tiledb_timestamp` parameter to `$reopen()` (#2866)

* [python] pin `pandas-stubs>=2` during pre-commit `mypy` hook (#2854)

* pin `pandas-stubs>=2` during pre-commit `mypy` hook

also simplify some `if TYPE_CHECKING` blocks

fixes #2839

* add r-ci.yml exclusions

* `actions{cache,checkout,setup-python}` upgrades (#2856)

* CI: use `pip` directly (instead of `python -m pip`)

* CI: checkout@v4, setup-python@v5 cache@v4

* disable setup-python cache for lint/pre-commit

See actions/setup-python#919; setup-python cache likely doesn't help much here, as we only `pip install pre-commit`, and cache `~/.cache/pre-commit` separately using `actions/cache`.

* bump to codecov-action@v4

rm coverage/uploader pin added in #2827, per codecov/uploader#1673 (comment)

* `s/MacOS/macOS/

* fix typo from #2854

* [python] Offer better guidance on attribute names with `.` (#2864)

* [python] Move `_update_column` into pybind11 (#2862)

* [r] Expose `tiledb_timestamp` parameter to `$reopen()`
Allow `$reopen()` to reopen at a particular timestamp; by default, the
timestamp is set to `NULL` to reopen at the curren time. This is needed
for compatibility between libtiledbsoma's timestamp handling and
resume-mode

[SC-52694](https://app.shortcut.com/tiledb-inc/story/52694/allow-reopen-to-take-a-timestamp-for-reopening)

---------

Co-authored-by: Ryan Williams <ryan.williams@tiledb.com>
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
Co-authored-by: nguyenv <vivian@tiledb.com>

* [r] [NO BACKPORT] Expose timestamps publicly and plumb through for resume-mode (#2871)

Expose timestamps publicly through a new active binding; replace calls to `private$tiledb_timestamp` with `self$tiledb_timestamp`

Also plumb timestamps through for `write_soma()` in resume-mode

* Timestamp range support for dataframe

* (First half) new timestamp test for SOMADataFrame

* Second half of new timestamp passing all but three reopen tests

* Test can now use factor

* Read and write DataFrame and {Dense,Sparse}Array under timestamps

* Quieter warnings

* Adapt one timestamped test

* Pause one test predicate for collections

* clang-format

as obsessing over one whitespace char before a comment is added value

* Micro-fix following code review

More changes to follow in wider changeset

* Update changelog
Bump develop version

---------

Co-authored-by: Paul Hoffman <mojaveazure@users.noreply.github.com>
Co-authored-by: Ryan Williams <ryan.williams@tiledb.com>
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
Co-authored-by: nguyenv <vivian@tiledb.com>
Co-authored-by: Paul Hoffman <paul.hoffman@tiledb.com>
mojaveazure added a commit that referenced this pull request Aug 26, 2024
* [python] pin `pandas-stubs>=2` during pre-commit `mypy` hook (#2854)

* pin `pandas-stubs>=2` during pre-commit `mypy` hook

also simplify some `if TYPE_CHECKING` blocks

fixes #2839

* add r-ci.yml exclusions

* `actions{cache,checkout,setup-python}` upgrades (#2856)

* CI: use `pip` directly (instead of `python -m pip`)

* CI: checkout@v4, setup-python@v5 cache@v4

* disable setup-python cache for lint/pre-commit

See actions/setup-python#919; setup-python cache likely doesn't help much here, as we only `pip install pre-commit`, and cache `~/.cache/pre-commit` separately using `actions/cache`.

* bump to codecov-action@v4

rm coverage/uploader pin added in #2827, per codecov/uploader#1673 (comment)

* `s/MacOS/macOS/

* fix typo from #2854

* [python] Offer better guidance on attribute names with `.` (#2864)

* [python] Move `_update_column` into pybind11 (#2862)

* [r] Expose `tiledb_timestamp` parameter to `$reopen()`
Allow `$reopen()` to reopen at a particular timestamp; by default, the
timestamp is set to `NULL` to reopen at the curren time. This is needed
for compatibility between libtiledbsoma's timestamp handling and
resume-mode

[SC-52694](https://app.shortcut.com/tiledb-inc/story/52694/allow-reopen-to-take-a-timestamp-for-reopening)

---------

Co-authored-by: Ryan Williams <ryan.williams@tiledb.com>
Co-authored-by: John Kerl <kerl.john.r@gmail.com>
Co-authored-by: nguyenv <vivian@tiledb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants