Repository navigation
Tags: DOI-USGS/dataretrieval-python
Tags
fix(waterdata): send the qualifier filter as a JSON array (#427) The Water Data API parses qualifier as a JSON array, so it responded with HTTP 400 to every value the getters sent (qualifier=ICE, or ICE,ESTIMATED for a list). _get_args now encodes qualifier as one compact JSON array: "ICE" becomes ["ICE"] and a list becomes one array. This also covers qualifier passed to get_peaks through **queryables. A string that starts with "[" is sent unchanged, and an empty list is still dropped from the request. The service matches the whole list, in order: "ICE" does not match an observation qualified ["ESTIMATED", "ICE"]. It responds with 400 to the CQL2 array operators, so there is no "contains" match; the qualifier docstrings now state the exact-match rule. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
feat(wateruse): add water-use module for the USGS NWDC API (#328) Add `dataretrieval.wateruse` for USGS National Water Availability Assessment Data Companion (NWDC) water-use estimates — modeled on a HUC12 grid and queryable by state, county, or hydrologic unit. This is the modern replacement for the defunct legacy NWIS water-use service (`nwis.get_water_use` now points callers here). from dataretrieval import wateruse df, md = wateruse.get_wateruse( model="wu-public-supply-wd", variable=["pswdtot", "pswdgw", "pswdsw"], state="RI", start_date="2020-01", time_resolution="monthly", ) The NWDC is a plain CSV REST service, not an OGC API Features collection, so the module supplies the NWDC-specific pieces (CSV parsing, the RFC 8288 Link-header pagination cursor, the `{detail}` error envelope, and state/county/huc location builders) but reuses the OGC engine's generic transport rather than re-implementing it: the shared pager (`_paginate`), the Jupyter-safe anyio sync bridge (`_run_sync`), response/frame aggregation, and `_default_headers`. It keeps the package conventions where they fit — a `(DataFrame, BaseMetadata)` return, the typed `DataRetrievalError` taxonomy (surfacing the NWDC `detail`), `API_USGS_PAT` token support, idiomatic snake_case params, and `state` / `county` / `huc` selectors that each accept a value or a list (a list fans out one concurrent request per location). Large areas paginate transparently. A `FutureWarning` flags the module as experimental, since the NWDC service is new and still changing. Extracting the reusable engine seams also de-duplicated the engine itself (~-66 LOC, behavior-preserving): `planning._merge_response` now backs both pagination and fan-out aggregation; a generic `utils.Ambient[T]` contextvar-with-scope helper collapses the per-call ambients; and `x-ratelimit-remaining` now reports the lowest value any concurrent sub-request saw (the quota actually left after a fan-out), fixing a latent inaccuracy in the OGC chunker too. Includes offline pytest-httpx coverage, a reference page, a README example, and a demo notebook. Claude-Session: https://claude.ai/code/session_01Sjb14HkwuCydKSKMsaXsgd Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
feat(waterdata): replace per-page logger.info with a single progress … …line (#288) Paginated and chunked Water Data queries display a progress bar that gets updated as data arrives: waterdata · chunk 2/5 · 14 pages · 8,421 rows · 4,870 requests left Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deprecate remaining active nwis functions ahead of 2027-05-06 removal (… …#271) The module-level "use waterdata instead" warning has been firing on import for a while; this PR makes the migration guidance actionable by emitting a per-function DeprecationWarning that names the specific waterdata replacement the user should switch to. Every active nwis function has a waterdata replacement, so all nine of them are deprecated here: nwis.get_dv -> waterdata.get_daily() nwis.get_iv -> waterdata.get_continuous() nwis.get_info -> waterdata.get_monitoring_locations() nwis.what_sites -> waterdata.get_monitoring_locations() nwis.get_stats -> waterdata.get_stats_por() / waterdata.get_stats_date_range() nwis.get_discharge_peaks -> waterdata.get_peaks() nwis.get_ratings -> waterdata.get_ratings() nwis.get_record -> the appropriate waterdata.get_*() nwis.query_waterdata -> a high-level waterdata.get_*() helper nwis.query_waterservices -> a high-level waterdata.get_*() helper (get_qwdata, get_discharge_measurements, get_gwlevels, get_pmcodes, and get_water_use are already defunct and raise NameError.) Implementation follows the nadp deprecation template (#243): a small _REPLACEMENTS dict + a _warn_deprecated(func_name) helper called as the first line of each public function. stacklevel=3 makes the warning point at the caller's code, not the helper's frame. Removal date is set to 2027-05-06. Maintainer can adjust if desired. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Small cleanups: idiomatic Python and a precedence fix (#224) * Small cleanups: idiomatic Python and a precedence fix - waterdata/utils.py: replace runtime `assert` in `_check_ogc_requests` with an explicit `ValueError` (assertions can be disabled with `-O`). - waterdata/utils.py: fix operator-precedence bug in `_format_datetime` where `len==1 and re.search(...) or "/" in datetime_input[0]` would short-circuit and return `datetime_input[0]` for 2-element inputs whose first element contained "/". Parenthesize so both branches require `len==1`. - nwis.py: `"dec_lat_va" in list(df)` -> `in df.columns` (drop needless list construction). - nwis.py: replace index-based `for i in range(len(index_list)-1)` loop with `zip(index_list[:-1], index_list[1:])`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Ruff format --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Make ref table function way simpler and more efficient (#209) * make ref table function way simpler and more efficient * add more documentation, an example * add in the deduplication line * add required packages for gpd.explore in docs and move a requirement to docs section of pyproject.toml * Update dataretrieval/waterdata/api.py Co-authored-by: Joe Zemmels (he/him) <jzemmels@gmail.com> * update notebook to pip install all required packages and fix small change --------- Co-authored-by: Joe Zemmels (he/him) <jzemmels@gmail.com>
PreviousNext