`nhsn` source does not respect `as_of` (and incorrect documentation?)

[damonbayer]

As an extreme example, here I request an as of date of Jan 1, 1800. I would expect this to produce no results, but I get results with an issue date and time_values from a few weeks ago. The results are the same if I use an as_of date from the recent past, or even the future.

library(epidatr)
library(tidyverse)
library(waldo)

nhsn_1800 <- pub_covidcast(
  source = "nhsn",
  signals = "confirmed_admissions_covid_ew_prelim",
  geo_type = "nation",
  time_type = "week",
  as_of = as_date("1800-01-01"))

nhsn_1800 |> distinct(issue)
#> # A tibble: 1 × 1
#>   issue     
#>   <date>    
#> 1 2025-06-15
nhsn_1800 |> summarize(max(time_value))
#> # A tibble: 1 × 1
#>   `max(time_value)`
#>   <date>           
#> 1 2025-06-08

nhsn_2025 <- pub_covidcast(
  source = "nhsn",
  signals = "confirmed_admissions_covid_ew_prelim",
  geo_type = "nation",
  time_type = "week",
  as_of = as_date("2025-01-01"))

nhsn_2050 <- pub_covidcast(
  source = "nhsn",
  signals = "confirmed_admissions_covid_ew_prelim",
  geo_type = "nation",
  time_type = "week",
  as_of = as_date("2050-01-01"))

compare(nhsn_1800, nhsn_2025)
#> ✔ No differences
compare(nhsn_1800, nhsn_2050)
#> ✔ No differences

^{Created on 2025-06-23 with reprex v2.1.1}

Per the documentation, the nhsn source has

Earliest issue available: November 19, 2024
Number of data revisions since 18 Nov 2024: 0
Date of last change: Never

This also does not agree with experience. It seems the data was updated as recently as two weeks ago.

[dshemetov]

Hi thanks for the note! This part of our API is a bit sharp: this particular signal's time values are stored in epiweek format YYYYWW, so as_of should be in that format:

r$> pub_covidcast(
          source = "nhsn",
          signals = "confirmed_admissions_covid_ew_prelim",
          geo_type = "nation",
          time_type = "week",
          as_of = "180001")
# A tibble: 0 × 0
Warning message:
epidata warning: `no results` 

r$> pub_covidcast(
          source = "nhsn",
          signals = "confirmed_admissions_covid_ew_prelim",
          geo_type = "nation",
          time_type = "week",
          as_of = "202501")
# A tibble: 230 × 15
   geo_value signal                               source geo_type time_type time_value direction issue        lag missing_value missing_stderr missing_sample_size value stderr sample_size
   <chr>     <chr>                                <chr>  <fct>    <fct>     <date>         <dbl> <date>     <dbl>         <dbl>          <dbl>               <dbl> <dbl>  <dbl>       <dbl>
 1 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-08-02        NA 2024-12-29   230             0              5                   5 39044     NA          NA
 2 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-08-09        NA 2024-12-29   229             0              5                   5 36630     NA          NA
 3 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-08-16        NA 2024-12-29   228             0              5                   5 36884     NA          NA
 4 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-08-23        NA 2024-12-29   227             0              5                   5 35479     NA          NA
 5 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-08-30        NA 2024-12-29   226             0              5                   5 32862     NA          NA
 6 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-09-06        NA 2024-12-29   225             0              5                   5 29806     NA          NA
 7 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-09-13        NA 2024-12-29   224             0              5                   5 27753     NA          NA
 8 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-09-20        NA 2024-12-29   223             0              5                   5 27983     NA          NA
 9 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-09-27        NA 2024-12-29   222             0              5                   5 31764     NA          NA
10 us        confirmed_admissions_covid_ew_prelim nhsn   nation   week      2020-10-04        NA 2024-12-29   221             0              5                   5 37341     NA          NA
# ℹ 220 more rows
# ℹ Use `print(n = ...)` to see more rows

r$> pub_covidcast(
          source = "nhsn",
          signals = "confirmed_admissions_covid_ew_prelim",
          geo_type = "nation",
          time_type = "week",
          as_of = "202501") %>% summarize(max(time_value))
# A tibble: 1 × 1
  `max(time_value)`
  <date>           
1 2024-12-22       

I can expand a bit more, if you'd like, but figured I'd try to get you an initial response quickly.

[damonbayer]

Thanks @dshemetov! So my as_date("1800-01-01")) was getting interpreted as epiweek 01 from the year 1800101 or something like that?

[dshemetov]

It's actually getting passed through directly as 18000101 to the API by the epidatr client. We haven't hard-coded these source-by-source date validity checks into the R client or the API. The absence of a guard rail here is something that's on our radar and we'd like to warn/error the user, but for now our efforts are focused on a big overhaul of our backend systems.

[dshemetov]

Also you can inspect the formed API call like this, it can sometimes help debug:

r$> pub_covidcast(
          source = "nhsn",
          signals = "confirmed_admissions_covid_ew_prelim",
          geo_type = "nation",
          time_type = "week",
          as_of = "202501", fetch_args = fetch_args_list(dry_run=TRUE))

── <epidata_call> object: ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
• Pipe this object into `fetch()` to actually fetch the data
• Request URL: https://api.delphi.cmu.edu/epidata/covidcast/?data_source=nhsn&signals=confirmed_admissions_covid_ew_prelim&geo_type=nation&time_type=week&geo_values=%2A&time_values=%2A&as_of=202501