Upcoming changes in time series tables #1250
Comments
CSSEGISandData
added
the
documentation
|
Will recovered cases still be reported on the daily CSV files? Will they reflect the daily recovered or aggregated? |
|
@DataChant No recovered cases will be reported in the daily reports and the time series tables. Update: we newly added recovered time series table for most countries. Thanks! |
|
Woah, major news. Let’s do this. Bummed about no recovered but seems to be difficult to collect. County level data is going to be massive. Thank you |
|
Thanks for your work. I'd like to know why you won't report or provide recovered cases. |
|
No reliable data source reporting recovered cases for many countries, such as the US. |
|
Can you please provide us a date/time for that cutover? Thank you, |
|
Thanks so much! I'm making a Power BI Report now, so it's good to know about these upcoming changes! |
|
Thanks @CSSEGISandData Changes look good - thanks for all the hard work - this is a very important data set! |
|
How do you count actice cases without having recovered available? |
|
You don’t, just confirmed, deaths, and testing. |
I'm just grouping the difference together into a group called "Active or Recovered". Like @paolinic03 said , it's the best we can do for the moment. |
|
THANK YOU!!! :) |
|
Will there be a release for those mentioned tables today? I don't see US tables yet. |
|
Thank you for this. This really is an amazing resource, and I'm excited for these changes. I recommend pinning this issue so that folks don't miss it. https://help.github.com/en/github/managing-your-work-on-github/pinning-an-issue-to-your-repository |
|
I cannot find |
|
I would also like to have access to this data.
Have you published it?
Atentamente / Best Regards
-------------------------------------------------
Oscar Rodas
*"Life's too short, stop fooling around."*
…On Tue, Apr 7, 2020 at 8:52 AM Alvaro Gil ***@***.***> wrote:
I cannot find time_series_covid19_testing_global can you point me out
where is that published? Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1250 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKAGX3JFC3IDZ2S2FCZMI3RLM42ZANCNFSM4LRMVWIQ>
.
|
|
In case anyone needs it, I have written some scripts that append the US data to the global files using the same format as the global file. I commit the concatenated files up to my github account once per day. https://github.com/jcampos8782/COVID19-API/tree/master/init/data/processed/covid19 This data is part of a bigger project which puts this data set into MongoDB and enables geolocation lookups of the series data. In other words, given a latitude and longitude, you can resolve regions associated with the geolcation and then find series data for those regions. The location data is from Google's APIs and falls back on the names provided in the data files if the region names cannot be resolved for the coordinates (for example, the Grand Princess data row). The documentation for the mongo database is here: There are also REST APIs using SpringBoot in this project and a React/Redux UI. If any particular part of this project is useful to you, feel free to fork or contribute. Thanks! |
|
Has anyone come across time series data for individual Mexico states by any chance? Looking for something similar to what we have here for US states. |
|
Very interesting 🤔 |
CSSEGISandData/COVID-19#1250 (comment) Remove disclaimer and whole oldrecovered data mess only disable recovered and sick for Canada and USA scopes with missing source
|
I have added Mexico state-level data to my data sets from some scraping. I can scrap daily to get data from this point forward, but I have no historical data. If anyone comes across it, could you please point me in the direction? https://github.com/jcampos8782/COVID19-API/tree/master/init/mongo-init/data/processed/covid19 |
|
@jcampos8782 check out https://github.com/open-covid-19/data. We data data all the way back to the first reported case. |
|
@owahltinez Thank you! I appreciate your work. It looks like you are using https://github.com/carranco-sga/Mexico-COVID-19 as the canonical source for Mexico so I am going to go with that. I've got to give credit where credit is due. |
|
@jcampos8782 I'm not trying to take any credit for the source, the Open COVID-19 dataset is not the canonical source of any of the data -- and neither is the repo you linked, canonical data comes from the local authorities. The main purpose of my repo is to provide a consistent dataset automatically populated with a source of data as close as possible to the official, local authority. |
|
@owahltinez I didn't mean to make it seem like you were. Sorry if it sounded like that. Thanks for your work! I just wanted to link to the data set you were deriving yours from to give him some credit. I realize that none of this is canonical data and most of its from screen scrapes from all over the web. Like you said, just trying to get as close as possible. |
Any update on this? |
Good decision, not sure about anywhere else in the world, but Australia / QLD was only tested travellers, or those showing severe respiratory. Many have been sent home untested, hence they may have recovered and there's no data to say they have. Our state is now moving to testing non-travelling with fevers or contact with others. While the health systems are mitigating loads, recovered data is a feel good factor and cant be gathered completely |
|
Is there any plan for GSSE to include counts of cases/comorbidities/hospitalisations/deaths by age-cohort? There is a very large change in age-group CFR - so much so that's it's criminal to report an age-agnostic CFR: it understates risk for over-70s by a factor of 4, and overstates risk for under50s by two orders of magnitude. Any metric with that characteristic is not worth having At the moment very few countries are making age-cohort data readily available: (in the US, age-cohort deaths and hospitalisations are available with a long lag; age-cohort cases are somewhat easier; age-cohort case-comorbidities data are like hen's teeth. NYC's API makes all of it available daily, so it's obviously doable. Hospitalisations by age-cohort are in CDC's COIVD-NET database - they are presented on a webpage - but the mechanism to download is like someone copied a udemy GUI assignment (clunky and not readily amenable to algorithmic download). There's an API endpoint at gis.cdc.gov/grasp/covid19_3_api but it's undocumented (easy enough to work out, tho). In Australia cases by age-cohort are updated daily, but put in a repository that's hard to find, in a way that's moderately difficult to scrape: another 'HelloWorld' effort (got it done tho: python requests FTW). The UK appears committed to not producing it (Oxford actually used Chinese age-cohort data to do a study in mid-March: why not their own?) Iceland has a good API. GSSE's efforts thus far seem more about giving HelloWorld coders a change to put coloured dots on maps, and to scare people who don't understand the irrelevance of CFR when 'F' has an inherently different probability structure in a priori identifiable cohorts of the population. Italian data look benign if you get age-cohort data - that's how important age-cohort data is are. |
|
Thank you for this great update!
Atentamente / Best Regards
-------------------------------------------------
Oscar Rodas
*"Life's too short, stop fooling around."*
…On Sat, Apr 11, 2020 at 6:33 PM Kratoklastes ***@***.***> wrote:
Is there any plan for GSSE to include counts of
cases/comorbidities/hospitalisations/deaths by age-cohort?
There is a very large change in age-group CFR - so much so that's it's
criminal to report an age-agnostic CFR: it *understates* risk for
over-70s by a factor of 4, and *overstates* risk for under50s by *two
orders of magnitude*. Any metric with that characteristic is not worth
having
At the moment very few countries are making age-cohort data readily
available: (in the US, age-cohort deaths and hospitalisations are available
with a long lag; age-cohort *cases* are somewhat easier; age-cohort
case-comorbidities data are like hen's teeth. NYC's API makes all of it
available *daily*, so it's obviously doable.
In Australia cases by age-cohort are updated daily, but put in a
repository that's hard to find, in a way that's moderately difficult to
scrape (got it done tho: python requests FTW). Iceland has a good API. The
UK appears committed to not producing it (Oxford actually used Chinese
age-cohort data to do a study in mid-March: why not their own?)
GSSE's efforts thus far seem more about giving HelloWorld coders a change
to put coloured dots on maps, and to scare people who don't understand the
irrelevance of CFR when 'F' has an inherently different probability
structure in *a priori* identifiable cohorts of the population.
Italian data look benign if you get age-cohort data - that's how important
age structures are.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1250 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKAGX75P5TUVJDLHHORCWTRMED5PANCNFSM4LRMVWIQ>
.
|
|
Maybe there is an error in the dataset of Germany. The number of deaths decreased today in the datafile „time_series_covid19_deaths_global.csv“. Please correct the numbers. |
|
What's the ETA on adding the ISO codes for time_series_covid19_deaths_global and time_series_covid19_confirmed_global? It would make things much easier to compare to maps rather than the Province/State Country/Region. I've also tried the lat and long as a way to get the ISO from the lookup table that you provide, but the numbers don't exactly match for a given country. Thanks for all of your hard work, this is amazing! |
|
@CaptainChemist -- as I've written a few weeks ago on this same issue, thus sorry for repeating this -- I have derived and augmented the JHU, NY Times and ECDC datasets, and among other augmentations I've also included the ISO country codes for all three datasets (and their variants). I have described this in #1281 and it is available at https://github.com/cipriancraciun/covid19-datasets Thus you could use these derived datasets until JHU does the updates you require. Moreover, you can also use and compare the ECDC dataset against the JHU one. |
|
The latest (cumulative) on 14-Apr-2020 Confirmed value for France is lower than the previous day value, typo? |
|
Worldometer marks: 143,303
Atentamente / Best Regards
-------------------------------------------------
Oscar Rodas
*"Life's too short, stop fooling around."*
…On Wed, Apr 15, 2020 at 1:03 AM Sound Spinning ***@***.***> wrote:
The latest (cumulative) on 14-Apr-2020 Confirmed value for France is lower
than the previous day value, typo?
[image: image]
<https://user-images.githubusercontent.com/12704331/79307897-8c0a7300-7eef-11ea-881a-d5d7932062c3.png>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1250 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKAGX6UVMWG6SSRM27D2L3RMVL4ZANCNFSM4LRMVWIQ>
.
|
I'm referring to this github database we are dealing with here. If you go to their live map app, and click on France see below the graph showing the lower latest point. Where did you get your value from? |
|
@SoundSpinning probably they have to adjust counters, for example in my country they adapted the number once they realized that you can't count total positive tests as total confirmed cases since some people have been tested twice. Not saying that the same is happening with France but is a possibility. |
Anything is possible since most countries are counting cases differently, and with non-consistent methods. Good point though, it could just be an adjust made on the last entry alone. Just odd to see a cumulative value lower than a previous one, but not that important. |
and others
We will update the time series tables in the following days, aiming to provide a cleaner and more organized dataset consistent with our new/current naming convention. We will also be reporting a new variable (i.e, testing), as well as data at the county level for the US. All files will continue to be updated daily around 11:59PM UTC.
The followiing specific changes will be made:
Three new time series tables will be added for the US. The first two will be the confirmed cases and deaths, reported at the county level. The third, number of tests conducted, will be reported at the state level. These new tables will be named
time_series_covid19_confirmed_US.csv,time_series_covid19_deaths_US.csv,time_series_covid19_testing_US.csv, respectively.Changes to the current time series include the removal of the US state and county-level entries, which will be replaced with a new single country level entry for the US. The tables will be renamed
time_series_covid19_confirmed_global.csvandtime_series_covid19_deaths_global.csv, andtime_series_covid19_testing_global.csv, respectively.The ISO code will be added in the global time series tables.
The FIPS code will be added in the new US time series tables.
We will no longer provide recovered cases.
The current set of time series files will be moved to our archive folder, and the new files will be added to the current folder.
Thanks!
Update:
time_series_covid19_recovered_global.csvis added.The text was updated successfully, but these errors were encountered: