Skip to content

Commit

Permalink
Merge branch 'main' into dev
Browse files Browse the repository at this point in the history
  • Loading branch information
@hswerdfe
hswerdfe authored Mar 15, 2021
2 parents f74a25a + 7056a14 commit 524e4d3
Show file tree
Hide file tree
Showing 3 changed files with 112 additions and 9 deletions.
12 changes: 12 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ See [license](LICENSE) for more information.

## Changelog


#### 2021-02-18

**v1.1.0**
Expand Down Expand Up @@ -107,6 +108,17 @@ Addresses issues: #59, #84, #90, #92, #93, #96, #97, #99, #101, #102, #103, #104
- SQL template updated to reflect v1.1.0 (and also v1.0.0). These files are now automatically generated from the metadata tables (above). The SQL tables are in SQLite format.
- Several small grammatical errors corrected in the English variable descriptions.

#### 2021-02-15

Metadata migrated to:

- `Tables.csv`: List of all tables.
- `Variables.csv`: List of all variables.
- `VariableCategory.csv`: List of all cteogies within variables.

Modify these tables for future additions and changes. `metadata.md` is automatically generated from `metadata_template.md`.


#### 2021-01-26

**v1.0.0 - Many additions and breaking changes. This version is recommended for widespread use.**
Expand Down
64 changes: 55 additions & 9 deletions data/wastewater_virus.csv
View file
Original file line number Diff line number Diff line change
Expand Up @@ -172,12 +172,58 @@ sampleDate,siteID,siteName,reportDate,covidN1_PMMV_meanNormal,covidN1_PMMV_SDNor
2021-01-15,Ottawa-1,Ottawa-ROPEC,2021-01-18,0.00051,0.00006,0.00065,0.0001,26.974
2021-01-16,Ottawa-1,Ottawa-ROPEC,2021-01-18,0.0006,0.00018,0.00061,0.00002,27.801
2021-01-17,Ottawa-1,Ottawa-ROPEC,2021-01-18,0.00077,0.00023,0.00086,0.00027,27.987
2021-01-18,Ottawa-1,Ottawa-ROPEC,2020-01-20,0.00036,0.00003,0.00018,0.00007,27.43
2021-01-19,Ottawa-1,Ottawa-ROPEC,2020-01-21,0.00034,0.0001,0.00058,0.00012,27.212
2021-01-20,Ottawa-1,Ottawa-ROPEC,2020-01-22,0.0007,0.00024,0.00116,0.00031,29.055
2021-01-21,Ottawa-1,Ottawa-ROPEC,2020-01-26,0.00044,0.00007,0.00025,0.00006,28.363
2021-01-22,Ottawa-1,Ottawa-ROPEC,2020-01-26,0.00058,0.00016,0.00038,0.00009,28.26
2021-01-23,Ottawa-1,Ottawa-ROPEC,2020-01-26,0.00033,0.00007,0.00021,0.00008,28.153
2021-01-24,Ottawa-1,Ottawa-ROPEC,2020-01-26,0.00029,0.00002,0.00015,0.00007,27.472
2021-01-25,Ottawa-1,Ottawa-ROPEC,2020-01-27,0.00019,0.00004,0.00074,0.00011,26.129
2021-01-26,Ottawa-1,Ottawa-ROPEC,2020-01-28,0.00027,0.0001,0.0002,0.00634,27.161
2021-01-18,Ottawa-1,Ottawa-ROPEC,2021-01-20,0.00036,0.00003,0.00018,0.00007,27.43
2021-01-19,Ottawa-1,Ottawa-ROPEC,2021-01-21,0.00034,0.0001,0.00058,0.00012,27.212
2021-01-20,Ottawa-1,Ottawa-ROPEC,2021-01-22,0.0007,0.00024,0.00116,0.00031,29.055
2021-01-21,Ottawa-1,Ottawa-ROPEC,2021-01-26,0.00044,0.00007,0.00025,0.00006,28.363
2021-01-22,Ottawa-1,Ottawa-ROPEC,2021-01-26,0.00058,0.00016,0.00038,0.00009,28.26
2021-01-23,Ottawa-1,Ottawa-ROPEC,2021-01-26,0.00033,0.00007,0.00021,0.00008,28.153
2021-01-24,Ottawa-1,Ottawa-ROPEC,2021-01-26,0.00029,0.00002,0.00015,0.00007,27.472
2021-01-25,Ottawa-1,Ottawa-ROPEC,2021-01-27,0.00019,0.00004,0.00074,0.00011,26.129
2021-01-26,Ottawa-1,Ottawa-ROPEC,2021-01-28,0.00041,0.0001,0.0002,0.00634,27.161
2021-01-27,Ottawa-1,Ottawa-ROPEC,2021-02-01,0.00028,0.00005,0.00016,0.00003,26.768
2021-01-28,Ottawa-1,Ottawa-ROPEC,2021-02-01,0.00031,0.00008,0.00037,0.00011,28.356
2021-01-29,Ottawa-1,Ottawa-ROPEC,2021-02-01,0.00043,0.00003,0.00058,0.00007,27.418
2021-01-30,Ottawa-1,Ottawa-ROPEC,2021-02-01,0.0006,0.00008,0.00078,0.00011,27.671
2021-01-31,Ottawa-1,Ottawa-ROPEC,2021-02-02,0.00043,0.00008,0.00053,0.0003,28.201
2021-02-01,Ottawa-1,Ottawa-ROPEC,2021-02-04,0.00016,0.00004,0.00033,0.00007,27.97
2021-02-02,Ottawa-1,Ottawa-ROPEC,2021-02-04,0.00018,0.00003,0.00015,0.00002,26.694
2021-02-03,Ottawa-1,Ottawa-ROPEC,2021-02-08,0.00055,0.00027,0.00053,0.00024,27.065
2021-02-04,Ottawa-1,Ottawa-ROPEC,2021-02-08,0.00031,0.00004,0.00026,0.00013,26.774
2021-02-05,Ottawa-1,Ottawa-ROPEC,2021-02-08,0.00039,0.00009,0.00045,0.00006,27.688
2021-02-06,Ottawa-1,Ottawa-ROPEC,2021-02-08,0.00035,0.00017,0.00033,0.00014,27.916
2021-02-07,Ottawa-1,Ottawa-ROPEC,2021-02-09,0.00042,0.00003,0.00039,0.00008,27.1
2021-02-08,Ottawa-1,Ottawa-ROPEC,2021-02-09,0.0002,0.00002,0.00026,0.00004,26.909
2021-02-09,Ottawa-1,Ottawa-ROPEC,2021-02-14,,,,,
2021-02-10,Ottawa-1,Ottawa-ROPEC,2021-02-14,0.00011,0.00004,0.00012,0.00004,26.365
2021-02-11,Ottawa-1,Ottawa-ROPEC,2021-02-15,0.0002,0.00004,0.00018,0.00003,27.227
2021-02-12,Ottawa-1,Ottawa-ROPEC,2021-02-15,0.00051,0.00012,0.00035,0.00006,28.02
2021-02-13,Ottawa-1,Ottawa-ROPEC,2021-02-15,0.00033,0.00006,0.00037,0.00002,26.767
2021-02-14,Ottawa-1,Ottawa-ROPEC,2021-02-19,0.00037,0.00003,0.00033,0.00003,28.133
2021-02-15,Ottawa-1,Ottawa-ROPEC,2021-02-19,0.00017,0.00005,0.00017,0.00007,27.86
2021-02-16,Ottawa-1,Ottawa-ROPEC,2021-02-19,0.00023,0.00003,0.00023,0.00005,27.294
2021-02-17,Ottawa-1,Ottawa-ROPEC,2021-02-19,0.00032,0.00006,0.00022,0.00006,28.616
2021-02-18,Ottawa-1,Ottawa-ROPEC,2021-02-22,0.00022,0.00004,0.00028,0.00005,27.448
2021-02-19,Ottawa-1,Ottawa-ROPEC,2021-02-22,0.00025,0.00004,0.00022,0.00009,28.063
2021-02-20,Ottawa-1,Ottawa-ROPEC,2021-02-22,0.00024,0.00004,0.00028,0.00005,26.958
2021-02-21,Ottawa-1,Ottawa-ROPEC,2021-02-24,0.00022,0.00006,0.00017,0.00003,28.394
2021-02-22,Ottawa-1,Ottawa-ROPEC,2021-02-24,0.00033,0.0001,0.00032,0.00014,27.602
2021-02-23,Ottawa-1,Ottawa-ROPEC,2021-02-25,0.00035,0.00008,0.00029,0.00009,30.357
2021-02-24,Ottawa-1,Ottawa-ROPEC,2021-02-26,0.00035,0.00002,0.00038,0.00009,28.758
2021-02-25,Ottawa-1,Ottawa-ROPEC,2020-03-01,,,,,
2021-02-26,Ottawa-1,Ottawa-ROPEC,2020-03-01,0.00029,0.00008,0.00022,0.00005,27.504
2021-02-27,Ottawa-1,Ottawa-ROPEC,2020-03-01,0.00043,0.00007,0.00048,0.00004,27.426
2021-02-28,Ottawa-1,Ottawa-ROPEC,2020-03-03,0.00092,0.0002,0.00067,0.00008,27.067
2021-03-01,Ottawa-1,Ottawa-ROPEC,2020-03-03,0.00049,0.00009,0.00044,0.00007,26.368
2021-03-02,Ottawa-1,Ottawa-ROPEC,2020-03-04,0.001,0.00012,0.00055,0.00005,26.82
2021-03-03,Ottawa-1,Ottawa-ROPEC,2020-03-05,0.0012,0.00021,0.00048,0.00014,26.746
2021-03-04,Ottawa-1,Ottawa-ROPEC,2020-03-08,,,,,
2021-03-05,Ottawa-1,Ottawa-ROPEC,2020-03-08,0.00058,0.00005,0.00034,0.0004,27.41
2021-03-06,Ottawa-1,Ottawa-ROPEC,2020-03-08,0.00058,0.0001,0.00061,0.00006,27.411
2021-03-07,Ottawa-1,Ottawa-ROPEC,2020-03-09,0.00081,0.00018,0.00025,0.00012,26.227
2021-03-08,Ottawa-1,Ottawa-ROPEC,2020-03-10,0.00052,0.00011,0.00022,0.00009,27.861
2021-03-09,Ottawa-1,Ottawa-ROPEC,2020-03-11,0.00013,0.00002,0.00009,0.00003,26.613
2021-03-10,Ottawa-1,Ottawa-ROPEC,2020-03-15,0.00232,0.00028,0.00209,0.00009,27.372
2021-03-11,Ottawa-1,Ottawa-ROPEC,2020-03-15,0.00012,0.00001,0.0002,0.00004,27.846
2021-03-12,Ottawa-1,Ottawa-ROPEC,2020-03-15,0.00011,0.00003,0.00011,0.00003,26.687
2021-03-13,Ottawa-1,Ottawa-ROPEC,2020-03-15,0.00013,0.00003,0.0001,0.00002,26.353
45 changes: 45 additions & 0 deletions roadmap.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Roadmap

The ODM Steering Group or Working Groups have identified the following features and tools for development.

See the [Roadmap](https://github.com/Big-Life-Lab/covid-19-wastewater/projects/3?add_cards_query=is%3Aopen) project for progress on the roadmap. Add new items or discuss on the discussion [thread](https://github.com/Big-Life-Lab/covid-19-wastewater/discussions/108). Issues with "future" tag are discussed for addition to the roadmap.

## Requirements

Requirements for future tools and features:

- **Support for English and French** - initially for the data model and then other documentation.
- **Open and cross platform** - Tools to support the data model will be generated using or support open software. The objective is to create pseudo-code that can form the basis to program code in R or Python. Rules as pseudo-code allow development teams to generate consistent data applications in other program platforms (i.e. MS 365 or ArcGIS).

## Tools

1. **Automatic updating tools** - scripts that can be used to automatically push the most recent updates of the data model to the necessary depending files (template, ERD, ...)
1. **Data mapping methods** - a collection of methods to:
Read in data from templates into a database (SQLite supported).
Conversion from "long" format to "wide" format needed for data analysis/ML.
1. **Data validation tool** - can validate a datafile in terms of formatting and context, and provides feedback of needed changes
1. **Data linking tools** - tools that will help ensure consistan summarization of public health data into summary results relevent for the WWTP catchment area.
1. **Data cleaning methods** - rules for cleaning raw ODM data.
1. **Visualization tools** - basic plots to visualize relevant signals. Simple visualization tools can be part of vignettes to show how to analyze wastewater data.
1. **Transformation functions** - functions to create derived variables and aggreagations. i.e. moving averages, normalisation,...
1. **Open, public data examples within the repository** - Example data to build
and test common tools and demonstrate ODM compliant data format.


## Documentation

Documentation that including tutorials, how-to guides, technical references and explanations. See [Divio documentation approach](https://documentation.divio.com).

1. Tutorials - learning-oriented.
1. How-to guides - problem-oriented.
1. Explanation - understanding-oriented.
1. Reference - information-oriented.

- Several presentations have introduced the ODM. Let's make the recordings available as a first step. Next, create additional visual documentation for specific functionalities.
- Peer-review description with a permalink/DOI reference which would facilitate sharing and referencing within the scientific community.
- Consider a website that organizes documentation for a wider audience of users that not necessarily know how to work with GitHub.

## Process

- Ensure Steering Committee and Working Groups have representation from a range discplines and regions.
- Add a code of conduct.

0 comments on commit 524e4d3

Please sign in to comment.