Repository navigation
Documentation/Data Dictionary written for the users so that they can analyse the parquet files #28
Description
Activity
- changed the title
[-]Test that output parquet files can be used in common data analysis software[/-][+]Documentation/Data Dictionary written for the users so that they can analyse the parquet files [/+]on Feb 4, 2026 This is parked until the project resumes.
@jeremyestein to include comments here about what is needed in the documentation.I imagine the data dictionary as being just the user-facing subset of a larger doc, which is a description of the data at each stage of the pipeline. That is, it is helpful for developers to know how a certain piece of data is traced through the pipeline:
HL7 -> wf-reader -> rabbitmq -> wf-controller -> wf-exporter -> DSH └> coreThe end users are mainly interested in the leaves of this tree, but really it's all the same thing.
Part of this is just tracing stuff through (eg. which field is the channel ID carried in?).
Especially on the data dict end of things, explain the interpretation, eg. what assumptions you are allowed to make.
Metadata
Metadata
Assignees
Labels
No labels
Description
Documentation/Data Dictionary written for the users so that they can analyse the data
Definition of Done
Can parquet files be loaded into R, Python, etc? Can the required data be easily located? Can a waveform be reconstructed, and does it look good? How would we validate whether it matched the input data.
Our users should guide us but I think time-based queries will be common. Ie. extract a particular waveform a few minutes either side of a particular time (when a medical event happened).