Skip to content

Commit

Permalink
Merge branch 'master' into platform/thetaurean/15645-partially-delivered
Browse files Browse the repository at this point in the history
  • Loading branch information
@thetaurean
thetaurean authored Oct 9, 2024
2 parents ecc6d50 + 5025bf5 commit dee5da9
Show file tree
Hide file tree
Showing 7 changed files with 199 additions and 69 deletions.
69 changes: 69 additions & 0 deletions .github/ISSUE_TEMPLATE/DevSecOps-issue.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
name: DevSecOps ticket
about: For new DevSecOps tickets on the board.
labels: ['DevSecOps', 'reportstream']
---
# DevSecOps Issue

## Summary

Context and description

## Environment

- [ ] Local
- [ ] Dev
- [ ] Stage
- [ ] Prod

## Priority

- [ ] **Critical** - affecting prod systems
- [ ] **Major** - blocking major functionality, deployment, etc
- [ ] **Minor** - improvements, bug fixes
- [ ] **Nice-to-have** - feature request

### Blocks the following

- List any blockers
- Link any Github relevant issues

### Blocked by the following

- Link to any tickets that add context, etc

## Contact

**Team**:

**Slack Team Channel**:

**Slack Contact (Full Name):**

**Technical Team Lead:**

## How to Reproduce

### Screenshots, links, etc. for context

## How to Test

## Definition of Done

- [ ]

## Context Links

**Git Repo:**

**Relevant Code Links:**

**Azure**:

**Site URL**:

**Other**:

## Notes

-
34 changes: 14 additions & 20 deletions frontend-react/src/content/about/roadmap.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import site from "../../content/site.json"
<Sidenav />
</LayoutSidenav>

<p className="usa-timestamp">Last updated: August 05, 2024</p>
<p className="usa-timestamp">Last updated: October 07, 2024</p>

## Objectives for FY24
Learn about our goals and how we are measuring success for this fiscal year (FY), which runs from October 2023 through September 2024.
Expand Down Expand Up @@ -46,37 +46,37 @@ Learn about our goals and how we are measuring success for this fiscal year (FY)
tag: "recently-completed",
body: (
<p>
Arizona, Colorado, and Kansas are able to receive flu and RSV data from ReportStream.
Arkansas, Florida, Guam, Idaho, Illinois, Louisiana, Maine, Massachusetts, Minnesota, and Oklahoma are able to receive flu and RSV data from ReportStream.
</p>
),
},
{
tag: "working-on-now",
body: (
<p>
Onboard or update connections for Arkansas, Connecticut, Florida, Idaho, Illinois, Iowa, Kentucky, Louisiana, Maine, Massachusetts, Michigan, Minnesota, Missouri, Montana, Oklahoma, Virginia, Washington D.C.
Onboard or update connections for Alabama, Connecticut, Delaware, Indiana, Iowa, Kentucky, Michigan, Mississippi, Missouri, Montana, New Hampshire, New Jersey, New Mexico, Ohio, Pennsylvania, Puerto Rico, Republic of Marshall Islands, South Dakota, Tennessee, Vermont, Virginia, Washington, Washington D.C., and Wyoming.
</p>
),
},
{
tag: "next",
body: (
<p>
Support remaining STLTs to connect or update their connections to be able to receive additional conditions.
</p>
<p>Support remaining STLTs to connect or update their connections to be able to receive additional conditions.</p>
),
},
{
title:<p>Partner with <A href="https://www.simplereport.gov/">SimpleReport</A> to help nontraditional and lower-resourced testing sites and facilities report test results and send data to STLTs and the CDC.</p>,
tag: "recently-completed",
body: (
<p>California, Nevada, and New York are receiving flu A/B or RSV data.</p>
<p>Contacted nontraditional and lower-resourced HIV senders in Texas as part of targeted health equity improvement work.</p>
),
},
{
tag: "recently-completed",
body: (
<p>Identify nontraditional and lower-resourced HIV senders in Texas as part of targeted health equity improvement work.</p>
<p>
Alaska, Arizona, California, Colorado, Florida, Hawaii, Idaho, Illinois, Louisiana, Massachusetts, Minnesota, Nevada, New York, and Rhode Island are receiving flu A/B or RSV data.
</p>
),
},
{
Expand All @@ -86,45 +86,39 @@ Learn about our goals and how we are measuring success for this fiscal year (FY)
),
},
{
tag: "next",
tag: "working-on-now",
body: (
<p>Identify more regions in California to exchange HIV test result data.</p>
<p>Add support for hepatitis C and other STI reporting through SimpleReport.</p>
),
},
{
tag: "next",
body: (
<p>Add support for hepatitis C reporting through SimpleReport.</p>
<p>Add support for more conditions reporting through SimpleReport.</p>
),
},
{
title:<p>Help at-home test takers and test manufacturers easily report results to STLTs and CDC.</p>,
tag: "recently-completed",
body: (
<p>Support file validation against different HL7 profiles, starting with the RADx MARS custom <A href="https://www.cdc.gov/vaccines/programs/iis/technical-guidance/downloads/hl7guide-1-5-2014-11.pdf" target="_blank">NIST HL7 2.5.1</A> profile.</p>
<p>Route test results from <A href="https://learn.makemytestcount.org/" target="_blank">MakeMyTestCount.org</A> to one STLT in test environment.</p>
),
},
{
tag: "working-on-now",
body: (
<p>Route test results from <A href="https://learn.makemytestcount.org/" target="_blank">MakeMyTestCount.org</A> to one STLT in test environment.</p>
<p>Route test results from <A href="https://learn.makemytestcount.org/" target="_blank">MakeMyTestCount.org</A> to one STLT in production.</p>
),
},
{
tag: "next",
body: (
<p>Route test results from <A href="https://learn.makemytestcount.org/" target="_blank">MakeMyTestCount.org</A> to one STLT in production.</p>
<p>Route test results from another OTC testing (point-of-care) sender to one STLT in production.</p>
),
},
{
title: "Facilitate electronic laboratory reporting (ELR) for the CDC infectious disease labs and transmit to STLTs. ",
tag: "recently-completed",
body: (
<p>Test data pipeline with two STLT partners to transmit test results.</p>
),
},
{
tag: "working-on-now",
body: (
<p>Complete final internal review of test result data transfer before moving to production.</p>
),
Expand Down
16 changes: 16 additions & 0 deletions prime-router/src/main/kotlin/azure/ActionHistory.kt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,22 @@ class ActionHistory(
}
}

/**
* Allows tracking of an empty report regardless of where it is generated in the pipeline
* @param report the details of the report
*/
fun trackEmptyReport(report: Report) {
generatingEmptyReport = true
val reportFile = ReportFile()
reportFile.reportId = report.id
reportFile.schemaTopic = report.schema.topic
reportFile.schemaName = "None"
reportFile.itemCount = 0
reportFile.bodyFormat = report.bodyFormat.toString()
reportFile.nextAction = TaskAction.none
reportsOut[reportFile.reportId] = reportFile
}

/**
* Use this to record history info about a newly generated empty [report] for sending to [receiver] that
* has requested an empty batch. The [event] will be batch or send.
Expand Down
33 changes: 3 additions & 30 deletions prime-router/src/main/kotlin/fhirengine/engine/FHIRConverter.kt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ import gov.cdc.prime.router.azure.LookupTableConditionMapper
import gov.cdc.prime.router.azure.ProcessEvent
import gov.cdc.prime.router.azure.db.Tables
import gov.cdc.prime.router.azure.db.enums.TaskAction
import gov.cdc.prime.router.azure.db.tables.pojos.ItemLineage
import gov.cdc.prime.router.azure.observability.bundleDigest.BundleDigestExtractor
import gov.cdc.prime.router.azure.observability.bundleDigest.FhirPathBundleDigestLabResultExtractorStrategy
import gov.cdc.prime.router.azure.observability.context.MDCUtils
Expand Down Expand Up @@ -277,41 +276,15 @@ class FHIRConverter(
}.collect(Collectors.toList()).filterNotNull()
}
} else {
val nextEvent = ProcessEvent(
Event.EventAction.NONE,
queueMessage.reportId,
Options.None,
emptyMap(),
emptyList()
)

// TODO: https://github.com/CDCgov/prime-reportstream/issues/15223
val report = Report(
MimeFormat.FHIR,
format,
emptyList(),
1,
0,
metadata = this.metadata,
topic = queueMessage.topic,
nextAction = TaskAction.none
)

// create item lineage
report.itemLineages = listOf(
ItemLineage(
null,
queueMessage.reportId,
1,
report.id,
1,
null,
null,
null,
report.getItemHashForRow(1)
)
)

// ensure tracking is set
actionHistory.trackCreatedReport(nextEvent, report)
actionHistory.trackEmptyReport(report)
reportEventService.sendReportProcessingError(
ReportStreamEventName.REPORT_NOT_PROCESSABLE,
report,
Expand Down
47 changes: 33 additions & 14 deletions prime-router/src/test/kotlin/common/UniversalPipelineTestUtils.kt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,24 @@ OBX|4|CWE|95421-4^Resides in a congregate care setting^LN^^^^2.69||N^No^HL70136|
OBX|5|CWE|95419-8^Has symptoms related to condition of interest^LN^^^^2.69||N^No^HL70136||||||F|||202102090000-0600|||||||||||||||QST
SPM|1|0cba76f5-35e0-4a28-803a-2f31308aae9b||258500001^Nasopharyngeal swab^SCT||||71836000^Nasopharyngeal structure (body structure)^SCT^^^^2020-09-01|||||||||202102090000-0600|202102090000-0600"""

// This report is trying to contain two items, but the HL7 is garbled, the first is missing an MSH segment and the second
// has a typo in its MSH segment
@Suppress("ktlint:standard:max-line-length")
const val garbledHL7Record =
"""FT|Centers for Disease Control and Prevention|0.1-SNAPSHOT|PRIME ReportStream|0.1-SNAPSHOT||20210210
PID|1||2a14112c-ece1-4f82-915c-7b3a8d152eda^^^Avante at Ormond Beach^PI||Buckridge^Kareem^Millie^^^^L||19580810|F||2106-3^White^HL70005^^^^2.5.1|688 Leighann Inlet^^South Rodneychester^TX^67071^^^^48077||7275555555:1:^PRN^^roscoe.wilkinson@email.com^1^211^2240784|||||||||U^Unknown^HL70189||||||||N
ORC|RE|73a6e9bd-aaec-418e-813a-0ad33366ca85^6^7^8&F^9|73a6e9bd-aaec-418e-813a-0ad33366ca85|||||||||1629082607^Eddin^Husam^^^^^^CMS&2.16.840.1.113883.3.249&ISO^^^^NPI||^WPN^^^1^386^6825220|20210209||||||Avante at Ormond Beach|170 North King Road^^Ormond Beach^FL^32174^^^^12127|^WPN^^jbrush@avantecenters.com^1^407^7397506|^^^^32174
OBR|1|73a6e9bd-aaec-418e-813a-0ad33366ca85|0cba76f5-35e0-4a28-803a-2f31308aae9b|94558-4^SARS-CoV-2 (COVID-19) Ag [Presence] in Respiratory specimen by Rapid immunoassay^LN|||202102090000-0600|202102090000-0600||||||||1629082607^Eddin^Husam^^^^^^CMS&2.16.840.1.113883.3.249&ISO^^^^NPI|^WPN^^^1^386^6825220|||||202102090000-0600|||F
OBX|1|CWE|94558-4^SARS-CoV-2 (COVID-19) Ag [Presence] in Respiratory specimen by Rapid immunoassay^LN||260415000^Not detected^SCT|||N^Normal (applies to non-numeric results)^HL70078|||F|||202102090000-0600|||CareStart COVID-19 Antigen test_Access Bio, Inc._EUA^^99ELR||202102090000-0600||||Avante at Ormond Beach^^^^^CLIA&2.16.840.1.113883.4.7&ISO^^^^10D0876999^CLIA|170 North King Road^^Ormond Beach^FL^32174^^^^12127
OBX|2|CWE|95418-0^Whether patient is employed in a healthcare setting^LN^^^^2.69||Y^Yes^HL70136||||||F|||202102090000-0600|||||||||||||||QST
OBX|3|CWE|95417-2^First test for condition of interest^LN^^^^2.69||Y^Yes^HL70136||||||F|||202102090000-0600|||||||||||||||QST
OBX|4|CWE|95421-4^Resides in a congregate care setting^LN^^^^2.69||N^No^HL70136||||||F|||202102090000-0600|||||||||||||||QST
OBX|5|CWE|95419-8^Has symptoms related to condition of interest^LN^^^^2.69||N^No^HL70136||||||F|||202102090000-0600|||||||||||||||QST
SPM|1|0cba76f5-35e0-4a28-803a-2f31308aae9b||258500001^Nasopharyngeal swab^SCT||||71836000^Nasopharyngeal structure (body structure)^SCT^^^^2020-09-01|||||||||202102090000-0600|202102090000-0600
SH|^~\&#!|CDC PRIME - Atlanta, Georgia (Dekalb)^2.16.840.1.114222.4.1.237821^ISO|Avante at Ormond Beach^10D0876999^CLIA|PRIME_DOH|Prime ReportStream|20210210170737||ORU^R01^ORU_R01|371784|P|2.5.1|||NE|NE|USA||||PHLabReportNoAck^ELR_Receiver^2.16.840.1.113883.9.99^ISO
SFT|Centers for Disease Control and Prevention|0.1-SNAPSHOT|PRIME ReportStream|0.1-SNAPSHOT||20210210
PID|1||2a14112c-ece1-4f82-915c-7b3a8d152eda^^^Avante at Ormond Beach^PI||Buckridge^Kareem^Millie^^^^L||19580810|F||2106-3^White^HL70005^^^^2.5.1|688 Leighann Inlet^^South Rodneychester^TX^67071^^^^48077||7275555555:1:^PRN^^roscoe.wilkinson@email.com^1^211^2240784|||||||||U^Unknown^HL70189||||||||N"""

@Suppress("ktlint:standard:max-line-length")
const val validRadxMarsHL7Message =
"""MSH|^~\&|MMTC.PROD^2.16.840.1.113883.3.8589.4.2.106.1^ISO|CAREEVOLUTION^00Z0000024^CLIA|AIMS.INTEGRATION.PRD^2.16.840.1.114222.4.3.15.1^ISO|AIMS.PLATFORM^2.16.840.1.114222.4.1.217446^ISO|20240403205305+0000||ORU^R01^ORU_R01|20240403205305_dba7572cc6334f1ea0744c5f235c823e|P|2.5.1|||NE|NE|||||PHLabReport-NoAck^ELR251R1_Rcvr_Prof^2.16.840.1.113883.9.11^ISO
Expand Down Expand Up @@ -257,12 +275,13 @@ object UniversalPipelineTestUtils {
)

/**
* fetch child reports associated with a [parent] report and ensure we find an [expected] number of children
* fetch child reports associated with a [parent] report and ensure we find an [expectedItems] number of children
*/
fun fetchChildReports(
parent: Report,
txn: DataAccessTransaction,
expected: Int? = null,
expectedItems: Int? = null,
expectedReports: Int = 1,
): List<ReportFile> {
val itemLineages = DSL
.using(txn)
Expand All @@ -271,15 +290,15 @@ object UniversalPipelineTestUtils {
.where(ItemLineage.ITEM_LINEAGE.PARENT_REPORT_ID.eq(parent.id))
.fetchInto(gov.cdc.prime.router.azure.db.tables.pojos.ItemLineage::class.java)

if (expected != null) {
assertThat(itemLineages).hasSize(expected)
assertThat(itemLineages.map { it.childIndex }).isEqualTo(MutableList(expected) { 1 })
if (expectedItems != null) {
assertThat(itemLineages).hasSize(expectedItems)
assertThat(itemLineages.map { it.childIndex }).isEqualTo(MutableList(expectedItems) { 1 })

// itemCount is on the report created by the test. It will not be null.
if (parent.itemCount > 1) {
assertThat(itemLineages.map { it.parentIndex }).isEqualTo((1..expected).toList())
assertThat(itemLineages.map { it.parentIndex }).isEqualTo((1..expectedItems).toList())
} else {
assertThat(itemLineages.map { it.parentIndex }).isEqualTo(MutableList(expected) { 1 })
assertThat(itemLineages.map { it.parentIndex }).isEqualTo(MutableList(expectedItems) { 1 })
}
}

Expand All @@ -290,9 +309,7 @@ object UniversalPipelineTestUtils {
.where(ReportLineage.REPORT_LINEAGE.PARENT_REPORT_ID.eq(parent.id))
.fetchInto(gov.cdc.prime.router.azure.db.tables.pojos.ReportLineage::class.java)

if (expected != null) {
assertThat(reportLineages).hasSize(expected)
}
assertThat(reportLineages).hasSize(expectedReports)

val childReportIds = reportLineages.map {
it.childReportId
Expand All @@ -307,11 +324,13 @@ object UniversalPipelineTestUtils {
)
)
.fetchInto(ReportFile::class.java)
if (expected != null) {
assertThat(reportFiles).hasSize(expected)

assertThat(reportFiles).hasSize(expectedReports)

if (expectedItems != 0) {
assertThat(itemLineages).transform { lineages -> lineages.map { it.childReportId }.sorted() }
.isEqualTo(reportFiles.map { it.reportId }.sorted())
}
assertThat(itemLineages).transform { lineages -> lineages.map { it.childReportId }.sorted() }
.isEqualTo(reportFiles.map { it.reportId }.sorted())

return reportFiles
}
Expand Down
Loading

0 comments on commit dee5da9

Please sign in to comment.