Add Audit API spec

[janedotx]

Eric Mai is the actual author. I am just moving the spec over to this fork to centralize spec PRs.

MDS Pull Request

Thank you for your contribution!

For most Pull Requests, the target branch should be dev. Please ensure you are targeting dev to avoid complications and help make the Review process as smooth as possible.

Explain pull request

Please provide a clear and concise reason for this pull request and the impact of the change

Is this a breaking change

A breaking change would require consumers or implementors of the API to modify their code for it to continue to function (ex: renaming of a required field or the change in data type of an existing field). A non-breaking change would allow existing code to continue to function (ex: addition of an optional field or the creation of a new optional endpoint).

• Yes, breaking
• No, not breaking
• I'm not sure

Impacted Spec

Which spec(s) will this pull request impact?

• agency
• policy
• provider

Additional context

Add any other context or screenshots about the feature request here.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[Retzoh]

@jfh01 , @schnuerle , @Karcass , Charles Noling please add comments to explain the use-cases you had in mind.

[schnuerle]

Here are some of the use cases I mentioned on the call. An Audit API can be beneficial to cities and providers to collectively discover discrepancies in data feeds vs. real world locations and come up with solutions. Examples include:

• Feedback from cities to providers on issues with data feeds.
• Conversations between cities and providers about details in the data and how to reach consensus on nuanced understanding.
• Helping the OMF improve MDS spec to address discovered issues.
• Collaboration to fix bugs or issues in the data feeds.
• Gives cities a way to trust and validate the data feed since otherwise all data only comes from providers (via MDS, GBFS, provider dashboards, provider monthly reports).
• Helps automate the manual work already being done in cities through field spot checks of device locations, trip behavior, policy, complaints, and safety investigations.

For an analogy, much like when cities publish open data, some departments can be reluctant to publish some datasets fearing data quality issues. But by publishing, cities allow others to look at it and find issues, comment on it, and suggest improvements. This feedback loop is important to help fix real problems and surface data quality issues.

[Retzoh]

@janedotx @whereissean is this PR a duplicate of #326?

[schnuerle]

Hi @janedotx @whereissean - checking back in to see if this is a duplicate of #326?

Also it looks like the CLA may have to be signed by @janedotx so we can move this forward.

We'd like to include the Audit work in the 1.0.0 release within the next two weeks.

[schnuerle]

Per our call today PR #326 is an older PR and is now closed in favor of this one.

Everyone on the call agreed this spec was valuable and should be part of OMF. The intent of Audit to to establish a shared ground truth between providers and cities, so you can trust the data. The USDOT has recognized this need as cities use data from outside sources to make policy decisions instead of relying only on data they collect.

Questions about:

Should this be part of the MDS repo, or its own new MDS Audit repo?

In MDS Repo

In the future it could be used to send data to providers, or in conjunction with the proposed Compliance API.

Note if included it needs documentation/link in the main MDS readme added to this PR.

Adding to this repo adds more complexity for spec maintainers.

In Audit Repo

Currently Audit is a city service only, without a communication point with providers. The results can be shared with providers, but that is not part of the spec.

Lives outside of the connections between Provider and Agency.

Don't want to over complicate the MDS repo and make implementation harder.

A new repo requires slightly more maintenance overhead for OMF admins.

Summary

Consensus that audit shouldn't be included as part of compliance mobile, since that is a reference implementation, like mds-core is. It can reference the Audit spec, but not be in the same repo.

Ride Report has their own audit system and they will provide feedback to ensure it covers their uses.

Looks like Audit may be pushed to 1.1.0 so that it can be published at the same time as the new Compliance API. But ran out of time and more discussion is needed.

Please leave your thoughts here.

[schnuerle]

Proposed to move this to its own, new OMF repo for now, called "MDS Audit", for the reasons stated in the above comment, instead of in this main MDS repo.

Agreed on by @jfh01 @schnuerle @Karcass. Would like feedback from others and @thekaveman @Retzoh specifically.

As Compliance gets fleshed out and Audit gets used, we can consider if it lives independently, or gets merged into the main MDS repo in the future.

[Retzoh]

I agree that this audit spec on how to verify data quality should be part of a separate repo for now.

MDS is about sharing data between providers and cities. Specifications on how to share data quality feedback between providers and cities definitely has a place there and would be a great match for the compliance API.
However this PR is not about how to share data quality but about one way to compute it. Having it separated from the MDS repo would make it easier to maintain. It will also keep MDS simpler for new providers / cities wanting to focus on core MDS functionalities.

[schnuerle]

The new Audit repo is located at https://github.com/openmobilityfoundation/mds-audit

@janedotx @Karcass Can you make a new PR against the new repo for the 2 files requested in this PR? Those files should go in the root level of that repo.

If that option is too cumbersome or it goes against how you have things architected on your end (eg, with a /audit folder) then let me know and I can create those files for you.

Then we can close this PR.

[schnuerle]

See the notes from the last WG call.

• in use in LA
• Validate ground truth by comparing with providers data
• Maybe for next release, need more support