Mock up an API endpoint that, given a Study.id and DataObject.data_object_type, returns the relevant DataObjects

[eecavanna]

This Issue (task) came out of today's metadata squad meeting (Wednesday, November 22, 2023).

---

Based on the first bullet point in #355...

...I'm envisioning an API endpoint that accepts a Study.id value and a DataObject.data_object_type value, and returns (i.e. responds with) a JSON array of all the DataObjects that are associated with that study and have that specific data_object_type value.

DataObject documentation:

• https://microbiomedata.github.io/nmdc-schema/DataObject/

Here's an example of a "long" query team members have run; not necessarily related to this Issue other than to serve as an example of a long query, which is something endpoints like this one could save people from having to write themselves:

• https://github.com/microbiomedata/notebook_hackathons/blob/soil-contig-tax/taxonomic_dist_by_soil_layer/python/mongodb_query.txt.js

[eecavanna]

Alternatively, mock up an API endpoint with the following inputs and outputs:

• Inputs:
  • Study.id (string)
  • WorkflowExecutionActivity.type (string)
• Outputs:
  • Array of DataObjects that are referenced by WorkflowExecutionActivity.has_output

The API endpoint would find all the WorkflowExecutionActivitys that have the specified type and are associated with the specified Study; then find the DataObjects referenced by those WorkflowExecutionActivitys.

Reference:

• https://microbiomedata.github.io/nmdc-schema/WorkflowExecutionActivity/

[PeopleMakeCulture]

Spun from #355

[eecavanna]

📓 FYI - I created this issue as a reminder to myself to mock up an endpoint that does what is described, so I could demo it to a teammate as a representative "endpoint that can abstract away a tedious query(ies)". It is not the case that I, personally, find myself doing this particular query often.

[aclum]

Is this going to be worked on this month?

[dwinston]

certainly can be. what would help immensely is knowing all of the paths from study to data object. I suppose this could be done via a graph algorithm on a graph representation of the schema.

@turbomam any ideas on how we could best do this?

[aclum]

I support using a graph representation of the schema, the berkeley refactor will add a number of new paths so this seems more robust than documenting the existing paths and having to add new ones in just a few weeks time.

[ssarrafan]

@dwinston @aclum let me know if this should be added to an NMDC or Infra sync meeting for discussion.

[aclum]

@ssarrafan please add to the infra sync meeting this week.

[shreddd]

Let's try and find a time when @dwinston can join us - 12-1p Pacific on Friday 2/9 could work.

[eecavanna]

This ticket was originally about mocking up a specific endpoint to demonstrate how people could delegate the graph traversal to the Runtime instead of having to write complex Mongo queries. I originally assigned it to myself.

The scope of the task seems to me to have grown into: implementing a real (not mock-up) endpoint that can traverse the graph and will be able to accommodate changes to the schema/graph over time.

If this ticket represents the task of implementing a "roll up" endpoint (where "roll up" is a word the referential integrity/roll up squad members have been using lately), I'd propose updating its title and maybe reassigning it to @sujaypatil96 and putting it on the squad board for the referential integrity/roll up squad (I don't think the squad has a squad board yet).

[sujaypatil96]

Agreed @eecavanna. I'll reassign this ticket to myself since I'm working on "rollup".

[ssarrafan]

@sujaypatil96 assuming you're still actively working on this issue? I'll move to the new sprint. Let me know if you're not planning to work on it.

[sujaypatil96]

@ssarrafan yup, i'm pushing up a draft PR for this issue just now.

[ssarrafan]

@sujaypatil96 it looks like you still need to review #608 so I'll move this to the new sprint. Let me know if you won't be working on it.

[aclum]

@sujaypatil96 @eecavanna @shreddd This endpoint on dev doesn't allow for any new arguments which were in the original request. If find_data_objects_for_study_data_objects_study__study_id__get was updated on the backend to use alldocs but no other arguments are allowed this needs to be reopened.