Mount public data in obi-one service

[james-isbister]

Description: Mount public entitycore data in obi-one service

Acceptance criteria: obi-one service endpoints can access mounted public entitycore data

[GianlucaFicarelli]

For reference, see also Mike's notes and work:

• https://github.com/openbraininstitute/prod-platform-architecture/issues/146
• https://github.com/openbraininstitute/aws-terraform-deployment/blob/staging/cells_svc/cells_ec2_ecs_user_data.sh

[ayimaok]

@GianlucaFicarelli - could we check with you whether this ticket is blocked by what's Mike's working on?
Would you be able to, please provide updates? Thanks

[GianlucaFicarelli]

This is in progress, not blocked. It will use the same approach used for cells service (EC2+ECS).

[evakennyobi]

This needs to be reviewed still @GianlucaFicarelli ? thanks

[GianlucaFicarelli]

Yes this still needs to be reviewed, the issue was automatically closed by github-project-automation :(

[GianlucaFicarelli]

A new service obi-one-v2 is deployed to staging using ECS on EC2: https://staging.openbraininstitute.org/api/obi-one-v2/version
The public directory of the entitycore bucket is mounted readonly under /public.
Note that the base path is probably going to change, so we can mount in a common base directory different buckets at the same time.
obi-one-v2 uses the same image specified for obi-one (staging or production), but this can be changed in aws-terraform-deployment.
cc @james-isbister

[GianlucaFicarelli]

Note that the mount points have changed with the last deployment of obi-one-v2 in staging:

• /data/aws_s3_internal/public (for public data in the entitycore bucket)
• /data/aws_s3_open (for openbluebrain data)

The base directory /data is exported in the env variable MOUNT_BASE_DIR.
These paths can be obtained in the application by concatenating the information from the asset metadata, for example:

path = Path(os.env["MOUNT_BASE_DIR"]) / asset["storage_type"] / asset["full_path"]

storage_type is already returned by entitycore, but it needs openbraininstitute/entitysdk#105 to be available in entitysdk.

cc @james-isbister @chr-pok