Feature/docker file for ecs#336
Merged
ascibisz merged 3 commits intostaging/cellpack-serverfrom May 15, 2025
Merged
Conversation
Contributor
Packing analysis reportAnalysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST
Packing imageDistance analysisExpected minimum distance: 50.00
|
rugeli
reviewed
May 15, 2025
docker/server.py
Outdated
| config = request.rel_url.query.get("config") | ||
| job_id = str(uuid.uuid4()) | ||
| os.environ["AWS_BATCH_JOB_ID"] = job_id | ||
| pack(recipe=recipe, config_path=config, docker=True) |
Collaborator
There was a problem hiding this comment.
should we add an exception here to prevent server crash if the pack function fails?
rugeli
reviewed
May 15, 2025
docker/server.py
Outdated
Comment on lines
8
to
9
| async def hello_world(request: web.Request) -> web.Response: | ||
| return web.Response(text="Hello world") |
Collaborator
There was a problem hiding this comment.
nit: really no biggie but hello world sounds a bit like a default place holder, how about changing the text to hello cell or hello cellPACK?
rugeli
approved these changes
May 15, 2025
ascibisz
added a commit
that referenced
this pull request
Jun 17, 2025
* Feature/aws cli in docker (#280) * Add basic Dockerfile and docker-entrypoint file * draft aws cli inside of docker container * access aws s3 with saved cred * update readme * update docker commands * update readme * update readme --------- Co-authored-by: ascibisz <alli.scibisz@alleninstitute.org> * Feature/pass recipe to docker (#285) * draft aws cli inside of docker container * access aws s3 with saved cred * update readme * update docker commands * update readme * update readme * add parameter for when we're using docker to default to using the staging firebase db * read recipe and config in from docker run params * use dockerignore file to prevent bloat in docker image * add `batch_job_id` to result metadata (#295) * retrieve job id * formatting * add a bucket for batch jobs * formatting * remove print statement and add comments --------- Co-authored-by: Ruge Li <rugeli0605@gmail.com> Co-authored-by: Ruge Li <91452427+rugeli@users.noreply.github.com> * pull changes for firebase configs handling * add docker option to config (#332) * Bug Fix: keep grid_file_path when loading recipe from firebase (#334) * keep grid_file_path when loading recipe from firebase * only add grid_file_path if it's in the firebase recipe * Feature/docker file for ecs (#336) * Add Dockerfile.ecs for our ECS image and rename old Dockerfile to Dockerfile.batch * add intentional error handling for if packing fails * change messaging from /hello endpoint * send response json using camelCase * add healthcheck endpoint and use port 80 instead of 8443 for load balancer requirements * add way for running ECS docker image locally without trying to publish results to cellpack-demo/ --------- Co-authored-by: ascibisz <alli.scibisz@alleninstitute.org> Co-authored-by: Alli <111383930+ascibisz@users.noreply.github.com>
ascibisz
added a commit
that referenced
this pull request
Jun 17, 2025
* Add Dockerfile.ecs for our ECS image and rename old Dockerfile to Dockerfile.batch * add intentional error handling for if packing fails * change messaging from /hello endpoint
ascibisz
added a commit
that referenced
this pull request
Jun 27, 2025
* Feature/aws cli in docker (#280) * Add basic Dockerfile and docker-entrypoint file * draft aws cli inside of docker container * access aws s3 with saved cred * update readme * update docker commands * update readme * update readme --------- Co-authored-by: ascibisz <alli.scibisz@alleninstitute.org> * Feature/pass recipe to docker (#285) * draft aws cli inside of docker container * access aws s3 with saved cred * update readme * update docker commands * update readme * update readme * add parameter for when we're using docker to default to using the staging firebase db * read recipe and config in from docker run params * use dockerignore file to prevent bloat in docker image * add `batch_job_id` to result metadata (#295) * retrieve job id * formatting * add a bucket for batch jobs * formatting * remove print statement and add comments --------- Co-authored-by: Ruge Li <rugeli0605@gmail.com> Co-authored-by: Ruge Li <91452427+rugeli@users.noreply.github.com> * pull changes for firebase configs handling * Feature/docker file for ecs (#336) * Add Dockerfile.ecs for our ECS image and rename old Dockerfile to Dockerfile.batch * add intentional error handling for if packing fails * change messaging from /hello endpoint * add healthcheck endpoint and use port 80 instead of 8443 for load balancer requirements * add way for running ECS docker image locally without trying to publish results to cellpack-demo/ * update server to respond from packing requests immediately, rather than waiting for the packing job to complete * revert accidential change to is_remote_path * remove is_remote flag * lint fix * temporarily use cellpack-results bucket * revert demo bucket name change * change to point cellpack install to main branch --------- Co-authored-by: Ruge Li <91452427+rugeli@users.noreply.github.com> Co-authored-by: Ruge Li <rugeli0605@gmail.com>
ascibisz
added a commit
that referenced
this pull request
Jul 7, 2025
* Add Dockerfile.ecs for our ECS image and rename old Dockerfile to Dockerfile.batch * add intentional error handling for if packing fails * change messaging from /hello endpoint
ascibisz
added a commit
that referenced
this pull request
Aug 13, 2025
* tFeature/aws cli in docker (#280) * Add basic Dockerfile and docker-entrypoint file * draft aws cli inside of docker container * access aws s3 with saved cred * update readme * update docker commands * update readme * update readme --------- Co-authored-by: ascibisz <alli.scibisz@alleninstitute.org> * Feature/pass recipe to docker (#285) * draft aws cli inside of docker container * access aws s3 with saved cred * update readme * update docker commands * update readme * update readme * add parameter for when we're using docker to default to using the staging firebase db * read recipe and config in from docker run params * use dockerignore file to prevent bloat in docker image * add `batch_job_id` to result metadata (#295) * retrieve job id * formatting * add a bucket for batch jobs * formatting * remove print statement and add comments --------- Co-authored-by: Ruge Li <rugeli0605@gmail.com> Co-authored-by: Ruge Li <91452427+rugeli@users.noreply.github.com> * pull changes for firebase configs handling * Feature/docker file for ecs (#336) * Add Dockerfile.ecs for our ECS image and rename old Dockerfile to Dockerfile.batch * add intentional error handling for if packing fails * change messaging from /hello endpoint * add healthcheck endpoint and use port 80 instead of 8443 for load balancer requirements * add way for running ECS docker image locally without trying to publish results to cellpack-demo/ * revert accidential change to is_remote_path * remove is_remote flag * temporarily use cellpack-results bucket * revert demo bucket name change * make changes to cellpack code to accept a remote recipe that closely resembles a local recipe * lint fix * cleanup code * use local variable to improve readability * lint fix --------- Co-authored-by: Ruge Li <91452427+rugeli@users.noreply.github.com> Co-authored-by: Ruge Li <rugeli0605@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
As part of our desire to speed up packings on the cellpack AWS server, we started testing working with ECS, instead of Batch. This requires a different Docker image to be built for us to run on ECS.
Solution
Created a new Dockerfile and docker-entrypoint for the image we'll have on ECS, as well as a basic python HTTP server for the client to send packings to.
For now, I want to keep both our ECS and our Batch Dockerfiles, because I think Batch can still have some really good use cases in the future. I moved everything into a /docker subdirectory and changed some file names, just to keep things organized and clear.
Also updated the README with instructions for building and running both docker images.