Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Create a new blob-store service for handling large temporal payloads #642

Merged
merged 3 commits into from
Oct 13, 2024

Conversation

creatorrr
Copy link
Contributor

@creatorrr creatorrr commented Oct 13, 2024

  • feat(blob-store): Add blob store for storing large blobs for agent workflows
  • feat(blob-store): Add authentication

Important

Introduces a new blob-store service using SeaweedFS for handling large temporal payloads, with Docker and authentication setup.

  • Blob Store Service:
    • Introduces a new blob-store service using SeaweedFS in blob-store/Dockerfile, docker-compose.yml, and docker-compose-ha.yml.
    • Adds entrypoint.sh for environment variable checks and configuration file generation.
    • Includes s3.json.template for S3 configuration.
  • Environment Configuration:
    • Updates .env.example with S3_ENDPOINT, S3_ACCESS_KEY, and S3_SECRET_KEY for blob store configuration.
    • Adds USE_BLOB_STORE_FOR_TEMPORAL and BLOB_STORE_CUTOFF_KB for temporal payload handling.
  • Authentication:
    • Implements authentication in s3.json.template with access and secret keys for the julep account.
  • Miscellaneous:
    • Adds .gitignore entry for /s3.json in blob-store/.gitignore.
    • Updates main docker-compose.yml to include blob-store/docker-compose.yml.

This description was created by Ellipsis for 736b9df. It will automatically update as commits are pushed.

…rkflows

Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 736b9df in 25 seconds

More details
  • Looked at 301 lines of code in 8 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. blob-store/Dockerfile:14
  • Draft comment:
    Consider removing build dependencies after use to reduce image size. You can do this by combining the apk del build_deps command with the previous commands in a single RUN statement.
  • Reason this comment was not posted:
    Comment looked like it was already resolved.
2. blob-store/entrypoint.sh:5
  • Draft comment:
    Consider checking if the DEBUG environment variable is set before using it. This will prevent potential issues if the variable is not set.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The entrypoint script checks for environment variables but does not check for the DEBUG variable, which is used later in the script.
3. blob-store/s3.json.template:4
  • Draft comment:
    Ensure that allowing anonymous read access is intentional and secure for your use case. If not, consider removing or restricting this access.
  • Reason this comment was not posted:
    Comment did not seem useful.

Workflow ID: wflow_ooMQtoDKoQIffK0E


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@creatorrr creatorrr merged commit e02ee06 into dev Oct 13, 2024
6 checks passed
@creatorrr creatorrr deleted the f/blob-store branch October 13, 2024 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant