object-storage-tests

A growing test suite for S3-like object storage implementations.

About

This project uses the load tests framework k6 (extended with some extensions) as the basis to execute test scenarios using popular open source command line tools like aws-cli, rclone, swift-cli as well as the open source k6 module k6-jslib-aws.

Roadmap

This is an early-stage project and the main focus is to test proprietary object storage providers, like AWS and Digital Ocean. But we would like to include open source/self-hosted object storage providers as well, some extra tests for OpenStack Swift providers are available and other free solutions like MinIO using the mc tool are in the roadmap.

An informal kanban board can be viewed here, card descriptions might be mixed in Portuguese and English.

Requirements

This tool was only tested on Linux machines with just and podman (or docker) installed.

Usage

Download latest version

git clone https://github.com/marmotitude/object-storage-tests.git
cd object-storage-tests

Setup remotes (config.yaml)

To run the test against a S3-compatible provider (we call them "remotes"), first copy the example.config.yaml to config.yaml and edit it to include your credentials.

cp example.config.yaml config.yaml
vim config.yaml #edit with your remotes

Run tests using a pre-made docker image

object-storage-tests is available as two docker images:

• tag latest is the main tests runner with its commands and requirements.
• tag devshell is a shell for developers to use it interactively and make contributions.

If you are on a machine with podman installed, you can use just run <command> to execute a command from within the main test runner image (tag latest). For example:

list available commands

just run

list available remotes

just run list-remotes

list available test scenarios

just run list-tests

run a single test scenario agains a single remote

just run test aws-east-1 boto3-presigned

The output of the tests are stored in a folder named results.

run the default s3 scenarions on a single remote

just run test aws-east-1

(Optional) mgc cli

If you have remotes that have a "mgc" config, copy the cli binary to the project path, renaming the executable to mgc.

Run commands from inside the devshell

if you are inside the devshell, or not using the OCI images, you don't need the word run (see the "Contributing" section below), you can use just <command> directly.

License

MIT License (c) 2023 Marmotidude and AUTHORS

Contributing

The easiest way to have a working developing environment is to use the provided dev-shell, using distrobox:

First assemble the distrobox with:

just assemble-dev

Then enter it with

just dev

To run the tests from inside the devshell use:

just test <remote name>

The devshell is a container with all the project tools installed, plus some extra developer tools like a code editor and a nice shell, to know more about this dev image see devshell.Dockerfile.

But you dont have to use it, feel free to install the tools used by this project:

• gotpl
• dasel
• k6 + xk6-exec + xk6-file + xk6-yaml
• aws-cli
• rclone
• Optional
  • swift-cli
  • mgc (unreleased)

Check Dockerfile for an up-to-date complete list.

Acknowledgments

• Luizalabs / @kikoreis / @cprov: for sponsoring @fczuardi and for supporting an open culture
• rclone: for being a powerful swiss army knife tool
• Grafana / K6: for open sourcing tools and for fostering a community around them
• NixOs: for planting the reproducible builds seeds
• Universal Blue / @castrojo / Boxkit Alpine: for promoting usability, community-driven projects and showcasing nice tools
• Microsoft / Github: for providing a great platform for developers for free