Skip to content

Integrate KiwixStorage #223

Open
Enhancement
@rgaudin

Description

Ours scrapers all depend on S3 and use KiwixStorage for it. That wrapper and repo are mostly untouched (yet working) and would great benefit from being integrated into scraperlib: tests, usage, dependencies update, better API, etc.

Now that we had experience using it, it probably make sense to reduce its API to what's actually used.

It's true that some non-scraper projects are using it (cardshop and zimfarm to upload to S3), but those are also using scraperlib and we've discussed making scraperlib's dependencies more modular via extras anyway.

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestquestionFurther information is requested

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions