This issue has been migrated from the CC Search Catalog repository

Author: aldenstpage
Date: Wed Jul 08 2020
Labels: 🙅 status: discontinued

When an image is added to cccatalog, we should schedule it for crawling so we can extract useful metadata about the image. That means producing a message to inbound_images in our Kafka cluster:

{
    "url": "https://example.gov/example.jpg",
    "uuid": "7563efd4-58d0-41eb-9a4f-3903d36a5225",
    "source": "example"
}

Let's finish our current big crawl for Rekognition before we implement this.

Original Comments:

mathemancer commented on Fri Jul 10 2020:

We could also do this in bulk by querying for images with 'created_on' greater than some date (along with filtering as before). This way, we don't have to introduce logic to know whether we're creating or updating an image into the loader workflow.

Maybe a weekly job to find any image 'created_on' after a certain date.
source