Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix advanced provider options for dags that override ingest records #4214

Merged
merged 13 commits into from
May 1, 2024
Merged

Fix advanced provider options for dags that override ingest records #4214

merged 13 commits into from
May 1, 2024

Commits on Apr 25, 2024

  1. Add support for fixed_query_params to ProviderDataIngester

    @stacimc
    stacimc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    a1ef682 View commit details
    Browse the repository at this point in the history

  2. Update ScienceMuseum to use new base class features

    @stacimc
    stacimc committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    724346b View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. Update other dags that override ingest_records to use new feature

    @stacimc
    stacimc committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    01670f0 View commit details
    Browse the repository at this point in the history

  2. Update TimeDelineatedProviderDataIngester to work with new features

    @stacimc
    stacimc committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    0a601de View commit details
    Browse the repository at this point in the history

  3. Fix tests

    @stacimc
    stacimc committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    b88c463 View commit details
    Browse the repository at this point in the history

  4. Wait to raise skipped ingestion errors until all rounds are done

    @stacimc
    stacimc committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    43c9454 View commit details
    Browse the repository at this point in the history

  5. Make TimeDelineatedProviderDataIngester more flexible to allow additi… 

    …onal fixed query params
    @stacimc
    stacimc committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    cbb2f19 View commit details
    Browse the repository at this point in the history

  6. Fix issue with Finnish museums initial_query_params

    @stacimc
    stacimc committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    ddf24a5 View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2024

  1. Add more test coverage

    @stacimc
    stacimc committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    d7552b6 View commit details
    Browse the repository at this point in the history

  2. Fix error when initial_query_params not provided for Finnish

    @stacimc
    stacimc committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    02d117c View commit details
    Browse the repository at this point in the history

  3. Base Science Museum page number off previous params

    @stacimc
    stacimc committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    4838215 View commit details
    Browse the repository at this point in the history

  4. Make logs more readable

    @stacimc
    stacimc committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    0775be3 View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. Fix typo 

    Co-authored-by: Krystle Salazar <krystle.salazar@automattic.com>
    @stacimc @krysal
    stacimc and krysal authored May 1, 2024
    Configuration menu
    Copy the full SHA
    b1ea828 View commit details
    Browse the repository at this point in the history