Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(wren-ai-service): spider metrics in evaluation #761

Merged
merged 36 commits into from
Oct 16, 2024
Merged

feat(wren-ai-service): spider metrics in evaluation #761

merged 36 commits into from
Oct 16, 2024

Commits on Oct 15, 2024

  1. restructure eval files

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    326def6 View commit details
    Browse the repository at this point in the history

  2. add spider_database

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    9f94d26 View commit details
    Browse the repository at this point in the history

  3. ignore spider database

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    d559465 View commit details
    Browse the repository at this point in the history

  4. update

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    61e670d View commit details
    Browse the repository at this point in the history

  5. generate mdl and question sql pairs by db

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    58d380a View commit details
    Browse the repository at this point in the history

  6. refactor

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    a7ecfa6 View commit details
    Browse the repository at this point in the history

  7. update

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    f77db0d View commit details
    Browse the repository at this point in the history

  8. add comments

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    695961c View commit details
    Browse the repository at this point in the history

  9. allow data curation app connects to duckdb

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    277f247 View commit details
    Browse the repository at this point in the history

  10. fix bug

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    d848ee0 View commit details
    Browse the repository at this point in the history

  11. revert

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    eb7894b View commit details
    Browse the repository at this point in the history

  12. fix relationship name

    @cyyeh
    cyyeh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    d2a947c View commit details
    Browse the repository at this point in the history

  13. feat: implement exact match accuracy metric of spider eval

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    8c875aa View commit details
    Browse the repository at this point in the history

  14. chore: move the metric to an independent module

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    8902f7a View commit details
    Browse the repository at this point in the history

  15. chore: remove redundant module

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    45ce662 View commit details
    Browse the repository at this point in the history

  16. chore: fix typo

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    e775c21 View commit details
    Browse the repository at this point in the history

  17. chore: update lock file

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    9befa19 View commit details
    Browse the repository at this point in the history

  18. chore: change the path of kmap and db dir

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    5cbe200 View commit details
    Browse the repository at this point in the history

  19. feat: put the catalog name into additional metadata in every prediction

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    00f1dce View commit details
    Browse the repository at this point in the history

  20. fix: get catalog from the wrong place

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    39e6d5c View commit details
    Browse the repository at this point in the history

  21. feat: support duckdb on prediction

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    bdb2cec View commit details
    Browse the repository at this point in the history

  22. feat: modify the metric config to use duckdb in eval

    @paopa
    paopa committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    5e72436 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. update README and command

    @cyyeh
    cyyeh committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    e011bf1 View commit details
    Browse the repository at this point in the history

  2. update

    @cyyeh
    cyyeh committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    fa862ad View commit details
    Browse the repository at this point in the history

  3. refine env name

    @cyyeh
    cyyeh committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    c88a182 View commit details
    Browse the repository at this point in the history

  4. fix eval bug using duckdb

    @cyyeh
    cyyeh committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    5548ac1 View commit details
    Browse the repository at this point in the history

  5. fix bug

    @cyyeh
    cyyeh committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    89d55b9 View commit details
    Browse the repository at this point in the history

  6. fix: wrong additional metadata

    @paopa
    paopa committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    6c049cd View commit details
    Browse the repository at this point in the history

  7. feat: implement Spider Execution Accuracy

    @paopa
    paopa committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    84dec12 View commit details
    Browse the repository at this point in the history

  8. feat: add execution accuracy into generation and end-to-end eval pipe… 

    …line
    @paopa
    paopa committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    036a6c7 View commit details
    Browse the repository at this point in the history

  9. fix: rewrite sql

    @paopa
    paopa committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    6583eed View commit details
    Browse the repository at this point in the history

  10. fix bug

    @cyyeh
    cyyeh committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    3643a79 View commit details
    Browse the repository at this point in the history

  11. fix mdl table construction bug

    @cyyeh
    cyyeh committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    c698550 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'main' into feat/spider-metrics

    @cyyeh
    cyyeh authored Oct 16, 2024
    Configuration menu
    Copy the full SHA
    679f3a5 View commit details
    Browse the repository at this point in the history

  13. fix: don't rewrite sql in execution accuracy metrci

    @paopa
    paopa committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    f3f40a3 View commit details
    Browse the repository at this point in the history

  14. try fix github actions issue

    @cyyeh
    cyyeh committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    7f95b7d View commit details
    Browse the repository at this point in the history