Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: instructlab/eval
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.4.2
Choose a base ref
...
head repository: instructlab/eval
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.5.0
Choose a head ref
  • 17 commits
  • 7 files changed
  • 3 contributors

Commits on Dec 10, 2024

  1. build(deps): bump pypa/gh-action-pypi-publish from 1.12.2 to 1.12.3

    Bumps [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) from 1.12.2 to 1.12.3.
    - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
    - [Commits](pypa/gh-action-pypi-publish@15c56db...67339c7)
    
    ---
    updated-dependencies:
    - dependency-name: pypa/gh-action-pypi-publish
      dependency-type: direct:production
      update-type: version-update:semver-patch
    ...
    
    Signed-off-by: dependabot[bot] <support@github.com>
    dependabot[bot] authored Dec 10, 2024
    Configuration menu
    Copy the full SHA
    81ab1a9 View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2024

  1. build(deps): bump hynek/build-and-inspect-python-package

    Bumps [hynek/build-and-inspect-python-package](https://github.com/hynek/build-and-inspect-python-package) from 2.10.0 to 2.11.0.
    - [Release notes](https://github.com/hynek/build-and-inspect-python-package/releases)
    - [Changelog](https://github.com/hynek/build-and-inspect-python-package/blob/main/CHANGELOG.md)
    - [Commits](hynek/build-and-inspect-python-package@f01e4d0...14c7e53)
    
    ---
    updated-dependencies:
    - dependency-name: hynek/build-and-inspect-python-package
      dependency-type: direct:production
      update-type: version-update:semver-minor
    ...
    
    Signed-off-by: dependabot[bot] <support@github.com>
    dependabot[bot] authored Dec 16, 2024
    Configuration menu
    Copy the full SHA
    aeb500e View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2024

  1. feat: update release-strategy to include discord

    In order to facilliate the addition of Discord as a new chat platform, we will be mirroring
    all announcements made regarding releases as currently happens on Slack to also take place
    on Discord. This commit updates our policy document to reflect this change
    
    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Dec 17, 2024
    Configuration menu
    Copy the full SHA
    742e05a View commit details
    Browse the repository at this point in the history
  2. Merge pull request #196 from instructlab/dependabot/github_actions/py…

    …pa/gh-action-pypi-publish-1.12.3
    
    build(deps): bump pypa/gh-action-pypi-publish from 1.12.2 to 1.12.3
    mergify[bot] authored Dec 17, 2024
    Configuration menu
    Copy the full SHA
    a8bd84d View commit details
    Browse the repository at this point in the history
  3. Merge pull request #198 from instructlab/dependabot/github_actions/hy…

    …nek/build-and-inspect-python-package-2.11.0
    
    build(deps): bump hynek/build-and-inspect-python-package from 2.10.0 to 2.11.0
    mergify[bot] authored Dec 17, 2024
    Configuration menu
    Copy the full SHA
    1aefd04 View commit details
    Browse the repository at this point in the history
  4. Merge pull request #199 from RobotSail/add-discord

    feat: update release-strategy to include discord
    mergify[bot] authored Dec 17, 2024
    Configuration menu
    Copy the full SHA
    0d44bbe View commit details
    Browse the repository at this point in the history
  5. feat: add discord e2e status reporting

    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Dec 17, 2024
    Configuration menu
    Copy the full SHA
    bc497ed View commit details
    Browse the repository at this point in the history
  6. Merge pull request #200 from RobotSail/add-discord-e2e

    feat: add discord e2e status reporting
    mergify[bot] authored Dec 17, 2024
    Configuration menu
    Copy the full SHA
    2e7e405 View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2025

  1. adds basic ragas eval

    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Jan 8, 2025
    Configuration menu
    Copy the full SHA
    3443ffa View commit details
    Browse the repository at this point in the history
  2. feat: add ability for ragas to read from a list

    We want ragas to read from both a list as well as a list of samples
    
    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Jan 8, 2025
    Configuration menu
    Copy the full SHA
    8568b13 View commit details
    Browse the repository at this point in the history
  3. feat: add ability for answers to be generated from user questions

    When a dataset is provided and is missing the `response` field, we will need to generate these responses. This commit ensures that when this case happens, we will error out when a student model is not configured. Otherwise, we will always generate these responses if the student model exists, regardless if `response` is in the dataframe or not.
    
    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Jan 8, 2025
    Configuration menu
    Copy the full SHA
    58880c3 View commit details
    Browse the repository at this point in the history
  4. chore: add unit tests for ragas evaluator

    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Jan 8, 2025
    Configuration menu
    Copy the full SHA
    04117dd View commit details
    Browse the repository at this point in the history
  5. feat: update the Ragas evaluator to have the OpenAI client as somethi…

    …ng that gets passed in to __init__
    
    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Jan 8, 2025
    Configuration menu
    Copy the full SHA
    c6b5a70 View commit details
    Browse the repository at this point in the history
  6. chore: decouple tests into more atomic units

    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Jan 8, 2025
    Configuration menu
    Copy the full SHA
    ab3d168 View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2025

  1. Merge pull request #193 from RobotSail/add-ragas

    adds basic ragas eval
    mergify[bot] authored Jan 9, 2025
    Configuration menu
    Copy the full SHA
    c437ef2 View commit details
    Browse the repository at this point in the history
  2. chore: update changelog for 0.5.0

    Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
    RobotSail committed Jan 9, 2025
    Configuration menu
    Copy the full SHA
    ab37b54 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #208 from RobotSail/update-changelog

    chore: update changelog for 0.5.0
    mergify[bot] authored Jan 9, 2025
    Configuration menu
    Copy the full SHA
    e31d19b View commit details
    Browse the repository at this point in the history
Loading