Replies: 1 comment 1 reply
-
Version control of datasets was raised a couple of times as an interesting extension of datasets. I have seen some teams work around this by copying the dataset whenever they make larger changes. Would this work for you as well? |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Describe the feature or potential improvement
When using a dataset for evaluation, it is often necessary to update the dataset items, especially the ground truth. After the update, the old ground truth is no longer visible. As a result, the evaluation history becomes difficult to interpret because the standard answers have been modified. Previously, the score was 80, but now, due to the changes in the answers, it has become 95. However, for the evaluation with a score of 80, we are unable to see the old standard answers, making it difficult to explain the score reason.
Additional information
No response
Beta Was this translation helpful? Give feedback.
All reactions