Merged
Conversation
Code allows person to go through and annotate, slowly reducing what they can do
It works! Retires clips after 2 people check it if it hits 75% accuracy. Things to consider: 2 users may be too few, perhaps 3. Next: I need to also take away annotations other users have no done for projects running iou metrics
Shows clips the users has never touched and clips that the user has completed themselves
recently annotated clips will be marked 1, older clips will be marked as 2
Added power to change the given threshold, set a max limit for number of users to see the data, and included the number of views + confidence information on data portal (for now, for debuging purposes)
Displays the current confidence threshold and max user limit in the edit project form
Added confidence and retirement information to csv. Also added counted status to csv. (Side note: fixed extra column being added by datetime object)
There was issues using next buttons with a diffrent active other than pending on an IOU project. Fixed this issue so active works as expected
Fixed bugs that appeared during demo today and made confidence metric run during saving
Prevented empty overlap dataframes from breaking apicalls with an if statement. Also changed manual_id from bird to labelCate.value. NOTE: THIS WILL BREAK FOR MUTLICLASS LABELS, NEED TO ADD IN THAT FUNCTIONALITY
Made note of mutliple-select labels
* Replaced IOU with Pairwise Compare IOU with each user instead of grouping all the old users into one group. * Removed old debug and added new note * Reworked pairwise matrix * increased timeoue * Fixed 4 line bug * Added IOU CSV Storage to SQL Added - A row in table to store iou score materix as pandas - ability to download these csvs as a zip folder for more data anaysis -Yeah
Put an general error handler around the clip confidence work just so clip successfully saves
Fixed bug were no Relevant Audio button didn't save the clip
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Created a clip retirement system! Based on the precision between users, Pyrenote will automatically retire clips with high degrees of overlap. This allows users to annotate previously poorly labeled clips in the attempt in improving metrics
Added
Testing:
TODO
Check to make sure that the quick load data button and other data viewing types (all, marked for review, etc) are working normallyAdd a retired clips view on frontend in the data portalMake sure this is limited to admins onlyadd a data column in sql tables that denotes which clips are considered "retired" for data results purposesAdd counted label to csvBUGS