Merged
Conversation
Compare IOU with each user instead of grouping all the old users into one group.
Added - A row in table to store iou score materix as pandas - ability to download these csvs as a zip folder for more data anaysis -Yeah
Sean1572
added a commit
that referenced
this pull request
Jun 14, 2022
* init * Fixed piha and pyrenote merger page * Working on rec system * Current code that works with a single annotator Code allows person to go through and annotate, slowly reducing what they can do * Fixed small bugs with second and third tester It works! Retires clips after 2 people check it if it hits 75% accuracy. Things to consider: 2 users may be too few, perhaps 3. Next: I need to also take away annotations other users have no done for projects running iou metrics * Added feature that only shows annotations made by you * Display clips that only the users annotated for IOU Shows clips the users has never touched and clips that the user has completed themselves * Update piha.py * Added ability determine most recent clip recently annotated clips will be marked 1, older clips will be marked as 2 * Added Threshold, Max User Limit, and debug view on data Added power to change the given threshold, set a max limit for number of users to see the data, and included the number of views + confidence information on data portal (for now, for debuging purposes) * show previous values when editing project Displays the current confidence threshold and max user limit in the edit project form * Added confidence and retirement status to csv Added confidence and retirement information to csv. Also added counted status to csv. (Side note: fixed extra column being added by datetime object) * Added see what clips are retired from data portal * Block retired clips from view * Fixed bugs with normal use There was issues using next buttons with a diffrent active other than pending on an IOU project. Fixed this issue so active works as expected * bugs and made it easier to run confidence Fixed bugs that appeared during demo today and made confidence metric run during saving * Changed confidence score from precision to global Iou * bug-proofed dataframes and added mutliple labels Prevented empty overlap dataframes from breaking apicalls with an if statement. Also changed manual_id from bird to labelCate.value. NOTE: THIS WILL BREAK FOR MUTLICLASS LABELS, NEED TO ADD IN THAT FUNCTIONALITY * Updated TODO messages Made note of mutliple-select labels * Update piha.py * Replaced IOU with Pairwise (#262) * Replaced IOU with Pairwise Compare IOU with each user instead of grouping all the old users into one group. * Removed old debug and added new note * Reworked pairwise matrix * increased timeoue * Fixed 4 line bug * Added IOU CSV Storage to SQL Added - A row in table to store iou score materix as pandas - ability to download these csvs as a zip folder for more data anaysis -Yeah * Error Handler for confidence Put an general error handler around the clip confidence work just so clip successfully saves * Fixed No Relevant Audio Bug Fixed bug were no Relevant Audio button didn't save the clip
Sean1572
added a commit
that referenced
this pull request
Jun 27, 2022
* init * Fixed piha and pyrenote merger page * Working on rec system * Current code that works with a single annotator Code allows person to go through and annotate, slowly reducing what they can do * Fixed small bugs with second and third tester It works! Retires clips after 2 people check it if it hits 75% accuracy. Things to consider: 2 users may be too few, perhaps 3. Next: I need to also take away annotations other users have no done for projects running iou metrics * Added feature that only shows annotations made by you * Display clips that only the users annotated for IOU Shows clips the users has never touched and clips that the user has completed themselves * Update piha.py * Added ability determine most recent clip recently annotated clips will be marked 1, older clips will be marked as 2 * Added Threshold, Max User Limit, and debug view on data Added power to change the given threshold, set a max limit for number of users to see the data, and included the number of views + confidence information on data portal (for now, for debuging purposes) * show previous values when editing project Displays the current confidence threshold and max user limit in the edit project form * Added confidence and retirement status to csv Added confidence and retirement information to csv. Also added counted status to csv. (Side note: fixed extra column being added by datetime object) * Added see what clips are retired from data portal * Block retired clips from view * Fixed bugs with normal use There was issues using next buttons with a diffrent active other than pending on an IOU project. Fixed this issue so active works as expected * bugs and made it easier to run confidence Fixed bugs that appeared during demo today and made confidence metric run during saving * Changed confidence score from precision to global Iou * bug-proofed dataframes and added mutliple labels Prevented empty overlap dataframes from breaking apicalls with an if statement. Also changed manual_id from bird to labelCate.value. NOTE: THIS WILL BREAK FOR MUTLICLASS LABELS, NEED TO ADD IN THAT FUNCTIONALITY * Updated TODO messages Made note of mutliple-select labels * Update piha.py * Replaced IOU with Pairwise (#262) * Replaced IOU with Pairwise Compare IOU with each user instead of grouping all the old users into one group. * Removed old debug and added new note * Reworked pairwise matrix * increased timeoue * Fixed 4 line bug * Added IOU CSV Storage to SQL Added - A row in table to store iou score materix as pandas - ability to download these csvs as a zip folder for more data anaysis -Yeah * Error Handler for confidence Put an general error handler around the clip confidence work just so clip successfully saves * Fixed No Relevant Audio Bug Fixed bug were no Relevant Audio button didn't save the clip * Improved save all feature - Changed save all feature so it sends all segmentation data to the backend so it processes all at once as opposed to calling the api call mutliple times - This improves proformance slightly with confidence work as now we don't need to run mutliple iou mertics for each indivual annotation * Quick patches to save all feature - Removed unneeded frontend console output - Fixed issue where error message appears if save all does not finish when user moves to a new page - Fixed issue where if a user moves to a new clip, batch saving does not work * Fixed non_iou recommendation bug Fixed issue where if a project did not have IOU control pipeline enabled, it would not show clips on the data pipeline * Fix bug with quick start button Fixed bug where quick start would take you to annotation page even if no data was ready to be annotated! * Update package.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compare IOU with each user instead of grouping all the old users into one group.