Skip to content

Updated metadata cherry-pick and management commands for learning activity and duration defaults#3408

Merged
rtibbles merged 17 commits intolearningequality:hotfixesfrom
rtibbles:cherry_pick_that_data
Jun 9, 2022
Merged

Updated metadata cherry-pick and management commands for learning activity and duration defaults#3408
rtibbles merged 17 commits intolearningequality:hotfixesfrom
rtibbles:cherry_pick_that_data

Conversation

@rtibbles
Copy link
Member

@rtibbles rtibbles commented Jun 7, 2022

Cherry-picks and copies model updates, dependency updates and migrations to hotfixes needed to launch DB schema updates for metadata changes in advance of production launch.

Adds default setting for new content nodes of learning activity based on kind.

Adds a management command to back fill default values for learning activities based on their kind.

Example output of management command:

Setting default learning activities for kind: exercise
INFO:command:Setting default learning activities for kind: exercise
Updated 601 content nodes of kind exercise with learning activity VwRCom7G
INFO:command:Updated 601 content nodes of kind exercise with learning activity VwRCom7G
Finished setting default learning activities for kind: exercise in 0.12742137908935547 seconds
INFO:command:Finished setting default learning activities for kind: exercise in 0.12742137908935547 seconds
Setting default learning activities for kind: video
INFO:command:Setting default learning activities for kind: video
Updated 497 content nodes of kind video with learning activity UD5UGM0z
INFO:command:Updated 497 content nodes of kind video with learning activity UD5UGM0z
Finished setting default learning activities for kind: video in 0.047971248626708984 seconds
INFO:command:Finished setting default learning activities for kind: video in 0.047971248626708984 seconds
Setting default learning activities for kind: audio
INFO:command:Setting default learning activities for kind: audio
Updated 527 content nodes of kind audio with learning activity mkA1R3NU
INFO:command:Updated 527 content nodes of kind audio with learning activity mkA1R3NU
Finished setting default learning activities for kind: audio in 0.01669907569885254 seconds
INFO:command:Finished setting default learning activities for kind: audio in 0.01669907569885254 seconds
Setting default learning activities for kind: document
INFO:command:Setting default learning activities for kind: document
Updated 507 content nodes of kind document with learning activity wA01urpi
INFO:command:Updated 507 content nodes of kind document with learning activity wA01urpi
Finished setting default learning activities for kind: document in 0.018065690994262695 seconds
INFO:command:Finished setting default learning activities for kind: document in 0.018065690994262695 seconds
Setting default learning activities for kind: html5
INFO:command:Setting default learning activities for kind: html5
Updated 557 content nodes of kind html5 with learning activity #j8L0eq3
INFO:command:Updated 557 content nodes of kind html5 with learning activity #j8L0eq3
Finished setting default learning activities for kind: html5 in 0.015359878540039062 seconds
INFO:command:Finished setting default learning activities for kind: html5 in 0.015359878540039062 seconds
Setting default learning activities for kind: h5p
INFO:command:Setting default learning activities for kind: h5p
Finished setting default learning activities for kind: h5p in 0.0013689994812011719 seconds
INFO:command:Finished setting default learning activities for kind: h5p in 0.0013689994812011719 seconds
Setting default learning activities for kind: slideshow
INFO:command:Setting default learning activities for kind: slideshow
Finished setting default learning activities for kind: slideshow in 0.0013191699981689453 seconds
INFO:command:Finished setting default learning activities for kind: slideshow in 0.0013191699981689453 seconds
Finished setting all null learning activities in 0.22896647453308105 seconds
INFO:command:Finished setting all null learning activities in 0.22896647453308105 seconds

Fixes #3400

Re-adds ffmpeg to dockerfile dependencies.
Adds management command to read files and set duration on all media files.

Example output of management command:

Setting default duration for media presets: ['audio', 'high_res_video', 'low_res_video', 'video_dependency', 'audio_dependency']
INFO:command:Setting default duration for media presets: ['audio', 'high_res_video', 'low_res_video', 'video_dependency', 'audio_dependency']
50585be4e3159a71c874c590d2ba12ec.mp3
File 50585be4e3159a71c874c590d2ba12ec.mp3 not found
WARNING:command:File 50585be4e3159a71c874c590d2ba12ec.mp3 not found
50585be4e3159a71c874c590d2ba12ec.mp4
storage/e/c/ecf46461d8376427137b7d78f1a76820.mp4
File ecf46461d8376427137b7d78f1a76820.mp4 not found
WARNING:command:File ecf46461d8376427137b7d78f1a76820.mp4 not found
storage/9/b/9bd33dd33ec83f77ee6c8fdd3f75e4bf.mp4
File 9bd33dd33ec83f77ee6c8fdd3f75e4bf.mp4 not found
WARNING:command:File 9bd33dd33ec83f77ee6c8fdd3f75e4bf.mp4 not found
storage/6/a/6a6f2dc374d99973e9162f9c22490eab.mp4
File 6a6f2dc374d99973e9162f9c22490eab.mp4 not found
WARNING:command:File 6a6f2dc374d99973e9162f9c22490eab.mp4 not found
Finished setting all null duration for 1058 files in 0.9561398029327393 seconds
INFO:command:Finished setting all null duration for 1058 files in 0.9561398029327393 seconds

Fixes #3401

dependabot bot and others added 11 commits June 7, 2022 13:44
Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.2.0 to 8.3.2.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/master/CHANGES.rst)
- [Commits](python-pillow/Pillow@8.2.0...8.3.2)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [django](https://github.com/django/django) from 3.2.4 to 3.2.5.
- [Release notes](https://github.com/django/django/releases)
- [Commits](django/django@3.2.4...3.2.5)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [django](https://github.com/django/django) from 3.2.5 to 3.2.13.
- [Release notes](https://github.com/django/django/releases)
- [Commits](django/django@3.2.5...3.2.13)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
@rtibbles rtibbles force-pushed the cherry_pick_that_data branch from 31933cd to 5d6711a Compare June 7, 2022 21:45
@rtibbles rtibbles changed the title Updated metadata cherry-pick Updated metadata cherry-pick and management command for learning activity defaults Jun 7, 2022
@rtibbles rtibbles requested a review from bjester June 7, 2022 22:48
@rtibbles rtibbles marked this pull request as ready for review June 7, 2022 22:48
@rtibbles rtibbles changed the title Updated metadata cherry-pick and management command for learning activity defaults Updated metadata cherry-pick and management commands for learning activity and duration defaults Jun 8, 2022
Copy link
Member

@bjester bjester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! One idea for the ffprobe duration extraction, but it may not work any better. We should test these on hotfixes post-deploy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants