Magpie thredds #107
Magpie thredds #107
Conversation
fmigneault
commented
Nov 25, 2020
•
fmigneault
commented
- add custom config to allow display of UI catalog elements of thredds service protected behind magpie/twitcher
- bump versions to 3.x to get custom config stored directly in db
secure config for thredds
…ce type to permission config
|
Can one of the admins verify this patch? |
|
@fmigneault reading the docs again, it was not clear to me if user omit the |
Just above the config in https://pavics-magpie.readthedocs.io/en/latest/services.html#servicethredds
|
fmigneault
changed the base branch from
bump_magpie_and_twitcher_to_3.2.0
to
master
I changed the base so that we only use that PR. The one will be included automatically. |
|
@dbyrns @tlvu @matprov I see failing tests for raven, but I'm not sure if this is normal or not as even our master branch doesn't evaluate them at the moment. |
|
@fmigneault |
|
Don't wait on Raven notebooks to pass to merge this PR. |
|
@fmigneault In fact, you pull the trigger too fast man. We have never done any upgrade testing ! All the Jenkins pipeline done was test a freshly new deployment. |
https://hirondelle.crim.ca/twitcher/ows/proxy/thredds/ AFAIK this is the closest that can be tested before deploy to prod |
Since we operate the production deployment, it would be kind if you let us do the merge this in repo so we have the chance to test against our datasets. Remember all the DB migration problems on an existing instance with the previous magpie upgrade? |
|
@nikola-rados FYI, if you want to block the autodeploy on your PAVICS instance, you just have to create a new uncommitted file in your birdhouse-deploy checkout and the autodeploy will refuse to deploy new changes. Hope you still have the time to do this. We have not yet tested the DB migration on existing instance of this PR. All the testing was done on a fresh new instances. |
|
@fmigneault I propose a revert of this PR so you take your time to flush out all the upgrade randomness. |
…hredds" This reverts commit 6ab2ae8, reversing changes made to 947ea40. Production has not been upgraded to the broken Magpie (PR bird-house/birdhouse-deploy#107) so this revert is so that Jenkins pass again on production.
Revert "Merge pull request #107 from bird-house/magpie-thredds" This reverts commit 7be3eea, reversing changes made to c0ffb41. PR #107 upgrade path is unstable, see issue Ouranosinc/Magpie#403 and issue Ouranosinc/Magpie#401. Revert so we can remove the autodeploy block on production instances and give us time to properly test any upcoming fixes. Also unblock Finch to go to production #138. Branch https://github.com/bird-house/birdhouse-deploy/tree/restore-previous-broken-magpie-upgrade-so-we-can-work-on-a-fix created so we have a place to work on a fix. Matching notebooks revert: Ouranosinc/pavics-sdi#211
…magpie-to-fix-jenkins-in-production Revert "Merge pull request #209 from Ouranosinc/magpie-unauthorized-thredds" This reverts commit 6ab2ae8, reversing changes made to 947ea40. Production has not been upgraded to the broken Magpie (PR bird-house/birdhouse-deploy#107) so this revert is so that Jenkins pass again on production. Jenkins build: https://daccs-jenkins.crim.ca/job/PAVICS-e2e-workflow-tests/job/master/242/console The `finch-usage.ipynb` failure is due to bird-house/finch#161 (comment), a new Finch will be deployed to prod soon (bird-house/birdhouse-deploy#138). Otherwise, with this revert, production will be green. The original #209 is back in branch https://github.com/Ouranosinc/pavics-sdi/tree/restore-change-for-new-broken-magpie-so-we-can-test-magpie-fix so we can test new Magpie with fix. Matching PAVICS stack revert: bird-house/birdhouse-deploy#137
…e-upgrade-so-we-can-work-on-a-fix Magpie upgrade strike II Strike II of this original PR #107. Matching notebook fix Ouranosinc/pavics-sdi#218 Performed test upgrade on staging (Medus) using prod (Boreas) Magpie DB, everything went well and Jenkins passed (http://jenkins.ouranos.ca/job/ouranos-staging/job/medus.ouranos.ca/80/parameters/). This Jenkins build uses the corresponding branch in Ouranosinc/pavics-sdi#218 and with `TEST_MAGPIE_AUTH` enabled. Manual upgrade migration procedure: 1. Save `/data/magpie_persist` folder from prod `pavics.ouranos.ca`: `cd /data; tar czf magpie_persist.prod.tgz magpie_persist` 2. scp `magpie_persist.prod.tgz` to `medus` 3. login to `medus` 4. `cd /path/to/birdhouse-deploy/birdhouse` 3. `./pavics-compose.sh down` 4. `git checkout master` 5. `cd /data` 2. `rm -rf magpie_persist` 3. `tar xzf magpie_persist.prod.tgz` # restore Magpie DB with prod version 4. `cd /path/to/birdhouse-deploy/birdhouse` 5. `./pavics-compose.sh up -d` 3. Update `env.local` `MAGPIE_ADMIN_PASSWORD` with prod passwd for Twitcher to be able to access Magpie since we juste restore the Magpie DB from prod 4. `./pavics-compose.sh restart twitcher` # for Twitcher to get new Magpie admin passwd 4. Baseline working state: trigger Jenkins test suite, ensure all pass except `pavics_thredds.ipynb` that requires new Magpie 5. Baseline working state: view existing services permissions on group Anonymous (https://medus.ouranos.ca/magpie/ui/groups/anonymous/default) 6. `git checkout restore-previous-broken-magpie-upgrade-so-we-can-work-on-a-fix` # This current branch 7. `./pavics-compose.sh up -d` # upgrade to new Magpie 8. `docker logs magpie`: check no DB migration error 9. Trigger Jenkins test suite again
Minor fix to `install-docker.sh` and comment update for other scripts due to Magpie upgrade Found when trying to bring up the new production host for PAVICS. `install-docker.sh`: fix to work with users with sudo priviledge. Before it needed user `root`. Other comments in scripts are due to new Magpie in PR #107.
