Skip to content

Add state check to verifyarchives helper and use to verify FINAL Archives only #267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1,016 commits into from
Closed

Add state check to verifyarchives helper and use to verify FINAL Archives only #267

wants to merge 1,016 commits into from

Conversation

jonasbardino
Copy link
Contributor

Implement a simple -s STATE option in the verifyarchives.py helper to make it fail if the verified archive is not actually in STATE. Use it by default in the generated migverifyarchives cron job with -s FINAL, so that in case a non-FINAL archive should enter the list of archives to verify it will fail and tell why.

jonasbardino added 30 commits July 1, 2024 13:09
@jonasbardino
tighten configuration generator to limit external dependencies and al…
59aa03a 
…ign better with secure-by-default considerations. In effect this reduces the set of enabled services unless explicitly requested. The examples in the README have been updated accordingly.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6067 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
merge git ignores from PR72 in specific git repos
759051f
@jonasbardino
manually merge PR72 with minor adjustments. Added missing license hea…
8432e47 
…der, adjusted long comment to fit line, removed pyoth from recommended.txt now that it's in required.txt and fix a couple of typos in comment.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6069 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
recommended should include core dependencies for Makefile logic to work
9ddc73e 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6070 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
fix a couple of typos in comments and values reaching test fixtures
7a701a3 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6071 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Fix legacy hard-coded cert dir in template
ee7ec5c 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6072 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Update fixture to fit recent fix for hard-coded apache path.
27cecf9
@jonasbardino
Disable import mangling in autopep8 when integrated in emacs as it br…
c707318 
…eaks some cases where migrid modules require sys.path mangling

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6074 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
update openid discovery note to point to new genoiddiscovery helper
b34745a 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6075 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
add missing docstrings and adjust import a bit after disabling autope…
e7585aa 
…p8 import mangling

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6076 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
update fixture to latest conf comment change about genoiddiscovery.py
a6c4eb4 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6077 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Actually use configuration.auto_add_oidc_user for OpenID Connect sign…
ec5f345 
… up attempts. Both kind of OpenID sign up used to rely on the original configuration.auto_add_oid_user .

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6078 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
proper format of autopep8 options in emacs
d33ded8 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6079 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
print error on stderr
efbe2a6 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6080 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
follow-up to PR72. It turns out that we hit a situation in docker-mig…
ea98d03 
…rid where the 'somearg is keyword_auto' comparison in generateconfs fails, when passing the default 'AUTO' values e.g. for salts. This a.o. breaks the CI genoiddiscovery.py call because 'AUTO' ends up in the MiGserver.conf and causes havoc on parsing. Comparing the values with 'somearg == keyword_auto' instead solves the problems. Further analysis may be needed but perhaps it's due to string argument passing throught the command line or any mangling like somearg.upper() or the like that the memory address comparison with 'is' fails. In most other cases it makes no difference, which is used for comparing string literals.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6081 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Manually merged github PR74 - thanks @albu-diku
c910228 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6082 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
widen common missing file error filters slightly to cover e.g. cp, too.
1feb65b 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6083 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
a few more common expected file error filters
21cea8a 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6084 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
update test fixture to latest migerrors helper
84bf1b3 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6085 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
manually merge PR78, thanks @albu-diku
1064c90 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6086 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
manually merge PR76
6cf5ee6 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6087 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
manually merge PR81
19f5699 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6088 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Fix a byte vs string issue in importusers.
72aeb53
@jonasbardino
Just fix a typo in the help text
d417216 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6089 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
manually merge the essence of PR84 - thanks @albu-diku. For the recor…
a782b43 
…d I made a number of adjustments to document said conf options and put them in a slightly more 'logical' location and order, which I realized was not at all obvious for anyone but us few long-term operators.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6090 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Fix a couple of symlink checks to actually see if the symlink itself …
5d8596f 
…exists rather than if the path pointed to exists. Addresses a regression in edituser with the unique id symlinks not being properly removed in preparation for relinking it to the new X509 ID.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6091 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
manually merged PR82 with minor adjustments in line with review comme…
56060a5 
…nts and added missing docstrings on new generate helper functions.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6092 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
manually merge PR 85 with FAQ entry about SSH and rsync
a9aa886 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6093 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Remove references to retired IDMC and retire temporary migrid.erda.dk…
6ff82e8 
… alias for migrid.org. Clean up stale system warnings

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6094 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Manually merged PR89 with some modifications in line with review comm…
80dece2 
…ents on github: fixed a comment typo, a PEP8 style recommendation and rearranged a few lines for tighter clustering of related code lines.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6096 b75ad72c-e7d7-11dd-a971-7dbc132099af
Martin-Rehr and others added 24 commits May 13, 2025 15:10
@Martin-Rehr
miglustrequota: Added 'quota_backend' prefix to quota paths
d6892f8
@Martin-Rehr
Manulally merge PR239 to show filedownload progress through browser
7bab6f0 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6232 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Attempt to filter fluff from release tarballs as described on
1e393ea 
https://softwareengineering.stackexchange.com/questions/366771/how-do-we-avoid-development-files-in-the-release
@jonasbardino
Manually merge PR198 to move addheader helper into bin/ now that we h…
01e3e49 
…ave it filtered from released source archives through the .gitattributes file.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6233 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Manually merge PR242 to update external account (migoid/migcert) expi…
6c41354 
…re warning emails to point to the new built-in Renew button at the Account page where applicable in order to limit operator overhead from account renewals.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6234 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Manually merge PR245 to redirect users to the vhost base URL after su…
71d5854 
…ccessfully logging out from OIDC and all sessions.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6235 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
add support for secondary upstream oidc logout
86d8c38 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6236 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Manually merge PR244 to add support for new include_sections option i…
8b07eae 
…n MiGserver.conf which will load SECTIONNAME into parsed configuration for any sectionname.conf files from the corresponding folder. Useful e.g. to keep rather involved cloud sections out of the main MiGserver.conf

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6237 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
screenshot helper relies on selenium so include it in recommended.txt
5785943 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6238 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Extend dependencies to recommended for pylint to e.g. pull in seleniu…
780f1df 
…m as well.
@jonasbardino
urlparse moved between py2 and 3. Use more robust import.
892faf1 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6239 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
manually merge PR250 to ignore some more frequent security scan noise…
9760d45 
… in migerrors cronjob

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6240 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Manually merge PR249 to follow-up on PR244 and allow overriding non-e…
c841d14 
…ssential GLOBAL and SITE conf options through include sections

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6241 b75ad72c-e7d7-11dd-a971-7dbc132099af
@rasmunk @jonasbardino
Merge the changes introduced in PR #247
7b75f6a 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6242 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
adjust logout to have double confirm for ucph but only one for mig op…
a75ee19 
…enid

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6243 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Switch to install recommended dependencies with ALLDEPS arg to make.
b8df8fe
@rasmunk @jonasbardino
Merge the changes introduces in PR #254
2bd24f6 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6244 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
try to fix dependency error with ALLDEPS in GH actions
bc23675 
git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6245 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Manually merge PR255 to fix type of jupyter_services_enable_proxy_htt…
d38179e 
…ps argument

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6246 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
fix version dependency for notebook_parameterizer package to address …
11381ba 
…failing GH Actions on rocky8

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6247 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
manually merge PR256 through svn to fix migadmin page on py3. Hopeful…
4c69537 
…ly last svn action.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6248 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino
Install python library dependencies for packages in recommended.txt t…
64418bb 
…o address

e.g. python sanity checks failing on Rocky8 due to missing enchant system lib.
@jonasbardino
Use apt rather than dnf in ubuntu container.
f9245cd
@jonasbardino
Final svn commit updating status a bit and informing about the switch…
0db398e 
… to python3 with future hosting at github.

git-svn-id: svn+ssh://svn.code.sf.net/p/migrid/code/trunk@6249 b75ad72c-e7d7-11dd-a971-7dbc132099af
@jonasbardino jonasbardino self-assigned this Jun 20, 2025
@jonasbardino jonasbardino added the enhancement New feature or request label Jun 20, 2025
@jonasbardino
Implement a simple -s STATE option in the verifyarchives.py helper …
684bfd2 
…to make it

fail if the verified archive is not actually in `STATE`. Use it by default in
the generated `migverifyarchives` cron job with `-s FINAL`, so that in case a
non-FINAL archive should enter the list of archives to verify it will fail and
tell why.
@jonasbardino
Copy link
Contributor Author

Sigh, PR of next commit against edge causes a lot of noise. Retired and fixed in PR #268 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@jonasbardino jonasbardino

Labels
enhancement New feature or request
Projects
None yet
Milestone
Python3 support for general research ...
Development

Successfully merging this pull request may close these issues.
4 participants
@jonasbardino @Martin-Rehr @albu-diku @rasmunk