Add python bandit

[tpansino]

Resolves #505

Proposed Changes

1. Adds support for Python's bandit security scanner

Readiness Checklist

Author/Contributor

• If documentation is needed for this change, has that been included in this pull request

Reviewing Maintainer

• Label as breaking if this is a large fundamental change
• Label as either automation, bug, documentation, enhancement, infrastructure, or performance

[codecov-commenter]

Codecov Report

Merging #533 (80ea1e4) into master (10e1465) will increase coverage by 1.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #533      +/-   ##
==========================================
+ Coverage   86.60%   87.64%   +1.03%     
==========================================
  Files         128      129       +1     
  Lines        2972     2978       +6     
==========================================
+ Hits         2574     2610      +36     
+ Misses        398      368      -30     

Impacted Files	Coverage Δ
...ests/test_megalinter/linters/python_bandit_test.py	100.00% <100.00%> (ø)
megalinter/MegaLinter.py	90.68% <0.00%> (+2.41%)	⬆️
megalinter/reporters/UpdatedSourcesReporter.py	89.18% <0.00%> (+2.70%)	⬆️
...alinter/tests/test_megalinter/helpers/utilstest.py	97.05% <0.00%> (+9.24%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 10e1465...80ea1e4. Read the comment docs.

[nvuillam]

Again a great contribution :)

• It seems bandit found an error within Mega-Linter own code , please can you correct it ?
• merge conflicts
• Please also add lines in changelog (insiders version), including about your previous PR, your work needs to be credited :)

[tpansino]

1. Actually it looks like there are 58 total errors reported by bandit. :( I will try to fix the error count parsing, but fixing 58 errors in project code will take more effort than I want to invest in this PR.
2. I think we discussed this before, but what is the point again of committing all these files when they are auto-generated? It seems like it only creates merge conflicts for contributors to resolve that are then overwritten on next rebuild. It feels very manual and wasteful, and there has to be a better way. For example, could the automation regenerate and commit/push the generated files after each PR is merged instead?
3. Thank you, but I'm not really interested in updating a changelog, even for my own contributions. The credit is nice but the changelog should be the responsibility of project maintainers to decide what goes in and write, in my opinion.

---

Just some friendly feedback.

As a contributor, my goals are to:

1. Understand how the project works quickly and identify where to make changes
2. Make the code, test, and doc changes necessary and get a PR published

So that I can add the value I want to see in the project.

I don't like it when:

• I have to repeatedly fix merge conflicts (especially when the files are auto-generated)
• I find out about undocumented requirements (updating a changelog, or that I will have to fix all errors found by any linters I might add to the project)

As a developer yourself, I imagine you share these dislikes :) And, you understand that these things can discourage contributors.

But also, as a maintainer, you want to have a high quality project. And you are just one person, so you want PRs to be as complete as possible and save you time and effort. I respect that very much, and your decision to reject a PR if it doesn't meet quality standards.

So with that, I will commit to fixing the error count if possible, and these 3 merge commits too since they are trivial (though annoying). I won't be adding changelog updates or fixing the 58 project errors, so if this is an issue, please kindly let me know, and I'll withdraw the PR.

Thank you for understanding. 🙇

[nvuillam]

Actually it looks like there are 58 total errors reported by bandit. :( I will try to fix the error count parsing, but fixing 58 errors in project code will take more effort than I want to invest in this PR.

When I looked yesterday there was only one error, saying to call yaml.safe_load instead of yaml.load, I'll check after CI job has run again
If there are really 58 errors, you can add PYTHON_BANDIT_DISABLE_ERRORS: true in Mega-Linter.yml and I or someone will correct them later :)

I have to repeatedly fix merge conflicts (especially when the files are auto-generated)

Usually just running bash build.sh recalculates everything so there are no conflicts to manually fix, except when it is in python code of course

I think we discussed this before, but what is the point again of committing all these files when they are auto-generated? It seems like it only creates merge conflicts for contributors to resolve that are then overwritten on next rebuild. It feels very manual and wasteful, and there has to be a better way. For example, could the automation regenerate and commit/push the generated files after each PR is merged instead?

Again, I agree with you... but it requires time that I don't really have these days because of my job as Salesforce Unit CTO in Hardis Group, job that I did not have when I started MegaLinter during Covid lockdown ^^
I don't say I'll never do it, but I can't tell when :/

But also, as a maintainer, you want to have a high quality project. And you are just one person, so you want PRs to be as complete as possible and save you time and effort. I respect that very much, and your decision to reject a PR if it doesn't meet quality standards.

I built Mega-Linter for open-source community, not for glory or money, my goal is to provide the best tool to the community, so I'm totally open to share the maintenance... maybe you are interested ? If so, let's make a call someday :)

So with that, I will commit to fixing the error count if possible, and these 3 merge commits too since they are trivial (though annoying). I won't be adding changelog updates or fixing the 58 project errors, so if this is an issue, please kindly let me know, and I'll withdraw the PR.

Do what you can, I'll obviously not reject this nice and useful PR, at worse I'll complete it myself :)

[nvuillam]

Note: It seems ansible-lint has been removed from apk alpine packages, I'll have to make a fix before this PR passes

[nvuillam]

FYI i'm still solving npm / WORKDIR issue in Dcokerfile, until that all PRs are blocked :/

[nvuillam]

@tpansino master is now ok for CI, and I added something that you my appreciate: now bash build.sh does not update doc, so less conflicts to manage :)
Auto-update-linters job now manages doc update, and if you wanna do it manually, it can be done with bash build.sh --doc :)

[nvuillam]

It seems bandit does not like Mega-Linter code :D
Before solving the issues, I made them non blocking :) (see added commit)

[nvuillam]

CI errors were due to the addition of a linter applicable to MegaLinter own code... I merged the PR, let's see if it works ^^

[tpansino]

@nvuillam apologies for silence, I had family visiting from out of town and some other business.

Again, I agree with you... but it requires time that I don't really have these days because of my job as Salesforce Unit CTO in Hardis Group, job that I did not have when I started MegaLinter during Covid lockdown ^^
I don't say I'll never do it, but I can't tell when :/

I fully appreciate that projects like this are free labor, and you can only give so much :) I think it's great what you have built so far, any improvements will just make it even better.

I built Mega-Linter for open-source community, not for glory or money, my goal is to provide the best tool to the community, so I'm totally open to share the maintenance... maybe you are interested ? If so, let's make a call someday :)

I am interested in becoming a maintainer! But maybe not right now :( I'd like to continue to contribute as I have time, and maybe fix a few things I've contributed like the error counts, maybe add some tests for the auto-fix...

My team uses this tool extensively, and I also use it personally. I think it is the best linter aggregator I have seen, and I want to see it continue to succeed, but given my current responsibilities, I would be foolish to commit to more things. 😅

Thank you for merging! I will open more PRs to fix the error count, etc.

[nvuillam]

@tpansino no problem, we all have priorities and family comes first ;)

Thanks for your nice words, I'm really glad MegaLinter fits your team' requirements :)
And your future PRs will be of course very welcome, MegaLinter needs all the help it can get, no matter the maintener or contributor status :)