Add new independent test to support running shell commands called "shellcommand"

[vanderpol]

Abstract

Add an independent test called "shellcommand" which supports running any arbitrary command or contents of a shell script on the target system. This feature should only be used if the content is trusted, ideally digitally signed either at the benchmark level, OVAL document level, or at the object level.

This allows for many more system configuration and vulnerability tests to be performed with OVAL without making the language overly large by adding numerous system commands to OVAL. This also will allow for security personal not well versed with OVAL to be able to create OVAL content. It should also help to simplify some of the overly complex binary trees of tests that has plagued OVAL in the past.

Link to Proposal
Once a proposal has been put into pull request form, add a link to the PR here. If and as alternate proposals or objections are added they should be linked here as well.

Additional context
Add any other context or screenshots about the enhancement.

[shawndwells]

1. Red Hat may have some opinions and lessons learned from implementation of Script Check Engine (https://www.open-scap.org/features/other-standards/sce/)

2. Can this also be implemented for all platforms? Specifically Windows via PowerShell and Cisco iOS and NX-OS?

[vanderpol]

@shawndwells Thanks for your input, we will check out the Script Check Engine for lessons learned there. As for Windows we plan for a windows specific windows;powershellcommand test (which essentially mirrors the unix:shellcommand, but allows for specific backend support for differences in the environments. We had not considered Cisco IOS, IOS XE or NX-OS as in scope for this, given the Cisco specific schemas which allow for running any global/line/interface command, I didn't see where Cisco was needing this type support.

[johnulmer-oval]

Please find the draft proposal for the unix shellcommand test attached. The introduction (Background) section of the attached PDF follows.

Background

OVAL test types have been relatively specific in their targeted functions and, typically, have been applicable to a specific system, sub-system, or application on a target machine. The time and effort required to add new tests to the OVAL specification and the relative scarcity of resources impede efficient development and support of the specification. To counter this and to broadly expand the breadth of OVAL tests in general, we propose two new OVAL tests, one each for the UNIX and Windows families, the Unix shell_comand test and the Windows powershell_command test. Each is much broader in their application than the previously narrow tests currently available in OVAL allowing a content author to use the full capability of the Unix shell (e.g. bash) or the Windows powershell to retrieve system characteristics.

This document proposes the Unix shell_command test, specifically. A second proposal covers the Windows powershell_command test.

Since this test requires the running of code supplied by content and since SCAP applications commonly run with elevated privileges, significant responsibility falls on the content author to do no harm. This also requires that any content stream that employs one of these tests MUST be from a known trusted source and be digitally signed. The use of any executables that are not supplied by the installed operating system is highly discouraged.

The advantages of these proposed tests include:
• dramatic flexibility in OVAL test design strategy.
• simple content and results structure.
• substantially reduced content development and testing resources.
• consistent the existing OVAL processing model.
• while initially proposed for the Unix and Windows Operating System families, this approach could later be easily extended to support other targets.

Once these tests are added, there will be a significant reduction in the need for other new OVAL tests and the resources required to develop and implement them.

The proposal document, example XML schema files, example content, and example results are attached.

Hmmm..... GH failing to accept my uploads and is not complaining. Supporting files will be available shortly.

[vanderpol]

Attaching proposald document, schemas, sample content and sample results for @johnulmer-oval
OVAL_Proposal_Unix_ShellCommand_test.zip

[vanderpol]

@OVAL-Community/oval-board-members we are looking for your high level feedback on the unix:shellcommand proposal, once we get some indication of approval, we will start the PR process.

[vanderpol]

@solind and/or @A-Biggs (feel free to tag others), but we would really like for you to beat up our proposal and let us know what you think. If we get a general thumbs up, we'll do a pull request and ask for a review there, but wanted overall gut feeling first. We pondered if we should include subexpression support, and our current design does not. Looking for feedback.

[solind]

Hi @vanderpol , I think you should consider adopting the CIS equivalent to this test, the shellcommand_test, which is a multi-platform equivalent that would work on both Windows and Unix. One advantage is that there are tools already able to support it, and content already using it.

You can find the schemas for that test here:
https://github.com/joval/jOVAL/tree/master/scap-extensions/schemas

[vanderpol]

@solind, we did review the CIS equivalent test, and had no idea from the documentation included that it was an independent test, we assumed it was a Unix only test for using a unix shell. Given the fact that there doesn't seem to be any proposal explaining the specification, or sample content/results and the documentation in the XSD are essentially "TO DO: Add some documentation", it's far from a spec that seems ready for prime time. Also given that CIS has indicated that they are dropping OVAL support going forward, and their content is not publicly available, and this is our "one chance" to update OVAL for the foreseeable future, I think we should adopt whatever specification will stand the test of time.

We pondered making a single shell test that would be independent and be for Windows and UNIX, and if you look at our proposals, they are nearly identical, but making separate tests allows for platform specific documentation, and the potential for platform specific improvements later in the future. And behind the scenes of every "independent" OVAL test is platform specific implementations, so the "savings" of an independent test are merely in the trivial amount of time required to make the XSD files, which is already done.

In reviewing the technical aspects of the CIS shellcommand test, we found the inclusion of the exit_status and the std_errrline to be interesting, but not really useful from a content/results perspective, so we intentionally excluded them, and let the application handle any errors. We didn't see any content needing to put those attributes in a state, nor any end user needing to see them. Once again, when looking at the documentation for the system characteristics item "TO DO: Add some documentation" didn't really feel ready for prime time.

If there is some content created for the CIS shellcommand, I feel it would be a trivial effort to write a conversion script/transform to convert it to whatever is deemed worthy of official inclusion in OVAL. The same would hold true for supporting a new/replacement shell test vs the CIS, most of underlying logic would be identical.

[wmunyan]

IIRC, there was never a formal proposal for it because the idea of a "shellcommand" construct was shot down pretty quickly, due mostly to reasoning around "you can run any command" including things that could potentially be malicious. I can only guess that is the reason for my FINE documentation of "TODO: Add some documentation". CIS did have some benchmarks that utilized the shellcommand constructs, and I've attached an (old) CentOS OVAL file that has some examples. Again, digging deep into my "memory archives", I believe the "exit_status" was to allow for tests that could potentially pass due to a failure in the command. If for whatever reason, no results were produced by the command and the exit_status was non-zero, perhaps that could be a "pass". As for the "stderr_line", I feel like that was used mostly for producing evidence in HTML reports, in case output was generated to stderr instead of stdout. It doesn't look like any of the tests in the attached OVAL use either of those elements, though... but I am pretty sure the "stderr_line" element was mostly for evidence and not for any state/item comparison purpose. Hope that helps! -Bill M.

…

On Wed, Oct 30, 2024 at 6:00 AM Jack Vander Pol ***@***.***> wrote: @solind <https://github.com/solind>, we did review the CIS equivalent test, and had no idea from the documentation included that it was an independent test, we assumed it was a Unix only test for using a unix shell. Given the fact that there doesn't seem to be any proposal explaining the specification, or sample content/results and the documentation in the XSD are essentially "TO DO: Add some documentation", it's far from a spec that seems ready for prime time. Also given that CIS has indicated that they are dropping OVAL support going forward, and their content is not publicly available, and this is our "one chance" to update OVAL for the foreseeable future, I think we should adopt whatever specification will stand the test of time. We pondered making a single shell test that would be independent and be for Windows and UNIX, and if you look at our proposals, they are nearly identical, but making separate tests allows for platform specific documentation, and the potential for platform specific improvements later in the future. And behind the scenes of every "independent" OVAL test is platform specific implementations, so the "savings" of an independent test are merely in the trivial amount of time required to make the XSD files, which is already done. In reviewing the technical aspects of the CIS shellcommand test, we found the inclusion of the exit_status and the std_errrline to be interesting, but not really useful from a content/results perspective, so we intentionally excluded them, and let the application handle any errors. We didn't see any content needing to put those attributes in a state, nor any end user needing to see them. Once again, when looking at the documentation for the system characteristics item "TO DO: Add some documentation" didn't really feel ready for prime time. If there is some content created for the CIS shellcommand, I feel it would be a trivial effort to write a conversion script/transform to convert it to whatever is deemed worthy of official inclusion in OVAL. The same would hold true for supporting a new/replacement shell test vs the CIS, most of underlying logic would be identical. — Reply to this email directly, view it on GitHub <#150 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7K53E7V5DMPCPKL22Q47LZ6CU35AVCNFSM6AAAAABOSQD5JKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBWGM4TSNRUGI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[solind]

@vanderpol The need to capture both STDOUT and STDERR is quite real, as is the need to capture the exit code. Processes on both Windows and Unix have both these character output streams, and attempting to collect them together through redirection always introduces a potential race condition. Similarly, checking the exit code makes it possible to write a check whose outcome is determined by the success or failure of that process's execution, rather than its character output.

These are all also collected in SCE, the "other" other way to run a command to produce a check result.

The one thing that for sure isn't needed, I think, is the <label/> field. The @comment attribute of the object should already contain that information.

[vanderpol]

Thanks @wmunyan, and I wasn't meaning to throw you under the bus with your old shellcommand test documentation, was just trying to explain why we decided to go forward with our own, slightly differently named, but very similar test. I'm not sure your attached content made it, if you can try to re-attach it again, that would be much appreciated, especially if there's some sample output to go with it.

[vanderpol]

@solind, thanks for the clarification on stdout/err and exit codes, I'll let @johnulmer-oval ponder this and chime in later. I'll also let John respond back on the 'label' element, as there's a subtle distinction between it and the "comment" attribute, and he can elaborate.