Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation error with dummy filereport and expression referencing type_as_text #176

Closed
michaelweiser opened this issue Aug 5, 2020 · 0 comments
Assignees
Labels
Milestone

Comments

@michaelweiser
Copy link
Contributor

Similar to #156, startup of Peekaboo fails with certain operators such as in because the dummy filereport's properties return None. Example expression:

"foo" in filereport.type_as_text -> ignore

Error message:

Traceback (most recent call last):
  File "/opt/peekaboo/bin/peekaboo", line 8, in <module>
    sys.exit(run())
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/daemon.py", line 364, in run
    engine = RulesetEngine(ruleset_config, db_con)
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/ruleset/engine.py", line 106, in __init__
    rule = rule_classes[rule](config, db_con)
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/ruleset/rules.py", line 58, in __init__
    self.get_config()
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/ruleset/rules.py", line 552, in get_config
    rule.eval(context=context)
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/ruleset/expressions.py", line 341, in eval
    ret = self.value[0].eval(context)
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/ruleset/expressions.py", line 452, in eval
    val1 = self.value[0].eval(context)
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/ruleset/expressions.py", line 457, in eval
    if not self.handle_regexes(function, val1, val2):
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/ruleset/expressions.py", line 449, in handle_regexes
    return function(val1, val2)
  File "/opt/peekaboo/local/lib/python3.6/site-packages/peekaboo/ruleset/expressions.py", line 419, in in_
    return a in b
TypeError: argument of type 'NoneType' is not iterable

I guess, eiither our dummies need to get more realistic or we need to scale back the level of evaluation we're doing against them.

@michaelweiser michaelweiser added this to the 2.1 milestone Aug 5, 2020
@michaelweiser michaelweiser self-assigned this Aug 5, 2020
michaelweiser added a commit to michaelweiser/PeekabooAV that referenced this issue Mar 17, 2022
Our logic operators 'and' and 'or' would not short-circuit if the result
was already conclusive. This would lead to unnecessary evaluation and
broke user assumption that we they could be used to run additional tests
only if required. We fix that by introducing short-circuiting and adding
tests to document the problem and catch regressions.

This now also allows tests to avoid error conditions like: foo is not
None and "bar" in foo (NoneType not iterable in this case).

Partially addresses scVENUS#176.
michaelweiser added a commit to michaelweiser/PeekabooAV that referenced this issue Mar 17, 2022
Python makes a distinction between something being contained in
something else (or not) and something not being able to be contained in
nothing (i.e. None) and raises a 'NoneType not iterable' exception in
the latter case. We can see no application of this distinction in our
use-case as of now. In fact it causes problems when writing tests
against object properties which are None. Therefore we handle that case
in our containment tests and interpret it as something (obviously) not
being contained in nothing.

Closes scVENUS#176.
michaelweiser added a commit to michaelweiser/PeekabooAV that referenced this issue Mar 18, 2022
Our logic operators 'and' and 'or' would not short-circuit if the result
was already conclusive. This would lead to unnecessary evaluation and
broke user assumption that they could be used to run additional tests
only if required. We fix that by introducing short-circuiting and adding
tests to document the problem and catch regressions.

This now also allows tests to avoid error conditions like: foo is not
None and "bar" in foo (NoneType not iterable in this case).

Partially addresses scVENUS#176.
michaelweiser added a commit to michaelweiser/PeekabooAV that referenced this issue Mar 18, 2022
Python makes a distinction between something being contained in
something else (or not) and something not being able to be contained in
nothing (i.e. None) and raises a 'NoneType not iterable' exception in
the latter case. We can see no application of this distinction in our
use-case as of now. In fact it causes problems when writing tests
against object properties which are None. Therefore we handle that case
in our containment tests and interpret it as something (obviously) not
being contained in nothing.

Closes scVENUS#176.
michaelweiser added a commit to michaelweiser/PeekabooAV that referenced this issue Mar 18, 2022
Our logic operators 'and' and 'or' would not short-circuit if the result
was already conclusive. This would lead to unnecessary evaluation and
broke user assumption that they could be used to run additional tests
only if required. We fix that by introducing explicit short-circuiting
and adding tests to document the problem and catch regressions.

We also fix over-eager short-circuiting for compound 'or' operations by
excluding both 'or' and 'and' from post-evaluation short-circuiting and
deferring their short-circuiting to the next iteration based on the
then-first operand.

Finally, we explicitly document the distinction between operand passing
within compound operations (such as 1 < 2 < 2) and their evaluation into
boolean results. We opt (as we did implictly before) to always return
booleans even for 'and' and 'or' which in python return the determing
operand (1 or 2 -> 1 instead of True).

In order to determine what operation we're looking at, we switch 'and'
and 'or' from lambda functions to static methods.

This now also allows tests to avoid error conditions like: foo is not
None and "bar" in foo (NoneType not iterable in this case).

Partially addresses scVENUS#176.
michaelweiser added a commit to michaelweiser/PeekabooAV that referenced this issue Mar 18, 2022
Python makes a distinction between something being contained in
something else (or not) and something not being able to be contained in
nothing (i.e. None) and raises a 'NoneType not iterable' exception in
the latter case. We can see no application of this distinction in our
use-case as of now. In fact it causes problems when writing tests
against object properties which are None. Therefore we handle that case
in our containment tests and interpret it as something (obviously) not
being contained in nothing.

Closes scVENUS#176.
michaelweiser added a commit to michaelweiser/PeekabooAV that referenced this issue Mar 18, 2022
Our logic operators 'and' and 'or' would not short-circuit if the result
was already conclusive. This would lead to unnecessary evaluation and
broke user assumption that they could be used to run additional tests
only if required. We fix that by introducing explicit short-circuiting
and adding tests to document the problem and catch regressions.

We also fix over-eager short-circuiting for compound 'or' operations by
excluding both 'or' and 'and' from post-evaluation short-circuiting and
deferring their short-circuiting to the next iteration based on the
then-first operand.

Finally, we explicitly document the distinction between operand passing
within compound operations (such as 1 < 2 < 2) and their evaluation into
boolean results. We opt (as we did implictly before) to always return
booleans even for 'and' and 'or' which in python return the determing
operand (1 or 2 -> 1 instead of True).

In order to determine what operation we're looking at, we switch 'and'
and 'or' from lambda functions to static methods.

This now also allows tests to avoid error conditions like: foo is not
None and "bar" in foo (NoneType not iterable in this case).

Partially addresses scVENUS#176.
michaelweiser added a commit to michaelweiser/PeekabooAV that referenced this issue Mar 18, 2022
Python makes a distinction between something being contained in
something else (or not) and something not being able to be contained in
nothing (i.e. None) and raises a 'NoneType not iterable' exception in
the latter case. We can see no application of this distinction in our
use-case as of now. In fact it causes problems when writing tests
against object properties which are None. Therefore we handle that case
in our containment tests and interpret it as something (obviously) not
being contained in nothing.

Closes scVENUS#176.
michaelweiser added a commit that referenced this issue Mar 21, 2022
Our logic operators 'and' and 'or' would not short-circuit if the result
was already conclusive. This would lead to unnecessary evaluation and
broke user assumption that they could be used to run additional tests
only if required. We fix that by introducing explicit short-circuiting
and adding tests to document the problem and catch regressions.

We also fix over-eager short-circuiting for compound 'or' operations by
excluding both 'or' and 'and' from post-evaluation short-circuiting and
deferring their short-circuiting to the next iteration based on the
then-first operand.

Finally, we explicitly document the distinction between operand passing
within compound operations (such as 1 < 2 < 2) and their evaluation into
boolean results. We opt (as we did implictly before) to always return
booleans even for 'and' and 'or' which in python return the determing
operand (1 or 2 -> 1 instead of True).

In order to determine what operation we're looking at, we switch 'and'
and 'or' from lambda functions to static methods.

This now also allows tests to avoid error conditions like: foo is not
None and "bar" in foo (NoneType not iterable in this case).

Partially addresses #176.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant