Signatures in limbo state

[PhilOrdo]

When attempting to create a signature, it will sometimes fall into a limbo state where it doesn't appear in ThreatKB but can be seen in the DB (attempting to recreate the rule will result in a duplicate rule name error message). Using the id listed for the rule in the DB, the rule can be accessed via https://threatkb.inquest.net/#!/yara_rules/{id} but changes to the rule will not save and results in the following stacktrace message:

Traceback (most recent call last):
File "/opt/threatkb38/env/lib64/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/opt/threatkb38/env/lib64/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/opt/threatkb38/env/lib64/python3.8/site-packages/flask_login/utils.py", line 272, in decorated_view
return func(*args, **kwargs)
File "/opt/threatkb38/app/routes/yara_rules.py", line 586, in update_yara_rule
db.session.commit()
File "", line 2, in commit
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 1435, in commit
self._transaction.commit(_to_root=self.future)
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 829, in commit
self._prepare_impl()
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
self.session.flush()
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 3367, in flush
self._flush(objects)
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 3507, in flush
transaction.rollback(capture_exception=True)
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
compat.raise(
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
raise exception
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/session.py", line 3467, in _flush
flush_context.execute()
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
rec.execute(self)
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
util.preloaded.orm_persistence.save_obj(
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 253, in save_obj
_finalize_insert_update_commands(
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1568, in _finalize_insert_update_commands
mapper.dispatch.after_update(mapper, connection, state)
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/event/attr.py", line 343, in call
fn(*args, **kw)
File "/opt/threatkb38/env/lib64/python3.8/site-packages/sqlalchemy/orm/events.py", line 743, in wrap
fn(*arg, **kw)
File "/opt/threatkb38/app/models/yara_rule.py", line 465, in yara_modified
state_activity_text = activity_log.get_state_change(target, target.name)
File "/opt/threatkb38/app/models/activity_log.py", line 86, in get_state_change
if o_state.is_release_state > 0 or o_state.is_retired_state > 0 or o_state.is_staging_state > 0
AttributeError: 'NoneType' object has no attribute 'is_release_state'

Example rule in limbo state:
SC_RDP_Properties_File_with_RemoteApp_Command_Line_Arguments
https://threatkb.inquest.net/#!/yara_rules/60769

[dcuellar322]

How was this signature created? The reason it's in limbo is because there is no State on the signature.

It's mandatory, so that's why I'm curious how it was able to be created to start with. But now that there's no signature, it can't update because the code assumes there always be a state.

I can fix the code, but the easiest way would be to possibly update it directly in the database. Are you able to do that?

Would you be able to see how many signatures don't have a State?

Also this particular signature happens to have the same EventID as https://threatkb.inquest.net/#!/yara_rules/60768

Where one of these created by Import?

[PhilOrdo]

This rule was created with Create Signature and set to "Sig Review" state before attempting to save. Unsure what could have happened to cause it to drop the state but still create the rule.

We should be able to update it directly in the database, though being able to get around the error by navigating to the rule by id and assigning a state would be ideal.

Querying for signatures where the state is NULL shows 14 rules total in this limbo state.

I recall Darren creating https://threatkb.inquest.net/#!/yara_rules/60768 around the same time and it being done by import. The activity log shows Darren importing his rule at 11:46 CST while my rule was created at 11:58 CST (assuming timing is the major factor here).

[dcuellar322]

I've merged in my changes. You should be able to see signatures without a state in the UI now... and you should be able to save now.

Please let me know.

[dcuellar322]

Visible in UI...

I'm going to change the State to Sig Review... FYI... you can change it to something else if you need. Just want to confirm it works.

[dcuellar322]

Seems to have worked https://threatkb.inquest.net/#!/yara_rules/60769

[PhilOrdo]

Tested with a few rules with NULL state, confirmed that it works.