Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] Analyzer reports dissapear in 4.1.4 (observable already exists error) #1982

Closed
androssfox opened this issue Apr 19, 2021 · 11 comments
Closed

[Bug] Analyzer reports dissapear in 4.1.4 (observable already exists error) #1982

androssfox opened this issue Apr 19, 2021 · 11 comments
Assignees
@To-om
Labels
bug TheHive4 TheHive4 related issues
Milestone
4.1.5

Comments

@androssfox
Copy link

Request Type

Bug

Work Environment

Question Answer
OS version (server) Ubuntu
OS version (client) Ubuntu
Virtualized Env. True
Dedicated RAM 48 GB
vCPU 12
TheHive version / git hash 4.1.4
Package Type DEB
Database Cassandra
Index type Lucene
Attachments storage HDFS

Problem Description

After running analyzers on an observable (and all finishing successfully), when loading the Observable page, the analyzer reports suddently dissapear and show "none", as if never run. The Hive log shows this error multiple times:

2021-04-19 07:36:10,082 [ERROR] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-13 [|10e08eec] uncaught error, not retrying
org.thp.scalligraph.CreateError: Observable already exists
        at org.thp.thehive.services.ObservableSrv.create(ObservableSrv.scala:95)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$5(JobSrv.scala:229)
        at scala.util.Success.flatMap(Try.scala:251)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$4(JobSrv.scala:228)
        at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$7(JanusDatabase.scala:241)
        at scala.util.Try$.apply(Try.scala:213)
        at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$6(JanusDatabase.scala:241)
        at scala.util.Try$.apply(Try.scala:213)
        at org.thp.scalligraph.utils.DelayRetry.withTry(Retry.scala:93)
        at org.thp.scalligraph.janus.JanusDatabase.tryTransaction(JanusDatabase.scala:238)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$2(JobSrv.scala:226)
        at scala.concurrent.Future$.$anonfun$traverse$1(Future.scala:850)
        at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
        at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
        at scala.collection.immutable.List.foldLeft(List.scala:91)
        at scala.concurrent.Future$.traverse(Future.scala:850)
        at org.thp.thehive.connector.cortex.services.JobSrv.importCortexArtifacts(JobSrv.scala:220)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$finished$5(JobSrv.scala:155)
        at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
        at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
        at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$2(ContextPropagatingDisptacher.scala:57)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at org.thp.scalligraph.DiagnosticContext$.$anonfun$withDiagnosticContext$2(ContextPropagatingDisptacher.scala:93)
        at org.thp.scalligraph.DiagnosticContext$.saveDiagnosticContext(ContextPropagatingDisptacher.scala:108)
        at org.thp.scalligraph.DiagnosticContext$.withDiagnosticContext(ContextPropagatingDisptacher.scala:91)
        at org.thp.scalligraph.DiagnosticContext$$anon$2.withContext(ContextPropagatingDisptacher.scala:76)
        at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$1(ContextPropagatingDisptacher.scala:57)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
2021-04-19 07:36:10,083 [ERROR] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-13 [|10e08eec] Exception raised, rollback (Observable already exists)

This error happens mainly with file analyzers, such as EMLparser.

As no error occurs in Cortex, we believe it is an issue in TheHive.

Steps to Reproduce

  1. Run analyzers on file observable, and ensure that the job in cortex finished sucessfully.
  2. Open the observable page to see reports.
  3. Error will happen and no reports will be shown, The Hive logs shows the same error multiple times.

Complementary information

We started observing this issue after upgrading The Hive 4.0.5 to 4.1.4.
@androssfox androssfox added TheHive4 TheHive4 related issues bug labels Apr 19, 2021
@mphbig
Copy link

mphbig commented Apr 19, 2021

Hello,

I think this issue is the same as this one : #1905

@kiz1
Copy link

kiz1 commented Apr 19, 2021

+1

@joseluratm
Copy link

joseluratm commented Apr 20, 2021 •
edited
Loading

we have observed the same bug in thehive4.1.4 gives us the errors:
An error occurs (java.lang. IllegalArgumentException: Neither sideEffects, map, nor path has a ff5e8828-ccc5-470e-8bae-8bae-8ba19b4f8f1f-key: WherePredicateStep(eq(ff5e8828-ccc5-470e-8bae-8ba19b4f8f1f)), retrying (1).

and later in other reports:
[�[33mwarn�[0m] o.t.s.u.Retry [00005425|40ec53c3] An error occurs (java.lang.IllegalArgumentException: The provided traverser does not map to a value: v[474452048]->[CoalesceStep([[JanusGraphVertexStep(IN,[AlertObservable],vertex), ProjectStep([2f573d25-3357-4617-80e5-f4e1d7a11e8e, 50b59f2b-9645-45d8-a84e-d63d53cbf923, 96cb0543-c9f4-4c58-b440-bc1a5f018038, 6948af53-9428-48f0-8876-63c4d64f7f20, 67e1ecef-6c8f-4a15-93d2-6b7f07d0e352],[identity, [JanusGraphVertexStep(OUT,[AlertCustomField],edge), ProjectStep([232573d1-b328-4572-a9dc-0f122f867862, 7cc67fa8-c761-4051-b189-719f338fdbdf],[identity, [EdgeVertexStep(IN)]]), FoldStep], [JanusGraphVertexStep(OUT,[AlertCase],vertex), RangeGlobalStep(0,1), IdStep, FoldStep], [JanusGraphVertexStep(OUT,[AlertCaseTemplate],vertex), JanusGraphPropertiesStep([name],value), RangeGlobalStep(0,1), FoldStep], [JanusGraphVertexStep(OUT,[AlertObservable],edge), CountGlobalStep]]), ProjectStep([coalesceIndex, coalesceValue],[[ConstantStep(0)], identity])], [JanusGraphVertexStep(IN,[ShareObservable],vertex), JanusGraphVertexStep(OUT,[ShareCase],vertex), NoOpBarrierStep(2500), ProjectStep([c4d68345-bad4-496b-b20b-4f764f71e722, 2de1f5d3-6cbc-4514-858a-c0314cab7333],[identity, [JanusGraphVertexStep(OUT,[CaseCustomField],edge), ProjectStep([7ca0107f-5595-4b10-804b-99846ffef96d, 6e482b3b-1819-49ff-bee6-ba4dd310bfa0],[identity, [EdgeVertexStep(IN)]]), FoldStep]]), ProjectStep([coalesceIndex, coalesceValue],[[ConstantStep(1)], identity])]])]), retrying (1)

Is happening to us with hashes when we consult a case report.
We are using elasticsearch as a engine.

@danniranderis
Copy link

danniranderis commented Apr 20, 2021 •
edited
Loading

We see the same issue for version 4.1.3 on Ubuntu 20.04.

When accessing observables page we get:

2021-04-20 11:07:12,879 [WARN] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-26 [|] An error occurs (java.lang.IllegalArgumentException: Neither the sideEffects, map, nor path has a 7b9c26f3-2a9c-403c-9c5f-11ec81329cce-key: WherePredicateStep(eq(7b9c26f3-2a9c-403c-9c5f-11ec81329cce))), retrying (1)

After trying to run analyzer again we briefly see the analyzer showing the last time the analyzer ran, but cannot open the report - when the analyzer is done, it shows again as not being run at all. For this we see:

2021-04-20 11:08:52,130 [ERROR] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-5 [|28ca54e8] Exception raised, rollback (Observable already exists)
2021-04-20 11:08:52,182 [ERROR] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-5 [|45369882] uncaught error, not retrying
org.thp.scalligraph.CreateError: Observable already exists
        at org.thp.thehive.services.ObservableSrv.create(ObservableSrv.scala:95)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$5(JobSrv.scala:229)
        at scala.util.Success.flatMap(Try.scala:251)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$4(JobSrv.scala:228)
        at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$7(JanusDatabase.scala:241)
        at scala.util.Try$.apply(Try.scala:213)
        at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$6(JanusDatabase.scala:241)
        at scala.util.Try$.apply(Try.scala:213)
        at org.thp.scalligraph.utils.DelayRetry.withTry(Retry.scala:93)
        at org.thp.scalligraph.janus.JanusDatabase.tryTransaction(JanusDatabase.scala:238)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$2(JobSrv.scala:226)
        at scala.concurrent.Future$.$anonfun$traverse$1(Future.scala:850)
        at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
        at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
        at scala.collection.immutable.List.foldLeft(List.scala:91)
        at scala.concurrent.Future$.traverse(Future.scala:850)
        at org.thp.thehive.connector.cortex.services.JobSrv.importCortexArtifacts(JobSrv.scala:220)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$finished$5(JobSrv.scala:155)
        at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
        at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
        at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$2(ContextPropagatingDisptacher.scala:57)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at org.thp.scalligraph.DiagnosticContext$.$anonfun$withDiagnosticContext$2(ContextPropagatingDisptacher.scala:93)
        at org.thp.scalligraph.DiagnosticContext$.saveDiagnosticContext(ContextPropagatingDisptacher.scala:108)
        at org.thp.scalligraph.DiagnosticContext$.withDiagnosticContext(ContextPropagatingDisptacher.scala:91)
        at org.thp.scalligraph.DiagnosticContext$$anon$2.withContext(ContextPropagatingDisptacher.scala:76)
        at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$1(ContextPropagatingDisptacher.scala:57)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

@torsolaso
Copy link

more (i think) relevant logs.

POST request that give the error:

[INFO]  [�[37minfo�[0m] o.t.s.AccessLogFilter [00005731|] 169.254.130.1 POST /api/v1/query?name=observable-jobs-~147857520 took 3ms and returned 200
[INFO]  [�[37minfo�[0m] o.t.s.AccessLogFilter [00005732|] 169.254.130.1 POST /api/v1/query?name=observable-jobs-~147857520.count took 1ms and returned 200 1 bytes
[INFO]  [�[33mwarn�[0m] o.t.s.u.Retry [|] An error occurs (java.lang.IllegalArgumentException: Neither the sideEffects, map, nor path has a 395a1fea-2f16-496c-8873-02c149dc2008-key: WherePredicateStep(eq(395a1fea-2f16-496c-8873-02c149dc2008))), retrying (1)
[INFO]  [�[33mwarn�[0m] o.t.s.u.Retry [|] An error occurs (java.lang.IllegalArgumentException: Neither the sideEffects, map, nor path has a 395a1fea-2f16-496c-8873-02c149dc2008-key: WherePredicateStep(eq(395a1fea-2f16-496c-8873-02c149dc2008))), retrying (2)

@torsolaso
Copy link

i observe that taxonomy continue to be attached to the observable on THv4

@LaZyDK
Copy link

LaZyDK commented Apr 26, 2021

We are seeing the same issues as explained above. Observable analysis reports are not shown.

@m5050
Copy link

m5050 commented May 2, 2021

also have the same issue here after upgrading Thehive 4.0.5 to 4.1.0

@mphbig
Copy link

mphbig commented May 4, 2021

still having the issue after upgrading to 4.1.4, is anything planned to fix this ?

do you (TheHive devs) need help figuring out what is happening ?

@To-om To-om self-assigned this May 10, 2021
@To-om To-om added this to the 4.1.5 milestone May 19, 2021
To-om added a commit that referenced this issue May 19, 2021
@To-om
#1982 Prevent failure when similar observable is in job report
c16c77e
@To-om
Copy link
Contributor

To-om commented May 19, 2021

This commit fixes the error The provided traverser does not map to a value

To-om added a commit that referenced this issue May 19, 2021
@To-om
#1982 Prevent failure when an observable cannot be imported from job
53e7c82
@To-om
Copy link
Contributor

To-om commented May 19, 2021

With this commit, the import of the job report won't fail if an observable can't be created (because is already exist, for example).
The observable duplication is no longer an error (just ignored)

@To-om To-om closed this as completed May 19, 2021
@ch0wm3in ch0wm3in mentioned this issue Jun 4, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@To-om To-om

Labels
bug TheHive4 TheHive4 related issues
Projects
None yet
Milestone
4.1.5
Development

No branches or pull requests
9 participants
@joseluratm @LaZyDK @danniranderis @m5050 @mphbig @To-om @androssfox @torsolaso @kiz1