Skip to content

[Bug] Analyzer reports dissapear in 4.1.4 (observable already exists error) #1982

Closed
@androssfox

Description

@androssfox

Request Type

Bug

Work Environment

Question Answer
OS version (server) Ubuntu
OS version (client) Ubuntu
Virtualized Env. True
Dedicated RAM 48 GB
vCPU 12
TheHive version / git hash 4.1.4
Package Type DEB
Database Cassandra
Index type Lucene
Attachments storage HDFS

Problem Description

After running analyzers on an observable (and all finishing successfully), when loading the Observable page, the analyzer reports suddently dissapear and show "none", as if never run. The Hive log shows this error multiple times:

2021-04-19 07:36:10,082 [ERROR] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-13 [|10e08eec] uncaught error, not retrying
org.thp.scalligraph.CreateError: Observable already exists
        at org.thp.thehive.services.ObservableSrv.create(ObservableSrv.scala:95)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$5(JobSrv.scala:229)
        at scala.util.Success.flatMap(Try.scala:251)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$4(JobSrv.scala:228)
        at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$7(JanusDatabase.scala:241)
        at scala.util.Try$.apply(Try.scala:213)
        at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$6(JanusDatabase.scala:241)
        at scala.util.Try$.apply(Try.scala:213)
        at org.thp.scalligraph.utils.DelayRetry.withTry(Retry.scala:93)
        at org.thp.scalligraph.janus.JanusDatabase.tryTransaction(JanusDatabase.scala:238)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$importCortexArtifacts$2(JobSrv.scala:226)
        at scala.concurrent.Future$.$anonfun$traverse$1(Future.scala:850)
        at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
        at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
        at scala.collection.immutable.List.foldLeft(List.scala:91)
        at scala.concurrent.Future$.traverse(Future.scala:850)
        at org.thp.thehive.connector.cortex.services.JobSrv.importCortexArtifacts(JobSrv.scala:220)
        at org.thp.thehive.connector.cortex.services.JobSrv.$anonfun$finished$5(JobSrv.scala:155)
        at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
        at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
        at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$2(ContextPropagatingDisptacher.scala:57)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at org.thp.scalligraph.DiagnosticContext$.$anonfun$withDiagnosticContext$2(ContextPropagatingDisptacher.scala:93)
        at org.thp.scalligraph.DiagnosticContext$.saveDiagnosticContext(ContextPropagatingDisptacher.scala:108)
        at org.thp.scalligraph.DiagnosticContext$.withDiagnosticContext(ContextPropagatingDisptacher.scala:91)
        at org.thp.scalligraph.DiagnosticContext$$anon$2.withContext(ContextPropagatingDisptacher.scala:76)
        at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$1(ContextPropagatingDisptacher.scala:57)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
2021-04-19 07:36:10,083 [ERROR] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-13 [|10e08eec] Exception raised, rollback (Observable already exists)

This error happens mainly with file analyzers, such as EMLparser.

As no error occurs in Cortex, we believe it is an issue in TheHive.

Steps to Reproduce

  1. Run analyzers on file observable, and ensure that the job in cortex finished sucessfully.
  2. Open the observable page to see reports.
  3. Error will happen and no reports will be shown, The Hive logs shows the same error multiple times.

Complementary information

We started observing this issue after upgrading The Hive 4.0.5 to 4.1.4.

Metadata

Metadata

Assignees

Labels

TheHive4TheHive4 related issuesbug

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions