Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Uncaught exception with duplicate mail type observables when added to case #1561

Closed
vi-or-die opened this issue Sep 30, 2020 · 1 comment
Assignees
Labels
bug TheHive4 TheHive4 related issues
Milestone

Comments

@vi-or-die
Copy link

Mail type observable has uncaught exception when duplicate is attempted to be added to case

Request Type

Bug

Work Environment

Question Answer
OS version (server) RedHat
OS version (client) MAC, Windows 10
TheHive version / git hash 4.0.0
Package Type RPM
Browser type & version Chrome, Firefox

Problem Description

Adding a list of mail observable fails when duplicate observable already exists in case. This prevents bulk addition of observables to a case.

Steps to Reproduce

  1. Create a case
  2. Add a mail observable i.e. first.last@domain.com, and first.last2@domain.com
  3. Add the duplicates:
    1. select "add observables" > set type to mail > add the same two observables in the "value" box as separate lines.
    2. leave one observable per line (1 unique observable) selected
  4. You will receive the error ObservableCreationCtrl: Observable already exists

Complementary information

trace logs:

2020-09-29 17:59:11,265 [DEBUG] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Begin of transaction
2020-09-29 17:59:11,267 [DEBUG] from org.thp.scalligraph.steps.Traversal in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Execution of [GraphStep(vertex,[43819240]), HasStep([_label.eq(Case), ~label.eq(Case)]), TraversalFilterStep([VertexStep(IN,[ShareCase],vertex), TraversalFilterStep([VertexStep(OUT,[ShareProfile],vertex), HasStep([permissions.eq(manageObservable)])]), VertexStep(IN,[OrganisationShare],vertex), HasStep([name.eq(TESTORG)])])] (headOption)
2020-09-29 17:59:11,272 [DEBUG] from org.thp.scalligraph.steps.Traversal in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Execution of [GraphStep(vertex,[]), HasStep([_label.eq(ObservableType), ~label.eq(ObservableType), name.eq(mail)])] (headOption)
2020-09-29 17:59:11,279 [DEBUG] from org.thp.scalligraph.steps.Traversal in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Execution of [GraphStep(vertex,[]), HasStep([_label.eq(Data), ~label.eq(Data), data.eq(victim.user)])] (headOption)
2020-09-29 17:59:11,964 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 122884240, message = Some(test)
2020-09-29 17:59:11,965 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 122884240, tlp = 2
2020-09-29 17:59:11,966 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 122884240, ioc = false
2020-09-29 17:59:11,966 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 122884240, sighted = false
2020-09-29 17:59:11,998 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Created vertex is v[122884240]
 - _createdAt = Tue Sep 29 17:59:11 UTC 2020 (class java.util.Date)
 - _label = Observable (class java.lang.String)
 - tlp = 2 (class java.lang.Integer)
 - _createdBy = first.last@TESTORG.com (class java.lang.String)
 - ioc = false (class java.lang.Boolean)
 - message = test (class java.lang.String)
 - sighted = false (class java.lang.Boolean)
2020-09-29 17:59:12,002 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Create edge ObservableObservableType from v[122884240] to v[8288]: e[3dxonm-215u34-74l-6e8][122884240-ObservableObservableType->8288]
 - _createdBy = first.last@TESTORG.com (class java.lang.String)
 - _createdAt = Tue Sep 29 17:59:12 UTC 2020 (class java.util.Date)
 - _label = ObservableObservableType (class java.lang.String)
2020-09-29 17:59:12,003 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Create edge ObservableData from v[122884240] to v[121974880]: e[3dxp1u-215u34-13it-20mcf4][122884240-ObservableData->121974880]
 - _createdBy = first.last@TESTORG.com (class java.lang.String)
 - _createdAt = Tue Sep 29 17:59:12 UTC 2020 (class java.util.Date)
 - _label = ObservableData (class java.lang.String)
2020-09-29 17:59:12,006 [DEBUG] from org.thp.scalligraph.steps.Traversal in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Execution of [GraphStep(vertex,[]), HasStep([_label.eq(Data), ~label.eq(Data), data.eq(cameron.kinsel)])] (headOption)
2020-09-29 17:59:12,678 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 40964136, data = cameron.kinsel
2020-09-29 17:59:12,683 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Created vertex is v[40964136]
 - _label = Data (class java.lang.String)
 - _createdBy = first.last@TESTORG.com (class java.lang.String)
 - _createdAt = Tue Sep 29 17:59:12 UTC 2020 (class java.util.Date)
 - data = cameron.kinsel (class java.lang.String)
2020-09-29 17:59:12,685 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 122888336, message = Some(test)
2020-09-29 17:59:12,685 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 122888336, tlp = 2
2020-09-29 17:59:12,685 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 122888336, ioc = false
2020-09-29 17:59:12,685 [DEBUG] from org.thp.thehive.services.IntegrityCheckActor in application-akka.actor.default-dispatcher-22 [|] An entity Data has been created
2020-09-29 17:59:12,685 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] set 122888336, sighted = false
2020-09-29 17:59:12,689 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Created vertex is v[122888336]
 - _label = Observable (class java.lang.String)
 - tlp = 2 (class java.lang.Integer)
 - _createdBy = first.last@TESTORG.com (class java.lang.String)
 - _createdAt = Tue Sep 29 17:59:12 UTC 2020 (class java.util.Date)
 - ioc = false (class java.lang.Boolean)
 - message = test (class java.lang.String)
 - sighted = false (class java.lang.Boolean)
2020-09-29 17:59:12,690 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Create edge ObservableObservableType from v[122888336] to v[8288]: e[3dxt02-215x8w-74l-6e8][122888336-ObservableObservableType->8288]
 - _createdBy = first.last@TESTORG.com (class java.lang.String)
 - _createdAt = Tue Sep 29 17:59:12 UTC 2020 (class java.util.Date)
 - _label = ObservableObservableType (class java.lang.String)
2020-09-29 17:59:12,691 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Create edge ObservableData from v[122888336] to v[40964136]: e[3dxtea-215x8w-13it-oe04o][122888336-ObservableData->40964136]
 - _createdBy = first.last@TESTORG.com (class java.lang.String)
 - _createdAt = Tue Sep 29 17:59:12 UTC 2020 (class java.util.Date)
 - _label = ObservableData (class java.lang.String)
2020-09-29 17:59:12,694 [DEBUG] from org.thp.scalligraph.steps.Traversal in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Execution of [GraphStep(vertex,[122884240]), HasStep([_label.eq(Observable), ~label.eq(Observable)]), AggregateStep(3cd99f16-6bc2-4016-bf0d-9b42a09503aa), UnionStep([[VertexStep(OUT,[ObservableData],vertex), VertexStep(IN,[ObservableData],vertex), EndStep], [VertexStep(OUT,[ObservableAttachment],vertex), VertexStep(IN,[ObservableAttachment],vertex), EndStep]]), WherePredicateStep(without([3cd99f16-6bc2-4016-bf0d-9b42a09503aa])), DedupGlobalStep, TraversalFilterStep([VertexStep(IN,[ShareObservable],vertex), VertexStep(IN,[OrganisationShare],vertex), HasStep([name.eq(TESTORG)])]), VertexStep(IN,[ShareObservable],vertex), VertexStep(OUT,[ShareCase],vertex), HasStep([~id.eq(43819240)])] (exists)
2020-09-29 17:59:12,705 [ERROR] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] uncaught error, not retrying
org.thp.scalligraph.CreateError: Observable already exists
	at org.thp.thehive.services.CaseSrv.addObservable(CaseSrv.scala:163)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$10(ObservableCtrl.scala:85)
	at org.thp.scalligraph.package$RichSeq.$anonfun$toTry$3(package.scala:25)
	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
	at scala.collection.immutable.List.foldLeft(List.scala:89)
	at org.thp.scalligraph.package$RichSeq.toTry(package.scala:24)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$9(ObservableCtrl.scala:83)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$7(ObservableCtrl.scala:79)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$5(ObservableCtrl.scala:76)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$4(ObservableCtrl.scala:75)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$3(ObservableCtrl.scala:71)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$authTransaction$2(Entrypoint.scala:77)
	at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$10(JanusDatabase.scala:199)
	at scala.util.Try$.apply(Try.scala:213)
	at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$8(JanusDatabase.scala:199)
	at scala.util.Try$.apply(Try.scala:213)
	at org.thp.scalligraph.utils.DelayRetry.withTry(Retry.scala:87)
	at org.thp.scalligraph.janus.JanusDatabase.tryTransaction(JanusDatabase.scala:193)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$authTransaction$1(Entrypoint.scala:77)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$auth$1(Entrypoint.scala:86)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$asyncAuth$3(Entrypoint.scala:107)
	at org.scalactic.Good.fold(Or.scala:1229)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$asyncAuth$2(Entrypoint.scala:107)
	at org.thp.scalligraph.DiagnosticContext$.$anonfun$withRequest$2(ContextPropagatingDisptacher.scala:99)
	at org.thp.scalligraph.DiagnosticContext$.saveDiagnosticContext(ContextPropagatingDisptacher.scala:106)
	at org.thp.scalligraph.DiagnosticContext$.withRequest(ContextPropagatingDisptacher.scala:97)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$asyncAuth$1(Entrypoint.scala:107)
	at org.thp.scalligraph.auth.SessionAuthSrv$$anon$1.$anonfun$invokeBlock$2(SessionAuthSrv.scala:89)
	at scala.Option.fold(Option.scala:251)
	at org.thp.scalligraph.auth.SessionAuthSrv$$anon$1.invokeBlock(SessionAuthSrv.scala:87)
	at org.thp.scalligraph.auth.SessionAuthSrv$$anon$1.invokeBlock(SessionAuthSrv.scala:84)
	at play.api.mvc.ActionBuilder$$anon$10.$anonfun$invokeBlock$2(Action.scala:408)
	at play.api.mvc.ActionBuilderImpl.invokeBlock(Action.scala:441)
	at play.api.mvc.ActionBuilderImpl.invokeBlock(Action.scala:439)
	at play.api.mvc.ActionBuilder$$anon$10.invokeBlock(Action.scala:408)
	at play.api.mvc.ActionBuilder$$anon$10.invokeBlock(Action.scala:404)
	at play.api.mvc.ActionBuilder$$anon$9.apply(Action.scala:379)
	at play.api.mvc.Action.$anonfun$apply$4(Action.scala:82)
	at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
	at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
	at org.thp.scalligraph.ContextPropagatingDisptacher$$anon$1.$anonfun$execute$2(ContextPropagatingDisptacher.scala:56)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.thp.scalligraph.DiagnosticContext$.$anonfun$withDiagnosticContext$2(ContextPropagatingDisptacher.scala:91)
	at org.thp.scalligraph.DiagnosticContext$.saveDiagnosticContext(ContextPropagatingDisptacher.scala:106)
	at org.thp.scalligraph.DiagnosticContext$.withDiagnosticContext(ContextPropagatingDisptacher.scala:89)
	at org.thp.scalligraph.DiagnosticContext$$anon$2.withContext(ContextPropagatingDisptacher.scala:74)
	at org.thp.scalligraph.ContextPropagatingDisptacher$$anon$1.$anonfun$execute$1(ContextPropagatingDisptacher.scala:56)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:163
2020-09-29 17:59:12,706 [ERROR] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-20 [00000020|44f816ea] Exception raised, rollback (Observable already exists)
2020-09-29 17:59:12,707 [WARN] from org.thp.scalligraph.ErrorHandler in application-akka.actor.default-dispatcher-20 [00000020|] POST /api/case/43819240/artifact returned 400
org.thp.scalligraph.CreateError: Observable already exists
	at org.thp.thehive.services.CaseSrv.addObservable(CaseSrv.scala:163)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$10(ObservableCtrl.scala:85)
	at org.thp.scalligraph.package$RichSeq.$anonfun$toTry$3(package.scala:25)
	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
	at scala.collection.immutable.List.foldLeft(List.scala:89)
	at org.thp.scalligraph.package$RichSeq.toTry(package.scala:24)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$9(ObservableCtrl.scala:83)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$7(ObservableCtrl.scala:79)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$5(ObservableCtrl.scala:76)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$4(ObservableCtrl.scala:75)
	at scala.util.Success.flatMap(Try.scala:251)
	at org.thp.thehive.controllers.v0.ObservableCtrl.$anonfun$create$3(ObservableCtrl.scala:71)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$authTransaction$2(Entrypoint.scala:77)
	at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$10(JanusDatabase.scala:199)
	at scala.util.Try$.apply(Try.scala:213)
	at org.thp.scalligraph.janus.JanusDatabase.$anonfun$tryTransaction$8(JanusDatabase.scala:199)
	at scala.util.Try$.apply(Try.scala:213)
	at org.thp.scalligraph.utils.DelayRetry.withTry(Retry.scala:87)
	at org.thp.scalligraph.janus.JanusDatabase.tryTransaction(JanusDatabase.scala:193)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$authTransaction$1(Entrypoint.scala:77)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$auth$1(Entrypoint.scala:86)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$asyncAuth$3(Entrypoint.scala:107)
	at org.scalactic.Good.fold(Or.scala:1229)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$asyncAuth$2(Entrypoint.scala:107)
	at org.thp.scalligraph.DiagnosticContext$.$anonfun$withRequest$2(ContextPropagatingDisptacher.scala:99)
	at org.thp.scalligraph.DiagnosticContext$.saveDiagnosticContext(ContextPropagatingDisptacher.scala:106)
	at org.thp.scalligraph.DiagnosticContext$.withRequest(ContextPropagatingDisptacher.scala:97)
	at org.thp.scalligraph.controllers.Entrypoint$EntryPointBuilder.$anonfun$asyncAuth$1(Entrypoint.scala:107)
	at org.thp.scalligraph.auth.SessionAuthSrv$$anon$1.$anonfun$invokeBlock$2(SessionAuthSrv.scala:89)
	at scala.Option.fold(Option.scala:251)
	at org.thp.scalligraph.auth.SessionAuthSrv$$anon$1.invokeBlock(SessionAuthSrv.scala:87)
	at org.thp.scalligraph.auth.SessionAuthSrv$$anon$1.invokeBlock(SessionAuthSrv.scala:84)
	at play.api.mvc.ActionBuilder$$anon$10.$anonfun$invokeBlock$2(Action.scala:408)
	at play.api.mvc.ActionBuilderImpl.invokeBlock(Action.scala:441)
	at play.api.mvc.ActionBuilderImpl.invokeBlock(Action.scala:439)
	at play.api.mvc.ActionBuilder$$anon$10.invokeBlock(Action.scala:408)
	at play.api.mvc.ActionBuilder$$anon$10.invokeBlock(Action.scala:404)
	at play.api.mvc.ActionBuilder$$anon$9.apply(Action.scala:379)
	at play.api.mvc.Action.$anonfun$apply$4(Action.scala:82)
	at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
	at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
	at org.thp.scalligraph.ContextPropagatingDisptacher$$anon$1.$anonfun$execute$2(ContextPropagatingDisptacher.scala:56)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.thp.scalligraph.DiagnosticContext$.$anonfun$withDiagnosticContext$2(ContextPropagatingDisptacher.scala:91)
	at org.thp.scalligraph.DiagnosticContext$.saveDiagnosticContext(ContextPropagatingDisptacher.scala:106)
	at org.thp.scalligraph.DiagnosticContext$.withDiagnosticContext(ContextPropagatingDisptacher.scala:89)
	at org.thp.scalligraph.DiagnosticContext$$anon$2.withContext(ContextPropagatingDisptacher.scala:74)
	at org.thp.scalligraph.ContextPropagatingDisptacher$$anon$1.$anonfun$execute$1(ContextPropagatingDisptacher.scala:56)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:163)
2020-09-29 17:59:12,707 [INFO] from org.thp.scalligraph.AccessLogFilter in application-akka.actor.default-dispatcher-20 [00000020|] 10.10.0.73 POST /api/case/43819240/artifact took 1526ms and returned 400 60 bytes
@vi-or-die vi-or-die added TheHive4 TheHive4 related issues bug labels Sep 30, 2020
@To-om To-om self-assigned this Oct 15, 2020
@To-om To-om added this to the 4.0.1 milestone Oct 15, 2020
@To-om To-om closed this as completed Oct 15, 2020
@vi-or-die
Copy link
Author

@To-om Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug TheHive4 TheHive4 related issues
Projects
None yet
Development

No branches or pull requests

2 participants