Skip to content

[Bug] Missing cases migrating from TH3 to TH4 #1682

Closed
@mamoedo

Description

@mamoedo

Request Type

Bug

Work Environment

Question Answer
OS version (server) Debian
TheHive version / git hash 4.0.2
Package Type Binary

Problem Description

Some cases are missing when migrating multiple index from TH3 to different TH4 organizations.

Steps to Reproduce

  1. Migrate the first index (about 100 cases):
    ./migrate --output /etc/thehive/application.conf --main-organisation Small-Org --es-index the_hive_small --es-uri http://elasticsearch:9200 --exclude-audit-actions Update,Creation,Delete
  2. Wait for the migration to finish. Migrate the second index (about 700 cases):
    ./migrate --output /etc/thehive/application.conf --main-organisation Medium-Org --es-index the_hive_medium --es-uri http://elasticsearch:9200 --exclude-audit-actions Update,Creation,Delete
  3. Wait for the migration to finish. Migrate the second index (about 1500 cases):
    ./migrate --output /etc/thehive/application.conf --main-organisation Main-Org --es-index the_hive --es-uri http://elasticsearch:9200 --exclude-audit-actions Update,Creation,Delete
  4. Compare the number of migrated cases with the TH3 number of cases for each organization.

Possible Solutions

I think it could be a problem with the case numeration (see add. info. 3). I tried a few cases searching them by number, and there's only one case with each number in one organization. For example: if you find case 50 in org A, you won't find it in org B or C. If you find case 250 in org B, it won't exist on org A or C and so on.

Complementary information

  1. Note that when using the --es-index option, _15 is always appended to the index name.

  2. Migration log seems to skip some cases:

[info] [Migrate cases and alerts] CaseTemplate/Task:55 Action:82/564 Case/Task:341/1626 Case:200/1303 Job:15026/304192 ObservableType:23/63 Alert:10692/48043(5ms) CaseTemplate:15/20 Alert/Observable:99725(13ms) Case/Observable:16026/227354 User:14/19 CustomField:28/29 Case/Task/Log
:143/606
[info] [Migrate cases and alerts] CaseTemplate/Task:55 Action:82/564 Case/Task:345/1626(6ms) Case:203/1303(6ms) Job:15070/304192(7ms) ObservableType:23/63 Alert:10711/48043(6ms) CaseTemplate:15/20 Alert/Observable:100122(7ms) Case/Observable:16146/227354(33ms) User:14/19 CustomFiel
d:28/29 Case/Task/Log:143/606
  1. At the end of the main organization, it shows:

Case: 338/1303 (965 exists) avg:10ms

As if 965 cases were already migrated, but they were not for this organization and from this index. (Altough it's a number very close to Small-Org + MediumOrg)

  1. When there's the same user in more than one index, it's only migrated the first time. So if user Bob exists in org A,B,C, it will only be migrated to orgA.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions