Timeout error when ingesting certain complex bundle

[romain-filigran]

Description

When consuming a TAXII stream exposed through an OpenCTI, I am confronted with an infinite loop of attempts by workers when ingesting a bundle with errors of the "MISSING_REFERENCE_ERROR" type.

{
    "category": "APP",
    "errors": [
        {
            "attributes": {
                "genre": "BUSINESS",
                "http_status": 404,
                "unresolvedIds": [
                    "domain-name--3ce2fdfc-e1be-5adb-9009-07275a620779",
                    "ipv4-addr--9c1fef00-081c-5189-a847-f50469c52a37"
                ]
            },
            "message": "Element(s) not found",
            "name": "MISSING_REFERENCE_ERROR",
            "stack": "MISSING_REFERENCE_ERROR: Element(s) not found\n    at error (/opt/opencti/build/src/config/errors.js:8:10)\n    at MissingReferenceError (/opt/opencti/build/src/config/errors.js:123:48)\n    at inputResolveRefsFn (/opt/opencti/build/src/database/middleware.js:732:13)\n    at createRelationRaw (/opt/opencti/build/src/database/middleware.js:2682:25)\n    at createRelation (/opt/opencti/build/src/database/middleware.js:2822:16)\n    at addStixCoreRelationship (/opt/opencti/build/src/domain/stixCoreRelationship.js:99:19)"
        }
    ],
    "inner_relation_creation": 0,
    "level": "warn",
    "message": "Element(s) not found",
    "operation": "Unspecified",
    "query_attributes": [
        [
            {
                "arguments": [
                    [
                        {
                            "is_empty": true,
                            "name": "input",
                            "type": "Variable"
                        }
                    ]
                ],
                "name": "stixCoreRelationshipAdd"
            }
        ]
    ],
    "size": 625,
    "source": "backend",
    "time": 56,
    "timestamp": "2024-08-13T06:14:02.546Z",
    "type": "WRITE_ERROR",
    "user": {
        "applicant_id": "628c63b2-bb71-4d45-b310-210afdc6f119",
        "call_retry_number": "2",
        "group_ids": [
            "44aefa36-d788-4f87-8c0b-ac4c43bcfdd4",
            "2f8ca9ae-93ba-4b7d-9045-2b29f4bd293d"
        ],
        "ip": "::ffff:172.20.0.22",
        "organization_ids": [],
        "socket": "query",
        "user_id": "628c63b2-bb71-4d45-b310-210afdc6f119",
        "user_metadata": {}
    },
    "version": "6.2.12"
}

Environment

OpenCTI: 6.2.12

Reproducible Steps

Try consuming the TAXII feed named "Test TAXII Bug" exposed on testing platform.

[richard-julien]

The retry from the global rabbitmq timeout.
The approach to resolve this is to split and requeue message with complex bundle