Skip to content

Add SOCRadar native alert feeder integration - #7

Open
Radargoger wants to merge 1 commit into
StrangeBeeCorp:mainfrom
Radargoger:feat/socradar-alert-feeder
Open

Radargoger wants to merge 1 commit into
StrangeBeeCorp:mainfrom
Radargoger:feat/socradar-alert-feeder

Conversation

@Radargoger

Copy link
Copy Markdown

Summary

Adds a native TheHive alert feeder integration for SOCRadar XTI. It polls the SOCRadar Incident API v4 and creates one TheHive alert per SOCRadar alarm.

  • integrations/vendors/SOCRadar/thehive/functions/function_Feeder_alertFromSOCRadar.js: feeder function (handle(input, context))
  • integrations/vendors/SOCRadar/use-cases/alert-ingestion.md: setup guide (license, feeder settings, function settings, field mapping, data coverage, troubleshooting)
  • integrations/vendors/SOCRadar/vendor.yml and assets/logo.png

.generated/ is untouched and left to the sync workflow.

Behavior

  • One alert per alarm, deduplicated on sourceRef = <company_id>:<alarm_id>.
  • Mapped fields: title, description (alarm text, mitigation, detection and analysis, related assets, compliance, technical content), severity (LOW..CRITICAL → 1..4), alarm creation date (UTC), main/sub type tags, and an externalLink to the alarm in SOCRadar.
  • Related assets and entities ({key, value} pairs) become typed observables (domain, hostname, url, ip, certificate serials as other) with ioc: false. They describe affected assets, not malicious indicators. Brand keywords, DNS record changes, ports and IDs stay in the description so they don't link unrelated alerts.
  • Credential values in the alarm content are redacted.
  • Every field is clipped to the limits of TheHive's POST /api/v1/alert schema (title 512, sourceRef 128, tags 128, externalLink 4096, description 1 MiB). Observable values over 4096 characters are listed in the description instead. The SOCRadar API doesn't truncate anything.

Design: self-paging feeder

SOCRadar alarms can become visible some time after they're created (for example, after analyst approval). They keep their original creation date, and the API returns alarms newest first, at most 100 per request. A feeder URL is static and a run reads one page, so a plain feeder can't page or move a start_date window.

The integration therefore pages through TheHive's own components:

  1. The feeder requests one page (limit=100&page=N&start_date=<epoch>&include_total_records=true).
  2. The function creates the missing alerts and writes the URL of the next request to a cursor alert (type socradar-feeder, status Ignored).
  3. A FilteredEvent notification on that cursor alert sends the URL to PUT /api/v1/connector/alert-feeder/{name} through an HttpRequest notifier. Updating a feeder runs it immediately, so the next page follows within seconds.

One feeder alternates recent passes (alarms since the newest one seen, page after page) with a lookback pass over the last 5 days every 60 minutes. When nothing changes, the URL stays the same, the notification doesn't fire, and the feeder waits for its 5-minute interval.

Other safeguards:

  • Existing alerts are checked with one _in query per 250 alarms instead of one alert.find per alarm (1,000 lookups: 0.45 s instead of 18 s).
  • Create errors are caught and logged, never rethrown, because a thrown error rolls back the whole run.
  • If the feeder ran an outdated URL (detected from alarm dates) or a requested hop never arrived, the function requests it again. After 3 retries it creates one health alert per day.

Testing

Local (Node, mocked context), 33/33 passing: mapping, dates, deduplication, error handling, field clipping, content truncation, the {key, value} related items, and the paging state machine (initial lookback, pass switching, burst paging, rest without hops, 150 alarms sharing one second, total_pages, outdated URL detection, lost-hop retries and the health alert).

Live, TheHive 5.8.0 (Docker, Platinum trial) with a mock SOCRadar API reproducing its sorting, paging and filters, 1-minute feeder and 3-minute lookback interval. After every scenario the chain must reach rest and keep the same cursor for two feeder intervals.

# Scenario Result
A First run: 300 alarms over 5 days plus a burst of 1,000 in the last hour 1,300 alerts, 0 duplicates, at rest after 185 s, feeder URL matches the cursor
B No new data Cursor unchanged; only scheduled runs and the scheduled lookback pass
C 250 new alarms in one second, 30 arriving mid-pass, 1 alarm created 26 h ago and visible late All 281 ingested, 0 duplicates
D Feeder URL overwritten with an outdated one Detected and repaired in 13 s
E Paging notification disabled while 150 alarms arrive in one second, then re-enabled Health alert after 3 retries, first page still ingested; full recovery, 0 duplicates, no loop

Live, TheHive 5.8.0 with the production SOCRadar API (one company, 287 approved alarms in the last 5 days):

Scenario Result
First run from the static setup URL, then a 3-page lookback pass and a switch to recent passes All within about 40 s
Completeness against the API 287 of 287 alarms in TheHive, 0 duplicates, nothing outside the 5-day window
Feeder configuration after the paging updates URL matches the cursor; interval, timeout, size and API-Key header preserved by the notification template
Idle scheduled runs One hop to the next start second, an empty response, then rest: no loop
Field sizes on real data (earlier run, 385 alarms) Longest title 88 characters, longest description 47,362 characters
Observables on real data Typed hostname, url, domain, ip and other, none flagged as IOC

scripts/validate-vendor-metadata.py --strict passes for SOCRadar. generate-catalogs.py and generate-docs.py pick up the vendor, use case and function.

Catalog listing request

We'd like SOCRadar to be listed in the TheHive integrations catalog with this use case. Please let us know if you need anything else for the listing, such as a different logo format, screenshots, or a vendor contact.

Feeder function that turns SOCRadar Incident API v4 alarms into TheHive
alerts, deduplicated on company_id:alarm_id, with severity, mitigation,
main/sub type tags, typed non-IOC observables, the full alarm content
(credentials redacted, long values shortened) and a link back to
SOCRadar. Every field is clipped to TheHive's alert input limits.

A feeder URL is static and a run reads one page, so the function stores
the next request in a cursor alert and a FilteredEvent notification
applies it to the feeder through the TheHive API. One feeder alternates
recent passes from the newest alarm seen with an hourly 5-day lookback
that catches late-approved alarms. Outdated URLs and lost hops are
retried, with a daily health alert after three retries.

Includes vendor metadata, logo and a setup guide.
@Radargoger

Copy link
Copy Markdown
Author

@nusantara-self Could you please help me to review our app?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant