Skip to content

Conversation

@yanivagman
Copy link
Collaborator

This PR migrates 34 security detection signatures from the legacy signature plugin system to the new detector framework, plus adds test infrastructure improvements.

Note for reviewers: This PR is best reviewed commit-by-commit, as each commit migrates one signature independently.

Changes

Migrated Signatures:

  • Anti-debugging & code injection: anti_debugging_ptraceme, ptrace_code_injection, proc_mem_code_injection, process_vm_write_code_injection, dynamic_code_loading, fileless_execution
  • System hooking: hooked_syscall, hooked_seq_ops, proc_fops_hooking, syscall_table_hooking
  • Persistence mechanisms: core_pattern_modification, default_loader_modification, ld_preload, rcd_modification, scheduled_task_modification, sudoers_modification
  • Container escapes: cgroup_notify_on_release_modification, cgroup_release_agent_modification, disk_mount, docker_abuse
  • Kubernetes security: k8s_service_account_token, kubernetes_api_connection, kubernetes_certificate_theft_attempt
  • System reconnaissance: aslr_inspection, proc_kcore_read, proc_mem_access, sched_debug_recon
  • Suspicious activities: dropped_executable, hidden_file_created, illegitimate_shell, stdio_over_socket, system_request_key_config_modification
  • Kernel security: kernel_module_loading

Test Infrastructure:

  • Added common detector test helpers
  • Fixed e2e-kernel-test to work with new detector framework

Impact

  • All signatures maintain identical detection logic
  • Tests updated to use new detector testing patterns
  • Signatures remain registered and functional

This is a refactoring migration with no behavioral changes to detection capabilities.

@yanivagman yanivagman self-assigned this Dec 29, 2025
Copilot AI review requested due to automatic review settings December 29, 2025 10:23
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates 34 security detection signatures from the legacy Go plugin signature system to the new detector framework, representing a significant architectural refactoring. The migration maintains identical detection logic while adopting a more modern, testable architecture with improved type safety and separation of concerns.

Key Changes

  • Migrated 34 signatures covering anti-debugging, code injection, persistence mechanisms, container escapes, Kubernetes security, system reconnaissance, and kernel security
  • Added detector test infrastructure with mock helpers for logger and data store registry
  • Updated e2e kernel test script to handle detector-produced events which use different event name patterns
  • Emptied the legacy signature export list as signatures transition to the detector framework

Reviewed changes

Copilot reviewed 132 out of 133 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/e2e-kernel-test.sh Added detector ID to event name mapping and logic to check for detector-produced events
detectors/test_helpers.go Created shared test infrastructure with mock logger and data store registry implementations
detectors/*_test.go (multiple files) Added comprehensive detector tests replacing legacy signature tests
detectors/*.go (multiple files) Implemented new detector versions of migrated signatures
signatures/golang/export.go Emptied ExportedSignatures list as signatures migrate to detector framework
signatures/golang/*.go (multiple files) Removed legacy signature implementations
signatures/golang/*_test.go (multiple files) Removed legacy signature test files

Comment on lines 156 to 170
# Map detector IDs to event names (detectors produce events with "name" field, not "signatureID")
declare -A detector_to_event=(
["TRC-102"]="anti_debugging"
["TRC-103"]="ptrace_code_injection"
["TRC-104"]="dynamic_code_loading"
["TRC-105"]="fileless_execution"
["TRC-107"]="ld_preload"
["TRC-1010"]="cgroup_release_agent"
["TRC-1014"]="disk_mount"
["TRC-1016"]="illegitimate_shell"
["TRC-1018"]="k8s_cert_theft"
["TRC-1022"]="dropped_executable"
)
Copy link

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hardcoded mapping between detector IDs and event names creates tight coupling between the test script and detector implementations. Consider generating this mapping dynamically from detector definitions or maintaining it in a shared configuration file that both the detectors and tests can reference. This would reduce maintenance burden when adding new detectors or modifying event names.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid point. Currently the test only covers 9 of the 34 migrated detectors, and the 1:1 mapping between detector IDs and event names is indeed always needed. As test coverage expands, maintaining a hardcoded map could become tedious. However, generating this dynamically from detector definitions would require either parsing Go code or invoking the compiled binary - both add complexity. A middle-ground might be a JSON config file that's easier to maintain and could be generated by a helper script. Open to suggestions on the best approach which we can implement in a future PR

@codecov
Copy link

codecov bot commented Dec 29, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 33.55%. Comparing base (235daa0) to head (16767da).
⚠️ Report is 36 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5146      +/-   ##
==========================================
+ Coverage   33.51%   33.55%   +0.04%     
==========================================
  Files         250      226      -24     
  Lines       28908    28442     -466     
==========================================
- Hits         9688     9544     -144     
+ Misses      18609    18322     -287     
+ Partials      611      576      -35     
Flag Coverage Δ
unit 33.55% <ø> (+0.04%) ⬆️
see 25 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yanivagman yanivagman force-pushed the signature_to_detector_migration branch from 58e2f2d to 8fc16a5 Compare December 29, 2025 10:34
Add shared test mocks for detector testing:
- mockLogger: implements detection.Logger interface
- mockDataStoreRegistry: implements datastores.Registry interface

These mocks are used across all detector test files.
Migrate derived event 'hooked_syscall' from pkg/events/derive to detector API.

Original: pkg/events/derive/hooked_syscall.go
Detector ID: DRV-002

Implementation:
- EventDetector interface with syscall_table_check input event
- SyscallStore (syscall) integration for syscall ID to name mapping
- KernelSymbolStore (symbol) integration for address resolution
- LRU cache (500 entries) for deduplication of reported hooks
- Multiple events produced when multiple symbols at same address

Testing:
- Unit tests with 86.7% coverage for hooked_syscall.go
- Verified output schema matches original (syscall, address, function, owner)
- Tests cover: symbol resolution, caching, multiple symbols, unknown syscalls

Migration notes:
- Uses newly created SyscallStore instead of pkg/events direct access
- Detector state managed via LRU cache (no eBPF map dependencies)
- Event name changed to 'hooked_syscall_detector' to avoid collision during transition
Migrate derived event 'hooked_seq_ops' from pkg/events/derive to detector API.

Original: pkg/events/derive/hooked_seq_ops.go
Detector ID: DRV-003

Implementation:
- EventDetector interface with print_net_seq_ops input event
- KernelSymbolStore (symbol) integration for address resolution
- Processes array of 24 seq_ops function pointers (6 structs * 4 functions)
- Produces map of hooked operations: "tcp4_seq_ops_show" -> {SymbolName, ModuleOwner}
- Skips unhooked functions (address == 0) automatically

Testing:
- Unit tests with 90.6% coverage for OnEvent
- Tests cover: single hook, multiple hooks, no hooks, empty array, unresolved symbols
- Verified output schema matches original (map[string]HookedSymbolData)
- Index math verified: seqOpsStruct = netSeqOps[i/4], seqOpsFunc = netSeqOpsFuncs[i%4]

Migration notes:
- Uses HookedSeqOps protobuf message type for output
- No state management needed (stateless detector)
- Event name changed to 'hooked_seq_ops_detector' to avoid collision during transition
…r framework

Migrate signature 'anti_debugging_ptraceme' from signatures/golang to detector API.

Original: signatures/golang/anti_debugging_ptraceme.go
Detector ID: TRC-102

Implementation:
- EventDetector interface with ptrace input event
- Uses DataFilter 'request=0' to receive only PTRACE_TRACEME events
- Simplified OnEvent logic - filter ensures only relevant events reach detector
- No external datastores required (stateless detector)
- Produces 'anti_debugging' event matching original signature output

Testing:
- Unit tests with 100% coverage for OnEvent
- Tests cover: PTRACE_TRACEME detection, multiple calls
- Verified output schema matches original signature
- Verified threat metadata (TRC-102, T1622, Defense Evasion)

Migration notes:
- Leverages DataFilter to move filtering logic to framework level
- Uses common/parsers for PTRACE_TRACEME constant (common module is safe dependency)
- Event name 'anti_debugging' matches original signature EventName
- Properties stored in threat.Properties map per protobuf schema
Migrate signature 'aslr_inspection' from signatures/golang to detector API.

Original: signatures/golang/aslr_inspection.go
Detector ID: TRC-109

Implementation:
- EventDetector interface with security_file_open input event
- DataFilter 'pathname=/proc/sys/kernel/randomize_va_space' to pre-filter events
- OnEvent checks if flags indicate read operation using parsers.IsFileRead()
- No external datastores required (stateless detector)
- Produces 'aslr_inspection' event matching original signature output

Testing:
- Unit tests with 100% coverage for OnEvent
- Tests cover: O_RDONLY, O_RDWR (read), O_WRONLY (write, no detection), missing flags
- Multiple operations tested for idempotency
- Verified output schema matches original signature
- Verified threat metadata (TRC-109, T1068, Privilege Escalation)

Migration notes:
- Leverages DataFilter for pathname to reduce events reaching detector
- Uses common/parsers.IsFileRead() for flag validation logic
- Event name 'aslr_inspection' matches original signature EventName
- Severity set to INFO (0 in original signature properties)
Migrate signature 'proc_kcore_read' from signatures/golang to detector API.

Original: signatures/golang/proc_kcore_read.go
Detector ID: TRC-1021

Implementation:
- EventDetector interface with security_file_open input event
- DataFilter 'pathname=/proc/kcore' to pre-filter events
- ScopeFilter 'container' to receive only container events (matching original Origin)
- OnEvent uses strings.HasSuffix() for pathname validation (edge cases like symlinks)
- OnEvent checks if flags indicate read operation using parsers.IsFileRead()
- No external datastores required (stateless detector)
- Produces 'proc_kcore_read' event matching original signature output

Testing:
- Unit tests with 100% coverage for OnEvent
- Tests cover: exact path, suffix match, O_RDONLY, O_RDWR, O_WRONLY, wrong path
- Missing fields tested (pathname, flags)
- Multiple operations tested for idempotency
- Verified output schema matches original signature
- Verified threat metadata (TRC-1021, T1611, Escape to Host)

Migration notes:
- Leverages both DataFilter and ScopeFilter to reduce events reaching detector
- Uses strings.HasSuffix to match original signature logic (not just exact match)
- Event name 'proc_kcore_read' matches original signature EventName
- Severity set to MEDIUM (2 in original signature properties)
- Scope filter 'container' matches original selector Origin: "container"
…amework

Migrate signature 'hidden_file_created' from signatures/golang to detector API.

Original: signatures/golang/hidden_file_created.go
Detector ID: TRC-1015

Implementation:
- EventDetector interface with magic_write input event
- OnEvent checks pathname contains '/." (hidden file pattern)
- OnEvent uses elf.IsElf() to validate ELF magic bytes
- No external datastores required (stateless detector)
- Produces 'hidden_file_created' event matching original signature output

Testing:
- Unit tests with 100% coverage for OnEvent
- Tests cover: various hidden paths, non-hidden paths, ELF vs non-ELF
- Missing fields tested (pathname, bytes)
- Multiple write operations tested for idempotency
- Verified output schema matches original signature
- Verified threat metadata (TRC-1015, T1564.001, Hidden Files and Directories)

Migration notes:
- Uses common/elf for ELF detection (common module is safe dependency)
- Uses strings.Contains for hidden path pattern (matching original logic)
- Event name 'hidden_file_created' matches original signature EventName
- Severity set to MEDIUM (2 in original signature properties)
- Cannot use DataFilter for complex conditions (pathname pattern + ELF check)
…ure to detector

Migrated from signatures/golang/cgroup_notify_on_release_modification.go
Detector ID: TRC-106
…to detector

Migrated from signatures/golang/cgroup_release_agent_modification.go
Detector ID: TRC-1010
Migrated from signatures/golang/core_pattern_modification.go
Detector ID: TRC-1011
…ector

Migrated from signatures/golang/default_loader_modification.go
Detector ID: TRC-1012
Migrated from signatures/golang/disk_mount.go
Detector ID: TRC-1014
Migrated from signatures/golang/docker_abuse.go
Detector ID: TRC-1019
Migrated from signatures/golang/dropped_executable.go
Detector ID: TRC-1022
Migrated from signatures/golang/dynamic_code_loading.go
Detector ID: TRC-104
Migrated from signatures/golang/fileless_execution.go
Detector ID: TRC-105
Migrated from signatures/golang/illegitimate_shell.go
Detector ID: TRC-1016
Migrated from signatures/golang/k8s_service_account_token.go
Detector ID: TRC-108
Migrated from signatures/golang/kernel_module_loading.go
Detector ID: TRC-1017
Migrated from signatures/golang/kubernetes_api_connection.go
Detector ID: TRC-1013
…re to detector

Migrated from signatures/golang/kubernetes_certificate_theft_attempt.go
Detector ID: TRC-1018
Migrated from signatures/golang/ld_preload.go
Detector ID: TRC-107
Migrated from signatures/golang/proc_fops_hooking.go
Detector ID: TRC-1020
Migrated from signatures/golang/proc_mem_access.go
Detector ID: TRC-1023
Migrated from signatures/golang/proc_mem_code_injection.go
Detector ID: TRC-1024
… detector

Migrated from signatures/golang/process_vm_write_code_injection.go
Detector ID: TRC-1025
Migrated from signatures/golang/ptrace_code_injection.go
Detector ID: TRC-103
Migrated from signatures/golang/rcd_modification.go
Detector ID: TRC-1026
Migrated from signatures/golang/sched_debug_recon.go
Detector ID: TRC-1029
…ector

Migrated from signatures/golang/scheduled_task_modification.go
Detector ID: TRC-1027
Migrated from signatures/golang/stdio_over_socket.go
Detector ID: TRC-101
Migrated from signatures/golang/sudoers_modification.go
Detector ID: TRC-1028
Migrated from signatures/golang/syscall_table_hooking.go
Detector ID: TRC-1030
…ture to detector

Migrated from signatures/golang/system_request_key_config_modification.go
Detector ID: TRC-1031
- Update test validation to search for detector event names instead of
  signatureID field (which doesn't exist in detector output)
- Add detector ID to event name mapping for all tested detectors
@yanivagman yanivagman force-pushed the signature_to_detector_migration branch from 8fc16a5 to 16767da Compare December 30, 2025 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant