added back `-h, --help` option to General Options #1266

hitenkoku · 2024-02-03T13:26:46Z

What Changed

Added back -h, --help option to General Options.

I would appreciate it if you could review when you have time.

…s://github.com/Yamato-Security/hayabusa into 1255-add-back-h-help-option-to-general-options

hitenkoku · 2024-02-03T13:29:44Z

Evidence

> ./1255.exe

╔╗ ╔╦═══╦╗  ╔╦═══╦══╗╔╗ ╔╦═══╦═══╗
║║ ║║╔═╗║╚╗╔╝║╔═╗║╔╗║║║ ║║╔═╗║╔═╗║
║╚═╝║║ ║╠╗╚╝╔╣║ ║║╚╝╚╣║ ║║╚══╣║ ║║
║╔═╗║╚═╝║╚╗╔╝║╚═╝║╔═╗║║ ║╠══╗║╚═╝║
║║ ║║╔═╗║ ║║ ║╔═╗║╚═╝║╚═╝║╚═╝║╔═╗║
╚╝ ╚╩╝ ╚╝ ╚╝ ╚╝ ╚╩═══╩═══╩═══╩╝ ╚╝
   by Yamato Security 

Hayabusa v2.13.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe <COMMAND> [OPTIONS]
  hayabusa.exe help <COMMAND>

Commands:
  computer-metrics     Print computer name metrics
  csv-timeline         Save the timeline in CSV format
  eid-metrics          Print event ID metrics
  json-timeline        Save the timeline in JSON/JSONL format
  level-tuning         Tune alert levels (default: ./rules/config/level_tuning.txt)
  list-contributors    Print the list of contributors
  list-profiles        List the output profiles
  logon-summary        Print a summary of successful and failed logons
  pivot-keywords-list  Create a list of pivot keywords
  search               Search all events by keyword(s) or regular expression
  set-default-profile  Set default output profile
  update-rules         Update to the latest rules in the hayabusa-rules github repository
  help                 Print this message or the help of the given subcommand(s)

> ./1255.exe csv-timeline -h
Hayabusa v2.13.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe csv-timeline <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder
  -J, --JSON-input       Scan JSON formatted logs instead of .evtx (.json or .jsonl)
  -x, --recover-records  Carve evtx records from slack space (default: disabled)

Filtering:
able,test)
      --exclude-tag <TAG...>            Do not load rules with specific tags (ex: sysmon)
      --include-category <CATEGORY...>  Only load rules with specified logsource categories (ex: process_creation,pipe_created)
      --include-computer <COMPUTER...>  Scan only specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --include-eid <EID...>            Scan only specified EIDs for faster speed (ex: 1) (ex: 1,4688)
      --include-tag <TAG...>            Only load rules with specific tags (ex: attack.execution,attack.discovery)
  -m, --min-level <LEVEL>               Minimum level for rules to load (default: informational)
  -P, --proven-rules                    Scan with only proven rules for faster speed (./rules/config/proven_rules.txt)
      --timeline-end <DATE>             End time of the event logs to load (ex: "2022-02-22 23:59:59 +09:00")
      --timeline-offset <OFFSET>        Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)
      --timeline-start <DATE>           Start time of the event logs to load (ex: "2020-02-22 00:00:00 +09:00")

Output:
  -G, --GeoIP <MAXMIND-DB-DIR>       Add GeoIP (ASN, city, country) info to IP addresses
  -H, --HTML-report <FILE>           Save Results Summary details to an HTML report (ex: results.html)
  -M, --multiline                    Output event field information in multiple rows
  -F, --no-field-data-mapping        Disable field data mapping
      --no-pwsh-field-extraction     Disable field extraction of PowerShell classic logs
  -o, --output <FILE>                Save the timeline in CSV format (ex: results.csv)
  -p, --profile <PROFILE>            Specify output profile
  -R, --remove-duplicate-data        Duplicate field data will be replaced with "DUP"
  -X, --remove-duplicate-detections  Remove duplicate detections (default: disabled)

Display Settings:
      --no-color            Disable color output
  -N, --no-summary          Do not display Results Summary for faster speed
  -q, --quiet               Quiet mode: do not display the launch banner
  -v, --verbose             Output verbose information
  -T, --visualize-timeline  Output event frequency timeline (terminal needs to support unicode)

General Options:
  -h, --help                           Show the help menu
  -C, --clobber                        Overwrite files when saving
  -w, --no-wizard                      Do not ask questions. Scan for all events and alerts
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -r, --rules <DIR/FILE>               Specify a custom rule directory or file (default: ./rules)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
      --ISO-8601          Output timestamp in ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

>  ./1255.exe csv-timeline --help
Hayabusa v2.13.0 - Dev Build
Yamato Security (https://github.com/Yamato-Security/hayabusa - @SecurityYamato)

Usage:
  hayabusa.exe csv-timeline <INPUT> [OPTIONS]

Input:
  -d, --directory <DIR>  Directory of multiple .evtx files
  -f, --file <FILE>      File path to one .evtx file
  -l, --live-analysis    Analyze the local C:\Windows\System32\winevt\Logs folder
  -J, --JSON-input       Scan JSON formatted logs instead of .evtx (.json or .jsonl)
  -x, --recover-records  Carve evtx records from slack space (default: disabled)

Filtering:
  -E, --EID-filter                      Scan only common EIDs for faster speed (./rules/config/target_event_IDs.txt)
  -D, --enable-deprecated-rules         Enable rules with a status of deprecated
  -n, --enable-noisy-rules              Enable rules set to noisy (./rules/config/noisy_rules.txt)
  -u, --enable-unsupported-rules        Enable rules with a status of unsupported
  -e, --exact-level <LEVEL>             Only load rules with a specific level (informational, low, medium, high, critical)
      --exclude-category <CATEGORY...>  Do not load rules with specified logsource categories (ex: process_creation,pipe_created)
      --exclude-computer <COMPUTER...>  Do not scan specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --exclude-eid <EID...>            Do not scan specific EIDs for faster speed (ex: 1) (ex: 1,4688)
      --exclude-status <STATUS...>      Do not load rules according to status (ex: experimental) (ex: stable,test)
      --exclude-tag <TAG...>            Do not load rules with specific tags (ex: sysmon)
      --include-category <CATEGORY...>  Only load rules with specified logsource categories (ex: process_creation,pipe_created)
      --include-computer <COMPUTER...>  Scan only specified computer names (ex: ComputerA) (ex: ComputerA,ComputerB)
      --include-eid <EID...>            Scan only specified EIDs for faster speed (ex: 1) (ex: 1,4688)
      --include-tag <TAG...>            Only load rules with specific tags (ex: attack.execution,attack.discovery)
  -m, --min-level <LEVEL>               Minimum level for rules to load (default: informational)
  -P, --proven-rules                    Scan with only proven rules for faster speed (./rules/config/proven_rules.txt)
      --timeline-end <DATE>             End time of the event logs to load (ex: "2022-02-22 23:59:59 +09:00")
      --timeline-offset <OFFSET>        Scan recent events based on an offset (ex: 1y, 3M, 30d, 24h, 30m)
      --timeline-start <DATE>           Start time of the event logs to load (ex: "2020-02-22 00:00:00 +09:00")

Output:
  -G, --GeoIP <MAXMIND-DB-DIR>       Add GeoIP (ASN, city, country) info to IP addresses
  -H, --HTML-report <FILE>           Save Results Summary details to an HTML report (ex: results.html)
  -M, --multiline                    Output event field information in multiple rows
  -F, --no-field-data-mapping        Disable field data mapping
      --no-pwsh-field-extraction     Disable field extraction of PowerShell classic logs
  -o, --output <FILE>                Save the timeline in CSV format (ex: results.csv)
  -p, --profile <PROFILE>            Specify output profile
  -R, --remove-duplicate-data        Duplicate field data will be replaced with "DUP"
  -X, --remove-duplicate-detections  Remove duplicate detections (default: disabled)

Display Settings:
      --no-color            Disable color output
  -N, --no-summary          Do not display Results Summary for faster speed
  -q, --quiet               Quiet mode: do not display the launch banner
  -v, --verbose             Output verbose information
  -T, --visualize-timeline  Output event frequency timeline (terminal needs to support unicode)

General Options:
  -h, --help                           Show the help menu
  -C, --clobber                        Overwrite files when saving
  -w, --no-wizard                      Do not ask questions. Scan for all events and alerts
  -Q, --quiet-errors                   Quiet errors mode: do not save error logs
  -r, --rules <DIR/FILE>               Specify a custom rule directory or file (default: ./rules)
  -c, --rules-config <DIR>             Specify custom rule config directory (default: ./rules/config)
      --target-file-ext <FILE-EXT...>  Specify additional evtx file extensions (ex: evtx_data)
  -t, --threads <NUMBER>               Number of threads (default: optimal number for performance)

Time Format:
      --European-time     Output timestamp in European time format (ex: 22-02-2022 22:00:00.123 +02:00)
      --ISO-8601          Output timestamp in ISO-8601 format (ex: 2022-02-22T10:10:10.1234567Z) (Always UTC)
      --RFC-2822          Output timestamp in RFC 2822 format (ex: Fri, 22 Feb 2022 22:00:00 -0600)
      --RFC-3339          Output timestamp in RFC 3339 format (ex: 2022-02-22 22:00:00.123456-06:00)
      --US-military-time  Output timestamp in US military time format (ex: 02-22-2022 22:00:00.123 -06:00)
      --US-time           Output timestamp in US time format (ex: 02-22-2022 10:00:00.123 PM -06:00)
  -U, --UTC               Output time in UTC format (default: local time)

codecov · 2024-02-03T13:45:00Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6eb5c73) 82.18% compared to head (269a198) 82.22%.
Report is 6 commits behind head on main.

❗ Current head 269a198 differs from pull request most recent head d0dc893. Consider uploading reports for the commit d0dc893 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1266      +/-   ##
==========================================
+ Coverage   82.18%   82.22%   +0.04%     
==========================================
  Files          27       27              
  Lines       24269    24330      +61     
==========================================
+ Hits        19945    20006      +61     
  Misses       4324     4324

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

fukusuket

I confirmed Integration-Test!
It looks like the update-rules command is failing. Could you please confirm?
https://github.com/Yamato-Security/hayabusa/actions/runs/7767289224/job/21183971409

…s://github.com/Yamato-Security/hayabusa into 1255-add-back-h-help-option-to-general-options

hitenkoku · 2024-02-04T12:45:59Z

@fukusuket Thanks for your revies.
I fixed following problem in 86d1f20.
Could you recheck it?

fukusuket

Thank you for quick fix :) I confirmed following commands! LGTM!!🚀
https://github.com/Yamato-Security/hayabusa/actions/runs/7774221587/job/21198783616

./hayabusa  computer-metrics -h
./hayabusa  csv-timeline -h
./hayabusa  eid-metrics -h
./hayabusa  json-timeline -h
./hayabusa  level-tuning -h
./hayabusa  list-contributors -h
./hayabusa  list-profiles -h
./hayabusa  logon-summary -h
./hayabusa  pivot-keywords-list -h
./hayabusa  search -h
./hayabusa  set-default-profile -h
./hayabusa  update-rules -h

YamatoSecurity · 2024-02-04T18:33:13Z

@hitenkoku Thanks so much!
Just one small thing. Could you put --help after --clobber to keep everything in alphabetical order?

General Options:
  -h, --help                           Show the help menu
  -C, --clobber                        Overwrite files when saving

hitenkoku · 2024-02-05T12:23:03Z

@YamatoSecurity Thanks for your commet.
I have fixed the points you pointed out to me with chttps://github.com//pull/1266/commits/d0dc8939388bd355fb84d7699535f4cbdacaa04d.

Colud you recheck it?

YamatoSecurity

@hitenkoku LGTM! Thanks so much!

hitenkoku added 3 commits February 3, 2024 22:16

feat(configs): added back -h, --help option to General Options #1255

80975d2

docs(CHANGELOG): added #1255

a8a9d8e

Merge branch '1255-add-back-h-help-option-to-general-options' of http…

005a669

…s://github.com/Yamato-Security/hayabusa into 1255-add-back-h-help-option-to-general-options

hitenkoku added the enhancement New feature or request label Feb 3, 2024

hitenkoku requested review from fukusuket and YamatoSecurity February 3, 2024 13:26

hitenkoku self-assigned this Feb 3, 2024

hitenkoku linked an issue Feb 3, 2024 that may be closed by this pull request

Add back -h, --help option to General Options #1255

Closed

test: fixed test error due to adding help option #1255

a74f01f

fukusuket reviewed Feb 3, 2024

View reviewed changes

YamatoSecurity and others added 5 commits February 4, 2024 08:06

Merge branch 'main' into 1255-add-back-h-help-option-to-general-options

ac06b1e

Merge branch 'main' into 1255-add-back-h-help-option-to-general-options

90bb836

fix(configs): fixed no help error in no required option command #1255

86d1f20

test: fixed test due to change of help variable type #1255

8611001

Merge branch '1255-add-back-h-help-option-to-general-options' of http…

269a198

…s://github.com/Yamato-Security/hayabusa into 1255-add-back-h-help-option-to-general-options

hitenkoku requested a review from fukusuket February 4, 2024 12:46

fukusuket approved these changes Feb 4, 2024

View reviewed changes

UI(configs): fixed help option order #1255

d0dc893

YamatoSecurity approved these changes Feb 5, 2024

View reviewed changes

YamatoSecurity merged commit 380376f into main Feb 5, 2024
5 checks passed

hitenkoku deleted the 1255-add-back-h-help-option-to-general-options branch February 8, 2024 03:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added back `-h, --help` option to General Options #1266

added back `-h, --help` option to General Options #1266

hitenkoku commented Feb 3, 2024

hitenkoku commented Feb 3, 2024

codecov bot commented Feb 3, 2024 •

edited

Loading

fukusuket left a comment

hitenkoku commented Feb 4, 2024

fukusuket left a comment

YamatoSecurity commented Feb 4, 2024

hitenkoku commented Feb 5, 2024

YamatoSecurity left a comment

added back -h, --help option to General Options #1266

added back -h, --help option to General Options #1266

Conversation

hitenkoku commented Feb 3, 2024

What Changed

hitenkoku commented Feb 3, 2024

Evidence

codecov bot commented Feb 3, 2024 • edited Loading

Codecov Report

fukusuket left a comment

Choose a reason for hiding this comment

hitenkoku commented Feb 4, 2024

fukusuket left a comment

Choose a reason for hiding this comment

YamatoSecurity commented Feb 4, 2024

hitenkoku commented Feb 5, 2024

YamatoSecurity left a comment

Choose a reason for hiding this comment

added back `-h, --help` option to General Options #1266

added back `-h, --help` option to General Options #1266

codecov bot commented Feb 3, 2024 •

edited

Loading