plugins: in_winevtlog: adds ability to ignore channels missing in Windows Event Log

[c0d3fau1t]

Additions to in_winevtlog plugin to allow scenarios where one or more channels are missing on Windows Event Log
ex: PowerShellCore/Operational needs the proper software installed to
appear under Application and Services Log

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• [N/A ] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A ] Attached local packaging test output showing all targets (including any new ones) build.

Documentation

• Documentation required for this feature

Added information in comments

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[c0d3fau1t]

Follows an example of the configuration file for the plugin.
The entry ignore_missing_channels defaults to false. In this scenario the plugin behaves as it currently does, hence it stops immediately when it fails to register to one of the channels.
When ignore_missing_channels is set to true a failure to register to one or more channels does not stops the plugin from starting as long as the resulting list of registered channels is not empty.

[INPUT]
    Name                     winevtlog
    Channels                Application,Security,System,Microsoft-Windows-PowerShell/Operational,Microsoft-Windows-TaskScheduler/Operational,Microsoft-Windows-AppLocker/EXE and DLL,Microsoft-Windows-AppLocker/Packaged app-Execution,Microsoft-Windows-AppLocker/Packaged app-Deployment,Microsoft-Windows-TerminalServices-RDPClient/Operational,Microsoft-Windows-SmartCard-Audit/Authentication,Microsoft-Windows-SMBClient/Operational,Microsoft-Windows-Windows Defender/Operational
    Interval_Sec             1
    String_Inserts           false
    Read_Existing_Events     false
    Render_Event_As_XML      true
    ignore_missing_channels  true
    db                    evt.db

[c0d3fau1t]

Follows an example of operation when a channel (BogusChannel) is present in the list of channels to register to. The plugin correctly throws an error indicating it could not subscribe to channel BogusChannel but continues with the other channels it was able to subscribe to.

[2022/10/10 19:55:40] [ info] [fluent bit] version=1.9.9, commit=, pid=31480
[2022/10/10 19:55:40] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2022/10/10 19:55:40] [ info] [storage] version=1.3.0, type=memory-only, sync=normal, checksum=disabled, max_chunks_up=128
[2022/10/10 19:55:40] [ info] [cmetrics] version=0.3.7
[2022/10/10 19:55:40] [debug] [winevtlog:winevtlog.0] created event channels: read=936 write=940
[2022/10/10 19:55:40] [error] [in_winevtlog] cannot subscribe 'BogusChannel' (0)
[2022/10/10 19:55:40] [debug] [stdout:stdout.0] created event channels: read=1204 write=1208
[2022/10/10 19:55:40] [debug] [router] match rule winevtlog.0:stdout.0
[2022/10/10 19:55:40] [ info] [sp] stream processor started
[2022/10/10 19:55:40] [ info] [output:stdout:stdout.0] worker #0 started
[2022/10/10 19:55:41] [debug] [input:winevtlog:winevtlog.0] read 4108 bytes from 'Microsoft-Windows-AppLocker/EXE and DLL'
[2022/10/10 19:55:41] [debug] [input chunk] update output instances with new chunk size diff=2159
[2022/10/10 19:55:48] [debug] [input:winevtlog:winevtlog.0] read 13956 bytes from 'Security'
[2022/10/10 19:55:48] [debug] [input chunk] update output instances with new chunk size diff=9858
[2022/10/10 19:55:48] [debug] [input:winevtlog:winevtlog.0] read 1722 bytes from 'Microsoft-Windows-TaskScheduler/Operational'
[2022/10/10 19:55:48] [debug] [input chunk] update output instances with new chunk size diff=997
[2022/10/10 19:55:51] [debug] [task] created task=011A7A98 id=0 OK
[2022/10/10 19:55:51] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2022/10/10 19:55:51] [ warn] [engine] service will shutdown in max 5 seconds
[2022/10/10 19:55:51] [debug] [out flush] cb_destroy coro_id=0
[2022/10/10 19:55:51] [debug] [task] destroy task=011A7A98 (task_id=0)
[2022/10/10 19:55:51] [ info] [engine] service has stopped (0 pending tasks)
[2022/10/10 19:55:52] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2022/10/10 19:55:52] [ info] [output:stdout:stdout.0] thread worker #0 stopped

[edsiper]

please sign off your commits (DCO error), you can do with at commit time with git commit -s -m ....

[cosmo0920]

Could you rebase off the current master? Some of commits seem to be contaminated.

[c0d3fau1t]

Could you rebase off the current master? Some of commits seem to be contaminated.

Will dig into that, Github and shared repositories are new to me.

[c0d3fau1t]

Debug level message was added:

[2022/10/12 16:58:28] [ info] [fluent bit] version=1.9.9, commit=, pid=25436
[2022/10/12 16:58:28] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2022/10/12 16:58:28] [ info] [storage] version=1.3.0, type=memory-only, sync=normal, checksum=disabled, max_chunks_up=128
[2022/10/12 16:58:28] [ info] [cmetrics] version=0.3.7
[2022/10/12 16:58:28] [debug] [winevtlog:winevtlog.0] created event channels: read=968 write=972
[2022/10/12 16:58:28] [error] [in_winevtlog] cannot subscribe 'BogusChannel' (0)
[2022/10/12 16:58:28] [debug] [in_winevtlog] channel 'BogusChannel' does not exist
[2022/10/12 16:58:28] [debug] [stdout:stdout.0] created event channels: read=1208 write=1212
[2022/10/12 16:58:28] [debug] [router] match rule winevtlog.0:stdout.0
[2022/10/12 16:58:28] [ info] [sp] stream processor started
[2022/10/12 16:58:28] [ info] [output:stdout:stdout.0] worker #0 started
[2022/10/12 16:58:29] [debug] [input:winevtlog:winevtlog.0] read 81334 bytes from 'Security'
[2022/10/12 16:58:29] [debug] [input chunk] update output instances with new chunk size diff=60481
[2022/10/12 16:58:29] [debug] [input:winevtlog:winevtlog.0] read 4112 bytes from 'Microsoft-Windows-AppLocker/EXE and DLL'
[2022/10/12 16:58:29] [debug] [input chunk] update output instances with new chunk size diff=2162
[2022/10/12 16:58:31] [debug] [input:winevtlog:winevtlog.0] read 2430 bytes from 'Security'
[2022/10/12 16:58:31] [debug] [input chunk] update output instances with new chunk size diff=1634
[2022/10/12 16:58:38] [debug] [task] created task=00EA8DF0 id=0 OK
[2022/10/12 16:58:38] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2022/10/12 16:58:38] [ warn] [engine] service will shutdown in max 5 seconds
[2022/10/12 16:58:38] [debug] [out flush] cb_destroy coro_id=0
[2022/10/12 16:58:38] [debug] [task] destroy task=00EA8DF0 (task_id=0)
[2022/10/12 16:58:39] [ info] [engine] service has stopped (0 pending tasks)
[2022/10/12 16:58:39] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2022/10/12 16:58:39] [ info] [output:stdout:stdout.0] thread worker #0 stopped

[cosmo0920]

Looks good. I'd confirmed that subscribing on non-existent channels does not cause fluent-bit termination. 👍

[edsiper]

thanks

[c0d3fau1t]

Thanks!