Skip to content

Conversation

@eschabell
Copy link
Contributor

@eschabell eschabell commented Dec 2, 2025

Add config_map entries for struct fields that were previously defined but not exposed as configuration options:

  • db.locking: set exclusive locking mode for the database
  • db.journal_mode: set the journal mode (DELETE, TRUNCATE,PERSIST, MEMORY, WAL, OFF)
  • dns_retries: dns lookup retries until network starts working
  • dns_wait_time: interval between network status checks
  • Also add the missing db.sync property reading code and journal_mode validation in kubernetes_events_conf.c to properly handle these database configuration options.

Fixes issue #11243


Testing
Before we can approve your change; please submit the following in a comment:

  • [N/A ] Example configuration file for the change
  • [X ] Debug log output from testing the change

Added tests in_kubernetes_events: Add test coverage for the new configuration parameters:

  • db.sync: test all valid values (extra, full, normal, off)
  • db.journal_mode: test all valid values (DELETE, TRUNCATE, PERSIST,
    MEMORY, WAL, OFF)
  • db.locking: test boolean values (true, false)
  • dns_retries and dns_wait_time: test custom values

Add test_ctx_create_with_config() helper function to support testing
optional configuration parameters without modifying the existing
test context creator.

Did full Fluent Bit build with tests and ran:

$ ./bin/flb-rt-in_kubernetes_events 

...
[ OK ]
SUCCESS: All unit tests have passed.
  • [N/A ] Attached Valgrind output that shows no leaks or memory corruption was found
  • [N/A ] Run local packaging test showing all targets (including any new ones) build.
  • [N/A ] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • [X ] Documentation required for this feature

Docs PR: fluent/fluent-bit-docs#2273

Backporting

  • [N/A ] Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features
    • Added runtime-configurable DB options for the Kubernetes events plugin: db.locking and db.journal_mode (with validation and defaults); db.sync is validated/normalized before DB open.
  • Changes
    • Removed DNS retry-related configuration (dns_retries and dns_wait_time) from the Kubernetes events plugin.
  • Tests
    • Added tests covering DB config permutations and validation.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link

coderabbitai bot commented Dec 2, 2025

Warning

Rate limit exceeded

@eschabell has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 15 minutes and 35 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 5f88c4c and 64b9ca3.

📒 Files selected for processing (4)
  • plugins/in_kubernetes_events/kubernetes_events.c (1 hunks)
  • plugins/in_kubernetes_events/kubernetes_events.h (0 hunks)
  • plugins/in_kubernetes_events/kubernetes_events_conf.c (1 hunks)
  • tests/runtime/in_kubernetes_events.c (2 hunks)

Walkthrough

Added SQLDB configuration options and validation for the Kubernetes events input, extended runtime tests for DB config permutations (including a duplicated helper), and removed two DNS-related fields from the plugin's public struct.

Changes

Cohort / File(s) Summary
Plugin config additions
plugins/in_kubernetes_events/kubernetes_events.c
Added SQLDB config entries db.locking (BOOL) and db.journal_mode (STR) under FLB_HAVE_SQLDB, and mapped them to new public struct fields.
Configuration validation
plugins/in_kubernetes_events/kubernetes_events_conf.c
Added defaulting/validation for db.sync (maps extra→3, full→2, normal→1, off→0; invalid values log error and abort init) and validation for db.journal_mode (accepts DELETE, TRUNCATE, PERSIST, MEMORY, WAL, OFF; invalid values log error and abort init). Performed before DB open.
Public struct change
plugins/in_kubernetes_events/kubernetes_events.h
Removed two fields from struct k8s_events: dns_retries and dns_wait_time.
Tests
tests/runtime/in_kubernetes_events.c
Added test_ctx_create_with_config helper (duplicated in file) and tests: flb_test_config_db_sync_values, flb_test_config_db_journal_mode_values, flb_test_config_db_locking_values. Tests create contexts with config overrides, start the pipeline, and clean up.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20–30 minutes

  • Inspect and consolidate the duplicated test_ctx_create_with_config helper in tests/runtime/in_kubernetes_events.c.
  • Verify that invalid db.sync and db.journal_mode values intentionally abort initialization and that error messages are clear.
  • Confirm new struct fields (db_locking, db_journal_mode) are initialized/defaulted and mapped correctly in config.
  • Search for any remaining references to the removed dns_retries and dns_wait_time fields to avoid build/runtime issues.

Suggested reviewers

  • edsiper
  • cosmo0920
  • leonardo-albertovich

Poem

🐰 I hopped through configs, found a shiny key,
DB locks and journals now dancing with me,
Tests sprout like carrots in tidy rows,
DNS fields hopped off where the cold wind blows,
Sniff, nibble, celebrate — into the code I go! 🥕

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title refers to adding missing config parameter descriptions, which aligns with the core changes (adding db.locking and db.journal_mode config entries with validation), though it understates the scope by not mentioning validation and test additions.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/runtime/in_kubernetes_events.c (1)

526-560: Test verifies configuration acceptance but not actual behavior.

The test correctly verifies that all valid db.sync values are accepted during initialization. However, it doesn't verify that the database is actually configured with the specified sync mode. Consider adding verification that reads back the SQLite PRAGMA to confirm the setting was applied.

This is acceptable for a configuration validation test, but for more thorough coverage, you could query the database after initialization:

/* After flb_start, could verify with SQL query: */
/* SELECT * FROM pragma_synchronous; */
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 10ebd3a and b4f88ba.

📒 Files selected for processing (3)
  • plugins/in_kubernetes_events/kubernetes_events.c (1 hunks)
  • plugins/in_kubernetes_events/kubernetes_events_conf.c (1 hunks)
  • tests/runtime/in_kubernetes_events.c (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
plugins/in_kubernetes_events/kubernetes_events_conf.c (1)
src/flb_input.c (1)
  • flb_input_get_property (776-780)
tests/runtime/in_kubernetes_events.c (2)
include/fluent-bit/flb_mem.h (2)
  • flb_calloc (84-96)
  • flb_free (126-128)
src/flb_lib.c (8)
  • flb_create (143-225)
  • flb_input (266-276)
  • flb_input_set (305-335)
  • flb_output (279-289)
  • flb_output_set (520-551)
  • flb_start (983-994)
  • flb_stop (1011-1055)
  • flb_destroy (228-263)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (31)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COMPILER_STRICT_POINTER_TYPES=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
  • GitHub Check: PR - fuzzing test
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-without-cxx (3.31.6)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-centos-7
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
🔇 Additional comments (9)
plugins/in_kubernetes_events/kubernetes_events_conf.c (1)

256-270: LGTM: db.journal_mode validation correctly aborts on invalid input.

The validation properly checks all allowed values and aborts initialization if an invalid value is provided, ensuring the database is never opened with an unsupported journal mode.

plugins/in_kubernetes_events/kubernetes_events.c (2)

1088-1099: LGTM: Database configuration entries are properly defined.

The db.locking and db.journal_mode configuration entries are correctly wired to their respective struct fields with appropriate defaults. These align with the validation logic added in kubernetes_events_conf.c.


1102-1111: Verify that dns_retries and dns_wait_time are actually used in the implementation.

The configuration entries are properly defined and mapped to struct fields, but ensure that the actual DNS retry logic references these values. These fields should be used in the DNS resolution logic; if they are not referenced anywhere in the implementation, the config map definitions should be removed or the implementation should be updated to use them.

tests/runtime/in_kubernetes_events.c (6)

306-383: LGTM: Test helper properly configures optional parameters.

The test_ctx_create_with_config function correctly creates a test context with optional database and DNS configuration parameters, using conditional logic to only set provided values. This is a clean approach for testing different configuration combinations.


562-596: Test coverage for db.journal_mode is appropriate.

The test validates all documented journal mode values (DELETE, TRUNCATE, PERSIST, MEMORY, WAL, OFF) and confirms they're accepted during initialization.


598-632: Test coverage for db.locking is complete.

The test validates both boolean values for the db.locking parameter.


634-664: DNS options test provides basic configuration validation.

The test confirms that dns_retries and dns_wait_time can be configured, though it doesn't verify their actual usage during DNS operations. This is sufficient for a configuration acceptance test.


670-673: Test suite properly registers all new configuration tests.

All four new test cases are correctly added to the TEST_LIST for execution.


306-383: The AI summary's claim of duplication appears unsupported by the provided code.

The function test_ctx_create_with_config shown in lines 306-383 appears only once in this snippet. Without access to verify the entire file, the claim of duplication cannot be confirmed from the evidence presented. If duplication exists elsewhere in the file, the line numbers should be explicitly cited.

@edsiper
Copy link
Member

edsiper commented Dec 3, 2025

thanks for this PR.

I noticed that while Database options have a proper code, there is no code handling for dns_wait_time and dns_retries, so the options are not being used, so they are a placeholder and should not be exposed to the user in the configuration.

There might be cases where in addition to the config map registration, we need to check if that option/flag is actually used.

@eschabell
Copy link
Contributor Author

@edsiper pushed changes for requested removal of dns_wait_time and dns_retries, including cleaning up of their testing code.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/runtime/in_kubernetes_events.c (1)

306-373: Reduce duplication by reusing existing test_ctx_create / test_ctx_destroy

test_ctx_create_with_config() largely duplicates test_ctx_create() plus a few extra flb_input_set calls for db.* options. To keep tests simpler to maintain, consider factoring this as:

  • Call test_ctx_create() to build the base context.
  • Then apply the optional db.sync, db.locking, and db.journal_mode overrides.
  • Reuse test_ctx_destroy() for cleanup in callers instead of open‑coding flb_stop/flb_destroy/flb_free in each test.

This avoids drift if the common setup/teardown ever changes.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b4f88ba and 9ca91b9.

📒 Files selected for processing (4)
  • plugins/in_kubernetes_events/kubernetes_events.c (1 hunks)
  • plugins/in_kubernetes_events/kubernetes_events.h (0 hunks)
  • plugins/in_kubernetes_events/kubernetes_events_conf.c (1 hunks)
  • tests/runtime/in_kubernetes_events.c (2 hunks)
💤 Files with no reviewable changes (1)
  • plugins/in_kubernetes_events/kubernetes_events.h
🚧 Files skipped from review as they are similar to previous changes (1)
  • plugins/in_kubernetes_events/kubernetes_events_conf.c
🧰 Additional context used
🧬 Code graph analysis (1)
tests/runtime/in_kubernetes_events.c (2)
include/fluent-bit/flb_mem.h (2)
  • flb_calloc (84-96)
  • flb_free (126-128)
src/flb_lib.c (8)
  • flb_create (143-225)
  • flb_service_set (652-678)
  • flb_input (266-276)
  • flb_input_set (305-335)
  • flb_output (279-289)
  • flb_start (983-994)
  • flb_stop (1011-1055)
  • flb_destroy (228-263)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (31)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COMPILER_STRICT_POINTER_TYPES=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
  • GitHub Check: PR - fuzzing test
  • GitHub Check: pr-compile-centos-7
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-without-cxx (3.31.6)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
🔇 Additional comments (1)
plugins/in_kubernetes_events/kubernetes_events.c (1)

1088-1099: DB config options wired correctly into config_map

The db.locking and db.journal_mode entries look consistent with the existing db / db.sync options: correct map types, sensible defaults, and offsets into struct k8s_events guarded by FLB_HAVE_SQLDB. No issues from this file’s side.

@edsiper
Copy link
Member

edsiper commented Dec 5, 2025

@eschabell thanks for the updates, as a last steps we just need to clean the history of commits

image

@edsiper edsiper removed this from the Fluent Bit v5.0 milestone Dec 5, 2025
@eschabell eschabell force-pushed the erics_in_kubernetes_events_config_fix branch 2 times, most recently from bee771b to 90533fc Compare December 10, 2025 09:22
@eschabell eschabell force-pushed the erics_in_kubernetes_events_config_fix branch from 90533fc to c71689f Compare December 10, 2025 09:25
@eschabell eschabell force-pushed the erics_in_kubernetes_events_config_fix branch from c71689f to 1439fa8 Compare December 10, 2025 09:25
@eschabell eschabell force-pushed the erics_in_kubernetes_events_config_fix branch from 1439fa8 to 383ad31 Compare December 10, 2025 09:30
Add db.locking, db.journal_mode, and db.sync configuration parameters with
proper validation. Remove dns_retries and dns_wait_time options as they were
never implemented in the plugin.

- Add config_map entries for db.locking, db.journal_mode
- Add db.sync property reading code and journal_mode validation
- Remove dns_retries and dns_wait_time (unused, unlike filter_kubernetes)
- Add test coverage for db.sync, db.journal_mode, and db.locking options
  and guard SQLDB config tests with FLB_HAVE_SQLDB

Fixes fluent#11243.

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell eschabell force-pushed the erics_in_kubernetes_events_config_fix branch from 383ad31 to 64b9ca3 Compare December 10, 2025 09:32
@eschabell
Copy link
Contributor Author

Can't pass the commit linter as there are two areas of work, one in_kubernetes_events: plugin and the other in tests:, so if I commit with the following they all fail:

commit message with tests:

tests: missing db config options

Subject prefix 'tests:' does not match files changed.
Expected one of: in_kubernetes_events:, tests:

commit message with in_kubernetes_events:

in_kubernetes_events: missing db config options

Subject prefix 'in_kubernetes_events:' does not match files changed.
Expected one of: in_kubernetes_events:, tests:

commit message with in_kubernetes_events: tests:

in_kubernetes_events: tests: missing db config options

Subject prefix 'in_kubernetes_events:' does not match files changed.
Expected one of: in_kubernetes_events:, tests:

@edsiper @cosmo0920 @lecaros What is the way to get past this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants