fix(unit tests): prevent missing components errors for memory tables in tests

[esensar]

Summary

This ensures that memory enrichment tables are always considered in tests, because we can't be sure if they are used.

Vector configuration

enrichment_tables:
  memory_table:
    type: memory
    ttl: 60
    flush_interval: 5
    inputs: ["cache_generator"]

sources:
  demo_logs_test:
    type: "demo_logs"
    format: "json"

transforms:
  demo_logs_processor:
    type: "remap"
    inputs: ["demo_logs_test"]
    source: |
      . = parse_json!(.message)
      userid = get!(., path: ["user-identifier"])

      existing, err = get_enrichment_table_record("memory_table", { "key": userid })

      if err == null {
        . = existing.value
        .source = "cache"
      } else {
        .referer = parse_url!(.referer)
        .referer.host = encode_punycode!(.referer.host)
        .source = "transform"
      }

  cache_generator:
    type: "remap"
    inputs: ["demo_logs_processor"]
    source: |
      existing, err = get_enrichment_table_record("memory_table", { "key": get!(., path: ["user-identifier"]) })
      if err != null {
        # We don't have this key cached, so we need to prepare it for the table
        data = .
        # Since the memory enrichment table takes in all key value pairs it receives and stores them
        # We want to produce an object that has the value of "user-identifier" as its key and
        # rest of the object as its value
        . = set!(value: {}, path: [get!(data, path: ["user-identifier"])], data: data)
      } else {
        . = {}
      }

sinks:
  console:
    inputs: ["demo_logs_processor"]
    target: "stdout"
    type: "console"
    encoding:
      codec: "json"


tests:
  - name: "Syslog merge out"
    inputs:
      - insert_at: demo_logs_processor
        type: log
        log_fields:
          "@timestamp": "2024-08-20T09:37:22.245597628Z"
          ."host": "localhost"
          ."service": "vector"
          ."message": "{\"host\":\"13.112.71.186\",\"user-identifier\":\"devankoshal\",\"datetime\":\"24/Oct/2025:06:55:38\",\"method\":\"OPTION\",\"request\":\"/apps/deploy\",\"protocol\":\"HTTP/2.0\",\"status\":\"550\",\"bytes\":44860,\"referer\":\"https://for.xn--rvc1e0am3e/secret-info/open-sesame\"}"
    outputs:
      - extract_from: demo_logs_processor
        conditions:
          - type: vrl
            source: |-
              assert!(.mac_address == "37:AC:16:7E:AF:48")
              assert!(.entity == "HOMELAB")
              assert!(.action == "accept")

How did you test this PR?

Ran vector test for the above configuration before this patch and received:

Failed to build test 'Syslog merge out':
  Input "cache_generator" for sink "memory_table" doesn't match any components..

After this patch vector test runs the tests and correctly fails, due to a failed assertion.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• Related: Q&A How to use memory table insides tests #23972

Notes

• Please read our Vector contributor resources.
• Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
• Some CI checks run only after we manually approve them.
  • We recommend adding a pre-push hook, please see this template.
  • Alternatively, we recommend running the following locally before pushing to the remote branch:
    • make fmt
    • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
    • make test
• After a review is requested, please avoid force pushes to help us review incrementally.
  • Feel free to push as many commits as you want. They will be squashed into one before merging.
  • For example, you can run git merge origin master and git push.
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
  run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

---

Sponsored by Quad9