tests: allow to test the number of events emitted by a transform

[hhromic]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Vector transforms have the ability to emit multiple events per input event if an array is assigned to the root . event:

sources:
  logs:
    type: demo_logs
    format: syslog

transforms:
  remap:
    type: remap
    inputs: [logs]
    drop_on_error: true
    source: |
      . = split(string!(.message), pattern: s',')

However, there seems to be no way to test the expected numbers of events emitted:

tests:
  - name: remap-test
    inputs:
      - insert_at: remap
        type: log
        log_fields:
          message: a,b,c
    outputs:
      - extract_from: remap
        conditions:
          - type: vrl
            source: |
              # there is no way here to test how many events were emitted by the transform

In our opinion, testing the number of events emitted by a transform is an important testing condition as well.

Attempted Solutions

There does not seem to be a way to test the number of events emitted by a transform.

Proposal

We propose to add an extra keyword to the outputs elements. For example:

tests:
  - name: remap-test
    inputs:
      - insert_at: remap
        type: log
        log_fields:
          message: a,b,c
    outputs:
      - extract_from: remap
        number_of_events: 3
        conditions:
          - type: vrl
            source: |
              assert!(.message != "d")

For most use cases, number_of_events will typically be 1. However, to fully preserve backwards compatibility, there should be no default and this check enforced only if number_of_events is present or non null.

References

No response

Version

vector 0.36.1 (x86_64-unknown-linux-gnu 2857180 2024-03-11 14:32:52.417737479)

[jszwedko]

Agree! This would be nice enhancement.

[hhromic]

After looking at the source code of the unit tests framework, maybe is not that hard for me to add this.
Let me know if anybody in Team Vector is already planning on addressing this, or if you would like me to take a stab at this.
If the latter, let me know what option name you would prefer for this.

[jszwedko]

After looking at the source code of the unit tests framework, maybe is not that hard for me to add this. Let me know if anybody in Team Vector is already planning on addressing this, or if you would like me to take a stab at this. If the latter, let me know what option name you would prefer for this.

It's in the backlog, but I'm not sure when we'll get to it so if it'd scratch an itch for you, have at it :) I would call it expected_event_count.