fix(file source) Fix a data corruption bug with multi-char delimiters

[lfrancke]

Summary

Fix problem where multi-chars delimiters fail to be parsed if they happen right at a buffer boundary.

Vector configuration

See https://github.com/lfrancke/vector-repro-24027 for a reproduction repository

How did you test this PR?

The repro repo contains a test case which I used.
In addition I added unit tests for 1-5 char delimiters.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• Closes: Multi-byte line delimiters split across buffer boundaries cause log event merging #24027

Notes

• Please read our Vector contributor resources.
• Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
• Some CI checks run only after we manually approve them.
  • We recommend adding a pre-push hook, please see this template.
  • Alternatively, we recommend running the following locally before pushing to the remote branch:
    • make fmt
    • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
    • make test
• After a review is requested, please avoid force pushes to help us review incrementally.
  • Feel free to push as many commits as you want. They will be squashed into one before merging.
  • For example, you can run git merge origin master and git push.
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
  run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

[thomasqueirozb]

Hi @lfrancke thanks for your contribution! Since this is something that alters Vector behavior it is considered a user facing change (I edited the PR description already). Could you please add a changelog? Thanks!

Also, your changes seem sound to me but I still need to review them more throughly. I will take a closer look soon

[lfrancke]

Will do! Thanks.
I see that I left two of my debugging statements in the test as well. I'll remove those too.

[lfrancke]

I pushed the changelog and removed the debug statements. It's ready for review I believe.