logfmt: don't omit last item if it does not have a value

[seveas]

A line like foo=bar baz should be parsed to {"foo"=>"bar", "baz"=>nil}, but baz was silently omitted by returning early from the parser.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• [N/A] Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

Documentation

• [N/A] Documentation required for this feature

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[seveas]

With this line in foo.log

foo=bar baz

The unpatched version outputs:

$ ./build/bin/fluent-bit -R parsers.conf -f 1 -i tail -p path=foo.log -p read_from_head=true -p parser=logfmt  -o stdout -m '*'
Fluent Bit v1.9.0
* Copyright (C) 2019-2021 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2021/11/22 20:37:30] [ info] [engine] started (pid=74288)
[2021/11/22 20:37:30] [ info] [storage] version=1.1.5, initializing...
[2021/11/22 20:37:30] [ info] [storage] in-memory
[2021/11/22 20:37:30] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2021/11/22 20:37:30] [ info] [cmetrics] version=0.2.2
[2021/11/22 20:37:30] [ info] [sp] stream processor started
[0] tail.0: [1637609850.350374000, {"foo"=>"bar"}]
^C[2021/11/22 20:37:34] [engine] caught signal (SIGINT)
[2021/11/22 20:37:34] [ info] [input] pausing tail.0
[2021/11/22 20:37:34] [ warn] [engine] service will shutdown in max 5 seconds
[2021/11/22 20:37:35] [ info] [engine] service has stopped (0 pending tasks)

And the fixed version outputs:

$ ./build/bin/fluent-bit -R parsers.conf -f 1 -i tail -p path=foo.log -p read_from_head=true -p parser=logfmt  -o stdout -m '*'
Fluent Bit v1.9.0
* Copyright (C) 2019-2021 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2021/11/22 20:38:10] [ info] [engine] started (pid=74953)
[2021/11/22 20:38:10] [ info] [storage] version=1.1.5, initializing...
[2021/11/22 20:38:10] [ info] [storage] in-memory
[2021/11/22 20:38:10] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2021/11/22 20:38:10] [ info] [cmetrics] version=0.2.2
[2021/11/22 20:38:10] [ info] [sp] stream processor started
[0] tail.0: [1637609890.374907000, {"foo"=>"bar", "baz"=>nil}]
^C[2021/11/22 20:38:13] [engine] caught signal (SIGINT)
[2021/11/22 20:38:13] [ info] [input] pausing tail.0
[2021/11/22 20:38:13] [ warn] [engine] service will shutdown in max 5 seconds
[2021/11/22 20:38:14] [ info] [engine] service has stopped (0 pending tasks)

[seveas]

# valgrind bin/fluent-bit -R parsers.conf -f 1 -i tail -p path=foo.log -p read_from_head=true -p parser=logfmt  -o stdout -m '*'
==20522== Memcheck, a memory error detector
==20522== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==20522== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==20522== Command: bin/fluent-bit -R parsers.conf -f 1 -i tail -p path=foo.log -p read_from_head=true -p parser=logfmt -o stdout -m *
==20522==
Fluent Bit v1.9.0
* Copyright (C) 2019-2021 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[0] tail.0: [1637611233.542604800, {"foo"=>"bar", "baz"=>nil}]
[2021/11/22 20:00:33] [ info] [engine] started (pid=20522)
[2021/11/22 20:00:33] [ info] [storage] version=1.1.5, initializing...
[2021/11/22 20:00:33] [ info] [storage] in-memory
[2021/11/22 20:00:33] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2021/11/22 20:00:33] [ info] [cmetrics] version=0.2.2
[2021/11/22 20:00:33] [ info] [sp] stream processor started
[2021/11/22 20:00:33] [ info] [input:tail:tail.0] inotify_fs_add(): inode=640070 watch_fd=1 name=foo.log
^C[2021/11/22 20:00:36] [engine] caught signal (SIGINT)
[2021/11/22 20:00:36] [ info] [input] pausing tail.0
[2021/11/22 20:00:36] [ warn] [engine] service will shutdown in max 5 seconds
[2021/11/22 20:00:37] [ info] [engine] service has stopped (0 pending tasks)
[2021/11/22 20:00:37] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=640070 watch_fd=1
==20522==
==20522== HEAP SUMMARY:
==20522==     in use at exit: 0 bytes in 0 blocks
==20522==   total heap usage: 1,174 allocs, 1,174 frees, 644,671 bytes allocated
==20522==
==20522== All heap blocks were freed -- no leaks are possible
==20522==
==20522== For lists of detected and suppressed errors, rerun with: -s
==20522== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

[nokute78]

Thank you for contribution.

A line like foo=bar baz should be parsed to {"foo"=>"bar", "baz"=>nil}

I think it should be {"foo"=>"bar", "baz"=>true}. I refered to golang implementation.
https://pkg.go.dev/github.com/kr/logfmt#pkg-overview

Note: Original PR to support logfmt is #871.
It treated true.
key%^as$=val1 %asd.5$^ a b c d e -> {"key%^as$"=>"val1", "%asd.5$^"=>true, "a"=>true, "b"=>true, "c"=>true, "d"=>true, "e"=>true}

#1737 might cause regression ?

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[seveas]

I had another look at this, and current logfmt gives all bare keys a nil value, it just misses the last one. So those are two independent bugs, and we should be able to merge this pr as-is. I'll file another pr for the nil vs true issue.