Skip to content

Commit 7bbf194

Browse files
authored
formatter_out_file: reduce memory usage (#4884)
**Which issue(s) this PR fixes**: Fixes # **What this PR does / why we need it**: When dealing with long logs at fast, it might improve memory efficient to clear the original strings used in string interpolation. ![chart](https://github.com/user-attachments/assets/42822460-b381-4ab0-8dc3-18e5e08e9b78) * config ``` <source> @type tail path "#{File.expand_path '~/tmp/access*.log'}" pos_file "#{File.expand_path '~/tmp/fluentd/access.log.pos'}" tag log refresh_interval 5s <parse> @type none </parse> </source> <match **> @type file path "#{File.expand_path '~/tmp/log'}" </match> ``` * script to generate log data ```ruby require "json" path = File.expand_path("~/tmp/access.log") File.open(path, "w") do |f| loop do log = { time: Time.now, message: "a" * 50 * 1024 }.to_json f.puts log sleep 0.0001 end end ``` **Docs Changes**: **Release Note**: Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
1 parent bcaaceb commit 7bbf194

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

lib/fluent/plugin/formatter_out_file.rb

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,10 +43,13 @@ def configure(conf)
4343
end
4444

4545
def format(tag, time, record)
46+
json_str = JSON.generate(record)
4647
header = ''
4748
header << "#{@timef.format(time)}#{@delimiter}" if @output_time
4849
header << "#{tag}#{@delimiter}" if @output_tag
49-
"#{header}#{JSON.generate(record)}#{@newline}"
50+
"#{header}#{json_str}#{@newline}"
51+
ensure
52+
json_str&.clear
5053
end
5154
end
5255
end

0 commit comments

Comments
 (0)