Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wildcard and quoting handling are not correct in Windows #1138

Open
tagomoris opened this issue Aug 3, 2016 · 1 comment
Open

Wildcard and quoting handling are not correct in Windows #1138

tagomoris opened this issue Aug 3, 2016 · 1 comment
Labels
bug Something isn't working v0.14 v1 windows

Comments

@tagomoris
Copy link
Member

Backslashes are handled as quoting character in path of in_tail.
But backslashes are also path separator in Windows environment, and it is not determined automatically whether a backslash is separator or quoting character.

We should have a way to handle it correctly in Windows environment.
Options:

  • add a configuration parameter not to extract globs and handle backslashes as path separators always
@daipom
Copy link
Contributor

daipom commented Jun 27, 2023

This problem also exists in v1.
We should need to be careful to use Slash / for the path separator for Windows.

I found the file buffer uses Dir.glob when resuming, so it can't find the existing chunk files if we use Backslash \ to the buffer path.

def resume
stage = {}
queue = []
exist_broken_file = false
patterns = [@path]
patterns.unshift @additional_resume_path if @additional_resume_path
Dir.glob(escaped_patterns(patterns)) do |path|

Dir.glob doesn't work with Backslash as the path separator for Windows, even if escaping it like \\.

Dir.glob("C:\\test\\fluentd\\buffer\\buffer.*.log")
=> []

Dir.glob("C:/test/fluentd/buffer/buffer.*.log")
=>  ["C:/test/fluentd/buffer/buffer.b5ff1d72f0a4b33c509f95a77bab5bf89.log", "C:/test/fluentd/buffer/buffer.b5ff1d74b9eb7a2e23696bf59405d3fce.log", "C:/test/fluentd/buffer/buffer.b5ff1d8af5a361ee8bc9a81ed39e877da.log"]

How to Reproduce

env

  • Windows 10 Home
  • Fluentd v1.16.1
  • ruby 3.2.0 (2022-12-25 revision a528908271) [x64-mingw-ucrt]

config

<source>
  @type sample
  tag test
</source>

<match test.**>
  @type stdout
  <buffer>
    @type file
    path C:\test\fluentd\buffer
    flush_at_shutdown false
    flush_interval 10
  </buffer>
</match>

steps

  • Run Fluentd: $ bundle exec fluentd -c /path/to/config
  • Stop Fluentd so that some chunk files remain.
  • Run Fluentd again.
  • We can find the existing chunk files not being loaded and remaining forever.

kenhys added a commit to kenhys/fluent-package-builder that referenced this issue Jun 29, 2023
kenhys added a commit to kenhys/fluent-package-builder that referenced this issue Jun 29, 2023
kenhys added a commit to kenhys/fluent-package-builder that referenced this issue Jun 29, 2023
kenhys added a commit to kenhys/fluent-package-builder that referenced this issue Jun 29, 2023
See fluent/fluentd#1138

Signed-off-by: Kentaro Hayashi <hayashi@clear-code.com>
kenhys added a commit to kenhys/fluent-package-builder that referenced this issue Jul 4, 2023
See fluent/fluentd#1138

Signed-off-by: Kentaro Hayashi <hayashi@clear-code.com>
kenhys added a commit to kenhys/fluent-package-builder that referenced this issue Jul 5, 2023
See fluent/fluentd#1138

Signed-off-by: Kentaro Hayashi <hayashi@clear-code.com>
daipom pushed a commit to kenhys/fluent-package-builder that referenced this issue Jul 10, 2023
See fluent/fluentd#1138

Signed-off-by: Kentaro Hayashi <hayashi@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working v0.14 v1 windows
Projects
None yet
Development

No branches or pull requests

2 participants