Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OSS Gate Workshop: online: 2023-09-30: daipom: Fluentd: Work log #1664

Closed
daipom opened this issue Sep 30, 2023 · 15 comments
Closed

OSS Gate Workshop: online: 2023-09-30: daipom: Fluentd: Work log #1664

daipom opened this issue Sep 30, 2023 · 15 comments

Comments

@daipom
Copy link
Contributor

daipom commented Sep 30, 2023

This is a work log of a "OSS Gate workshop".
"OSS Gate workshop" is an activity to increase OSS developers.
Here's been discussed in Japanese. Thanks.

作業ログ作成時の説明

以下のテンプレートを埋めてタイトルに設定します。埋め方例はスクロールすると見えてきます。

OSS Gate Workshop: ${LOCATION}: ${YEAR}-${MONTH}-${DAY}: ${ACCOUNT_NAME}: ${OSS_NAME}: Work log

タイトル例↓：

OSS Gate Workshop: Tokyo: 2017-01-16: kou: Rabbit: Work log

OSS Gateワークショップ関連情報

  • スライド：ワークショップの進行に使っているスライドがあります。
  • チャット：OSS開発に関することならなんでも相談できます。ワークショップが終わった後もオンラインで相談しながら継続的にOSSの開発に参加しましょう！
  • シナリオ：ワークショップの目的・内容・進め方の詳細が書いています。
  • 過去のビギナーの作業ログ：他の人の作業ログから学べることがいろいろあるはずです。
@daipom daipom changed the title OSS Gate Workshop: online: 2023-09-${DAY}: ${ACCOUNT_NAME}: ${OSS_NAME}: Work log OSS Gate Workshop: online: 2023-09-30: daipom: ${OSS_NAME}: Work log Sep 30, 2023
@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

作業メモできた

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

最近Fluentdを触っているので、本日はこれを対象としたい

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

GitHubリポジトリー: https://github.com/fluent/fluentd
公式webサイト: https://www.fluentd.org/

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

GitHubリポジトリーをみると、ライセンスはApache License 2.0らしい

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

このライセンスは https://opensource.org/licenses にあるので、FluentdはOSSだと思います。
今日はこれを触ってみます

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

公式ドキュメント: https://docs.fluentd.org/

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023 •
edited
Loading

https://docs.fluentd.org/ を読みながらインストールしてみる

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

https://docs.fluentd.org/installation/install-from-source の方法でインストールを試してみる

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

次のコマンドが次のようにエラーになる。

$ fluentd -c ./fluent/fluent.conf -vv
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="./fluent/fluent.conf"
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: gem 'fluent-plugin-concat' version '2.5.0'
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: gem 'fluent-plugin-forest' version '0.3.3'
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: gem 'fluent-plugin-record-modifier' version '2.1.1'
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: gem 'fluent-plugin-sigdump' version '1.0.1'
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.1'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered output plugin 'stdout'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered buffer plugin 'memory'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered formatter plugin 'stdout'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered formatter plugin 'json'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered output plugin 'forward'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered sd plugin 'static'
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: adding forwarding server '192.168.0.12:24224' host="192.168.0.12" port=24224 weight=60 plugin_id="object:898"
2023-09-30 11:16:16 +0900 [debug]: fluent/log.rb:341:debug: rebuilding weight array lost_weight=0
2023-09-30 11:16:16 +0900 [info]: [forward_output] adding forwarding server '192.168.0.11:24224' host="192.168.0.11" port=24224 weight=60 plugin_id="forward_output"
2023-09-30 11:16:16 +0900 [debug]: [forward_output] rebuilding weight array lost_weight=0
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered parser plugin 'in_http'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered input plugin 'http'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered parser plugin 'msgpack'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered parser plugin 'json'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered input plugin 'monitor_agent'
2023-09-30 11:16:16 +0900 [trace]: fluent/log.rb:319:trace: registered input plugin 'debug_agent'
2023-09-30 11:16:16 +0900 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
  <source>
    @type forward
    @id forward_input
  </source>
  <source>
    @type http
    @id http_input
    port 8888
  </source>
  <source>
    @type monitor_agent
    @id monitor_agent_input
    port 24220
  </source>
  <source>
    @type debug_agent
    @id debug_agent_input
    bind "127.0.0.1"
    port 24230
  </source>
  <match debug.**>
    @type stdout
    @id stdout_output
  </match>
  <match system.**>
    @type forward
    @id forward_output
    <server>
      host "192.168.0.11"
    </server>
    <secondary>
      <server>
        host "192.168.0.12"
      </server>
    </secondary>
  </match>
</ROOT>
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=2420356 ruby="3.2.2"
2023-09-30 11:16:16 +0900 [info]: fluent/log.rb:362:info: spawn command to main:  cmdline=["/home/daipom/.rbenv/versions/3.2.2/bin/ruby", "-Eascii-8bit:ascii-8bit", "/home/daipom/.rbenv/versions/3.2.2/bin/fluentd", "-c", "./fluent/fluent.conf", "-vv", "--under-supervisor"]
2023-09-30 11:16:17 +0900 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2023-09-30 11:16:17 +0900 [info]: fluent/log.rb:362:info: adding match pattern="debug.**" type="stdout"
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered output plugin 'stdout'
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered buffer plugin 'memory'
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered formatter plugin 'stdout'
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered formatter plugin 'json'
2023-09-30 11:16:17 +0900 [info]: fluent/log.rb:362:info: adding match pattern="system.**" type="forward"
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered output plugin 'forward'
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered sd plugin 'static'
2023-09-30 11:16:17 +0900 [info]: #0 fluent/log.rb:362:info: adding forwarding server '192.168.0.12:24224' host="192.168.0.12" port=24224 weight=60 plugin_id="object:898"
2023-09-30 11:16:17 +0900 [debug]: #0 fluent/log.rb:341:debug: rebuilding weight array lost_weight=0
2023-09-30 11:16:17 +0900 [info]: #0 [forward_output] adding forwarding server '192.168.0.11:24224' host="192.168.0.11" port=24224 weight=60 plugin_id="forward_output"
2023-09-30 11:16:17 +0900 [debug]: #0 [forward_output] rebuilding weight array lost_weight=0
2023-09-30 11:16:17 +0900 [info]: fluent/log.rb:362:info: adding source type="forward"
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2023-09-30 11:16:17 +0900 [info]: fluent/log.rb:362:info: adding source type="http"
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered parser plugin 'in_http'
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'http'
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered parser plugin 'msgpack'
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered parser plugin 'json'
2023-09-30 11:16:17 +0900 [info]: fluent/log.rb:362:info: adding source type="monitor_agent"
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'monitor_agent'
2023-09-30 11:16:17 +0900 [info]: fluent/log.rb:362:info: adding source type="debug_agent"
2023-09-30 11:16:17 +0900 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'debug_agent'
2023-09-30 11:16:17 +0900 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2023-09-30 11:16:17 +0900 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=2420375 ppid=2420356 worker=0
2023-09-30 11:16:17 +0900 [debug]: #0 [forward_output] buffer started instance=2180 stage_size=0 queue_size=0
2023-09-30 11:16:17 +0900 [info]: #0 [debug_agent_input] listening dRuby uri="druby://127.0.0.1:24230" object="Fluent::Engine" worker=0
2023-09-30 11:16:17 +0900 [debug]: #0 [forward_output] flush_thread actually running
2023-09-30 11:16:17 +0900 [debug]: #0 [forward_output] enqueue_thread actually running
2023-09-30 11:16:17 +0900 [trace]: #0 [forward_output] enqueueing all chunks in buffer instance=2180
2023-09-30 11:16:17 +0900 [debug]: #0 [monitor_agent_input] listening monitoring http server on http://0.0.0.0:24220/api/plugins for worker0
2023-09-30 11:16:17 +0900 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=ArgumentError error="unknown keyword: :logger"
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/async-2.6.2/lib/async/scheduler.rb:32:in `initialize'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/async-2.6.2/lib/async/reactor.rb:19:in `initialize'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/plugin_helper/http_server/server.rb:42:in `new'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/plugin_helper/http_server/server.rb:42:in `initialize'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/plugin_helper/http_server.rb:69:in `new'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/plugin_helper/http_server.rb:69:in `http_server_create_http_server'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/plugin/in_monitor_agent.rb:213:in `start'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/root_agent.rb:203:in `block in start'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/root_agent.rb:191:in `each'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/root_agent.rb:191:in `block in lifecycle'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/root_agent.rb:178:in `each'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/root_agent.rb:178:in `lifecycle'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/root_agent.rb:202:in `start'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/engine.rb:248:in `start'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/engine.rb:147:in `run'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/supervisor.rb:617:in `block in run_worker'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/supervisor.rb:962:in `main_process'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/supervisor.rb:608:in `run_worker'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/work/ossgateworkshop/demo/fluentd/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/home/daipom/.rbenv/versions/3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/home/daipom/.rbenv/versions/3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/.rbenv/versions/3.2.2/bin/fluentd:25:in `load'
  2023-09-30 11:16:17 +0900 [error]: #0 fluent/supervisor.rb:962:main_process: /home/daipom/.rbenv/versions/3.2.2/bin/fluentd:25:in `<main>'

@daipom daipom changed the title OSS Gate Workshop: online: 2023-09-30: daipom: ${OSS_NAME}: Work log OSS Gate Workshop: online: 2023-09-30: daipom: Fluentd: Work log Sep 30, 2023
@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

ドキュメントのチュートリアルに方法通りにやってエラーがでるのは困るので、報告したい。

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

報告方法: https://github.com/fluent/fluentd/blob/master/CONTRIBUTING.md

これを読むとドキュメントに関する報告はこっち

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

Fluentd本体のバグかドキュメントがおかしいのか判断できない。
ので、まずはとりあえずドキュメントの方に報告をしてみる

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

テンプレートがある。
これにそった下書きを作る。

https://github.com/fluent/fluentd-docs-gitbook/issues/new?assignees=&labels=&projects=&template=bug_report.yaml

@daipom
Copy link
Contributor Author

daipom commented Sep 30, 2023

報告した

fluent/fluentd-docs-gitbook#475

@github-actions github-actions bot closed this as completed Oct 4, 2023
@github-actions
Copy link

github-actions bot commented Oct 4, 2023

おつかれさまでした！

ワークショップの終了にともないissueを閉じますが、このまま作業メモとして使っても構いません 👌

ワークショップの感想を集めています！

ブログなどに書かれた際は、このページへリンクの追加をお願いします 🙏

またの参加をお待ちしています！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@daipom