Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fluentforwardreceiver : logs cannot be received due to wrong timestamp type #11435

Closed
Bhogayata-Keval opened this issue Jun 22, 2022 · 4 comments
Labels
bug Something isn't working data:logs Logs related issues good first issue Good for newcomers help wanted Extra attention is needed receiver/fluentforward Stale

Comments

@Bhogayata-Keval
Copy link

Bhogayata-Keval commented Jun 22, 2022

Bug Description
Fluent Forward Receiver cannot receive logs in case the timestamp follows uint64

Steps to reproduce

Case 1 : If you use "fluent-logger" npm package and try to send logs directly to fluentreceiver endpoint (Ex. localhost:8006)

receivers:
  fluentforward:
    endpoint: localhost:8006
const FluentClient = require("@fluent-org/logger").FluentClient;
const logger = new FluentClient("tag_prefix", {
  socket: {
    host: "localhost",
    port: 8006,
    timeout: 3000, // 3 seconds
  }
});

// send an event record with 'tag.label'
logger.emit('label', {record: 'this is a log'});

This setup does not work unless I add a uint64 case in conversion.go > timeFromTimestamp

func timeFromTimestamp(ts interface{}) (time.Time, error) {
	switch v := ts.(type) {
	case uint64:
		fmt.Println("---- this case needs to be added !")
		return time.Unix(int64(v), 0), nil
	case int64:
		return time.Unix(v, 0), nil
	case *eventTimeExt:
		return time.Time(*v), nil
	default:
		return time.Time{}, fmt.Errorf("unknown type of value: %v", ts)
	}
}

Case 2 : If you use "fluent" protocol for both input & output configuration

[SERVICE]
    Flush         5
    Daemon        off
    
[INPUT]
    Name    forward
    Host    localhost
    Port    24224
    
[OUTPUT]
    Name   stdout
    Match  *

[OUTPUT]
    Name    forward
    Host    localhost
    Port    8006

Same thing happens as case 1 : logs does not pass unless you add a case to handle uint64 timestamp !!

@Bhogayata-Keval Bhogayata-Keval added the bug Something isn't working label Jun 22, 2022
@mx-psi mx-psi added the data:logs Logs related issues label Jun 22, 2022
@github-actions
Copy link
Contributor

Pinging code owners: @dmitryax

@benewen-brav
Copy link

I'm also experiencing this issue on otel/opentelemetry-collector-contrib:0.58.0

Can also confirm that the fix above works for me.

@Bhogayata-Keval I noticed your SO post describing the issue also: https://stackoverflow.com/questions/72709477/can-i-use-fluent-logger-with-otel-fluentforwardreceiver

Glad to see you found it possible to directly use the fluentforward receiver as the log driver for your container. However, I couldn't get this to work with older versions of docker, <=18 seem unsupported (a constraint of one of our current environments, rancher 1.6).

Therefore I am keen to see a fix to use fluentbit as an intermediary. I might read up the contribution steps for this repo when I have the time.

@dmitryax dmitryax added help wanted Extra attention is needed good first issue Good for newcomers labels Aug 17, 2022
@dmitryax
Copy link
Member

@Bhogayata-Keval @benewen-brav since we have the solution suggested already, does any of you want to submit a PR?

@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working data:logs Logs related issues good first issue Good for newcomers help wanted Extra attention is needed receiver/fluentforward Stale
Projects
None yet
Development

No branches or pull requests

5 participants