Skip to content

Cannot use message_in_json_format=false #46

@elemask

Description

@elemask

Hi,

Thank you for your implementation of a Loki client that is up to date and supports structured Loki metadata.

I would like to log string messages to Loki, not JSON structures but I cannot get the message_in_json_format=false option to work.

I believe there is broken logic in stream.py:

    def append_value(self, value, metadata=None):
        """
        Append a value to the stream with a timestamp.
        
        Args:
            value (dict): A dictionary representing the value to be appended. 
                          It should contain a 'timestamp' key.
        """
        try:
            # Convert the timestamp to nanoseconds and ensure it's a string
            timestamp = str(int(value.get("timestamp") * 1e9))
        except (TypeError, ValueError):
            # Fallback to the current time in nanoseconds if the timestamp is missing or invalid
            timestamp = str(time_ns())
        
        formatted_value = json.dumps(value, ensure_ascii=False) if self.message_in_json_format else value

It seems that value is expected to be a python dictionary but if message_in_json_format is false then formatted_value will also become this (unchanged) dictionary. This dictionary will be then be sent (un-encoded) as the log message which fails with a parsing error when received by Loki.

If value is a string, then the code above fails (the value.get("timestamp") will raise AttributeError which is not caught (for good reason) and the message will not get sent to Loki.

Unless I am missing something, I cannot get the message_in_json_format=false option to work as intended.

I have fixed this locally by deriving two custom classes, one from LokiLoggerHandler and one from Stream. I override the LokiLoggerHandler._send() method to instantiate a custom Stream with a custom append_value method. This is an ugly hack and would rather this be fixed upstream :)

Thanks again for your work!

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions