logcli: a way to color log lines based on labels or pattern matching

[Morriz]

Is your feature request related to a problem? Please describe.

We can't easily distinguish (in either streaming nor standard mode) what source the logs lines come from. We have to interpret the source column text now (my brain is too slow for that ;p) which gives headaches.

Describe the solution you'd like

Loki should color the log lines by giving query sources each a preconfigured distinguishable color and reflect those in the source and message part only (to still easily see the standard DEBUG|INFO|WARN|ERR log level colors). The query sources would then preferably have (pre-filled) color selectors to still be able to select another color (to create better readability for individuals with different color preferences).

And all filtering features that reflect visually should of course work also in streaming mode to get insights into the applications working together.

Describe alternatives you've considered

No alternatives exist for Loki as it does not allow for 3rd partly plugins I presume?

[slim-bean]

Hey @Morriz thanks for the issue!

I think this would need to be filed with Grafana, Loki only returns raw results via the API without any indications of color.

However before closing this and asking you to open it over there I have a question.

What are you referring to as source in this context? as in the different label values on the streams themselves? or are you creating multiple separate queries in explore?

[j-zimnowoda]

Hi @slim-bean,
With @Morriz we would love to be able to see logs from containers (source) that are grouped by colors.
A given color represents containers that belong to a specific workload. It helps to follow logs and make it easier to understand what is really happening with workloads.

A good example of it can be found in stern CLI tool.

[cyriltovena]

I used stern, it’s cool you’re right. But making the color default can be tricky as some terminal may not support it.

[Morriz]

Hey @Morriz thanks for the issue!

I think this would need to be filed with Grafana, Loki only returns raw results via the API without any indications of color.

However before closing this and asking you to open it over there I have a question.

What are you referring to as source in this context? as in the different label values on the streams themselves? or are you creating multiple separate queries in explore?

I would think the easiest road to this would be having separate query source colors (so yes, multiple query entry lines, each with a prefilled color select input).

Because there is no good way for Loki to know which labels comprise a "source app", right? That is why we can create custom queries that make up what we are interested in. Loki simply can't be guessing what the user is interested in. It's a bumpy road to preference hell.

Or you could just group by pod name if one entry is given. That would serve most use cases. And if multiple entries are given, you start giving those colors as well (colored dots in front of the lines?).

[slim-bean]

@Morriz can you open a feature request in the Grafana repo for this?

I believe you are asking to have a color picker for each line in the screenshot above ^ ?

This would be a feature request for Grafana (and I think a good one!)

From the CLI perspective (logcli) that would fall into this repo, however this would be no small task I'm afraid, getting colors correct on different terminals is really hard as @cyriltovena mentioned, additionally, defining the colors in an easy way is going to be a bit tricky too.

That being said, it would be awesome to have this so I'm gonna rename the issue and keep this open.

[Morriz]

Alright. I do not think why the word logcli is involved, and I have no clue if that refers to an actual thing. Could you elaborate?

[Morriz]

And isn’t it possible to map the lines coming back to the initially requested labels? That would be poor. Maybe promtail should return those for every line by default

[cyriltovena]

Yeah we could add a flag and have a color per stream, unique labels set. Would that work ?

[Morriz]

That would be awesome, but it would be of course easier if we would be able to distinguish sources even from one query