[R&D] Logs embeddables: what are your use cases?

[jasonrhodes]

I've heard rumblings of people wanting various logs-related embeddable UI components to drop into other parts of Kibana. What are the use cases for this kind of embeddable? If you have one, please describe it in a comment on this issue so we can collect the various use case requirements and draft a plan for creating the ones that have the best simple to useful ratios.

Please use the following format:

Component Name
Filtered Logs List

Inputs
Date range, field-based filters, results per page (optional)

Output description
List of logs matching the description. Should be paginated if results exceed the optional "results per page" value. Each result should have its own action menu with "view in context" option, and should link back to the logs stream with these filters applied.

Proposed placement?
Metrics UI, on a host view, filtered by host, etc, etc

Thanks!

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[jasonrhodes]

Here's one case: #67611

[cyrille-leclerc]

Thanks, I'll work with @alex-fedotyev to capture the integrations between APM and logs Alex is looking at for the "APM Service Landing Page".
For the integration between APM Distributed Traces and Logs, I'll iterate with @formgeist .

[sorenlouv]

APM would like to display related log lines for a particular trace (see details here). Log lines will be read from the index already specified in the Log ui settings.

This is our suggested interface for a react component:

<LogStream  
 timestamp="1590690626648" 
 filter={'trace.id:"0570667f4e27e2cac0d6c5b311c65918"'} 
/>

It could also take startTime and endTime instead:

<LogStream  
 startTime="1590690626648" 
 endTime="1590690636648" 
 filter={'trace.id:"0570667f4e27e2cac0d6c5b311c65918"'} 
/>

[drewpost]

Synthetics would like to have

• Logs that contain a specific unique test-run identifier that's passed through for each synthetic run so we can then display them in the "test results" details page in Client Side monitoring and also in the errors details page
• Time filtered view of all logs +/- a configurable window of time
• Log anomalies detected by ML filtered by time and matched in the Client Side monitoring UI to relevant error details pages and test results

Considerations:
Ideally these are stateful - I'm not sure we want them changing out underneath a user.
We also need to figure out how to preserve relevant anomalies and logs to match a "full fidelity" window of data retention in client side monitoring

[weltenwort]

Ideally these are stateful - I'm not sure we want them changing out underneath a user.

@drewpost could you elaborate on that? Which state are you referring to?

[drewpost]

@weltenwort - sorry for the delay, tweaking my mail filters.

What I mean by that is that the logs shown in the error details page in synthetics don't change out underneath the user. Perhaps we take a copy of the relevant log entries? I don't want to get into implementation but from a requirement perspective, I'd like to avoid logs disappearing from the details page if they've been shown there.

I'm happy to tie this to a time window though and we can handle errors outside of this period of retention in the UI.

[weltenwort]

That makes sense, thanks for explaining. I see three options right now to achieve that, most of which you already alluded to:

• Rely on the time filter and ensure the retention of the original log entries is sufficient.
• Copy the relevant log entries to a different index with longer retention and get the logs view to read the entries from there.
• Store the log entries somewhere else and directly pass them to the embeddable as data.

They all have various tradeoffs which we'd have to discuss in detail.

[jasonrhodes]

@drewpost it sounds like this would be good enough for your use case?

Rely on the time filter and ensure the retention of the original log entries is sufficient.