[Logs UI] View log message in context

[katrin-freihofner]

We defined a way for the Logs UI to allow users to quickly see a log message in context. For now, context is defined as log lines within the same file.

Find more background information and related discussions in the design issue
#40808

View in context

The context can be triggered from each of the log lines. On hover (and touch) of a specific log line, the round button appears. This button triggers a popover menu with the following actions:

• View in context
• View monitor status (link to Uptime)
• View details (which should trigger the flyout that is currently triggered by the expand icon)

Figma/designs link https://www.figma.com/file/PvXWESp9YhkJMFy4l3Judo/40808-Logs-view-in-context

Overlay

The context itself is shown in an overlay/modal. Besides the selected log line, it also shows logs before and after the selected one.
Additionally, I used a white backdrop with 90% opacity to emphasize the overlay.

Actions

• A show more button (EUI button empty) on the top and on the bottom of the overlay extend the number of shown log lines.
• A button (EUI button empty) on the top, next to the headline should set this context as a filter for the log stream.
• A button (EUI icon btn) in the top right corner to close the overlay. (Additionally, users should be able to close the overlay with the Esc key and clicking somewhere outside the overlay.

When closing the overlay, the log line should still be selected!

Open questions

• How many log lines should we show in the context overlay? -> let's start with 10
• By how many log lines should we extend each time the user clicks Show more-> let's start with 10
• What should we call the label that adds the context as a filter to the log stream? ('Filter for this context' in mockup)

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[weltenwort]

I like the look of this 👏 I'd like to ask some questions to explore the edge cases if I may:

• What does the overlay look like when the log line is at the very top or bottom?
• The loading of the might take a noticeable amount of time if the cluster is slow. How is the initial loading state of the overlay visualized?
• How can the overlay be closed? Is there a UI element for that? Will the Esc key close it?
• Which field is used to determine the file? Filebeat writes log.file.path, but it's not an ECS field.
• How are log entries handled that don't have the file name field?
• Should "Log lines that appeared in the same file" mention the filename? (e.g. "Log entries from ${filename}")

[katrin-freihofner]

@weltenwort very good questions:

• What does the overlay look like when the log line is at the very top or bottom?

Let me create an example of this.

• The loading of the might take a noticeable amount of time if the cluster is slow. How is the initial loading state of the overlay visualized?

Super cool would be a skeleton loader - we could reuse this also for the stream. I'm thinking of something like this

Do you think we could do something like this? Otherwise I would use our standard EUI loader (I can add an example for this too.)

• How can the overlay be closed? Is there a UI element for that? Will the Esc key close it?

1. Clicking somewhere in the background
2. Esc key is a great idea
3. I think we should additionally add a X-button in the upper right corner

• Which field is used to determine the file? Filebeat writes log.file.path, but it's not an ECS field.
• How are log entries handled that don't have the file name field?

I don't think that I can help here...is there another context we could use? Or maybe we just disable the functionality for these log messages for now?

• Should "Log lines that appeared in the same file" mention the filename? (e.g. "Log entries from ${filename}")

Yes, I would like that very much! 🤩

[katrin-freihofner]

As discussed via slack, the loading could look like this for a more fluent interaction. 🤩

This would also mean that we move the selected log message to the center of the screen (which means we have more space to expand the overlay.

Furthermore, we need to define a max. height of the overlay and scroll within as soon as the log lines do not fit anymore.

[katrin-freihofner]

Summary of the feedback I got in yesterdays design weekly:

• show more (~10) log lines when we initially display the overlay (@roncohen)
• highlight the selected log line when closing the overlay (updated the video in the comment above)
  I'm going to update the description accordingly.

[katrin-freihofner]

Casper just told me that we have something like a skeleton loader for text in EUI: https://elastic.github.io/eui/#/display/loading - this might be useful.

[mukeshelastic]

@katrin-freihofner and I discussed the current UX and typical user journey associated with viewing surrounding logs and few minor enhancements to the current UX. Below is the summary of our discussion:

Example use case for logs in troubleshooting workflow:

Background
Elastic stack is often used as a centralized logging system for ingesting logs from various components of IT infrastructure. For example: IT infrastructure that runs apps on K8s clusters ingests logs from master and worker nodes of all clusters, all the K8s processes running on every master node, pod and daemonset logs from all worker nodes and application logs from every container running on all clusters.

Problem description
In such case, when users are trying to troubleshoot a problem, it is often within the context of a specific component misbehaving. e.g scheduler metric for pending unscheduled pods exceeding a threshold.

User journey
In this case, users may come to logs either through other monitoring mechanisms alerts on scheduler metric linking them to logs or they come to logs directly because a developer complained about not being able to schedule pods.. Either ways, they are going to filter logs by the K8s cluster and master node associated with the misbehaving scheduler. Then they are going to look at logs for the time frame when the problem was reported, and search for text associated with pods not being scheduled. Once they see a log line that resembles the error, they'd like to see the logs surrounding the error text but retain the other filters like K8s cluster, master node etc. They either find what they are looking for in the surrounding logs or they don't. In latter case, they go back to the stream view and look at the another log line that resembles the text associated with the error and follow the same journey as before..

UX

1. When the user looking at logs stream, filtered by K8s cluster, master node and some free form text, spots an interesting log line, she wants to see logs surrounding the interesting log line but just for the scheduler from the specific K8s cluster and master node. So the surrounding logs UI should filter logs by the terms in stream filters section.
2. If user wants to view the surrounding logs for another interesting log line from stream view, then they can close the surrounding logs overlay which will take them to stream view and then follow step 1.
3. If they want to see surrounding logs for different master node within the same K8s cluster they can remove the master node filter and follow step 1.

[weltenwort]

Thanks for the use case description. I have a question for clarification, if I may:

So the surrounding logs UI should filter logs by the terms in stream filters section.

This implies that the Logs UI can somehow distinguish which specific part of the filter to ignore for the context. What would be the criteria so that the remaining filter is predictable for the user?

[mukeshelastic]

My assumption of a typical user journey for troubleshooting is: start with one or more key:value filter and text for search. When users wants to see surrounding logs, they want to remove the text search but retain what ever they have in key:value filters. So the criteria is to always remove the free text search..

[weltenwort]

Technically the free text search is just an implicit search against all fields, but I get what you mean. We should be able to distinguish these in the query AST somehow. I wonder, though, how we can communicate that to the user so they are not surprised and so they can learn to use key-value queries for the context view to be of value.

[phantasia15]

Hi, sorry to chime in but I think it will be pretty limited if the context only ignores the free text.
As an user my most common use case for context is:

• Search for the logs with log.level field equals Exception
• View the surrounding logs with log.level set to Debug.

The Google stackdriver has an interesting feature called anchor. Basically if a log is anchored, it will be always displayed and centered in the results (as long as it matches the search criteria). I think a flow like this would be nice to have:

• User clicks on the log he want to view the context
• An overlay shows the anchored log line.
• All surrounding logs are shown (ignore the original search criteria)
• There is a filter input box on the overlay. User can filter the surrounding logs and the anchored log line will always be present and centered. (eg: set the log.level to Debug, remove the free text)
• For convinience, there should be a button to quickly copy the original search criteria into the filter box and the filter should also shows a drop-down of the recent input in case the user want to view the context for several logs successively.

[weltenwort]

@phantasia15 thank you for taking the time to contribute! What you describe makes sense and offers a lot of flexibility: The user can incrementally narrow the context by constructing a new filter expression.

Maybe we can unify that with the approach that @mukeshelastic described by initializing that context filter with the current global filter so the user can widen the context. Then the user can choose whether to remove the full-text search terms or some of the field-specific terms. 🤔

[mukeshelastic]

Thank you @phantasia15 for taking time to describe your workflows. Like @weltenwort pointed out, we will provide clarity on which filters are still applied by showing them in the overlay. We will explore how to accomplish that UX elegantly.

When the user clicks on show surrounding logs, they are intentional about removing some filter. We are removing the free text search because we believe that is usually the last step in narrowing down logs to find a log line that interests users and they want to see what is happening before and after that log line. If they want to remove other filters, then they can remove them from the context filter.

[mukeshelastic]

@katrin-freihofner and I discussed this UX further and we made a minor adjustment to previous UX. Earlier when users clicked on surrounding logs, we only dropped the 'text search' filter.. If users just had key value filter and no text search filter then the surrounding logs will show same logs as that of stream view, which is misleading. So we think the the right UX is to drop the last filter, independent of whether it is text search or key value filter. For now, we will assume the last filter is the right most filter. If our assumption is wrong then we may provide a way for users to edit the filter in the surrounding logs overlay.

[katrin-freihofner]

Quick update:

Some notes

• the overlay panel should be scrollable which means it could be bigger than shown here and would be able to hold much more log lines
• loading more log lines should follow the same pattern as we are aiming for with the new date picker [Logs UI] Replace current timepicker with EUI SuperDatePicker #49154
• We have to think about the filter action in the log details flyout (Where should this filter be applied?)
• Are there any additional context actions?

[weltenwort]

loading more log lines should follow the same pattern as we are aiming for with the new date picker

That means when the user scrolls to the end of the date interval in the overlay there is an option to extend the overall time range? Or does the overlay have a custom date range picker?

[katrin-freihofner]

Yes, I think extending the timeframe could work. I don't think that the overlay should have its own time picker.

Maybe something like the overlay initially shows the selected log line +-30 min (or now) and this timeframe can be extended like the stream?

Let me know what's your opinion on that.

[weltenwort]

I think it should have at least the same initial interval as the main date picker. Otherwise it might exclude items that are visible on the stream.

I'd probably recommend to start out with sharing the date state between the two for simplicity. If we then learn that it leads to a bad experience we can give it a separate date range state.

[jasonrhodes]

I had a good chat with @mukeshelastic today about this feature. One thing I keep getting stuck on is the assumption around removing the “last filter” applied, etc. It feels like this UI as shown is a lot of ceremony for only making this one small assumption around removing one of the applied filters and it’s not clear to me yet that will always be useful.

One thing I’m wondering, if “surrounding logs” is really about “view my current set of filters but with one removed”, is could we change the way filter state is represented so that when you add a filter, it is applied in an enabled state. But added filters can be a) disabled/re-enabled, or b) removed. This would let users achieve this kind of exploration mode without necessarily needing a new modal to do it in, and it would let users choose how many and which filters they need to “disable” in order to explore some log context and then quickly return to the fully filtered list they were looking at a moment ago.

Just some thoughts, curious what others think. ping @katrin-freihofner

[katrin-freihofner]

A few updates from our discussions in Slack and on Zoom:

• the filters in the context overlay can be enabled/disabled
• we show appropriate filters (because our UI is very smart 😉) that can be quickly enabled/disabled by clicking them.

[weltenwort]

As I already mentioned on Slack, I have doubts that we can generically convert KQL expressions into useful filters. How would we treat more complex KQL expressions that are more than just single-level conjunctions?

[katrin-freihofner]

Quick update of yesterday's discussion. Viewing a logline in context means:

• Setting this log lines host.name and file.path as filters in the stream's filter bar.
• This also means, there won't be an overlay, we will use the UI we already have.
• This context can be adjusted by opening the details and selecting a different filter. Please be aware, this is not a new feature. This is already possible today.

I created a quick video to show this in action:

[katrin-freihofner]

This is an update according to our discussion in yesterdays meetings:

I updated the engineering issue accordingly.