[Dataset quality] States for missing permissions

[yngrdyn]

📓 Summary

When users have viewer role, they are able to see only Datasets Quality summary

If they use a direct url for accessing dataset details (flyout opened), this is what they see

Errors

Error from the endpoint dataset_quality/data_streams/stats?type=logs

security_exception
  Root causes:
    security_exception: action [indices:monitor/data_stream/stats] is unauthorized for user [viewer] with effective roles [viewer] on indices [logs-*], this action is granted by the index privileges [monitor,manage,all]

Error form endpoint dataset_quality/data_streams/estimated_data?type=logs&start=2024-03-27T14%3A51%3A17.556Z&end=2024-03-28T14%3A51%3A17.556Z

security_exception
  Root causes:
    security_exception: action [indices:monitor/stats] is unauthorized for user [viewer] with effective roles [viewer] on indices [logs-*-*], this action is granted by the index privileges [monitor,cross_cluster_replication,manage,all]

Error from endpoint ``

security_exception
  Root causes:
    security_exception: action [indices:monitor/data_stream/stats] is unauthorized for user [viewer] with effective roles [viewer] on indices [logs-*system.syslog-default*], this action is granted by the index privileges [monitor,manage,all]

Adding a role with index monitor privileges over logs-*-* fixes the failures in the aforementioned endpoints and the user is able to see all the information

Since we still can fetch information about degraded Docs we should present the users with this information, and point them to documentation on how to fix their permissions problems instead of just failing the page when the requests have fall because of permissions problems. For doing that Isa has proposed the following mock up

For achieving this new state I propose the following tasks

Tasks

Preview Give feedback

1. [Dataset quality] Split integration request from dataStreamStats #180560
   backport:skip release_note:skip v8.14.0 +3
   
2. Hide Size and Last Activity columns when user doesn't have required permissions
3. Hide Show inactive datasets toggle
4. Modify content of Active datasets and Estimated data to point users to some documentation where we tell them how to solve the privileges problem

✔️ Acceptance criteria

When users don't have the required privileges to have all the information we show in dataset quality, they should be able to see a reduced version of the page:

• Table with Dataset name with integration information related (icon), Namespace, Degraded Docs, Actions.
• Content inside Active datasets and Estimated data replaced by You don't have the necessary privileges to see this information. Learn more. and a link to the dataset quality page documentation where they can check the required permission for the full version of the page.
• Flyout should be accessible presenting the information we can show. Consider replacing Dataset details information with the message about the missing privileges as well

[yngrdyn]

@mdbirnstiehl, additionally we should document the permission model needed to access dataset quality feature, I was thinking in something similar on how it's done for security Data quality dashboard, wdyt?

[mohamedhamed-ahmed]

lets also keep in mind when deciding on the acceptance criteria to handle pasting urls where the flyout state is set to open for example

[elasticmachine]

Pinging @elastic/unified-observability (Team:Observability)

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[ruflin]

What part of the table needs the stats call? As we are able to show a summary, I assume we could also show some info for each dataset, just not all columns?

[yngrdyn]

As of now we are relying in the dataStreamStats api for most of the table columns, without that call we might be able to show, with some changes, rawName of the dataset (no integration information or human-readable name), namespace and degraded Docs column.

[flash1293]

@yngrdyn will the the metering API for serverless provide all the information we need eventually? Or are there gaps we won't be able to close this way?

[yngrdyn]

We'll miss information to correlate to an integration. I think we could flip it around eventually and get the correlation from fleet, where we could know which datasets should belong to a certain integration, but in any case we would need certain level of permissions to get that information.

[gbamparop]

As of now we are relying in the dataStreamStats api for most of the table columns, without that call we might be able to show, with some changes, rawName of the dataset (no integration information or human-readable name), namespace and degraded Docs column.

The list of integrations and datasets in the data source selector can be displayed with the viewer role, so I guess it's possible to correlate datasets with integrations in the table as well?

[isaclfreire]

@ruflin has a point. If there's already something we could show, then we shouldn't preclude it to the user. We could also dedicate a column to quality (now it's just communicated as part of the degraded docs column), so it will remain in the version with less privileges as well. This column could also be filterable if users only want to see poor quality datasets, for example. See below the full access example:

Another thing to consider is: should we completely remove the "active datasets" and "estimated data" box from the summary, or should we invest in an empty state? Can users eventually request access? Should we propose them with any available documentation? See below a very quick mock

I wonder in the case of restricted permissions, if it still makes sense to keep the flyout...

[gbamparop]

Another thing to consider is: should we completely remove the "active datasets" and "estimated data" box from the summary, or should we invest in an empty state? Can users eventually request access? Should we propose them with any available documentation? See below a very quick mock

+1 for empty states where it makes sense, to make it easier for users to see what's missing and request the necessary permissions from their admins, we could display the missing privileges in the tooltip.

[yngrdyn]

The list of integrations and datasets in the data source selector can be displayed with the viewer role, so I guess it's possible to correlate datasets with integrations in the table as well?

will have a closer look on how can we get this information, we might be getting this info using an internal user rather than current user (?)

I wonder in the case of restricted permissions, if it still makes sense to keep the flyout...

They will be able to see, for example, the evolution of the degradation, but without permissions they probably wouldn't be able to act on any future recommendation/action proposed by us. If we can get Integration info they could also be able to navigate to them from there