Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DataUsage][Serverless] Data usage metrics page enhancements #195556

Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,36 +5,50 @@
* 2.0.
*/

import { RequestHandler } from '@kbn/core/server';
import { type ElasticsearchClient, RequestHandler } from '@kbn/core/server';
import { DataUsageContext, DataUsageRequestHandlerContext } from '../../types';

import { errorHandler } from '../error_handler';

export interface MeteringStats {
name: string;
num_docs: number;
size_in_bytes: number;
}

interface MeteringStatsResponse {
datastreams: MeteringStats[];
}

const getMeteringStats = (client: ElasticsearchClient) => {
return client.transport.request<MeteringStatsResponse>({
method: 'GET',
path: '/_metering/stats',
});
};

export const getDataStreamsHandler = (
dataUsageContext: DataUsageContext
): RequestHandler<never, unknown, DataUsageRequestHandlerContext> => {
const logger = dataUsageContext.logFactory.get('dataStreamsRoute');

return async (context, _, response) => {
logger.debug(`Retrieving user data streams`);
logger.debug('Retrieving user data streams');

try {
const core = await context.core;
const esClient = core.elasticsearch.client.asCurrentUser;

const { data_streams: dataStreamsResponse } = await esClient.indices.dataStreamsStats({
name: '*',
expand_wildcards: 'all',
});
const { datastreams: meteringStats } = await getMeteringStats(
core.elasticsearch.client.asSecondaryAuthUser
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@neptunian I copied some of this from x-pack/plugins/index_management/server/routes/api/data_streams/register_get_route.ts#L90.

Do you happen to know how asSecondaryAuthUser is different from asCurrentUser. I also tried with asCurrentUser here and it yields the same set of datastreams/data. I'm not sure if we should just use asCurrentUser.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. Lets ask @elastic/kibana-management .

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @mattkime! Do you or anyone on your team happen to know?

Copy link
Contributor

@mattkime mattkime Oct 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting up to speed on this but to answer your direct question - https://github.com/elastic/kibana/blob/main/packages/core/elasticsearch/core-elasticsearch-server/src/client/scoped_cluster_client.ts#L28


I doubt my above comment was useful so I'll just report what Alison shared in chat - #190131 (comment)

);

const sorted = dataStreamsResponse
.sort((a, b) => b.store_size_bytes - a.store_size_bytes)
.map((dataStream) => ({
name: dataStream.data_stream,
storageSizeBytes: dataStream.store_size_bytes,
const body = meteringStats
.sort((a, b) => b.size_in_bytes - a.size_in_bytes)
.map((stat) => ({
name: stat.name,
storageSizeBytes: stat.size_in_bytes,
}));

return response.ok({
body: sorted,
body,
});
} catch (error) {
return errorHandler(logger, response, error);
Expand Down