[eem] _search endpoint / initial entity manager UI #199609
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
klacabane
added
release_note:skip
miltonhultgren
approved these changes
Nov 21, 2024
afharo
approved these changes
Nov 21, 2024
tommyers-elastic
approved these changes
Nov 22, 2024
| sourceCommand(source), | ||
| ...filterCommands(source), | ||
| statsCommand(source), | ||
| `SORT entity.last_seen_timestamp DESC`, |
There was a problem hiding this comment.
if a source doesn't have a timestamp field, this will error out (i think). we should handle that case.
There was a problem hiding this comment.
The timestamp field set on a source is mandatory in this PR so the query will not run if it's not defined
| function mergeEntities(entity1: Entity, entity2: Entity): Entity { | ||
| const merged: Entity = { | ||
| ...entity1, | ||
| 'entity.last_seen_timestamp': new Date( |
There was a problem hiding this comment.
again, i think we have to handle the case of missing last_seen_timestamp
💚 Build Succeeded
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Public APIs missing exports
Page load bundle
Unknown metric groupsAPI count
async chunk count
History
cc @klacabane |
|
Starting backport for target branches: 8.x https://github.com/elastic/kibana/actions/runs/11975189293 |
kibanamachine
added
the
backport missing
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
1 similar comment
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
paulinashakirova
pushed a commit
to paulinashakirova/kibana
that referenced
this pull request
Nov 26, 2024
## Summary
- create `_search` endpoint to discover entities with esql queries. It
currently reads sources of the provided `type` from
`kibana_entity_definitions` index. Run this query to insert a
definition:
```
POST kibana_entity_definitions/_doc
{
"entity_type": "service",
"index_patterns": ["remote_cluster:logs-*"],
"metadata_fields": [],
"identity_fields": ["service.name"],
"filters": [],
"timestamp_field": "@timestamp"
}
```
By default `_search` will look at data in the last 5m. The lookup period
can be overriden by providing `start`/`end` parameters in ISO format. It
also accepts a `limit` to specify the number of entities returned which
defaults to 10
```
POST kbn:/internal/entities/v2/_search
{
"type": "service",
"start": "2024-11-19T20:40:00.000Z",
"end": "2024-11-19T20:50:00.000Z",
"limit": 20
}
```
- create `_search/preview` endpoint to preview output of entity sources
without persisting them
- create UI to preview results of an entity definition at
`/app/entity_manager`. The application is living in its own plugin at
`observability_solution/entity_manager_app`
---------
Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Milton Hultgren <[email protected]>
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
klacabane
added a commit
to klacabane/kibana
that referenced
this pull request
Nov 27, 2024
## Summary
- create `_search` endpoint to discover entities with esql queries. It
currently reads sources of the provided `type` from
`kibana_entity_definitions` index. Run this query to insert a
definition:
```
POST kibana_entity_definitions/_doc
{
"entity_type": "service",
"index_patterns": ["remote_cluster:logs-*"],
"metadata_fields": [],
"identity_fields": ["service.name"],
"filters": [],
"timestamp_field": "@timestamp"
}
```
By default `_search` will look at data in the last 5m. The lookup period
can be overriden by providing `start`/`end` parameters in ISO format. It
also accepts a `limit` to specify the number of entities returned which
defaults to 10
```
POST kbn:/internal/entities/v2/_search
{
"type": "service",
"start": "2024-11-19T20:40:00.000Z",
"end": "2024-11-19T20:50:00.000Z",
"limit": 20
}
```
- create `_search/preview` endpoint to preview output of entity sources
without persisting them
- create UI to preview results of an entity definition at
`/app/entity_manager`. The application is living in its own plugin at
`observability_solution/entity_manager_app`
---------
Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Milton Hultgren <[email protected]>
(cherry picked from commit 0b3f4fb)
# Conflicts:
# .github/CODEOWNERS
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
1 task
klacabane
added a commit
that referenced
this pull request
Nov 28, 2024
…202050) # Backport This will backport the following commits from `main` to `8.x`: - [[eem] _search endpoint / initial entity manager UI (#199609)](#199609) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Kevin Lacabane","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-22T15:12:04Z","message":"[eem] _search endpoint / initial entity manager UI (#199609)\n\n## Summary\r\n\r\n- create `_search` endpoint to discover entities with esql queries. It\r\ncurrently reads sources of the provided `type` from\r\n`kibana_entity_definitions` index. Run this query to insert a\r\ndefinition:\r\n```\r\nPOST kibana_entity_definitions/_doc\r\n{\r\n \"entity_type\": \"service\",\r\n \"index_patterns\": [\"remote_cluster:logs-*\"],\r\n \"metadata_fields\": [],\r\n \"identity_fields\": [\"service.name\"],\r\n \"filters\": [],\r\n \"timestamp_field\": \"@timestamp\"\r\n}\r\n```\r\n\r\nBy default `_search` will look at data in the last 5m. The lookup period\r\ncan be overriden by providing `start`/`end` parameters in ISO format. It\r\nalso accepts a `limit` to specify the number of entities returned which\r\ndefaults to 10\r\n\r\n```\r\nPOST kbn:/internal/entities/v2/_search\r\n{\r\n \"type\": \"service\",\r\n \"start\": \"2024-11-19T20:40:00.000Z\",\r\n \"end\": \"2024-11-19T20:50:00.000Z\",\r\n \"limit\": 20\r\n}\r\n```\r\n\r\n- create `_search/preview` endpoint to preview output of entity sources\r\nwithout persisting them\r\n \r\n- create UI to preview results of an entity definition at\r\n`/app/entity_manager`. The application is living in its own plugin at\r\n`observability_solution/entity_manager_app`\r\n\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: Milton Hultgren <[email protected]>","sha":"0b3f4fbd3cd60663289fc13f8f01e3f4c9131479","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport missing","v9.0.0","backport:prev-minor","ci:project-deploy-observability","Team:obs-entities"],"number":199609,"url":"https://github.com/elastic/kibana/pull/199609","mergeCommit":{"message":"[eem] _search endpoint / initial entity manager UI (#199609)\n\n## Summary\r\n\r\n- create `_search` endpoint to discover entities with esql queries. It\r\ncurrently reads sources of the provided `type` from\r\n`kibana_entity_definitions` index. Run this query to insert a\r\ndefinition:\r\n```\r\nPOST kibana_entity_definitions/_doc\r\n{\r\n \"entity_type\": \"service\",\r\n \"index_patterns\": [\"remote_cluster:logs-*\"],\r\n \"metadata_fields\": [],\r\n \"identity_fields\": [\"service.name\"],\r\n \"filters\": [],\r\n \"timestamp_field\": \"@timestamp\"\r\n}\r\n```\r\n\r\nBy default `_search` will look at data in the last 5m. The lookup period\r\ncan be overriden by providing `start`/`end` parameters in ISO format. It\r\nalso accepts a `limit` to specify the number of entities returned which\r\ndefaults to 10\r\n\r\n```\r\nPOST kbn:/internal/entities/v2/_search\r\n{\r\n \"type\": \"service\",\r\n \"start\": \"2024-11-19T20:40:00.000Z\",\r\n \"end\": \"2024-11-19T20:50:00.000Z\",\r\n \"limit\": 20\r\n}\r\n```\r\n\r\n- create `_search/preview` endpoint to preview output of entity sources\r\nwithout persisting them\r\n \r\n- create UI to preview results of an entity definition at\r\n`/app/entity_manager`. The application is living in its own plugin at\r\n`observability_solution/entity_manager_app`\r\n\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: Milton Hultgren <[email protected]>","sha":"0b3f4fbd3cd60663289fc13f8f01e3f4c9131479"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199609","number":199609,"mergeCommit":{"message":"[eem] _search endpoint / initial entity manager UI (#199609)\n\n## Summary\r\n\r\n- create `_search` endpoint to discover entities with esql queries. It\r\ncurrently reads sources of the provided `type` from\r\n`kibana_entity_definitions` index. Run this query to insert a\r\ndefinition:\r\n```\r\nPOST kibana_entity_definitions/_doc\r\n{\r\n \"entity_type\": \"service\",\r\n \"index_patterns\": [\"remote_cluster:logs-*\"],\r\n \"metadata_fields\": [],\r\n \"identity_fields\": [\"service.name\"],\r\n \"filters\": [],\r\n \"timestamp_field\": \"@timestamp\"\r\n}\r\n```\r\n\r\nBy default `_search` will look at data in the last 5m. The lookup period\r\ncan be overriden by providing `start`/`end` parameters in ISO format. It\r\nalso accepts a `limit` to specify the number of entities returned which\r\ndefaults to 10\r\n\r\n```\r\nPOST kbn:/internal/entities/v2/_search\r\n{\r\n \"type\": \"service\",\r\n \"start\": \"2024-11-19T20:40:00.000Z\",\r\n \"end\": \"2024-11-19T20:50:00.000Z\",\r\n \"limit\": 20\r\n}\r\n```\r\n\r\n- create `_search/preview` endpoint to preview output of entity sources\r\nwithout persisting them\r\n \r\n- create UI to preview results of an entity definition at\r\n`/app/entity_manager`. The application is living in its own plugin at\r\n`observability_solution/entity_manager_app`\r\n\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: Milton Hultgren <[email protected]>","sha":"0b3f4fbd3cd60663289fc13f8f01e3f4c9131479"}}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <[email protected]>
kibanamachine
added
v8.18.0
and removed
backport missing
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Dec 12, 2024
## Summary
- create `_search` endpoint to discover entities with esql queries. It
currently reads sources of the provided `type` from
`kibana_entity_definitions` index. Run this query to insert a
definition:
```
POST kibana_entity_definitions/_doc
{
"entity_type": "service",
"index_patterns": ["remote_cluster:logs-*"],
"metadata_fields": [],
"identity_fields": ["service.name"],
"filters": [],
"timestamp_field": "@timestamp"
}
```
By default `_search` will look at data in the last 5m. The lookup period
can be overriden by providing `start`/`end` parameters in ISO format. It
also accepts a `limit` to specify the number of entities returned which
defaults to 10
```
POST kbn:/internal/entities/v2/_search
{
"type": "service",
"start": "2024-11-19T20:40:00.000Z",
"end": "2024-11-19T20:50:00.000Z",
"limit": 20
}
```
- create `_search/preview` endpoint to preview output of entity sources
without persisting them
- create UI to preview results of an entity definition at
`/app/entity_manager`. The application is living in its own plugin at
`observability_solution/entity_manager_app`
---------
Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Milton Hultgren <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
_searchendpoint to discover entities with esql queries. It currently reads sources of the providedtypefromkibana_entity_definitionsindex. Run this query to insert a definition:By default
_searchwill look at data in the last 5m. The lookup period can be overriden by providingstart/endparameters in ISO format. It also accepts alimitto specify the number of entities returned which defaults to 10create
_search/previewendpoint to preview output of entity sources without persisting themcreate UI to preview results of an entity definition at
/app/entity_manager. The application is living in its own plugin atobservability_solution/entity_manager_app