-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fleet] Setup fleet server indices in Kibana without packages #90658
[Fleet] Setup fleet server indices in Kibana without packages #90658
Conversation
Pinging @elastic/fleet (Team:Fleet) |
@elasticmachine merge upstream |
@elasticmachine merge upstream |
@elasticmachine merge upstream |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
'Extra setup is needed to be able to use central management for agent, please visit the Fleet app in Kibana.' | ||
); | ||
} | ||
if (appContextService.getConfig()?.agents?.fleetServerEnabled) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we should have another condition to check on if central management is enabled yet? setting up central management === setting up Fleet server, according to the code here. what do you think? of course, this would mean we trigger setup after user clicks to enable central management too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's okay to create the indices even if central management is not enabled later, and doing it here outside of central management setup allow us to have a seamless migration
x-pack/plugins/fleet/server/services/fleet_server/elastic_index.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/fleet/server/services/fleet_server/elastic_index.ts
Outdated
Show resolved
Hide resolved
['.fleet-policies', ESFleetPoliciesIndex], | ||
['.fleet-policies-leader', ESFleetPoliciesLeaderIndex], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"policies" here is a bit confusing as we already have the concept of agent and integration policies within fleet. can you help me understand the purpose of these indices?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will try to explain what are these indices, first Fleet Server do not know about kibana saved object so all the data used by fleet server come from these indices:
.fleet-policies
contains a full policy as it's going to be distributed to the agent (except the API keys)
.fleet-policies-leader
it's not used in Kibana is used to have a policy coordinator per policy in Fleet server for future features.
Co-authored-by: Jen Huang <[email protected]>
…a-setup-fleet-server
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this works as described. After starting Kibana with xpack.fleet.agents.fleetServerEnabled: true
, and without triggering the Fleet setup endpoints, I see the following indices and aliases being created.
Indices
.fleet-actions_1
.fleet-policies_1
.fleet-servers_1
.fleet-policies-leader_1
.fleet-enrollment-api-keys_1
.fleet-agents_1
Aliases
.fleet-servers .fleet-servers_1 - - - -
.fleet-agents .fleet-agents_1 - - - -
.fleet-policies .fleet-policies_1 - - - -
.fleet-actions .fleet-actions_1 - - - -
.fleet-enrollment-api-keys .fleet-enrollment-api-keys_1 - - - -
.fleet-policies-leader .fleet-policies-leader_1 - - - -
Is this expected behavior? Would it be possible to document in the initial PR descriptions what these indices are for?
@skh yes creating the indices and the aliases is the expected behavior this will allow us to run destructive migrations in the future if needed I will try to describe what indice is used for what
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nchaulet Thanks for the additional explanation.
…a-setup-fleet-server
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the changes and additional explanation 🚀
💚 Build SucceededMetrics [docs]Async chunks
Page load bundle
History
To update your PR or re-run it, just comment with: |
Summary
Related to #87372
Replace the setup of fleet server indices by the integration package, by creating them using the
kibana_system
during the plugin start.Details
Foreach index we create an index like this
.fleet-agents_1
and an alias to allow for destructive migrations in the future if we need them.For each index we create a hash of the mapping and we update the index only if the hash do not match.
The
kibana_system
role asall
access to.fleet-*
so it can create the indices.How to test
Starting from a fresh ES and kibana with this flag
You should see the
.fleet
indices created.