Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deprecation warnings without an obvious cause or mitigation #81589

Closed
rudolf opened this issue Dec 9, 2021 · 7 comments
Closed

Deprecation warnings without an obvious cause or mitigation #81589

rudolf opened this issue Dec 9, 2021 · 7 comments
Labels
>bug :Core/Infra/Core Core issues without another label :Security/Security Security issues without another label Team:Core/Infra Meta label for core/infra team Team:Security Meta label for security team

Comments

@rudolf
Copy link
Contributor

rudolf commented Dec 9, 2021 •
edited
Loading

After reviewing the deprecation logs from Kibana CI elastic/kibana#120043 I came across several requests which triggered deprecations around system indices even though the request doesn't explicitly access system indices. Are these false positives? Could we provide a message that helps users resolve these?

POST /_async_search?batched_reduce_size=64&wait_for_completion_timeout=1ms&keep_on_completion=false&keep_alive=1m&ignore_unavailable=true&track_total_hits=true\n{\"query\":{\"match_all\":{}}}

this request accesses system indices: [.security-7], but in a future major version, direct access to system indices will be prevented by default

POST /_async_search?batched_reduce_size=64&wait_for_completion_timeout=1ms&keep_on_completion=true&keep_alive=604800000ms&ignore_unavailable=true&track_total_hits=true\n{\"query\":{\"term\":{\"agent\":\"1\"}}}

this request accesses system indices: [.async-search, .security-7], but in a future major version, direct access to system indices will be prevented by default

PUT /_cluster/settings\n{\"persistent\":{\"cluster\":{\"remote\":{\"ccr-api-integration-tests-cluster\":{\"skip_unavailable\":null,\"mode\":null,\"proxy_address\":null,\"proxy_socket_connections\":null,\"server_name\":null,\"seeds\":null,\"node_connections\":null}}}}}

Elasticsearch deprecation: 299 Elasticsearch-7.16.1-SNAPSHOT-847309f2a3c2c1b504da0c59a68936dbb9c617a1 "[xpack.monitoring.collection.enabled] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.",
299 Elasticsearch-7.16.1-SNAPSHOT-847309f2a3c2c1b504da0c59a68936dbb9c617a1 "[xpack.monitoring.collection.interval] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.

PUT /_rollup/job/rcbyzc-1639047624062\n{\"index_pattern\":\"tvkmxued-1639047624007\",\"rollup_index\":\"rollup_index\",\"cron\":\"0 0 0 ? * 7\",\"page_size\":1000,\"groups\":{\"date_histogram\":{\"interval\":\"24h\",\"delay\":\"1d\",\"time_zone\":\"UTC\",\"field\":\"testCreatedField\"},\"terms\":{\"fields\":[\"testTotalField\",\"testTagField\"]},\"histogram\":{\"interval\":\"7\",\"fields\":[\"testTotalField\"]}},\"metrics\":[{\"field\":\"testTotalField\",\"metrics\":[\"avg\",\"value_count\"]},{\"field\":\"testCreatedField\",\"metrics\":[\"max\",\"min\"]}]}

299 Elasticsearch-7.16.1-SNAPSHOT-847309f2a3c2c1b504da0c59a68936dbb9c617a1 "this request accesses system indices: [.async-search, .logstash, .security-7, .tasks], but in a future major version, direct access to system indices will be prevented by default

GET /.siem-signals-default-*/_alias

299 Elasticsearch-7.16.1-SNAPSHOT-847309f2a3c2c1b504da0c59a68936dbb9c617a1 "this request accesses aliases with names reserved for system indices: [.security], but in a future major version, direct access to system indices and their aliases will not be allowed\
@rudolf rudolf added the >bug label Dec 9, 2021
@astefan astefan added the :Security/Security Security issues without another label label Dec 22, 2021
@elasticmachine elasticmachine added the Team:Security Meta label for security team label Dec 22, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-security (Team:Security)

@tvernum tvernum added the :Core/Infra/Core Core issues without another label label Dec 22, 2021
@elasticmachine elasticmachine added the Team:Core/Infra Meta label for core/infra team label Dec 22, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@ywangd
Copy link
Member

ywangd commented Dec 23, 2021

I had a brief look at the case of
GET /.siem-signals-default-*/_alias

And I can confirm it does issue deprecation warning if the user has access to system indices. The reason is that GetAliasRequest sees the above request to be equivalent to GET /.siem-signals-default-*/_alias/_all and loads all authorized aliases:

elasticsearch/x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/authz/IndicesAndAliasesResolver.java

Lines 383 to 385 in 8eb23b6

if (aliases.length == 0) {
finalAliases.addAll(authorizedAliases);
}

It then issues warnings based on all of the authorized aliases without taking in account of the actual index pattern (.siem-signals-default-*) on the request:

elasticsearch/server/src/main/java/org/elasticsearch/action/admin/indices/alias/get/TransportGetAliasesAction.java

Line 198 in 8eb23b6

checkSystemAliasAccess(request, systemIndices, systemIndexAccessLevel, threadContext);

The straightforward fix seems to be restricting the checking to only aliases that have actually been resolved for the requested index patterns. In fact, the check for system indices (not aliases) access does filter out for the requested indices only.

@cjcenizal cjcenizal mentioned this issue Dec 23, 2021
Closed
@jakelandis
Copy link
Contributor

Elasticsearch deprecation: 299 Elasticsearch-7.16.1-SNAPSHOT-847309f2a3c2c1b504da0c59a68936dbb9c617a1 "[xpack.monitoring.collection.enabled] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.",
299 Elasticsearch-7.16.1-SNAPSHOT-847309f2a3c2c1b504da0c59a68936dbb9c617a1 "[xpack.monitoring.collection.interval] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.

This is from the use of internal monitoring, which is deprecated but still in use pending a migration to metricbeat monitoring. I will follow up to see if we should hide that deprecation warning on platforms still impacted.

@jakelandis
Copy link
Contributor

jakelandis commented Jan 6, 2022 •
edited
Loading

I will follow up to see if we should hide that deprecation warning on platforms still impacted.

OK... it look like xpack.monitoring.* is flagged to be hidden, but support to hide dynamic settings isn't available until 7.17: #81836

@gwbrown
Copy link
Contributor

gwbrown commented Jan 14, 2022 •
edited
Loading

It then issues warnings based on all of the authorized aliases without taking in account of the actual index pattern (.siem-signals-default-*) on the request:

Part of this is that good ol' thing Security does of replacing the index patterns in the request with a list of authorized indices, so we can't do anything based on the actual index pattern because we don't have it.

Anyway, what this check is doing is checking for requests that try to access aliases that have names reserved for system indices/aliases. I'm not sure why we were doing that, as we don't try to do that anywhere else - we only warn about accessing things that actually exist, rather than things that would be problematic to access if they did exist. Anyway, I think the solution here is just to remove the warning entirely and I'll have a PR up to do so shortly.

@gwbrown gwbrown mentioned this issue Jan 19, 2022
Merged
@rudolf rudolf mentioned this issue Jan 24, 2022
Merged
1 task
elasticsearchmachine pushed a commit that referenced this issue Jan 25, 2022
@gwbrown
Remove unnecessary deprecation warning in Get Alias API (#82773)
9be18cb 
The check removed by this commit checks specifically for requests for
aliases that *would* be system aliases, if they existed. I'm not sure
why we were doing this, as we don't try to do this anywhere else. The
only test that this seems to make fail is the test explicitly checking
that behavior, which I don't think is really what we want. So I'm just
removing it.

Relates #81589
gwbrown added a commit to gwbrown/elasticsearch that referenced this issue Jan 25, 2022
@gwbrown
Remove unnecessary deprecation warning in Get Alias API (elastic#82773)
43c096f 
The check removed by this commit checks specifically for requests for
aliases that *would* be system aliases, if they existed. I'm not sure
why we were doing this, as we don't try to do this anywhere else. The
only test that this seems to make fail is the test explicitly checking
that behavior, which I don't think is really what we want. So I'm just
removing it.

Relates elastic#81589
elasticsearchmachine pushed a commit that referenced this issue Jan 25, 2022
@gwbrown
Remove unnecessary deprecation warning in Get Alias API (#82773) (#83009
adc1a25 
)

The check removed by this commit checks specifically for requests for
aliases that *would* be system aliases, if they existed. I'm not sure
why we were doing this, as we don't try to do this anywhere else. The
only test that this seems to make fail is the test explicitly checking
that behavior, which I don't think is really what we want. So I'm just
removing it.

Relates #81589
gwbrown added a commit to gwbrown/elasticsearch that referenced this issue Jan 25, 2022
@gwbrown
Remove unnecessary deprecation warning in Get Alias API (elastic#82773)
1009b05 
The check removed by this commit checks specifically for requests for
aliases that *would* be system aliases, if they existed. I'm not sure
why we were doing this, as we don't try to do this anywhere else. The
only test that this seems to make fail is the test explicitly checking
that behavior, which I don't think is really what we want. So I'm just
removing it.

Relates elastic#81589
elasticsearchmachine pushed a commit that referenced this issue Jan 25, 2022
@gwbrown
Remove unnecessary deprecation warning in Get Alias API (#82773) (#83090
6d541d9 
)

The check removed by this commit checks specifically for requests for
aliases that *would* be system aliases, if they existed. I'm not sure
why we were doing this, as we don't try to do this anywhere else. The
only test that this seems to make fail is the test explicitly checking
that behavior, which I don't think is really what we want. So I'm just
removing it.

Relates #81589
@gwbrown
Copy link
Contributor

gwbrown commented Mar 2, 2022

As noted above, some of these are expected, and others (the _alias one in particular) we addressed with #82773. As such, I don't think there's anything more to do on this issue so I'm going to close it, but please feel free to comment and reopen if I'm wrong.

@gwbrown gwbrown closed this as completed Mar 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>bug :Core/Infra/Core Core issues without another label :Security/Security Security issues without another label Team:Core/Infra Meta label for core/infra team Team:Security Meta label for security team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@astefan @jakelandis @rudolf @gwbrown @tvernum @ywangd @elasticmachine