Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Detection Engine][9.0] Deprecate legacy siem signals migration endpoints #195523

Open
1 of 4 tasks
yctercero opened this issue Oct 9, 2024 · 4 comments
Open
1 of 4 tasks

[Detection Engine][9.0] Deprecate legacy siem signals migration endpoints #195523

yctercero opened this issue Oct 9, 2024 · 4 comments
Assignees
@vitaliidm
Labels
Team:Detection Engine Security Solution Detection Engine Area Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v9.0.0

Comments

@yctercero
Copy link
Contributor

yctercero commented Oct 9, 2024 •
edited by vitaliidm
Loading

See doc for info on deprecating API - https://docs.google.com/document/d/1W7csjn6QYjrBjmbXMzSz_JUD4KcmWz8jTTtAWFwgUJM/edit#heading=h.tui2zvb9gca6

Summary

We propose removing the siem signals migration endpoints from Kibana in 10.0. These include the following routes:

POST /api/detection_engine/signals/migrations

DELETE /api/detection_engine/signals/migrations

POST /api/detection_engine/signals/finalize_migrations

GET /api/detection_engine/signals/migration_status

To prepare for this we need to:

Docs checklist
Preview Give feedback

QA checklist
Preview Give feedback
@yctercero yctercero added v9.0.0 Team:Detection Engine Security Solution Detection Engine Area labels Oct 9, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-engine (Team:Detection Engine)

@yctercero yctercero added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Oct 9, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@vitaliidm vitaliidm self-assigned this Oct 21, 2024
@yctercero
Copy link
Contributor Author

Update:

Core has merged the PR that enables teams to register deprecated HTTP APIs. Per the latest instructions, we'll need to:

  • Registering our APIs as deprecated
  • Send it a request to ensure that it will be surfaced in Upgrade Assistant along with a link to your release note (see our docs).

@vitaliidm vitaliidm mentioned this issue Oct 29, 2024
Closed
This was referenced Dec 3, 2024
Merged
Merged
Open
vitaliidm added a commit that referenced this issue Dec 10, 2024
@vitaliidm @kibanamachine
[Security Solution][Detection Engine] deprecates siem signals migrati…
3d46ead 
…on APIs (#202662)

## Summary

 - addresses part of #195523
- deprecates
https://www.elastic.co/guide/en/security/current/signals-migration-api.html
APIs according to internal 9.x readiness
[guideline](https://docs.google.com/document/d/1W7csjn6QYjrBjmbXMzSz_JUD4KcmWz8jTTtAWFwgUJM/edit?tab=t.0#heading=h.tui2zvb9gca6)
 
#### How to test deprecated APIs?

1. Run API
https://www.elastic.co/guide/en/security/current/signals-migration-api.html
2. Observe warning deprecation on Kibana Upgrade page

<img width="2540" alt="Screenshot 2024-12-03 at 10 43 59"
src="https://github.com/user-attachments/assets/24fcebb9-2d31-4ca3-a0dc-4ed7861d26a2">

---------

Co-authored-by: kibanamachine <[email protected]>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Dec 10, 2024
@vitaliidm
[Security Solution][Detection Engine] deprecates siem signals migrati…
9f5255c 
…on APIs (elastic#202662)

## Summary

 - addresses part of elastic#195523
- deprecates
https://www.elastic.co/guide/en/security/current/signals-migration-api.html
APIs according to internal 9.x readiness
[guideline](https://docs.google.com/document/d/1W7csjn6QYjrBjmbXMzSz_JUD4KcmWz8jTTtAWFwgUJM/edit?tab=t.0#heading=h.tui2zvb9gca6)

#### How to test deprecated APIs?

1. Run API
https://www.elastic.co/guide/en/security/current/signals-migration-api.html
2. Observe warning deprecation on Kibana Upgrade page

<img width="2540" alt="Screenshot 2024-12-03 at 10 43 59"
src="https://github.com/user-attachments/assets/24fcebb9-2d31-4ca3-a0dc-4ed7861d26a2">

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 3d46ead)
vitaliidm added a commit that referenced this issue Dec 11, 2024
@vitaliidm @rylnd
[Security Solution][Detection Engine] adds legacy siem signals teleme…
8821e03 
…try (#202671)

## Summary

- partly addresses #195523
- adds snapshot telemetry that shows number of legacy siem signals and
number of spaces they are in
- while working on PR, discovered and fixed few issues in APIs
- get migration status API did not work correctly with new `.alerts-*`
indices, listing them as outdated
- finalize migration API did account for spaces, when adding alias to
migrated index
- remove migration API failed due to lack of permissions to removed
migration task from `.tasks` index

### How to test

#### How to create legacy siem index?

run script that used for FTR tests

```bash
node scripts/es_archiver --kibana-url=http://elastic:changeme@localhost:5601 --es-url=http://elastic:changeme@localhost:9200 load x-pack/test/functional/es_archives/signals/legacy_signals_index

```
These would create legacy siem indices. But be aware, it might break
Kibana .alerts indices creation. But sufficient for testing


#### How to test snapshot telemetry

Snapshot
For snapshot telemetry use
[API](https://docs.elastic.dev/telemetry/collection/snapshot-telemetry#telemetry-usage-payload-api)
call
OR
Check snapshots in Kibana adv settings -> Global Settings Tab -> Usage
collection section -> Click on cluster data example link -> Check
`legacy_siem_signals ` fields in flyout

<details>
<summary> Snapshot telemetry </summary>


<img width="2549" alt="Screenshot 2024-12-03 at 13 08 03"
src="https://github.com/user-attachments/assets/28ffe983-01c7-4435-a82a-9a968d32d5e0">


 </details>

---------

Co-authored-by: Ryland Herrick <[email protected]>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Dec 11, 2024
@vitaliidm
[Security Solution][Detection Engine] adds legacy siem signals teleme…
4740371 
…try (elastic#202671)

## Summary

- partly addresses elastic#195523
- adds snapshot telemetry that shows number of legacy siem signals and
number of spaces they are in
- while working on PR, discovered and fixed few issues in APIs
- get migration status API did not work correctly with new `.alerts-*`
indices, listing them as outdated
- finalize migration API did account for spaces, when adding alias to
migrated index
- remove migration API failed due to lack of permissions to removed
migration task from `.tasks` index

### How to test

#### How to create legacy siem index?

run script that used for FTR tests

```bash
node scripts/es_archiver --kibana-url=http://elastic:changeme@localhost:5601 --es-url=http://elastic:changeme@localhost:9200 load x-pack/test/functional/es_archives/signals/legacy_signals_index

```
These would create legacy siem indices. But be aware, it might break
Kibana .alerts indices creation. But sufficient for testing

#### How to test snapshot telemetry

Snapshot
For snapshot telemetry use
[API](https://docs.elastic.dev/telemetry/collection/snapshot-telemetry#telemetry-usage-payload-api)
call
OR
Check snapshots in Kibana adv settings -> Global Settings Tab -> Usage
collection section -> Click on cluster data example link -> Check
`legacy_siem_signals ` fields in flyout

<details>
<summary> Snapshot telemetry </summary>

<img width="2549" alt="Screenshot 2024-12-03 at 13 08 03"
src="https://github.com/user-attachments/assets/28ffe983-01c7-4435-a82a-9a968d32d5e0">

 </details>

---------

Co-authored-by: Ryland Herrick <[email protected]>
(cherry picked from commit 8821e03)
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@vitaliidm @kibanamachine @CAWilson94
[Security Solution][Detection Engine] deprecates siem signals migrati…
f257939 
…on APIs (elastic#202662)

## Summary

 - addresses part of elastic#195523
- deprecates
https://www.elastic.co/guide/en/security/current/signals-migration-api.html
APIs according to internal 9.x readiness
[guideline](https://docs.google.com/document/d/1W7csjn6QYjrBjmbXMzSz_JUD4KcmWz8jTTtAWFwgUJM/edit?tab=t.0#heading=h.tui2zvb9gca6)
 
#### How to test deprecated APIs?

1. Run API
https://www.elastic.co/guide/en/security/current/signals-migration-api.html
2. Observe warning deprecation on Kibana Upgrade page

<img width="2540" alt="Screenshot 2024-12-03 at 10 43 59"
src="https://github.com/user-attachments/assets/24fcebb9-2d31-4ca3-a0dc-4ed7861d26a2">

---------

Co-authored-by: kibanamachine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@vitaliidm @rylnd @CAWilson94
[Security Solution][Detection Engine] adds legacy siem signals teleme…
04a6bcc 
…try (elastic#202671)

## Summary

- partly addresses elastic#195523
- adds snapshot telemetry that shows number of legacy siem signals and
number of spaces they are in
- while working on PR, discovered and fixed few issues in APIs
- get migration status API did not work correctly with new `.alerts-*`
indices, listing them as outdated
- finalize migration API did account for spaces, when adding alias to
migrated index
- remove migration API failed due to lack of permissions to removed
migration task from `.tasks` index

### How to test

#### How to create legacy siem index?

run script that used for FTR tests

```bash
node scripts/es_archiver --kibana-url=http://elastic:changeme@localhost:5601 --es-url=http://elastic:changeme@localhost:9200 load x-pack/test/functional/es_archives/signals/legacy_signals_index

```
These would create legacy siem indices. But be aware, it might break
Kibana .alerts indices creation. But sufficient for testing


#### How to test snapshot telemetry

Snapshot
For snapshot telemetry use
[API](https://docs.elastic.dev/telemetry/collection/snapshot-telemetry#telemetry-usage-payload-api)
call
OR
Check snapshots in Kibana adv settings -> Global Settings Tab -> Usage
collection section -> Click on cluster data example link -> Check
`legacy_siem_signals ` fields in flyout

<details>
<summary> Snapshot telemetry </summary>


<img width="2549" alt="Screenshot 2024-12-03 at 13 08 03"
src="https://github.com/user-attachments/assets/28ffe983-01c7-4435-a82a-9a968d32d5e0">


 </details>

---------

Co-authored-by: Ryland Herrick <[email protected]>
@vitaliidm
Copy link
Contributor

Telemetry was added in #202671
API deprecated in #202662

Docs ticket: elastic/security-docs#6284

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vitaliidm vitaliidm

Labels
Team:Detection Engine Security Solution Detection Engine Area Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v9.0.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@yctercero @elasticmachine @vitaliidm