Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KB article with opensearch documentation #2572

Merged
merged 13 commits into from
Dec 6, 2024

Conversation

pburkholder
Copy link
Contributor

@pburkholder pburkholder commented Nov 27, 2024

Changes proposed in this pull request:

  • Adds a KB article with information on migrating to OpenSearch
  • Adds a bunch of supporting screenshots
  • Updates the original announcement with a link to the new KB article

Security Considerations

Safe. Documents how to use a customer-facing service, without going into sensitive implementation details.

@pburkholder pburkholder force-pushed the peterb/migrating-to-opensearch branch from 3b196e3 to 616b582 Compare December 6, 2024 16:39
@pburkholder pburkholder changed the title WIP for opensearch documentation KB article with opensearch documentation Dec 6, 2024
@pburkholder pburkholder marked this pull request as ready for review December 6, 2024 17:24
@pburkholder pburkholder requested a review from a team as a code owner December 6, 2024 17:24
@pburkholder
Copy link
Contributor Author

Issue:

  • I outlined the Kibana export screen-by-screen, probably too exhaustively, then was much more abridged in the OpenSearch import steps (since many of the steps are the same). We could either keep as-is, or trim the Kibana part, or expand the OpenSearch part. I think as-is is good enough, but I can be convinced otherwise.

The screenshot below show some of the major changes to the user interfaces, such as:
1. The drop down menus have moved from the upper left to the upper right
2. The "Top 5 values" for a field view is now an option to the right of the field, instead of a double-click
3. There are a lot more values gathered for container metrics
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is true, but it's not a user interface change

* If you share the same saved object across multiple orgs, you will need to import it into each of your orgs.
* Better handling of large log messages. Both Kibana/ELK and OpenSearch have a 32kb limit on message size. The older system dropped such messages from Kibana (although they were still retained in cold storage), the newer system keeps the first 32kb and discards the rest
* Truncated messages are tagged with `_messagetrimmed`.
* Extremely large log messages (over 1Gb) are trimmed and tagged `_logtrimmed` -- such message are probably indicative of a coding error in your application.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought 1Gb sounded high, so I must have mis-heard Jason. Fixing...

* Definitions of saved searches and visualizations are now isolated by OpenSearch tenants that correspond to Cloud.gov organizations.
* You no longer need to worry about choosing a globally unique name.
* If you share the same saved object across multiple orgs, you will need to import it into each of your orgs.
* Better handling of large log messages. Both Kibana/ELK and OpenSearch have a 32kb limit on message size. The older system dropped such messages from Kibana (although they were still retained in cold storage), the newer system keeps the first 32kb and discards the rest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently we're only truncating JSON messages that are over 32KB, but we probably should do it for all messages


---

## What's Changing in December 2024
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the title of the post ("Migrating to the OpenSearch Dashboard for Cloud.gov log") renders as second-level heading, for semantics it could make sense to render these as third level headings and so on, but not a blocking suggestion

@pburkholder pburkholder added this pull request to the merge queue Dec 6, 2024
Merged via the queue into main with commit 0165349 Dec 6, 2024
6 checks passed
@pburkholder pburkholder deleted the peterb/migrating-to-opensearch branch December 6, 2024 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants