Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

increase index.mapping.nested_fields.limit in opensearch index template #180

Closed
mmguero opened this issue Apr 4, 2023 · 0 comments
Closed
Assignees
Labels
bug Something isn't working opensearch Relating to Malcolm's use of OpenSearch regression It worked at one point...

Comments

@mmguero
Copy link
Collaborator

mmguero commented Apr 4, 2023

Malcolm's opensearch template uses ECS to compose itself, in addition to our custom zeek fields.

Recent changes (I think to ECS itself) have caused us to exceed the default max for nested fields, which is 60. I've increased it to 250 (see mmguero-dev/Malcolm@f0f81fe).

@mmguero mmguero added bug Something isn't working opensearch Relating to Malcolm's use of OpenSearch regression It worked at one point... labels Apr 4, 2023
@mmguero mmguero self-assigned this Apr 4, 2023
@mmguero mmguero added this to Malcolm Apr 4, 2023
@mmguero mmguero changed the title increase index.mapping.nested_fields.limit increase index.mapping.nested_fields.limit in opensearch index template Apr 4, 2023
@mmguero mmguero moved this to Done in Malcolm Apr 4, 2023
@mmguero mmguero closed this as completed Apr 4, 2023
mmguero added a commit that referenced this issue Apr 5, 2023
Malcolm v23.04.0 is a release with enhancements, component version updates and bug fixes.

IMPORTANT NOTE: In March 2023 Docker Inc. announced its decision to sunset the "Docker Free Team" plan, which prompted us to decide to migrate away from Docker Hub to the Github Container Registry or "ghcr" (see #163). Due to public backlash, Docker Inc. reversed its decision. However, the Malcolm project will continue with the decision to use GHCR beginning with this release (Malcolm v23.04.0) and moving forward. If you're updating an existing instance of Malcolm, it's recommended that you back up your `docker-compose.yml` and `docker-compose-standalone.yml` files, replace them with the ones from this release and re-run `./scripts/install.py --configure` to ensure that you're pointing at the latest images (this is actually always good practice when moving to a new release of Malcolm).

v23.03.0...v23.04.0

* Enhancements
    - autostart `install.py --configure` on Malcolm ISO first boot (#157)
    - clarify information about auth_setup's use of external OpenSearch connections (#160)
    - migrate away from DockerHub container registry (#163)
    - give easier option for transferring SSL client files from Malcolm to forwarder (#177)
        + added `tx-rx-secure.sh` script as wrapper around [croc](https://github.com/schollz/croc) automatically creating and using a local-only relay

* Component version updates
    - [Zeek v5.2.0](https://github.com/zeek/zeek/releases) (#161)
    - [fluent bit v2.0.10](https://fluentbit.io/announcements/v2.0.10/)
    - [NetBox v3.4.7](https://github.com/netbox-community/netbox/releases/tag/v3.4.7)

* Fixes
    - XFCE4's "save session on exit" causes conflict with Hedgehog kiosk mode if firefox instance is started upon session restore (#164)
    - docker-compose move from go-yaml/v3 breaks Malcolm's docker-compose YAML files (#178, docker/compose#10411)
    - increase index.mapping.nested_fields.limit in opensearch index template (#180)
mmguero added a commit to cisagov/Malcolm that referenced this issue Apr 5, 2023
Malcolm v23.04.0 is a release with enhancements, component version updates and bug fixes.

IMPORTANT NOTE: In March 2023 Docker Inc. announced its decision to sunset the "Docker Free Team" plan, which prompted us to decide to migrate away from Docker Hub to the Github Container Registry or "ghcr" (see idaholab#163). Due to public backlash, Docker Inc. reversed its decision. However, the Malcolm project will continue with the decision to use GHCR beginning with this release (Malcolm v23.04.0) and moving forward. If you're updating an existing instance of Malcolm, it's recommended that you back up your `docker-compose.yml` and `docker-compose-standalone.yml` files, replace them with the ones from this release and re-run `./scripts/install.py --configure` to ensure that you're pointing at the latest images (this is actually always good practice when moving to a new release of Malcolm).

v23.03.0...v23.04.0

* Enhancements
    - autostart `install.py --configure` on Malcolm ISO first boot (idaholab#157)
    - clarify information about auth_setup's use of external OpenSearch connections (idaholab#160)
    - migrate away from DockerHub container registry (idaholab#163)
    - give easier option for transferring SSL client files from Malcolm to forwarder (idaholab#177)
        + added `tx-rx-secure.sh` script as wrapper around [croc](https://github.com/schollz/croc) automatically creating and using a local-only relay

* Component version updates
    - [Zeek v5.2.0](https://github.com/zeek/zeek/releases) (idaholab#161)
    - [fluent bit v2.0.10](https://fluentbit.io/announcements/v2.0.10/)
    - [NetBox v3.4.7](https://github.com/netbox-community/netbox/releases/tag/v3.4.7)

* Fixes
    - XFCE4's "save session on exit" causes conflict with Hedgehog kiosk mode if firefox instance is started upon session restore (idaholab#164)
    - docker-compose move from go-yaml/v3 breaks Malcolm's docker-compose YAML files (idaholab#178, docker/compose#10411)
    - increase index.mapping.nested_fields.limit in opensearch index template (idaholab#180)
@mmguero mmguero moved this from Done to Released in Malcolm Apr 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working opensearch Relating to Malcolm's use of OpenSearch regression It worked at one point...
Projects
Status: Released
Development

No branches or pull requests

1 participant