Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

integration:cisco_secure_endpoint Fix noisy error log entry when pagination completes #10527

Open
agmic opened this issue Jul 18, 2024 · 1 comment
Labels
bug Something isn't working, use only for issues Integration:cisco_secure_endpoint Cisco Secure Endpoint Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations]

Comments

@agmic
Copy link
Contributor

agmic commented Jul 18, 2024

The Cisco Secure Endpoint integration collects event data via httpjson. The integration throws an error when it hits the last page, and this gets logged in the Elastic Agent error log with something similar to:

error processing response: template: :1:17: executing "" at <.last_response.body.metadata.links.next>: map has no entry for key "next"
This fills up the Elastic Agent dashboard with agent errors, as it is raised every time the Secure Endpoint data is polled (every 2 minutes/default) and the noise makes it more difficult to find actual errors.

I initially raised this in (elastic/beats#38291) and a new way of handling pagination was merged in (elastic/beats#39929)

The integration now needs to be updated to incorporate this new way of paginating.

existing code
https://github.com/elastic/integrations/blob/129752c5db99035ccc6c8f8e70606e54268b392a/packages/cisco_secure_endpoint/data_stream/event/agent/stream/httpjson.yml.hbs#L36C1-L41C1

response.pagination:
- set:
    target: url.value
    value: '[[ .last_response.body.metadata.links.next ]]'
    fail_on_template_error: true

example from 39929, may need to be adapted

response.pagination_until: '[[not (index .last_response.body.meta "cursor")]]'
response.pagination:
- set:
    target: url.params.cursor
    value: '[[.last_response.body.meta.cursor]]
@jamiehynds jamiehynds added Integration:cisco_secure_endpoint Cisco Secure Endpoint Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations] labels Jul 18, 2024
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@agmic agmic changed the title integration:cisco_secure_endpoint Fix noisy error log entry when pagination fails integration:cisco_secure_endpoint Fix noisy error log entry when pagination completes Aug 5, 2024
@narph narph added the bug Something isn't working, use only for issues label Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working, use only for issues Integration:cisco_secure_endpoint Cisco Secure Endpoint Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations]
Projects
None yet
Development

No branches or pull requests

4 participants