Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x-pack/filebeat/input/entityanalytics/provider/azuread: allow fine-grain control of API requests #36441

Merged
merged 3 commits into from
Sep 5, 2023

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Aug 29, 2023

Proposed commit message

This adds support for specifying which of users/devices to collect from the AzureAD API endpoints in order to reduce network costs for users who do not need a full set of entities.

The current change does not change the behaviour of device collection of registered owners and registered users; when the "devices" dataset is selected there user entities will still be collected as they are considered here as an attribute of the device, rather than a component of the users dataset.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@efd6 efd6 added enhancement Filebeat Filebeat Team:Security-External Integrations backport-skip Skip notification from the automated backport with mergify 8.11-candidate labels Aug 29, 2023
@efd6 efd6 self-assigned this Aug 29, 2023
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Aug 29, 2023
@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 29, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-09-05T21:26:15.517+0000

  • Duration: 74 min 46 sec

Test stats 🧪

Test Results
Failed 0
Passed 3168
Skipped 176
Total 3344

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@efd6 efd6 marked this pull request as ready for review August 29, 2023 06:44
@efd6 efd6 requested a review from a team as a code owner August 29, 2023 06:44
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@efd6
Copy link
Contributor Author

efd6 commented Aug 29, 2023

@jamiehynds Please take a look at the intended behaviour documented here.

if err != nil {
return updatedUsers, updatedDevices, err
}
p.logger.Debugf("Received %d devices from API", len(changedUsers))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
p.logger.Debugf("Received %d devices from API", len(changedUsers))
p.logger.Debugf("Received %d devices from API", len(changedDevices))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. That was incorrect in the previous code. Good catch.

case "", "all", "users", "devices":
default:
return errors.New("dataset must be 'all', 'users', 'devices' or empty")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional comment: If validation is required , then can we extend the unit test here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That feels like testing that addition works.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not super important may be..!

[float]
===== `dataset`

The datasets to collect from the API. This can be one of "all", "users" or "devices",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@efd6 this approach looks ok to me and we can set the default to 'all'. To be safe we can add a section to the integration docs to outline the three options, and the fact that we'll include registered users/owners as part of the device data collection.

FYI @piyush-elastic

@efd6 efd6 changed the title x-pack/filebeat/input/entityanalytics/azuread: allow fine-grain control of API requests x-pack/filebeat/input/entityanalytics/provider/azuread: allow fine-grain control of API requests Sep 3, 2023
@mergify

This comment was marked as outdated.

@mergify

This comment was marked as outdated.

efd6 added 2 commits September 5, 2023 19:02
…ol of API requests

This adds support for specifying which of users/devices to collect from
the AzureAD API endpoints in order to reduce network costs for users who
do not need a full set of entities.

The current change does not change the behaviour of device collection of
registered owners and registered users; when the "devices" dataset is
selected there user entities will still be collected as they are
considered here as an attribute of the device, rather than a component
of the users dataset.
Copy link
Contributor

@taylor-swanson taylor-swanson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach LGTM

I think testing is good enough, although an extra unit test case for config validation wouldn't hurt (could just be an invalid value to make sure that branch is counted in coverage).

@efd6 efd6 merged commit 220a4e2 into elastic:main Sep 5, 2023
8 checks passed
Scholar-Li pushed a commit to Scholar-Li/beats that referenced this pull request Feb 5, 2024
…ain control of API requests (elastic#36441)

This adds support for specifying which of users/devices to collect from
the AzureAD API endpoints in order to reduce network costs for users who
do not need a full set of entities.

The current change does not change the behaviour of device collection of
registered owners and registered users; when the "devices" dataset is
selected there user entities will still be collected as they are
considered here as an attribute of the device, rather than a component
of the users dataset.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.11-candidate backport-skip Skip notification from the automated backport with mergify enhancement Filebeat Filebeat
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants