[SOAR-17529] Palo alto Cortex XDR Task #2754

cmcnally-r7 · 2024-08-30T14:36:46Z

Proposed Changes

Description

Task implementation for the monitor_alerts task. The task connection test and unit tests (started) has also been added and testing outputs have been provided below.

Describe the proposed changes:

Task Implementation for monitor_alerts
Task Connection Test added
Unit tests

`PluginValidator` is failing

The validator is failing because the connection_version is bumping from 2 -> 4 and the validator checks for +1 increments. We can ignore this in the context of this plugin and raise a ticket to fix this in the validators

Testing

Task

Called the Task locally and it pulls the first 100 logs successfully whilst also returning has_more_pages as true.

Task Connection Test

I tested by hitting the endpoint locally via Postman requests to ensure the correct exceptions show when inserting invalid data, and that when hitting the endpoint successfully the correct message shows.

Successful test:
Invalid URL
Invalid Secret Key/ Key ID

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/tasks/monitor_alerts/task.py

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/tasks/monitor_incident_events/task.py

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/tasks/monitor_alerts/task.py

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/tasks/monitor_incident_events/task.py

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/tasks/monitor_alerts/task.py

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/util/api.py

plugins/palo_alto_cortex_xdr/plugin.spec.yaml

plugins/palo_alto_cortex_xdr/help.md

plugins/palo_alto_cortex_xdr/plugin.spec.yaml

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/tasks/monitor_alerts/task.py

ekelly-r7 · 2024-09-20T08:59:09Z

Looks good, @ablakley-r7 comments around plugin exceptions may need to be addressed before merging

Validator is also failing on connection_version, is that expected?

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/connection/connection.py

cmcnally-r7 · 2024-09-20T09:33:16Z

Looks good, @ablakley-r7 comments around plugin exceptions may need to be addressed before merging

Validator is also failing on connection_version, is that expected?

Yeah we need to raise a ticket around this, since the validator is looking for connection_version + 1. Since v4.0.0 has not be released, and we're releasing this alongside it, we're keeping the connection version the same.

ablakley-r7 · 2024-09-20T12:02:20Z

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/util/api.py

+        url = f"{fqdn}{endpoint}"
+
+        request = requests.Request(method="post", url=url, headers=headers, json=post_body)
+        response = make_request(_request=request, timeout=60, exception_data_location=ResponseExceptionData.RESPONSE)


We should make a ticket to update the error handling to include the custom plugin exceptions for each status code.

plugins/palo_alto_cortex_xdr/icon_palo_alto_cortex_xdr/connection/connection.py

* Logging to find credential headers * test * Fix logging * Spec file refresh * start parse custom config * Successful API Call * Update checksum * Get Alerts * API Call & Custom Config * Some updates * Change limit to 7500 * Return total count * Get timestamp if max limit * Custom testing for pagination * Rough Draft for pagination & switching out get_alerts_two with a better method * Tidy up * Convert alert_time to right format * Convert alert_time to right format * Convert alert_time to right format * Filters * Organise methods & start making request in task.py * Making api call in task.py * Checkpoint * Add headers retrievla in task * It's working * Loads of logging and notes * Ryan checkpoint * Ryan fibannoci murphy * More logging * Setting states * Pagination * Unit test framework * More unit test progress * Task Connection Test * Black . * Some changes and unit testing * Removing a few logger lines/ print statements * Removing monitor_incident_event task * More refactoring * Filter's added + Working * custom config and sort by working * Exception handling improved, still needs minor tweaks * Unit tests * Add url back * Remove lots of comments in task | fix unit test * Some abtracting | refactoring * help.md/plugin.spec updated with SDK bump and examples added. Working through comments * Exception handling working. removed unused constants/lines * changing connection_version, CHECKSUM and reverting changed methods in apy.py to original (change not needed) * Refactoring & making datetime/epoch conversions cleaner and easier * Fix connection SCA * Pagination progress * trying to debug pagination error * Fix SCA in action, refactor request to be in api.py * Remove unused imports * Improve comments * Debugging * Conversion custom_config bug fix * Pagination issue resolved * Drop pagination state * Add len * has_more_pages flag set to false after finished paginating * Tidy a bit * ;-; * Now to fix up the state * Strip Empty Spaces when Inputting API Keys * Tidyed state management * Rename search state variables * Remove testing items and update comments * Fix sca * CHECKSUM FIX * Unit Test Test / Possible workaround? * Unit Test * Change import path for a unit test * Version change and unit test * unit test fixing in progress * strip removed for int * Rename state variables, remove total count from state * Fix pagination again * Woops remove print statement * Comment out unit test * Remove last hash alert from drop state * Change version back to 4.1.0, make connection version 4 * Improving state * Some comments addressed * Remove some repeated logging, add another safety check in api.py * Remove all new unit tests for task * Rename vars, improve logging * Fix pagination again * Streamline datetimes, improve logging * Change version back to 4.0.0 * Fix the spec file * Fixed Task Connection test (updated to new method and params) * Tidy datetime more * Done * Remove authenitcation error in api.py, change post body logging to debug * Datetime fixed * Simplfy task connection test, remove some things in error handling * Fix SCA * Remove logger in connection --------- Co-authored-by: rjmurray <[email protected]>

* Palo Alto Cortex - Get Alerts: Fixed issue where trigger was failing due to empty or wrong typed output fields (#2535) * [SOAR-17528] Palo alto Plugin Refresh (#2740) * SDK Bump * Reverting Version * Updating help.md * [SOAR-17529] Palo alto Cortex XDR Task (#2754) * Logging to find credential headers * test * Fix logging * Spec file refresh * start parse custom config * Successful API Call * Update checksum * Get Alerts * API Call & Custom Config * Some updates * Change limit to 7500 * Return total count * Get timestamp if max limit * Custom testing for pagination * Rough Draft for pagination & switching out get_alerts_two with a better method * Tidy up * Convert alert_time to right format * Convert alert_time to right format * Convert alert_time to right format * Filters * Organise methods & start making request in task.py * Making api call in task.py * Checkpoint * Add headers retrievla in task * It's working * Loads of logging and notes * Ryan checkpoint * Ryan fibannoci murphy * More logging * Setting states * Pagination * Unit test framework * More unit test progress * Task Connection Test * Black . * Some changes and unit testing * Removing a few logger lines/ print statements * Removing monitor_incident_event task * More refactoring * Filter's added + Working * custom config and sort by working * Exception handling improved, still needs minor tweaks * Unit tests * Add url back * Remove lots of comments in task | fix unit test * Some abtracting | refactoring * help.md/plugin.spec updated with SDK bump and examples added. Working through comments * Exception handling working. removed unused constants/lines * changing connection_version, CHECKSUM and reverting changed methods in apy.py to original (change not needed) * Refactoring & making datetime/epoch conversions cleaner and easier * Fix connection SCA * Pagination progress * trying to debug pagination error * Fix SCA in action, refactor request to be in api.py * Remove unused imports * Improve comments * Debugging * Conversion custom_config bug fix * Pagination issue resolved * Drop pagination state * Add len * has_more_pages flag set to false after finished paginating * Tidy a bit * ;-; * Now to fix up the state * Strip Empty Spaces when Inputting API Keys * Tidyed state management * Rename search state variables * Remove testing items and update comments * Fix sca * CHECKSUM FIX * Unit Test Test / Possible workaround? * Unit Test * Change import path for a unit test * Version change and unit test * unit test fixing in progress * strip removed for int * Rename state variables, remove total count from state * Fix pagination again * Woops remove print statement * Comment out unit test * Remove last hash alert from drop state * Change version back to 4.1.0, make connection version 4 * Improving state * Some comments addressed * Remove some repeated logging, add another safety check in api.py * Remove all new unit tests for task * Rename vars, improve logging * Fix pagination again * Streamline datetimes, improve logging * Change version back to 4.0.0 * Fix the spec file * Fixed Task Connection test (updated to new method and params) * Tidy datetime more * Done * Remove authenitcation error in api.py, change post body logging to debug * Datetime fixed * Simplfy task connection test, remove some things in error handling * Fix SCA * Remove logger in connection --------- Co-authored-by: rjmurray <[email protected]> * Update cloud enabled (#2809) * Reset offsets when changing lookback (#2811) * Reset offsets when changing lookback * Reset query endtime back to now * Make it store the int not the datetime object * Pop last search to * Add default none incase of key error * Update pop * Palo Alto | Update error to return error.data (#2823) * Update error to return error.data * Remove print statements * Wrap it in a plugin exception * Palo Alto Cortex XDR - 17008 - Get Alerts: Fixed issue with triggers output validation (#2824) * Rename response to new response * Take headers away from initialisation (#2827) * Palo Alto | Fix start time moving bug, Refactor dedupe logic (#2833) * Add logging for final page, Ensure start & end time are fixed when paginating * Add now time into get alerts * End time and start time static again * Refactor dedupe logic * Remove custom config * Update SDK * Update checksum for task schema --------- Co-authored-by: igorski-r7 <[email protected]> Co-authored-by: rmurray-r7 <[email protected]>

cmcnally-r7 force-pushed the palo_alto_task branch 2 times, most recently from 7242f11 to ce4945c Compare September 2, 2024 14:47

rmurray-r7 force-pushed the palo_alto_task branch 2 times, most recently from 825cab8 to 1ef0954 Compare September 2, 2024 15:54

rbowden-r7 force-pushed the develop branch from 55f59d5 to d54ef90 Compare September 3, 2024 09:44

cmcnally-r7 force-pushed the palo_alto_task branch from 6179bb7 to cd46e39 Compare September 3, 2024 13:26

igorski-r7 force-pushed the develop branch from c09db6e to c29d8d6 Compare September 5, 2024 07:44

cmcnally-r7 force-pushed the palo_alto_task branch 2 times, most recently from 43e6241 to 6c22f28 Compare September 6, 2024 10:19

rmurray-r7 changed the title ~~Palo alto task~~ [SOAR-17529] Palo alto Cortex XDR Task Sep 10, 2024

rmurray-r7 added the Work in Progress 🚧 PR is in progress label Sep 10, 2024

rmurray-r7 requested review from igorski-r7, rbowden-r7, ablakley-r7, dlaverty-r7 and joneill-r7 September 10, 2024 09:05