Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Palo Alto | Update error to return error.data #2823

Merged
merged 3 commits into from
Sep 25, 2024
Merged

Conversation

cmcnally-r7
Copy link
Collaborator

Proposed Changes

Description

Describe the proposed changes:

  • Change return error -> return error.data

PR Requirements

Developers, verify you have completed the following items by checking them off:

Testing

Unit Tests

Review our documentation on generating and writing plugin unit tests

  • Unit tests written for any new or updated code

In-Product Tests

If you are an InsightConnect customer or have access to an InsightConnect instance, the following in-product tests should be done:

  • Screenshot of job output with the plugin changes
  • Screenshot of the changed connection, actions, or triggers input within the InsightConnect workflow builder

Style

Review the style guide

  • For dependencies, pin OS package and Python package versions
  • For security, set least privileged account with USER nobody in the Dockerfile when possible
  • For size, use the slim SDK images when possible: rapid7/insightconnect-python-3-38-slim-plugin:{sdk-version-num} and rapid7/insightconnect-python-3-38-plugin:{sdk-version-num}
  • For error handling, use of PluginException and ConnectionTestException
  • For logging, use self.logger
  • For docs, use changelog style
  • For docs, validate markdown with insight-plugin validate which calls icon_validate to lint help.md

Functional Checklist

  • Work fully completed
  • Functional
    • Any new actions/triggers include JSON test files in the tests/ directory created with insight-plugin samples
    • Tests should all pass unless it's a negative test. Negative tests have a naming convention of tests/$action_bad.json
    • Unsuccessful tests should fail by raising an exception causing the plugin to die and an object should be returned on successful test
    • Add functioning test results to PR, sanitize any output if necessary
      • Single action/trigger insight-plugin run -T tests/example.json --debug --jq
      • All actions/triggers shortcut insight-plugin run -T all --debug --jq (use PR format at end)
    • Add functioning run results to PR, sanitize any output if necessary
      • Single action/trigger insight-plugin run -R tests/example.json --debug --jq
      • All actions/triggers shortcut insight-plugin run --debug --jq (use PR format at end)

Assessment

You must validate your work to reviewers:

  1. Run insight-plugin validate and make sure everything passes
  2. Run the assessment tool: insight-plugin run -A. For single action validation: insight-plugin run tests/{file}.json -A
  3. Copy (insight-plugin ... | pbcopy) and paste the output in a new post on this PR
  4. Add required screenshots from the In-Product Tests section

@@ -71,14 +71,20 @@ def run(self, params={}, state={}, custom_config: dict = {}): # pylint: disable
f"A PluginException has occurred. Status code {status_code} returned. Error: {error}. "
f"Existing state: {existing_state}"
)
return [], existing_state, False, status_code, error
return [], existing_state, False, status_code, error.data
Copy link
Collaborator

@ekelly-r7 ekelly-r7 Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the context behind this change?

Is error.data a PluginException? I thought an Exception (or None) should be returned in run, not data of an exception?

Copy link
Collaborator Author

@cmcnally-r7 cmcnally-r7 Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error is of type Response. The error that Palo Alto returns is nested and to access the information within it we need to specify the data.

By returning just error, the sdk throws this error:
TypeError: Object of type Response is not JSON serializable

Causing it to return 500 and continuously retry despite the 401 initially returned

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we return error.data, this is what we get

{
    "body": {
        "has_more_pages": false,
        "log": "Connect: Connecting...\nrapid7/Palo Alto Cortex XDR:4.0.0. Step name: monitor_alerts\nNo previous alert time within state.\nApplying the following start time='2024-09-24T09:59:19.000000Z'.\nStarting to download alerts...\nQuery start time: 2024-09-24T09:59:19.000000Z, Query end time: 2024-09-25T09:59:19.000000Z\nA PluginException has occurred. Status code 401 returned. Error: An error occurred during plugin execution!\n\nAuthentication failed: invalid credentials. Please verify the credentials for your account and try again.. Existing state: {}\n",
        "meta": {},
        "output": [],
        "state": {},
        "status": "ok",
        "status_code": 401
    },
    "type": "task_event",
    "version": "v1"
}

Copy link
Collaborator

@ekelly-r7 ekelly-r7 Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may work in this specific scenario, but we shouldn't be returning response data where it should be an Exception

I think you'll need to wrap it in a PluginException

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay good to know, I can add that in now

@cmcnally-r7 cmcnally-r7 merged commit 2138f18 into develop Sep 25, 2024
8 of 10 checks passed
@cmcnally-r7 cmcnally-r7 deleted the palo_alto_500_fix branch September 25, 2024 10:34
cmcnally-r7 added a commit that referenced this pull request Sep 25, 2024
* Update error to return error.data

* Remove print statements

* Wrap it in a plugin exception
cmcnally-r7 added a commit that referenced this pull request Oct 2, 2024
* Palo Alto Cortex - Get Alerts: Fixed issue where trigger was failing due to empty or wrong typed output fields (#2535)

* [SOAR-17528] Palo alto Plugin Refresh (#2740)

* SDK Bump

* Reverting Version

* Updating help.md

* [SOAR-17529] Palo alto Cortex XDR Task (#2754)

* Logging to find credential headers

* test

* Fix logging

* Spec file refresh

* start parse custom config

* Successful API Call

* Update checksum

* Get Alerts

* API Call & Custom Config

* Some updates

* Change limit to 7500

* Return total count

* Get timestamp if max limit

* Custom testing for pagination

* Rough Draft for pagination & switching out get_alerts_two with a better method

* Tidy up

* Convert alert_time to right format

* Convert alert_time to right format

* Convert alert_time to right format

* Filters

* Organise methods & start making request in task.py

* Making api call in task.py

* Checkpoint

* Add headers retrievla in task

* It's working

* Loads of logging and notes

* Ryan checkpoint

* Ryan fibannoci murphy

* More logging

* Setting states

* Pagination

* Unit test framework

* More unit test progress

* Task Connection Test

* Black .

* Some changes and unit testing

* Removing a few logger lines/ print statements

* Removing monitor_incident_event task

* More refactoring

* Filter's added + Working

* custom config and sort by working

* Exception handling improved, still needs minor tweaks

* Unit tests

* Add url back

* Remove lots of comments in task | fix unit test

* Some abtracting | refactoring

* help.md/plugin.spec updated with SDK bump and examples added. Working through comments

* Exception handling working. removed unused constants/lines

* changing connection_version, CHECKSUM and reverting changed methods in apy.py to original (change not needed)

* Refactoring & making datetime/epoch conversions cleaner and easier

* Fix connection SCA

* Pagination progress

* trying to debug pagination error

* Fix SCA in action, refactor request to be in api.py

* Remove unused imports

* Improve comments

* Debugging

* Conversion custom_config bug fix

* Pagination issue resolved

* Drop pagination state

* Add len

* has_more_pages flag set to false after finished paginating

* Tidy a bit

* ;-;

* Now to fix up the state

* Strip Empty Spaces when Inputting API Keys

* Tidyed state management

* Rename search state variables

* Remove testing items and update comments

* Fix sca

* CHECKSUM FIX

* Unit Test Test / Possible workaround?

* Unit Test

* Change import path for a unit test

* Version change and unit test

* unit test fixing in progress

* strip removed for int

* Rename state variables, remove total count from state

* Fix pagination again

* Woops remove print statement

* Comment out unit test

* Remove last hash alert from drop state

* Change version back to 4.1.0, make connection version 4

* Improving state

* Some comments addressed

* Remove some repeated logging, add another safety check in api.py

* Remove all new unit tests for task

* Rename vars, improve logging

* Fix pagination again

* Streamline datetimes, improve logging

* Change version back to 4.0.0

* Fix the spec file

* Fixed Task Connection test (updated to new method and params)

* Tidy datetime more

* Done

* Remove authenitcation error in api.py, change post body logging to debug

* Datetime fixed

* Simplfy task connection test, remove some things in error handling

* Fix SCA

* Remove logger in connection

---------

Co-authored-by: rjmurray <[email protected]>

* Update cloud enabled (#2809)

* Reset offsets when changing lookback (#2811)

* Reset offsets when changing lookback

* Reset query endtime back to now

* Make it store the int not the datetime object

* Pop last search to

* Add default none incase of key error

* Update pop

* Palo Alto | Update error to return error.data (#2823)

* Update error to return error.data

* Remove print statements

* Wrap it in a plugin exception

* Palo Alto Cortex XDR - 17008 - Get Alerts: Fixed issue with triggers output validation (#2824)

* Rename response to new response

* Take headers away from initialisation (#2827)

* Palo Alto | Fix start time moving bug, Refactor dedupe logic (#2833)

* Add logging for final page, Ensure start & end time are fixed when paginating

* Add now time into get alerts

* End time and start time static again

* Refactor dedupe logic

* Remove custom config

* Update SDK

* Update checksum for task schema

---------

Co-authored-by: igorski-r7 <[email protected]>
Co-authored-by: rmurray-r7 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants