Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YR-Fix-documentation/FeedRecordeduture/XSUP-29069 #31015

Merged
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ configuration:
required: false
- display: ""
name: feedExpirationPolicy
defaultvalue: suddenDeath
defaultvalue: indicatorType
type: 17
options:
- never
Expand Down Expand Up @@ -195,7 +195,7 @@ script:
- contextPath: RecordedFutureFeed.RiskRule.Criticality
description: The risk rule criticality.
type: String
dockerimage: demisto/python3:3.10.13.75921
dockerimage: demisto/python3:3.10.13.80593
feed: true
runonce: false
script: '-'
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
## Notes
1. It is highly recommended to not create multiple instances of the same indicator type, even when fetching both from fusion and connectApi. Creating multiple instances with same indicator type will lead to duplicate indicators being fetched which can cause performance issues for the server.
2. Recommended interval for fetching indicators according to Recorded Future documentation:
2. Because of restrictions on the API side, it is strongly advisable to maintain the number of indicators below 100,000 per instance. Exceeding this limit may result in unforeseen expiration of indicators beyond that threshold.
3. Recommended interval for fetching indicators according to Recorded Future documentation:

| **Indicator Type** | **Recommended Fetch Interval**
| --- | --- |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ This integration was integrated and tested with Recorded Future Feed
4. Click **Test** to validate the URLs, token, and connection.
## Notes
1. It is highly recommended to not create multiple instances of the same indicator type, even when fetching both from fusion and connectApi. Creating multiple instances with same indicator type will lead to duplicate indicators being fetched which can cause performance issues for the server.
2. Recommended interval for fetching indicators according to Recorded Future documentation:
2. Because of restrictions on the API side, it is strongly advisable to maintain the number of indicators below 100,000 per instance. Exceeding this limit may result in unforeseen expiration of indicators beyond that threshold.
3. Recommended interval for fetching indicators according to Recorded Future documentation:

| **Indicator Type** | **Recommended Fetch Interval**
| --- | --- |
Expand All @@ -38,8 +39,8 @@ This integration was integrated and tested with Recorded Future Feed
| Hash | 1 Day. |
| URL | 2 Hours. |
| Vulnerability | 2 Hours. |
3. Per instance configuration, it is recommended to use either `connectApi` or `fusion` as a service for chosen indicator type, and not both, as most of the data between both services is duplicated.
4. The feed size can be change according to the chosen indicator type:
4. Per instance configuration, it is recommended to use either `connectApi` or `fusion` as a service for chosen indicator type, and not both, as most of the data between both services is duplicated.
5. The feed size can be change according to the chosen indicator type:
RosenbergYehuda marked this conversation as resolved.
Show resolved Hide resolved
- IP - As of September 24, 2020, this risk list includes over 5.9k records.
- Domain - Due to additional sources of malicious domains added recently, the number of high risk domains collected and analyzed in Recorded Future has dramatically increased. As a result, now cap this risklist at 100,000 domains.
- Hash - In the second half of 2018, improvements and enhancements to our hash collection and analysis processes led to a dramatic increase in risky hashes that meet the above criteria. As a result, now cap this risklist at 100,000 hashes.
Expand Down Expand Up @@ -92,3 +93,8 @@ To limit the 'connectApi' service indicators list.
| RecordedFutureFeed.RiskRule.Name | String | The risk rule name. |
| RecordedFutureFeed.RiskRule.Description | String | The risk rule description. |
| RecordedFutureFeed.RiskRule.Criticality | String | The risk rule criticality. |


### Troubleshooting

If indicators expire unexpectedly, please ensure that the feed is not receiving more than 100,000 indicators per fetch. As it is discouraged to use "large" as a risk rule, we currently receive indicators in a single large CSV file containing up to 100,000 indicators. If Recorded Future has additional indicators to send, the CSV will be sorted in descending order based on the highest score. Consequently, some indicators may not pass through, leading to their expiration, particularly if the expiration is configured as "When removed from the feed" and they were present in our system from previous fetches.
RosenbergYehuda marked this conversation as resolved.
Show resolved Hide resolved
7 changes: 7 additions & 0 deletions Packs/FeedRecordedFuture/ReleaseNotes/1_0_31.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@

#### Integrations

##### Recorded Future RiskList Feed
- Updated the integration description to include a notice advising users against fetching 100,000 or more indicators per instance.
JasBeilin marked this conversation as resolved.
Show resolved Hide resolved

- Updated the Docker image to: *demisto/python3:3.10.13.80593*.
2 changes: 1 addition & 1 deletion Packs/FeedRecordedFuture/pack_metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "Recorded Future Feed",
"description": "Ingests indicators from Recorded Future feeds into Demisto.",
"support": "xsoar",
"currentVersion": "1.0.30",
"currentVersion": "1.0.31",
"author": "Cortex XSOAR",
"url": "https://www.paloaltonetworks.com/cortex",
"email": "",
Expand Down
Loading