Remove non standard header findings and add deprecated headers findings

[noamblitz]

Changes

Remove findings from non standard headers as stated in #2731

Issue link

Closes #2731

Demo

Please add some proof in the form of screenshots or screen recordings to show (off) new functionality, if there are interesting new features for end-users.

QA notes

Please add some information for QA on how to test the newly created code.

---

Code Checklist

• All the commits in this PR are properly PGP-signed and verified.
• This PR only contains functionality relevant to the issue.
• I have written unit tests for the changes or fixes I made.
• I have checked the documentation and made changes where necessary.
• I have performed a self-review of my code and refactored it to the best of my abilities.

• Tickets have been created for newly discovered issues.
• For any non-trivial functionality, I have added integration and/or end-to-end tests.
• I have informed others of any required .env changes files if required and changed the .env-dist accordingly.
• I have included comments in the code to elaborate on what is not self-evident from the code itself, including references to issues and discussions online, or implicit behavior of an interface.

---

Checklist for code reviewers:

Copy-paste the checklist from the docs/source/templates folder into your comment.

---

Checklist for QA:

Copy-paste the checklist from the docs/source/templates folder into your comment.

[ammar92]

No remarks

[underdarknl]

Seeing that these headers are not longer required, they are however still not-standard.
Maybe we should introduce a finding for those (if don't already have one) and raise that for all of these non-standard headers.
The following not standard headers would warrant this I'd think:

• X-Forwarded-For
• X-Forwarded-Host
• X-Forwarded-Proto
• X-DNS-Prefetch-Control
• X-Robots-Tag

There's also the deprecated headers that might warrant a similar warning

• Pragma
• Warning

[underdarknl]

Nice! looks good to me.

[stephanie0x00]

As discussed with @noamblitz the title of the finding is renamed to "KAT-NONSTANDARD-HEADERS" for all these headers to include the headers listed below. In the future it would be nice to have these split into two categories: Non-standard findings and deprecated findings.

Seeing that these headers are not longer required, they are however still not-standard. Maybe we should introduce a finding for those (if don't already have one) and raise that for all of these non-standard headers. The following not standard headers would warrant this I'd think:

* [X-Forwarded-For](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For)

* [X-Forwarded-Host](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Host)

* [X-Forwarded-Proto](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Proto)

* [X-DNS-Prefetch-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-DNS-Prefetch-Control)

* [X-Robots-Tag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Robots-Tag)

There's also the deprecated headers that might warrant a similar warning

* [Pragma](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Pragma)

* [Warning](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Warning)

[stephanie0x00]

Checklist for QA:

• I have checked out this branch, and successfully ran a fresh make reset.
• I confirmed that there are no unintended functional regressions in this branch:
  • I have managed to pass the onboarding flow
  • Objects and Findings are created properly
  • Tasks are created and completed properly
• I confirmed that the PR's advertised feature or hotfix works as intended.
• I checked the logs for errors and/or warnings and made issues where necessary

What works:

Looks good. Manually added the various deprecated and non-standard hosts. They are now all gathered under the'Non-standard header' finding. Created a new ticket to split these into non-standard and deprecated headers, as that would be more ideal. #3188

What doesn't work:

n/a

Bug or feature?:

n/a