Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

implicit thread count when not specified in payloads + threads support in dns,network #4715

Merged
merged 4 commits into from
Feb 1, 2024
Merged

implicit thread count when not specified in payloads + threads support in dns,network #4715

merged 4 commits into from
Feb 1, 2024

Conversation

tarunKoyalwar
Copy link
Member

@tarunKoyalwar tarunKoyalwar commented Feb 1, 2024 •
edited
Loading

Proposed Changes

  • Add threads (i.e payload concurrency) support in dns and network protocols
  • When threads value is not specified it defaults to value of -c flag i.e template concurrency
  • Adds Optional ThreadSetter Callback in ExecuterOptions to allow overriding logic on demand from SDK
  • closes Set default threads for payloads #4711

Example run before change

$  cmdutil nuclei -id zip-backup-files -stats -u https://scanme.sh                         1 ↵

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.1.8

		projectdiscovery.io

[INF] Current nuclei version: v3.1.8 (latest)
[INF] Current nuclei-templates version: v9.7.5 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 106
[INF] Templates loaded for current scan: 1
[INF] Executing 1 signed templates from projectdiscovery/nuclei-templates
[INF] Targets loaded for current scan: 1
[0:00:05] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 24/1440 (1%)
[0:00:10] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 48/1440 (3%)
[0:00:15] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 73/1440 (5%)
[0:00:20] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 98/1440 (6%)
[0:00:25] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 123/1440 (8%)
[0:00:30] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 147/1440 (10%)
[0:00:35] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 172/1440 (11%)
[0:00:40] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 197/1440 (13%)
[0:00:45] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 220/1440 (15%)
[0:00:50] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 245/1440 (17%)
[0:00:55] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 270/1440 (18%)
[0:01:00] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 295/1440 (20%)
[0:01:05] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 314/1440 (21%)
[0:01:10] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 339/1440 (23%)
[0:01:15] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 365/1440 (25%)
[0:01:20] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 390/1440 (27%)
[0:01:25] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 410/1440 (28%)
[0:01:30] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 435/1440 (30%)
[0:01:35] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 460/1440 (31%)
[0:01:40] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 485/1440 (33%)
[0:01:45] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 510/1440 (35%)
[0:01:50] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 536/1440 (37%)
[0:01:55] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 561/1440 (38%)
[0:02:00] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 586/1440 (40%)
[0:02:05] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 611/1440 (42%)
[0:02:10] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 636/1440 (44%)
[0:02:15] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 661/1440 (45%)
[0:02:20] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 686/1440 (47%)
[0:02:25] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 711/1440 (49%)
[0:02:30] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 736/1440 (51%)
[0:02:35] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 761/1440 (52%)
[0:02:40] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 787/1440 (54%)
[0:02:45] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 812/1440 (56%)
[0:02:50] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 837/1440 (58%)
[0:02:55] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 862/1440 (59%)
[0:03:00] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 887/1440 (61%)
[0:03:05] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 912/1440 (63%)
[0:03:10] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 936/1440 (65%)
[0:03:15] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 961/1440 (66%)
[0:03:20] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 987/1440 (68%)
[0:03:25] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1013/1440 (70%)
[0:03:30] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1038/1440 (72%)
[0:03:35] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1063/1440 (73%)
[0:03:40] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1088/1440 (75%)
[0:03:45] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1113/1440 (77%)
[0:03:50] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1138/1440 (79%)
[0:03:55] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1162/1440 (80%)
[0:04:00] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1188/1440 (82%)
[0:04:05] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1212/1440 (84%)
[0:04:10] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1237/1440 (85%)
[0:04:15] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1262/1440 (87%)
[0:04:20] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1287/1440 (89%)
[0:04:25] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1310/1440 (90%)
[0:04:30] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1335/1440 (92%)
[0:04:35] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1360/1440 (94%)
[0:04:40] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1385/1440 (96%)
[0:04:45] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1410/1440 (97%)
[0:04:50] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1436/1440 (99%)
[INF] No results found. Better luck next time!
[0:04:51] | Templates: 1 | Hosts: 1 | RPS: 4 | Matched: 0 | Errors: 0 | Requests: 1440/1440 (100%)

------------------------------
Command: nuclei -id zip-backup-files -stats -u https://scanme.sh
Max RSS: 204 MB
Sys Time: 456.389µs
User Time: 976.316µs
Actual Time: 4m51.99417775s
Voluntary Context Switch (nvcsw): 135

Example run after change

$  cmdutil ./nuclei -id zip-backup-files -stats -u https://scanme.sh

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.1.8

		projectdiscovery.io

[INF] Current nuclei version: v3.1.8 (latest)
[INF] Current nuclei-templates version: v9.7.5 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 106
[INF] Templates loaded for current scan: 1
[INF] Executing 1 signed templates from projectdiscovery/nuclei-templates
[INF] Targets loaded for current scan: 1
[0:00:05] | Templates: 1 | Hosts: 1 | RPS: 287 | Matched: 0 | Errors: 0 | Requests: 1440/1440 (100%)
[0:00:10] | Templates: 1 | Hosts: 1 | RPS: 143 | Matched: 0 | Errors: 0 | Requests: 1440/1440 (100%)
[INF] No results found. Better luck next time!
[0:00:11] | Templates: 1 | Hosts: 1 | RPS: 125 | Matched: 0 | Errors: 0 | Requests: 1440/1440 (100%)

------------------------------
Command: ./nuclei -id zip-backup-files -stats -u https://scanme.sh
Max RSS: 224 MB
Sys Time: 883.114µs
User Time: 33.076µs
Actual Time: 12.621391459s
Voluntary Context Switch (nvcsw): 94

@tarunKoyalwar tarunKoyalwar self-assigned this Feb 1, 2024
@tarunKoyalwar tarunKoyalwar changed the title issue 4711 threads in payloads implicit thread count when not specified in payloads + threads support in dns,network Feb 1, 2024
@tarunKoyalwar tarunKoyalwar marked this pull request as ready for review February 1, 2024 15:35
@ehsandeep ehsandeep merged commit ead58f4 into dev Feb 1, 2024
9 of 12 checks passed
@ehsandeep ehsandeep deleted the issue-4711-threads-in-payloads branch February 1, 2024 20:35
@BrewTestBot BrewTestBot mentioned this pull request Feb 2, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ehsandeep ehsandeep ehsandeep approved these changes

Assignees

@tarunKoyalwar tarunKoyalwar

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Set default threads for payloads
2 participants
@tarunKoyalwar @ehsandeep