Let users provide full URLs instead of just IPs for auxiliary / exploit http modules

[ErikWynter]

Basic Idea

I think it could be useful for http auxiliary and exploit http modules to let users provide not just a file/list of IPs via RHOSTS, but also a file/list containing full URLs via RHOSTS or an additional parameter (eg RURLS or TARGET_URLS) so that modules can set the correct settings for each target at runtime and users don't have to set the PORT, SSL and TARGETURI options separately for different targets. The point is to make mass scanning / exploitation possible for web apps.

Example

So let's say a user wants to run auxiliary/scanner/http/tomcat_mgr_login against several Tomcat instances that run on different ports at different web roots and with different SSL configurations. Currently, this requires configuring and running the module separately for each target, or writing a rather complex resource file, neither of which is ideal. It would be great if it would instead be possible to target all applications in a single run by providing a targets file such as the one show below:

http://192.168.1.1:8080/manager/html
https://192.168.1.2:9090/tomcat/manager/html
http://192.168.1.3:8081/manager/html
...

Then the module would read the targets from the file, and for each target it would automatically set the correct PORT, SSL and TARGETURI options.

What do you think?

I'm curious to know who else thinks it would be worth adding this, and if there are any reasons why it might not be such a great idea? I will not be able to take on any new projects for several months but I could have a go at this after the summer if people are open to it.

[jmartin-tech]

#13961. Testing and feedback on if this meets the needs would be greatly appreciated.

[jmartin-tech]

I am not sure if this can already support a file for targeting, however that might be a reasonable enhancement.

[ErikWynter]

Thanks for the reference! From what I can see, #13961 only supports a single URL at a time. What I have in mind is a full alternative to RHOSTS that can process multiple URLs, which can be provided directly or indeed via a file as you mentioned. It's great that the basis for this already exists within the framework. I probably won't have time to test this for a while, but I will look into this after the summer if no one else has picked it up by then.

[adfoster-r7]

or writing a rather complex resource file

#13961 helps simplify this a lot, but it's currently behind a feature flag. For example, a resource script would look like:

msf6 > cat resource.rc
[*] exec: cat resource.rc

features set RHOST_HTTP_URL true
<ruby>

module_name = "auxiliary/scanner/http/tomcat_mgr_login"
http_urls = [
   "http://127.0.0.1/manager/html",
   "http://127.0.0.1/tomcat/manager/html",
   "https://127.0.0.1/b"
]
mod = framework.modules.create(module_name)

http_urls.each do |url|
  mod.datastore['RHOST_HTTP_URL'] = url
  mod.run_simple(
     'LocalInput' => input,
     'LocalOutput' => output,
     'RunAsJob' => false,
  )
end

</ruby>

msf6 > resource resource.rc

After dogfooding the currently feature flagged RHOST_HTTP_URL option we've learnt a lot more about the drawbacks that it has, and the complexity of the existing RHOSTS implementation. There was also a page of possible implementations over in the wiki https://github.com/rapid7/metasploit-framework/wiki/RFC---Metasploit-URL-support, at the time this seemed like the best solution - but I don't think it's quite ready to enable the functionality by default

[adfoster-r7]

The initial work of #13961 was reverted here - #15241

There's now a new PR that's in progress that should support this exact use case #15253

use auxiliary/scanner/http/tomcat_mgr_login
set rhosts http://192.168.1.1:8080/manager/html https://192.168.1.2:9090/tomcat/manager/html http://192.168.1.3:8081/manager/html
run

It supports loading urls from a file too with:

echo "http://192.168.1.1:8080/manager/html\nhttps://192.168.1.2:9090/tomcat/manager/html\nhttp://192.168.1.3:8081/manager/html" > rhosts.txt
use auxiliary/scanner/http/tomcat_mgr_login
set rhosts file:rhosts.txt
run

It needs a bit more work / testing / dogfooding, but if you have any cycles to check it out / verify your use case works as expected - that would be great! 🎉

[adfoster-r7]

This should now be resolved by #15253

[ErikWynter]

It certainly is :)