Feature request: Support for the file:// scheme?

[porglezomp]

I'm unsure if reqwest is necessarily the right place to put this since I could probably easily implement it in a very small wrapper, but it would be nice to have support for reading files with the file:// scheme so that I can use it for testing without firing up an HTTP server.

If this is out of scope for reqwest, I completely understand.

[seanmonstar]

I have wondered this myself. On one hand, there is std::fs::File which is pretty simple to use. And pretty much all of the HTTP stuff doesn't apply when just reading a file (headers are worthless).

At the same time, curl offers it, and people seem to like that. Is that a good thing? Or is it a mistake that curl just has to live with forever now?

[clarfonthey]

IMHO Url::to_file_path is pretty much all we need: https://docs.rs/url/1.5.1/url/struct.Url.html#method.to_file_path

[seanmonstar]

@clarcharr "all we need" for what? To implement this feature? Or do you mean the feature isn't needed because "all we need" is a Path?

[clarfonthey]

Sorry, that was actually very vague.

I meant that people wishing to support file:// schemes could just call that method on the URL and then separate out file paths if necessary. Of course, this eliminates the boilerplate of parsing the URL that reqwest removes, and that might be a reason to add this feature, but I dunno.

[mathstuf]

What does it mean to POST to a file:// URL? Would it just emulate a 405 Method Not Supported error for any non-GET call? What would HEAD return?

[porglezomp]

I think the answers to lots of these questions are probably best resolved by seeing what curl does, but also I'm not longer sure that this is particularly necessary, checking the scheme and using Url::to_file_path seems like it'd do most of what I want.

[mathstuf]

Huh, looks like HEAD returns stat info:

Content-Length: 123
Accept-ranges: bytes
Last-Modified: Thu, 08 May 2014 19:44:22 GMT

POST just has an automatic closed connection. GET returns the contents. Basically, it appears as though curl is acting like a static file webserver to itself…

[crlf0710]

Personally I think since reqwest description is "An easy and powerful Rust HTTP Client", this feature is out of scope. Would love to see "reqwest_ftp: An easy and powerful Rust FTP Client" and series of libraries like that (file schema handling is one of them) though.

[kykc]

I think that should be done in the very least is: do not panic for file:/// or all other unsupported but completely valid URIs. For now if I don't filter user input on my end user can input valid but unsupported URI and crash the whole program. Example:

extern crate reqwest;

fn main() {
    let uri_str = "file:///etc/resolv.conf";
    let mut _resp = reqwest::get(uri_str);
    println!("Hello, world!");
}

Will panic:

kykc@submarine:/media/kykc/System/automatl/rust-sandbox/reqwest_uri_panic_reproduce$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.15s                                                                                                                                                                                 
     Running `target/debug/reqwest_uri_panic_reproduce`
thread 'reqwest-internal-sync-core' panicked at 'a parsed Url should always be a valid Uri: UriError(MissingAuthority)', libcore/result.rs:945:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.
thread 'main' panicked at 'core thread panicked', /home/kykc/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/reqwest-0.8.8/src/client.rs:487:17

Repo with this sample

[seanmonstar]

@kykc Oh my, yea that is a bug! I've filed #347, as it shouldn't panic.

[martinvonz]

std::fs::File makes sense if you know that you're going to use only file URLs. However, if you want to let the user specify a URL to fetch something from, it would be nice to be able to just pass that to Client::get().

[sotrh]

This feature would be extremely useful in my project. I want to use a "file://" url to load resources when the user is using the desktop version, and an "http://" when using Web Assembly.

[bryanlarsen]

Yes, this feature would be would be super useful for tests. I won't be using file:// URL's in production, but it's super useful for tests. If my code under test does something different for file:// URL's, then using file:// URL's in tests doesn't test all of my code. OTOH, if the support is in reqwest it does...