FR: Support Data URIs

[fonsp]

Increasingly common in webworld is the use of Data URIs, which contain the data in the URL itself.

For example (from wikipedia), this URL points to an image of a red dot (copy and open it in a new tab):

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==

Currently, the data: protocol is not supported in Downloads.jl, nor the old built-in download, but it would be great to have it added!

[giordano]

If you're ok to special-case this kind of URL for the time being, you can use Base64.base64decode to get the image of that:

julia> using Base64

julia> write("/tmp/file.png", base64decode("iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg=="))
85

[StefanKarpinski]

Does libcurl support this?

[StefanKarpinski]

I don't really want to start hacking Downloads to support additional URI schemas in addition to what libcurl support in an ad hoc way, so the way to make forward progress on this is to make a feature request to libcurl to add support for this and then Downloads would inherit that support automatically.

[fonsp]

I implemented a wrapper around download that supports data URLs with testing here if anyone needs it!

https://github.com/fonsp/Pluto.jl/blob/a6b15d20fa761ad57739ac76caf890db8cb3a7e5/src/webserver/data_url.jl#L61-L95

[fonsp]

(I'm too intimidated by libcurl's mailing list to submit the feature request myself)

[StefanKarpinski]

Eh, maybe we can just special case this. I'll reopen for consideration.

[fonsp]

@StefanKarpinski Should I start a PR?

[StefanKarpinski]

Sure. I would handle it in the body of the request function. It's a little unclear to me what should happen if there is input or if the method is something other than GET. It needs to emulate what would happen with a different kind of URL, like a normal http:// url that serves the same data.

[KristofferC]

Any previous art in other languages with a download function?

[fonsp]

• fetch from JS supports data URLs 💕
• requests in Python does not
• WebClient in .NET does not

[fonsp]

About the request methods, it looks like fetch ignores body, and responds with the data unless HEAD is used.

> const url = "data:;base64,aGVsbG8="
undefined
> await (await fetch(url, {method: "GET"})).text()
"hello"
> await (await fetch(url, {method: "HEAD"})).text()
""
> await (await fetch(url, {method: "PUT"})).text()
"hello"
> await (await fetch(url, {method: "POST"})).text()
"hello"
> await (await fetch(url, {method: "POST", body: "world"})).text()
"hello"

(open a new tab and press F12 to run this code)

Looking at the spec, it looks like nothing is returned for HEAD or CONNECT (§4.1.19). I will search the spec for more data-url-related ideas :)