feat: support loading init img from url

[wseagar]

This isn't the best implementation but it's a start.

Some problems with this approach is:

• Mixing paths/urls isn't great and if this is directly exposed from a web interface someone could do a path traversal
• If looping with the same init_image it'll be downloaded N times.

Another option but more work for the caller is to allow a user to supply a PIL image in the ImaginePrompt

[brycedrennan]

yeah I need to think about this for a minute. I'm leaning towards supporting an input PIL image. Having a web request be possible inside the library doesn't feel quite right

[brycedrennan]

on the other hand, just putting in a url would be so convenient... but yeah seems like a not clean api and also less secure. Trying to think how to have it both ways.

[wseagar]

If we want to make it super easy to use then just exposing a raw PIL image interface is not ideal. Here's an idea to get the flexibility I need but still keeping it easy to use.

A new class ImagineImage, change init_image to take a ImagineImage remove init_image_strength and make that a member property

Static Methods (or however you do this in python):
ImagineImage.from_url("https://website.com/image.png")
ImagineImage.from_file("./image.png")
ImagineImage.from_PIL(pil_image)

Also ideally we can share one ImagineImage across prompts meaning that some of this work that's currently done like resizing etc can live in this class and only be done once.

Usage:

init_image = ImagineImage.from_url("https://website.com/image.png", init_image_strength=0.75)

prompts = [
    ImaginePrompt("a scenic landscape", seed=1, init_image=init_image),
    ImaginePrompt("a bowl of fruit",  init_image=init_image),
]
for result in imagine(prompts):
    # do something
    result.save("my_image.jpg")

[brycedrennan]

Yes. I was thinking something similar although I think it needs to lazy load the images or otherwise you end up consuming lots of memory when the queue of prompts is large.

[wseagar]

Yes. I was thinking something similar although I think it needs to lazy load the images or otherwise you end up consuming lots of memory when the queue of prompts is large.

Understand your point but probably an optimization that can come later, remember when you call imagine we do load at least 7GB from the internet/into memory. I don't think anyone will ever queue up enough prompts to even get within an order of magnitude of the current memory/network usage.

[brycedrennan]

Really rough code here. I don't like naming and maybe interface. but i think the idea works

class ImagineImage:
    def __init__(self, filepath=None, url=None, pil_img=None):
        self.filepath = filepath
        # etc
        # add validation checks
   
    def as_pil_img(self):
        if self.pil_img is not None:
            return self.pil_img
        if self.url:
            self.pil_img = load_img_from_url(self.url)
        elif self.filepath:
            self.pil_img = load_Img_from_path(self.filepath)
        return self.pil_img

[brycedrennan]

Especially because we're already using lots of memory we should be careful about using more.

[brycedrennan]

I'm on the fence about where the image strength argument should exist. Especially since I'm playing with alternative img2img methods which may take different arguments