Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Improvements to wasm support and yew example #3

Merged
merged 1 commit into from
Dec 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 41 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,51 @@ As with the img2img function, you can use the StableDiffusionParameters struct t

Both the img2img and text2img functions return a Result type, which allows you to handle any errors that may occur during the image generation process.

## Examples
## WASM Support

This library supports WASM targets. To use it in a WASM project, you will need to use the library `wasm-bindgen-futures`:

```toml
[dependencies]
wasm-bindgen-futures = "0.4.33"
```

Invoke the library inside a spawn_local block:

```rust
use stable_diffusion::{StableDiffusionClient, StableDiffusionParameters};
use wasm_bindgen_futures::spawn_local;

let client = StableDiffusionClient::new("http://example.com".to_string());
let params = StableDiffusionParameters {
prompt: "Some prompt".to_string(),
..Default::default()
};
spawn_local(async move {
let response = client.text2img(params).await.unwrap();
});
```

You can view a full example using the Yew framework in the `examples/yew-app` folder. Open the folder in a terminal window and run `trunk serve` to start the development server. You can then view the example at `http://localhost:8080.

### Troubleshooting WASM

If you are using the library in a WASM project and are getting a HTTP 405 error you will need to change your server configuration to emit the CORS headers, or use a nginx server to proxy the API requests, applying the desired CORS headers.

You can view an example configuration in the `examples/yew-app` folder that starts a docker nginx server using a custom configuration file.

```bash
cd examples/yew-app && docker-compose up -d
```

Feel free to modify the configuration file to suit your needs.

## Other Examples

There are some examples available in the `examples` folder:


This example calls the text2img function and display the API response without any parsing:
This example calls the `text2img` function and display the API response without any parsing:

```bash
cargo run --example text2img
Expand Down
9 changes: 8 additions & 1 deletion examples/clap/img2img.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,14 @@ async fn main() {
match sd_api.img2img(params).await {
Ok(res) => {
let image = base64_to_png(res.images[0].clone());
save_image_to_disk(image, "image.png".to_string());
match image {
Ok(image) => {
save_image_to_disk(image, "image.png".to_string());
}
Err(err) => {
println!("{:?}", err);
}
}
}
Err(err) => {
println!("{:?}", err);
Expand Down
9 changes: 8 additions & 1 deletion examples/clap/text2img.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,14 @@ async fn main() {
match sd_api.text2img(params).await {
Ok(res) => {
let image = base64_to_png(res.images[0].clone());
save_image_to_disk(image, "image.png".to_string());
match image {
Ok(image) => {
save_image_to_disk(image, "image.png".to_string());
}
Err(err) => {
println!("{:?}", err);
}
}
}
Err(err) => {
println!("{:?}", err);
Expand Down
4 changes: 2 additions & 2 deletions examples/yew-app/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
yew = { version = "0.20.0", features = ["csr"] }
yew = "0.17.0"
sd_client = { path = "../../" }
wasm-bindgen-futures = "0.4.33"
wasm-bindgen = { version = "0.2.83", features = ["serde-serialize"] }
web-sys = { version = "0.3.60" }
dotenv = "0.15.0"
load-dotenv = "0.1.2"
load-dotenv = "0.1.2"
9 changes: 9 additions & 0 deletions examples/yew-app/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
version: '3'
services:
nginx:
image: nginx
ports:
- "80:80"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf

41 changes: 41 additions & 0 deletions examples/yew-app/nginx.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
http {
server {
listen 80;
location / {

if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';

add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';

add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';

add_header 'Access-Control-Max-Age' 86400;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
return 204; break;
}

if ($request_method = 'POST') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
if ($request_method = 'GET') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}

proxy_pass https://ba23c154a29d80bb.gradio.app;
# proxy_set_header Host $host:$server_port;
# proxy_set_header X-Real-IP $remote_addr;
}
}
}
events {
worker_connections 1024;
}
78 changes: 60 additions & 18 deletions examples/yew-app/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@ use yew::prelude::*;
use sd_client::stable_diffusion::{
StableDiffusionClient, StableDiffusionParameters,
};
use wasm_bindgen_futures::spawn_local;
use web_sys::console;
use load_dotenv::load_dotenv;

load_dotenv!();

pub async fn call_sd() -> String {


pub async fn call_sd(prompt: String) -> String {

let sd_api_url = std::env!("SD_API_URL").to_string();

let params = StableDiffusionParameters {
prompt: Some("A prompt".to_string()),
prompt: Some(prompt),
steps: Some(50),
..Default::default()
};
Expand All @@ -30,25 +30,67 @@ pub async fn call_sd() -> String {
}
}

#[function_component]
fn App() -> Html {
enum Msg {
CallSd(String),
SetImage(String),
}

struct App {
link: ComponentLink<Self>,
image: Option<String>,
}

impl Component for App {
type Message = Msg;
type Properties = ();

let app_title: String = "Yew Example".into();
fn create(_: Self::Properties, link: ComponentLink<Self>) -> Self {
Self {
link,
image: None,
}
}

// Spawn a task to call the SD API
// This is needed to call async api in a wasm context
spawn_local(async {
let res = call_sd().await;
console::log_1(&res.into());
});
fn update(&mut self, msg: Self::Message) -> ShouldRender {
match msg {
Msg::SetImage(image) => {
self.image = Some(image);
true
}
Msg::CallSd(prompt) => {
let link = self.link.clone();
wasm_bindgen_futures::spawn_local(async move {
let image = call_sd(prompt).await;
link.send_message(Msg::SetImage(image));
});
false
}
}
}

html! {
<div>
<p>{ &app_title }</p>
</div>
fn view(&self) -> Html {
let onclick = self.link.callback(|_| Msg::CallSd("A gorilla with a baseball hat".to_string()));
html! {
<div>
<button onclick=onclick>{ "Call Stable Diffusion" }</button>
<br />
<br />
{
if let Some(image) = &self.image {
html! {
<img src=format!("data:image/png;base64,{}", image) />
}
} else {
html! {}
}
}
</div>
}
}

fn change(&mut self, _: <Self as yew::Component>::Properties) -> bool { todo!() }
}

fn main() {
yew::Renderer::<App>::new().render();
yew::start_app::<App>();
}
29 changes: 22 additions & 7 deletions src/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,30 @@ pub mod image {

}

pub fn base64_to_png(base64_text: String) -> image::DynamicImage {
pub fn base64_to_png(base64_text: String) -> Result<image::DynamicImage, Box<dyn std::error::Error>> {
// Decode the base64 text
let decoded = base64::decode(base64_text).unwrap();

// Create a cursor from the decoded data
let cursor = Cursor::new(decoded);
match base64::decode(base64_text) {
Ok(decoded) => {
// Create a cursor from the decoded data
let cursor = Cursor::new(decoded);

// Read the image from the cursor
match image::load(cursor, ImageFormat::Png) {
Ok(image) => {
return Ok(image);
},
Err(err) => {
println!("Error loading image: {:?}", err);
return Err(Box::new(err));
}
}
},
Err(err) => {
println!("Error decoding base64: {:?}", err);
return Err(Box::new(err));
}
}

// Read the image from the cursor
image::load(cursor, ImageFormat::Png).unwrap()
}

pub fn save_image_to_disk(image: image::DynamicImage, file_name: String) {
Expand Down