Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using S3 presigned URL's with gr.Video or gr.Model3D fails #10375

Closed
1 task done
XnetLoL opened this issue Jan 16, 2025 · 1 comment · Fixed by #10406
Closed
1 task done

Using S3 presigned URL's with gr.Video or gr.Model3D fails #10375

XnetLoL opened this issue Jan 16, 2025 · 1 comment · Fixed by #10406
Labels
bug Something isn't working good first issue Good for newcomers python Backend-related issue (Python)

Comments

@XnetLoL
Copy link

XnetLoL commented Jan 16, 2025

Describe the bug

Description

Using presigned URLs with certain Gradio components like gr.Video or gr.Model3D fails, resulting in the following error:

OSError: [Errno 22] Invalid argument: 'C:\\Users\\<username>\\AppData\\Local\\Temp\\gradio\\<hashed_filename>\\.mp4?response-content-disposition=inline&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Security-Token=<truncated_token>&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=<truncated_credential>&X-Amz-Date=20250116T133649Z&X-Amz-Expires=1800&X-Amz-SignedHeaders=host&X-Amz-Signature=<truncated_signature>'

Steps to Reproduce

  1. Generate a presigned URL for a file stored in S3.
  2. Use the presigned URL as input to a Gradio component, such as gr.Video or gr.Model3D.
  3. Observe the error during file handling.

Expected Behavior

Gradio components should seamlessly handle presigned URLs and load the respective content without issues. For example this GitHub's RAW filepath does work: https://github.com/XnetLoL/test/raw/813ce5b531308d88f1a0c4256849fefe024c4d9e/breakdance.glb

Actual Behavior

The presigned URL, which includes query parameters, causes an invalid file path error when Gradio tries to process it.

Possible Cause

It seems Gradio attempts to interpret the entire presigned URL (including query parameters) as a file path, which results in an invalid argument error.

Environment

  • Gradio version: 5.12.0
  • OS: Windows 11
  • Python version: 3.13

Additional Notes

Maybe this issue could be resolved by properly handling URLs with query parameters in Gradio components.

Thanks.

Have you searched existing issues? 🔎

  • I have searched and found no existing issues

Reproduction

import gradio as gr

# Replace this with an actual presigned S3 URL
presigned_url = "https://your-bucket.s3.amazonaws.com/sample.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=..."

with gr.Blocks() as demo:
    gr.Markdown("### Video Test with Presigned URL")
    gr.Video(presigned_url )

if __name__ == "__main__":
    demo.launch()

Screenshot

No response

Logs

System Info

Gradio Environment Information:
------------------------------
Operating System: Windows
gradio version: 5.12.0
gradio_client version: 1.5.4

------------------------------------------------
gradio dependencies in your environment:

aiofiles: 23.2.1
anyio: 4.8.0
audioop-lts: 0.2.1
fastapi: 0.115.6
ffmpy: 0.5.0
gradio-client==1.5.4 is not installed.
httpx: 0.28.1
huggingface-hub: 0.27.1
jinja2: 3.1.5
markupsafe: 2.1.5
numpy: 2.2.1
orjson: 3.10.13
packaging: 24.2
pandas: 2.2.3
pillow: 10.4.0
pydantic: 2.10.4
pydub: 0.25.1
python-multipart: 0.0.20
pyyaml: 6.0.2
ruff: 0.8.6
safehttpx: 0.1.6
semantic-version: 2.10.0
starlette: 0.41.3
tomlkit: 0.12.0
typer: 0.15.1
typing-extensions: 4.12.2
urllib3: 2.3.0
uvicorn: 0.34.0
authlib; extra == 'oauth' is not installed.
itsdangerous; extra == 'oauth' is not installed.


gradio_client dependencies in your environment:

fsspec: 2024.12.0
httpx: 0.28.1
huggingface-hub: 0.27.1
packaging: 24.2
typing-extensions: 4.12.2
websockets: 12.0

Severity

I can work around it

@XnetLoL XnetLoL added the bug Something isn't working label Jan 16, 2025
@freddyaboulton freddyaboulton added good first issue Good for newcomers python Backend-related issue (Python) labels Jan 16, 2025
@XnetLoL
Copy link
Author

XnetLoL commented Jan 20, 2025

@freddyaboulton Hi there, I am unable to run tests locally (so I can't push a PR), but this change worked for me

async def async_ssrf_protected_download(url: str, cache_dir: str) -> str:
:

async def async_ssrf_protected_download(url: str, cache_dir: str) -> str:
    temp_dir = Path(cache_dir) / hash_url(url)
    temp_dir.mkdir(exist_ok=True, parents=True)

    # Extract base filename without query parameters
    parsed_url = urlparse(url)
    base_path = parsed_url.path.rstrip("/")
    filename = (
        client_utils.strip_invalid_filename_characters(Path(base_path).name) or "file"
    )

    # Perhaps here we could add an extension based on content-type if filename has none (with sh.get)
    # if "." not in filename:
    #     response = await sh.get(
    #         url,
    #         domain_whitelist=PUBLIC_HOSTNAME_WHITELIST,
    #         _transport=async_transport,
    #         follow_redirects=True,
    #     )
    #     content_type = response.headers.get("content-type", "")
    #     ext = mimetypes.guess_extension(content_type) or ""
    #     filename = f"{filename}{ext}"

    full_temp_file_path = str(abspath(temp_dir / filename))

    if Path(full_temp_file_path).exists():
        return full_temp_file_path

    hostname = parsed_url.hostname

    response = await sh.get(
        url, domain_whitelist=PUBLIC_HOSTNAME_WHITELIST, _transport=async_transport
    )

    while response.is_redirect:
        redirect_url = response.headers["Location"]
        redirect_parsed = urlparse(redirect_url)

        if not redirect_parsed.hostname:
            redirect_url = f"{parsed_url.scheme}://{hostname}{redirect_url}"

        response = await sh.get(
            redirect_url,
            domain_whitelist=PUBLIC_HOSTNAME_WHITELIST,
            _transport=async_transport,
        )

    if response.status_code != 200:
        raise Exception(f"Failed to download file. Status code: {response.status_code}")

    async with aiofiles.open(full_temp_file_path, "wb") as f:
        async for chunk in response.aiter_bytes():
            await f.write(chunk)

    return full_temp_file_path

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers python Backend-related issue (Python)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants