Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add depth estimation pipeline #18446

Closed
NielsRogge opened this issue Aug 3, 2022 · 5 comments · Fixed by #18618
Closed

Add depth estimation pipeline #18446

NielsRogge opened this issue Aug 3, 2022 · 5 comments · Fixed by #18618

Comments

@NielsRogge
Copy link
Contributor

NielsRogge commented Aug 3, 2022

Feature request

We currently have 2 monocular depth estimation models in the library, namely DPT and GLPN.

It would be great to have a pipeline for this task, with the following API:

from transformers import pipeline

pipe = pipeline("depth-estimation")
pipe("cats.png")

This pipeline could default to the https://huggingface.co/Intel/dpt-large checkpoint. Also check out the Space that showcases the model.

This can be implemented similar to other pipelines. For an example PR that added a pipeline, see #11598.

Motivation

Pipelines are a great way to quickly perform inference with a model for a given task, abstracting away all the complexity.

Your contribution

I can assist with this, together with @Narsil.

@Narsil
Copy link
Contributor

Narsil commented Aug 3, 2022

What would be the output like @NielsRogge ?

My understanding is that depth is just a gray scale image (black = infinitely far, white = infinitely close).

If that's the case It seems really close to image-segmentation in the sense that it's generating a new image from the original image, so we should try and reuse as much as possible.

Also maybe we could have something like image-generation to try and keep the name generic ? (And have an alias for depth-estimation for instance ?)

@nandwalritik
Copy link
Contributor

Hi @NielsRogge I would like to add this pipeline.

@NielsRogge
Copy link
Contributor Author

NielsRogge commented Aug 4, 2022

Hi @Narsil,

I'm not sure whether we should add this to the existing image-segmentation pipeline. Depth estimation is basically pixel regression, rather than pixel classification (the latter is image segmentation). It would be quite confusing to add it there.

Depth estimation is quite a different field, see e.g. https://paperswithcode.com/task/depth-estimation

And hi @nandwalritik, thanks for your interest in this. Feel free to start a draft PR.

@nandwalritik
Copy link
Contributor

Thanks I will start working on it.

@Narsil
Copy link
Contributor

Narsil commented Aug 4, 2022

I'm not sure whether we should add this to the existing image-segmentation pipeline.

I said we should inspire from it, not reuse it, but I suggested using an image-generationone. (Just to be slightly more general)
The output is a grayscale image, right ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants