Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow click prompt for SAM 2 #7993

Merged
merged 30 commits into from
Aug 20, 2024
Merged

Allow click prompt for SAM 2 #7993

merged 30 commits into from
Aug 20, 2024

Conversation

MichaelBuessemeyer
Copy link
Contributor

@MichaelBuessemeyer MichaelBuessemeyer commented Aug 14, 2024

URL of deployed dev instance (used for testing):

  • https://___.webknossos.xyz

Steps to test:

  • use quick-select in ai mode with click and bounding box

Issues:


(Please delete unneeded items, merge only when none are left open)

Copy link
Contributor Author

@MichaelBuessemeyer MichaelBuessemeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for implementing the frontend part so fast @philippotto.

Please find my potential improvements below

.circleci/not-on-master.sh Outdated Show resolved Hide resolved
conf/application.conf Outdated Show resolved Hide resolved
@philippotto philippotto changed the title Single Click Sam2 Allow click prompt for SAM 2 Aug 15, 2024
@philippotto philippotto requested a review from fm3 August 15, 2024 12:41
@philippotto philippotto marked this pull request as ready for review August 15, 2024 12:41
selectionBottomRightXOpt: Option[Int],
selectionBottomRightYOpt: Option[Int],
pointXOpt: Option[Int],
pointYOpt: Option[Int],
dataShape: Vec3Int, // two of the axes will be 1024, the other is the "depth". Axis order varies depending on viewport
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: remove / adapt comment

Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backend LGTM, excepting the remaining todo

@philippotto
Copy link
Member

@MichaelBuessemeyer could you have a look at my latest commits? I think, afterwards the PR might be good to merge.

Copy link
Contributor Author

@MichaelBuessemeyer MichaelBuessemeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Frontend changes look good and testing worked well.

But I myself cannot approve this pr :)

@MichaelBuessemeyer MichaelBuessemeyer merged commit 756dc3e into master Aug 20, 2024
2 checks passed
@MichaelBuessemeyer MichaelBuessemeyer deleted the sam2-point branch August 20, 2024 08:50
dieknolle3333 pushed a commit that referenced this pull request Sep 2, 2024
* backend: add support for point interaction

* format backend

* fix buffer size calculation of buffer sent to torchserve

* support point interaction with sam-based quick select

* tmp enable SAM and disable most of ci

* improve comments

* update nux text

* tweak width of nux

* incorporate feedback

* undo tmp changes

* fix busy-mutex violated

* make backend accept sam bboxes with width / height less than 1024

* refactor a bit (e.g.,DEV flags)

* use a local bounding box for SAM by default (can be toggled via webknossos.DEV.flags.sam.useLocalMask = false)

* tune busy blocking msg

* refactor WkDevFlags further

* remove unused import

* tmp: disable some ci steps

* update comment in backend about torchserver request params

* fix that the padding could make the bbox too large

* use viewport extent in correct mag as minimum for mask size

* Revert "tmp: disable some ci steps"

This reverts commit 8362003.

* also mention clicking in quick select tooltip (and docs)

* update changelog

* add tooltip with hints for sam

* improve styling of hint

* improve tooltip text

---------

Co-authored-by: Philipp Otto <[email protected]>
Co-authored-by: Philipp Otto <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants