Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixed an issue in SQR-Deformable DETR 's inference pipeline #9

Open
Fangyi-Chen opened this issue Jun 12, 2023 · 0 comments
Open

fixed an issue in SQR-Deformable DETR 's inference pipeline #9

Fangyi-Chen opened this issue Jun 12, 2023 · 0 comments

Comments

@Fangyi-Chen
Copy link
Owner

Fangyi-Chen commented Jun 12, 2023

we found and fixed an unseen bug in SQR-Deformable-DETR:

The code logic of SQR-Deformable-DETR (and the original Deformable-DETR) is like:

  1. calculate and update queries stage by stage in

    class QRDeformableDetrTransformerDecoder(DeformableDetrTransformerDecoder):

  2. collect the calculated queries.

  3. feed the collected queries to the cls branch and box branch in

    class QRDeformableDETRHead(DETRHead):

  4. for the original Deformable-DETR's training and inference, and for sqr's inference, we will have 6 queries and 6 lvl of corresponding cls branches and box branches. so the alignment is simply like

    outputs_class = self.cls_branches[lvl](hs[lvl])

But for sqr training, we will have to align the collected queries (32 queries) with the corresponding 6 lvl of cls branch and box branch

elif qid < 1: # training mode sqr query-stage alignment

While this alignment should not be applied to sqr's inference pipeline, we accidentally applied it. However, for (SQR)Deformable-DETR, the cls branches and box branches are shared across all levels. So, this bug only exists logically, but did not actually affect anything.

We have added two lines of code to force SQR-Deformable-DETR to use the original pipeline, as in

if not feat.requires_grad: # test mode

@Fangyi-Chen Fangyi-Chen changed the title fixed an issue in deformable detr fixed an issue in SQR-Deformable DETR 's inference pipeline Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant