PtndArrayEx.multiboxDetection() implementation

[juliangamble]

Description

The big goal here is to implement PtNDArrayEx.multiBoxDetection() (currently throws NotImplementException) so that you can run PikachuTraining with Pytorch and so you can run SSD Training on M1 Macs.

The implementation for this is derived from:
https://github.com/apache/mxnet/blob/master/src/operator/contrib/multibox_detection.cc
and
https://github.com/apache/mxnet/blob/master/src/operator/contrib/multibox_detection-inl.h

This implementation is based on implementing the unit test on the C++ side and comparing the steps internally.

[lanking520]

Is that possible to achieve below code under matrix calculation through using existing NDArray functions?

[juliangamble]

Thanks, that’s helpful. I’ll take a look.

…

On Thu, 7 Sep 2023 at 2:34 am, Qing Lan ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/engine/PtNDArrayEx.java <#2769 (comment)>: > + float temp = pClsProb[j * numAnchors + i]; + if (temp > score) { + score = temp; + id = j; + } + } + + if (id > 0 && score < threshold) { + id = 0; + } + + // [id, prob, xmin, ymin, xmax, ymax] + outputs[i][0] = id - 1; + outputs[i][1] = score; + int offset = i * 4; + float[] pAnchorRow4 = new float[4]; Is that possible to achieve below code under matrix calculation through using existing NDArray functions? — Reply to this email directly, view it on GitHub <#2769 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB734FANB5BWQTUINR4WDDXZCQXPANCNFSM6AAAAAA4KH6HHQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[zachgk]

I made a few updates to merge this in for our next release. This includes some cleanup and a few bug fixes (including with freezing parameters in PtModel loading outside the scope of this PR).

One thing to note is I got the tests running in SingleShotDetectionTest. Right now, it passes the SingleShotDetectionTest.testLoadPredict using the new code. However, it does not fully support SSD. It won't pass SingleShotDetectionTest.testLoadTrain or other SSD training without also implementing PtNDArrayEx.multiBoxTarget

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 91.89189% with 9 lines in your changes missing coverage. Please review.

Project coverage is 72.21%. Comparing base (bb5073f) to head (61597e5).
Report is 1331 commits behind head on master.

Files with missing lines	Patch %	Lines
...c/main/java/ai/djl/pytorch/engine/PtNDArrayEx.java	91.26%	0 Missing and 9 partials ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #2769      +/-   ##
============================================
+ Coverage     72.08%   72.21%   +0.12%     
- Complexity     5126     7111    +1985     
============================================
  Files           473      702     +229     
  Lines         21970    31680    +9710     
  Branches       2351     3284     +933     
============================================
+ Hits          15838    22879    +7041     
- Misses         4925     7237    +2312     
- Partials       1207     1564     +357     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[juliangamble]

Thank you Zach. I’ll give this a try. JG

…

On Thu, 28 Sep 2023 at 3:04 pm, Zach Kimberg ***@***.***> wrote: I made a few updates to merge this in for our next release. This includes some cleanup and a few bug fixes (including with freezing parameters in PtModel loading outside the scope of this PR). One thing to note is I got the tests running in SingleShotDetectionTest. Right now, it passes the SingleShotDetectionTest.testLoadPredict using the new code. However, it does not fully support SSD. It won't pass SingleShotDetectionTest.testLoadTrain or other SSD training without also implementing PtNDArrayEx.multiBoxTarget — Reply to this email directly, view it on GitHub <#2769 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB734ACDRJVE7UU63H2KBDX4UAPDANCNFSM6AAAAAA4KH6HHQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>