A conformance suite with disallowed intra-op examples would be helpful for hardening.

[quidity]

Much of the security of an implementation of a spec will revolve around rejecting badly formed models, and rejecting changes to tensors that might result in out of bounds accesses.

A suite of failure cases would be helpful to ensure invalid input is properly rejected, and will form useful input to a fuzzer corpus.

(very possible this lives elsewhere!)

[anssiko]

The group discussed this issue and it was proposed to formalize these failure cases in the test suite that is under development.

[huningxin]

Thanks @quidity .

A suite of failure cases would be helpful to ensure invalid input is properly rejected, and will form useful input to a fuzzer corpus.

+1.

The interfaces interact with user-supplied buffers including MLGraphBuilder::constant, MLGraph::compute. Besides the invalid input buffers, for MLGraph::compute, the invalid output buffers probably also need to be considered. The implementation should reject producing the results into the invalid user-supplied output buffers (e.g., whose size is smaller than expected for output operand) and avoid the out of bounds accesses.

[anssiko]

@huningxin if you think you have enough information, feel free to work with @BruceDai to prototype test cases for invalid input and invalid user-supplied output buffers. Any findings or blockers to be documented in this issue.

@quidity please chime in here if you think our plan needs any course correction, or if you have any additional feedback.

[anssiko]

@BruceDai please chime in on this issue if you need more information or guidance from the group on how to add the proposed failure cases into the test suite.

[huningxin]

As discussed in Chromium CL-5017758 review for gather op, there is a request to create conformance tests which explicitly perform out-of-bounds access and validates the result conforms to the spec text.

/cc @quidity @RafaelCintron @BruceDai @fdwr

[fdwr]

create conformance tests which explicitly perform out-of-bounds access

I agree with this, to ensure no crashes/errors.

validates the result conforms to the spec text

Not sure about this though, if we're expecting the output tensor to exactly match values, as different implementations will have different behaviors for this garbage-in-garbage-out case. Some I can fathom include:

• Clamp all buffer offset reads to the end of the linear buffer (a simple security measure - DML does this).
• Clamp all reads to the last element of the axis or 0th element of the axis.
• Read the 0th element of the axis.
• Return a special out-of-bounds value (e.g. 0, -1...)