Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul or deprecate data layout inference #173

Open
maltanar opened this issue Feb 25, 2025 · 0 comments
Open

Overhaul or deprecate data layout inference #173

maltanar opened this issue Feb 25, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@maltanar
Copy link
Collaborator

Some QONNX and FINN flows require data layout inference, i.e. explicitly identify which tensor dimension is the batch, channels, image height/width and similar. See InferDataLayouts for more details on how this currently works.

The current approach is quite ad-hoc and breaks for almost any previously-unseen op or pattern, potentially causing invalid transformations later on due to incorrect data layout annotations. We should consider a major overhaul of data layout inference to address this.

Some thoughts on the matter:

  1. It may not even be possible to always define sane data layouts for all tensors in an arbitrary NN topology. As just one example, it is possible to create arbitrary dimensions with Reshape that don't clearly correspond to any meaningful dimension.
  2. We don't necessarily care about identifying all dimensions as part of data layout inference - mostly, the location of the channels dimension for convnets, and perhaps the location of the batch dimension are the ones that matter. So perhaps an overhauled system should allow for incomplete specifications like ['N', '_', '_', 'C'].
  3. Certain ops (due to how they are defined in the standard) dictate where the batch and channels dimensions are located, e.g. BatchNormalization and Conv. This could be used to propagate layout annotations both backwards and forwards in the graph. If it clashes with the result from the propagation of user-provided input annotation, this should raise an error.
@maltanar maltanar added the enhancement New feature or request label Feb 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant