[TOSA] Fix aten.view and aten.slice.tensor #1768

AmosLewis · 2023-01-03T07:14:14Z

Find this error in nod-ai/SHARK-Studio#494
Deal with aten.view input -1
Deal with aten.slice start -1

slice op explaination: https://cran.r-project.org/web/packages/torch/vignettes/indexing.html

AmosLewis · 2023-01-17T18:50:01Z

func.func @torch.aten.slice(%arg0: !torch.vtensor<[1,128,2],f32>) -> !torch.vtensor<[1,1,2],f32> {
  %int-1 = torch.constant.int -1
  %int1 = torch.constant.int 1
  %int0 = torch.constant.int 0
  %0 = torch.aten.slice.Tensor %arg0, %int1, %int-1, %int0, %int1 : !torch.vtensor<[1,128,2],f32>, !torch.int, !torch.int, !torch.int, !torch.int -> !torch.vtensor<[1,1,2],f32>
  return %0 : !torch.vtensor<[1,1,2],f32>
}

➜ ~ torch-mlir-opt -convert-torch-to-tosa /tmp/slice.mlir

module {
  func.func @torch.aten.slice(%arg0: !torch.vtensor<[1,128,2],f32>) -> !torch.vtensor<[1,1,2],f32> {
    %0 = torch_c.to_builtin_tensor %arg0 : !torch.vtensor<[1,128,2],f32> -> tensor<1x128x2xf32>
    %int-1 = torch.constant.int -1
    %int1 = torch.constant.int 1
    %int0 = torch.constant.int 0
    %1 = "tosa.slice"(%0) {size = array<i64: 1, 1, 2>, start = array<i64: 0, 127, 0>} : (tensor<1x128x2xf32>) -> tensor<1x1x2xf32>
    %2 = torch_c.from_builtin_tensor %1 : tensor<1x1x2xf32> -> !torch.vtensor<[1,1,2],f32>
    return %2 : !torch.vtensor<[1,1,2],f32>
  }
}

AmosLewis · 2023-01-17T18:52:26Z

func.func @torch.aten.view(%arg0: !torch.vtensor<[1,128],si64>) -> !torch.vtensor<[1,128],si64> {
  %int-1 = torch.constant.int -1
  %int128 = torch.constant.int 128
  %0 = torch.prim.ListConstruct %int-1, %int128 : (!torch.int, !torch.int) -> !torch.list<int>
  %1 = torch.aten.view %arg0, %0 : !torch.vtensor<[1,128],si64>, !torch.list<int> -> !torch.vtensor<[1,128],si64>
  return %1 : !torch.vtensor<[1,128],si64>
}

➜ ~ torch-mlir-opt -convert-torch-to-tosa /tmp/view_torchbackend.mlir

module {
  func.func @torch.aten.view(%arg0: !torch.vtensor<[1,128],si64>) -> !torch.vtensor<[1,128],si64> {
    %0 = torch_c.to_builtin_tensor %arg0 : !torch.vtensor<[1,128],si64> -> tensor<1x128xi64>
    %int-1 = torch.constant.int -1
    %int128 = torch.constant.int 128
    %1 = torch.prim.ListConstruct %int-1, %int128 : (!torch.int, !torch.int) -> !torch.list<int>
    %2 = "tosa.reshape"(%0) {new_shape = array<i64: 1, 128>} : (tensor<1x128xi64>) -> tensor<1x128xi64>
    %3 = torch_c.from_builtin_tensor %2 : tensor<1x128xi64> -> !torch.vtensor<[1,128],si64>
    return %3 : !torch.vtensor<[1,128],si64>
  }
}

AmosLewis · 2023-01-17T18:54:13Z

➜  ~ torch-mlir-opt -pass-pipeline='builtin.module(torch-backend-to-tosa-backend-pipeline)'   /tmp/view_torchbackend.mlir
module {
  func.func @torch.aten.view(%arg0: tensor<1x128xi64>) -> tensor<1x128xi64> {
    return %arg0 : tensor<1x128xi64>
  }
}

➜  ~ torch-mlir-opt -pass-pipeline='builtin.module(torch-backend-to-tosa-backend-pipeline)'   /tmp/slice.mlir            
module {
  func.func @torch.aten.slice(%arg0: tensor<1x128x2xf32>) -> tensor<1x1x2xf32> {
    %0 = "tosa.slice"(%arg0) {size = array<i64: 1, 1, 2>, start = array<i64: 0, 127, 0>} : (tensor<1x128x2xf32>) -> tensor<1x1x2xf32>
    return %0 : tensor<1x1x2xf32>
  }
}

lib/Conversion/TorchToTosa/TorchToTosa.cpp

ramiro050 · 2023-01-19T19:12:10Z

lib/Conversion/TorchToTosa/TorchToTosa.cpp

@@ -3061,13 +3079,18 @@ LogicalResult ConvertAtenOp<AtenSliceTensorOp>::matchAndRewrite(
  if (!matchPattern(op.getStart(), m_TorchConstantInt(&start)))
    return rewriter.notifyMatchFailure(op, "start must be a Scalar constant");

-  if (start < 0)


Can you add an e2e test that checks these changes?

ramiro050 · 2023-01-19T19:12:27Z

test/Conversion/TorchToTosa/basic.mlir

+// CHECK:           %[[VAL_6:.*]] = torch_c.from_builtin_tensor %[[VAL_5]] : tensor<1x1x2xf32> -> !torch.vtensor<[1,1,2],f32>
+// CHECK:           return %[[VAL_6]] : !torch.vtensor<[1,1,2],f32>
+// CHECK:         }
+func.func @torch.aten.slice(%arg0: !torch.vtensor<[1,128,2],f32>) -> !torch.vtensor<[1,1,2],f32> {


ramiro050 · 2023-01-19T19:21:57Z

lib/Conversion/TorchToTosa/TorchToTosa.cpp

@@ -2645,6 +2645,24 @@ LogicalResult ConvertAtenOp<AtenViewOp>::matchAndRewrite(
    return rewriter.notifyMatchFailure(op,
                                       "size must consist of Scalar constants");

+  // # the size -1 is inferred from other dimensions


You need to first make sure there is at most one -1 in the list and return a notifyMatchFailure if that is not the case

ramiro050 · 2023-01-19T19:22:30Z

lib/Conversion/TorchToTosa/TorchToTosa.cpp

+  }
+  for (size_t i = 0; i < outShape.size(); i++) {
+    if (outShape[i] < 0) {
+      outShape[i] = totalSize / otherSize;


I would add a break after this line to make it very clear that this is expected to only runs once

ramiro050 · 2023-01-19T19:23:59Z

lib/Conversion/TorchToTosa/TorchToTosa.cpp

-  if (start < 0)
-    return rewriter.notifyMatchFailure(op, "Currently unsupported: start < 0");
+  if (start < 0) {
+    start = start + selfType.getShape()[dim];


You need to check that it is positive after this

ramiro050 · 2023-01-19T19:26:02Z

lib/Conversion/TorchToTosa/TorchToTosa.cpp


  int64_t end;
  if (!matchPattern(op.getEnd(), m_TorchConstantInt(&end)))
    return rewriter.notifyMatchFailure(op, "end must be a Scalar constant");

+  if (end <= 0) {


I don't think this is correct. For the end == 0 case, end should remain zero. Make sure to also e2e test this edge case

AmosLewis · 2023-01-20T09:04:22Z

Split the aten.view op into a new patch #1815

AmosLewis requested review from ramiro050 and vivekkhandelwal1 January 3, 2023 07:14

AmosLewis mentioned this pull request Jan 3, 2023

DistilGPT2 to TOSA nod-ai/SHARK-Studio#494

Closed

AmosLewis requested a review from eric-k256 January 3, 2023 07:17

AmosLewis force-pushed the slice branch 2 times, most recently from e8021a4 to 694fc38 Compare January 3, 2023 22:24

ramiro050 requested changes Jan 18, 2023

View reviewed changes

lib/Conversion/TorchToTosa/TorchToTosa.cpp Show resolved Hide resolved

AmosLewis force-pushed the slice branch 2 times, most recently from b0a5f34 to 7feb66a Compare January 18, 2023 01:19

AmosLewis requested a review from ramiro050 January 18, 2023 04:01

AmosLewis force-pushed the slice branch 6 times, most recently from 4232ba5 to e18993c Compare January 19, 2023 01:21

ramiro050 requested changes Jan 19, 2023

View reviewed changes

[TOSA] Fix aten.view and aten.slice.tensor

bb8374e

AmosLewis force-pushed the slice branch from e18993c to bb8374e Compare January 20, 2023 06:24

AmosLewis marked this pull request as draft January 20, 2023 06:33

AmosLewis mentioned this pull request Jan 20, 2023

[TOSA] Add aten.view shape -1 support #1815

Merged

AmosLewis closed this Jan 31, 2023

AmosLewis deleted the slice branch January 19, 2024 19:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TOSA] Fix aten.view and aten.slice.tensor #1768

[TOSA] Fix aten.view and aten.slice.tensor #1768

AmosLewis commented Jan 3, 2023 •

edited

Loading

AmosLewis commented Jan 17, 2023 •

edited

Loading

AmosLewis commented Jan 17, 2023

AmosLewis commented Jan 17, 2023

ramiro050 Jan 19, 2023

ramiro050 Jan 19, 2023

ramiro050 Jan 19, 2023

AmosLewis Jan 20, 2023

ramiro050 Jan 19, 2023

AmosLewis Jan 20, 2023

ramiro050 Jan 19, 2023

AmosLewis Jan 20, 2023

ramiro050 Jan 19, 2023

AmosLewis commented Jan 20, 2023

[TOSA] Fix aten.view and aten.slice.tensor #1768

[TOSA] Fix aten.view and aten.slice.tensor #1768

Conversation

AmosLewis commented Jan 3, 2023 • edited Loading

AmosLewis commented Jan 17, 2023 • edited Loading

AmosLewis commented Jan 17, 2023

AmosLewis commented Jan 17, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AmosLewis commented Jan 20, 2023

AmosLewis commented Jan 3, 2023 •

edited

Loading

AmosLewis commented Jan 17, 2023 •

edited

Loading