Add dtype functions for two tensor promotion ops

[ramiro050]

This commit adds dtype functions for ops in RefineTypes under the category of "Promote the two dtypes". The only ops not added here are convolution ops, since they take an optional tensor argument, and the dtype pipeline currently does not correctly handle that case. I will add a follow up patch fixing this.

This commit also adds two helper functions that perform a very thorough testing of dtype functions. The helper function _check_two_tensor_op is able to independently test invalid input dtypes and invalid output dtypes.

Lastly, this commit also XFAILs "MobilenetV3Module_basic".

[ramiro050]

This is for #1807

[ramiro050]

@li-plus, this is the patch with the new helper functions to test the dtype functions.

[ramiro050]

@li-plus can you review this patch, since you're familiar with most of the changes?

[li-plus]

@ramiro050 LGTM, except for the float16 dtype. I think the shape function shouldn't raise an error for float16 since it is implemented on cuda.

[ramiro050]

@ramiro050 LGTM, except for the float16 dtype. I think the shape function shouldn't raise an error for float16 since it is implemented on cuda.

Because our testing framework in the CI currently runs only on CPU, we need to raise an error for f16. However, there are discussions about moving the CI testing to GPU instead. See: #1669. If that happens in the future, we can remove the f16 assert checks

[li-plus]

Ok I see, but the problem is that we have lots of gpu models to convert using torch-mlir. Most of them are half()ed and full of float16 ops. Currently we are running torch-mlir with cuda-version pytorch to convert these models, and float16 can be correctly inferred by RefineTypes.cpp.

If shape functions raise at float16, all our pipelines will fail. So I was wondering if we could skip float16 checking on cpu machine in CI, and we don't need to raise an error for float16. In fact, float16 always fail on cpu, so maybe there is no point testing it.

[qingyunqu]

Hi, we strongly recommend to support float16 in shape infer and dtype infer. At least not to raise an error.

[ramiro050]

It turns out that if we perform the dtype checks using meta tensors, we can avoid this issue of cpu not supporting f16.

In [6]: torch.tanh(a)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Input In [6], in <cell line: 1>()
----> 1 torch.tanh(a)

RuntimeError: "tanh_vml_cpu" not implemented for 'Half'

In [7]: torch.tanh(a.to("meta"))
Out[7]: tensor(..., device='meta', size=(3, 4, 5), dtype=torch.float16)

I will create a PR soon adding this change.

[qingyunqu]

How about the progress of pytorch's migration to meta tensor? I have commited some ops's meta migration to pytorch two years ago.
BTW, is there any discussion about implementing dynamic shape in torch-mlir? Including dynamic inputs and dynamic ops(like non_zero)?

[ramiro050]

How about the progress of pytorch's migration to meta tensor? I have commited some ops's meta migration to pytorch two years ago.

Are meta tensors not finished in PyTorch?

BTW, is there any discussion about implementing dynamic shape in torch-mlir? Including dynamic inputs and dynamic ops(like non_zero)?

What do you mean by dynamic inputs and dynamic ops? We currently support at the torch dialect level dynamic sizes for the tensors. In the case of torch.nonzero, the op should work without a problem in the case where as_tuple=False. It's just a matter of the backends to torch-mlir being able to handle the behavior.

The case of as_tuple=True is not yet supported in torch-mlir, since currently need to have a known size at compile time. As far as I know, there is currently no discussion for adding support for tuples that don't have a known size at compile time. However, if this is something that is needed, we can start such a discussion