Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arm64/SVE: Implemented ConvertToint64 and ConvertToUInt64 #104069

Merged
merged 45 commits into from
Jun 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
c738b77
Added ConverToInt32 and ConvertToUInt32 for float inputs.
ebepho Jun 5, 2024
35d39d9
Added flags to handle only low predicate registers.
ebepho Jun 5, 2024
7a781e1
Fixed merge conflicts.
ebepho Jun 5, 2024
1378d60
Fix whitespace
ebepho Jun 5, 2024
10c7a15
Remove special codegen flag
ebepho Jun 7, 2024
8004868
Added new test template for operations with different return types.
ebepho Jun 10, 2024
af7ccd4
Merge branch 'main' into ConvertToInt32
ebepho Jun 10, 2024
8cb76da
Add new test template.
ebepho Jun 11, 2024
abe25fc
Added api for ConvertToInt32 and ConvertToUInt 32 for double.
ebepho Jun 13, 2024
0f51f38
fix merge conflicts.
ebepho Jun 13, 2024
7fabb91
Merge branch 'dotnet:main' into main
ebepho Jun 14, 2024
11affed
Completed SVE Apis for ConvertToInt64 and ConvertToUInt64.
ebepho Jun 15, 2024
d5374ca
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 15, 2024
711b28a
Merge branch 'main' into converttoint64
ebepho Jun 15, 2024
fe32a2f
ConvertToSingle for int and uint.
ebepho Jun 15, 2024
478b969
ConvertToSingle for long and ulong.
ebepho Jun 15, 2024
4aa224d
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 15, 2024
cc63edf
Merge branch 'main' into ConvertToSingleDouble
ebepho Jun 15, 2024
ff54068
Started ConvertToDouble.
ebepho Jun 15, 2024
56601b4
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 17, 2024
422068b
Merge branch 'main' into converttoint64
ebepho Jun 17, 2024
5b4c4f3
Changed Validation Template Test name.
ebepho Jun 17, 2024
04071a3
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 18, 2024
ffcd267
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 18, 2024
d4b8dc3
Merge branch 'main' into converttoint64
ebepho Jun 18, 2024
5ac4a05
Merge branch 'main' into converttosingledouble
ebepho Jun 18, 2024
f055d0c
Merge branch 'converttosingledouble' into ConvertTo_Int64_Single_Double
ebepho Jun 18, 2024
33626b3
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 19, 2024
0327fa6
Merge branch 'main' into ConvertTo_Int64_Single_Double
ebepho Jun 19, 2024
da441d1
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 21, 2024
229017b
Merge branch 'main' into ConvertTo_Int64_Single_Double
ebepho Jun 21, 2024
f98fd84
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 24, 2024
51c9bf1
Merge branch 'main' into ConvertTo_Int64_Single_Double
ebepho Jun 24, 2024
1e68ff6
Merge branch 'main' of github.com:ebepho/runtime
ebepho Jun 26, 2024
d053d13
Merge branch 'main' into ConvertTo_int64_single_double
ebepho Jun 26, 2024
b2a777e
ConvertToInt64.
ebepho Jun 27, 2024
66abcaa
ConvertToInt64 passes optimized tests.
ebepho Jun 27, 2024
a0c7333
Added cases for ConvertToSingle and ConvertToDouble.
ebepho Jun 27, 2024
37e1da1
double or long to 32 bit value.
ebepho Jun 27, 2024
2c13be7
Removed ConvertToDouble and ConvertToSingle.
ebepho Jun 28, 2024
2f3c901
Removed more of ConvertToSingle and ConvertToDouble.
ebepho Jun 28, 2024
990c75b
all tests pass.
ebepho Jun 28, 2024
eaf3905
addressed comments.
ebepho Jun 28, 2024
81f11cb
jit format:
kunalspathak Jun 28, 2024
c2d5d15
Remove trailing space
kunalspathak Jun 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/coreclr/jit/hwintrinsic.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1731,6 +1731,8 @@ GenTree* Compiler::impHWIntrinsic(NamedIntrinsic intrinsic,
{
case NI_Sve_ConvertToInt32:
case NI_Sve_ConvertToUInt32:
case NI_Sve_ConvertToInt64:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expected to have entries for ConvertToSingle and ConvertToDouble here as well. Did you confirm this works for scenario where maskSize != operSize in lowering?

case NI_Sve_ConvertToUInt64:
// Save the base type of return SIMD. It is used to contain this intrinsic inside
// ConditionalSelect.
retNode->AsHWIntrinsic()->SetAuxiliaryJitType(getBaseJitTypeOfSIMDType(sig->retTypeSigClass));
Expand Down
17 changes: 14 additions & 3 deletions src/coreclr/jit/hwintrinsiccodegenarm64.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -492,12 +492,22 @@ void CodeGen::genHWIntrinsic(GenTreeHWIntrinsic* node)

// Special handling for ConvertTo* APIs
// Just need to change the opt here.
insOpts embOpt = opt;
switch (intrinEmbMask.id)
{
case NI_Sve_ConvertToInt32:
case NI_Sve_ConvertToUInt32:
{
opt = intrinEmbMask.baseType == TYP_DOUBLE ? INS_OPTS_D_TO_S : INS_OPTS_SCALABLE_S;
embOpt = emitTypeSize(intrinEmbMask.baseType) == EA_8BYTE ? INS_OPTS_D_TO_S
: INS_OPTS_SCALABLE_S;
break;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also needs handling of ConvertToSingle

embOpt = intrinEmbMask.baseType == TYP_DOUBLE ? INS_OPTS_D_TO_S : INS_OPTS_SCALABLE_S;

and ConvertToDouble:

embOpt = intrinEmbMask.baseType == TYP_FLOAT ? INS_OPTS_S_TO_D : INS_OPTS_SCALABLE_D;

which means you can combine ConvertToSingle with Convert*32 and ConvertToDouble with Convert*64.


case NI_Sve_ConvertToInt64:
case NI_Sve_ConvertToUInt64:
{
embOpt = emitTypeSize(intrinEmbMask.baseType) == EA_4BYTE ? INS_OPTS_S_TO_D
: INS_OPTS_SCALABLE_D;
break;
}
default:
Expand Down Expand Up @@ -536,7 +546,8 @@ void CodeGen::genHWIntrinsic(GenTreeHWIntrinsic* node)

// We cannot use use `movprfx` here to move falseReg to targetReg because that will
// overwrite the value of embMaskOp1Reg which is present in targetReg.
GetEmitter()->emitIns_R_R_R(insEmbMask, emitSize, targetReg, maskReg, embMaskOp1Reg, opt);
GetEmitter()->emitIns_R_R_R(insEmbMask, emitSize, targetReg, maskReg, embMaskOp1Reg,
embOpt);

GetEmitter()->emitIns_R_R_R_R(INS_sve_sel, emitSize, targetReg, maskReg, targetReg,
falseReg, opt);
Expand All @@ -550,7 +561,7 @@ void CodeGen::genHWIntrinsic(GenTreeHWIntrinsic* node)
}
}

GetEmitter()->emitIns_R_R_R(insEmbMask, emitSize, targetReg, maskReg, embMaskOp1Reg, opt);
GetEmitter()->emitIns_R_R_R(insEmbMask, emitSize, targetReg, maskReg, embMaskOp1Reg, embOpt);
break;
}

Expand Down
2 changes: 2 additions & 0 deletions src/coreclr/jit/hwintrinsiclistarm64sve.h
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ HARDWARE_INTRINSIC(Sve, Compute64BitAddresses,
HARDWARE_INTRINSIC(Sve, Compute8BitAddresses, -1, 2, true, {INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_sve_adr, INS_invalid, INS_sve_adr, INS_invalid, INS_invalid}, HW_Category_SIMD, HW_Flag_Scalable|HW_Flag_SpecialCodeGen)
HARDWARE_INTRINSIC(Sve, ConditionalSelect, -1, 3, true, {INS_sve_sel, INS_sve_sel, INS_sve_sel, INS_sve_sel, INS_sve_sel, INS_sve_sel, INS_sve_sel, INS_sve_sel, INS_sve_sel, INS_sve_sel}, HW_Category_SIMD, HW_Flag_Scalable|HW_Flag_ExplicitMaskedOperation|HW_Flag_SupportsContainment)
HARDWARE_INTRINSIC(Sve, ConvertToInt32, -1, -1, false, {INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_sve_fcvtzs, INS_sve_fcvtzs}, HW_Category_SIMD, HW_Flag_Scalable|HW_Flag_BaseTypeFromFirstArg|HW_Flag_EmbeddedMaskedOperation|HW_Flag_LowMaskedOperation)
HARDWARE_INTRINSIC(Sve, ConvertToInt64, -1, -1, false, {INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_sve_fcvtzs, INS_sve_fcvtzs}, HW_Category_SIMD, HW_Flag_Scalable|HW_Flag_BaseTypeFromFirstArg|HW_Flag_EmbeddedMaskedOperation|HW_Flag_LowMaskedOperation)
HARDWARE_INTRINSIC(Sve, ConvertToUInt32, -1, -1, false, {INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_sve_fcvtzu, INS_sve_fcvtzu}, HW_Category_SIMD, HW_Flag_Scalable|HW_Flag_BaseTypeFromFirstArg|HW_Flag_EmbeddedMaskedOperation|HW_Flag_LowMaskedOperation)
HARDWARE_INTRINSIC(Sve, ConvertToUInt64, -1, -1, false, {INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_sve_fcvtzu, INS_sve_fcvtzu}, HW_Category_SIMD, HW_Flag_Scalable|HW_Flag_BaseTypeFromFirstArg|HW_Flag_EmbeddedMaskedOperation|HW_Flag_LowMaskedOperation)
HARDWARE_INTRINSIC(Sve, Count16BitElements, 0, 1, false, {INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_sve_cnth, INS_invalid, INS_invalid, INS_invalid}, HW_Category_Scalar, HW_Flag_Scalable|HW_Flag_HasEnumOperand|HW_Flag_SpecialCodeGen|HW_Flag_NoFloatingPointUsed)
HARDWARE_INTRINSIC(Sve, Count32BitElements, 0, 1, false, {INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_sve_cntw, INS_invalid, INS_invalid, INS_invalid}, HW_Category_Scalar, HW_Flag_Scalable|HW_Flag_HasEnumOperand|HW_Flag_SpecialCodeGen|HW_Flag_NoFloatingPointUsed)
HARDWARE_INTRINSIC(Sve, Count64BitElements, 0, 1, false, {INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_invalid, INS_sve_cntd, INS_invalid, INS_invalid, INS_invalid}, HW_Category_Scalar, HW_Flag_Scalable|HW_Flag_HasEnumOperand|HW_Flag_SpecialCodeGen|HW_Flag_NoFloatingPointUsed)
Expand Down
4 changes: 3 additions & 1 deletion src/coreclr/jit/lowerarmarch.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3389,7 +3389,9 @@ void Lowering::ContainCheckHWIntrinsic(GenTreeHWIntrinsic* node)
// For now, make sure that we get here only for intrinsics that we are
// sure about to rely on auxiliary type's size.
assert((embOp->GetHWIntrinsicId() == NI_Sve_ConvertToInt32) ||
(embOp->GetHWIntrinsicId() == NI_Sve_ConvertToUInt32));
(embOp->GetHWIntrinsicId() == NI_Sve_ConvertToUInt32) ||
(embOp->GetHWIntrinsicId() == NI_Sve_ConvertToInt64) ||
(embOp->GetHWIntrinsicId() == NI_Sve_ConvertToUInt64));

uint32_t auxSize = genTypeSize(embOp->GetAuxiliaryType());
if (maskSize == auxSize)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -869,6 +869,26 @@ internal Arm64() { }
public static unsafe Vector<int> ConvertToInt32(Vector<float> value) { throw new PlatformNotSupportedException(); }


/// ConvertToInt64 : Floating-point convert

/// <summary>
/// svint64_t svcvt_s64[_f64]_m(svint64_t inactive, svbool_t pg, svfloat64_t op)
/// FCVTZS Ztied.D, Pg/M, Zop.D
/// svint64_t svcvt_s64[_f64]_x(svbool_t pg, svfloat64_t op)
/// FCVTZS Ztied.D, Pg/M, Ztied.D
/// svint64_t svcvt_s64[_f64]_z(svbool_t pg, svfloat64_t op)
/// </summary>
public static unsafe Vector<long> ConvertToInt64(Vector<double> value) { throw new PlatformNotSupportedException(); }

/// <summary>
/// svint64_t svcvt_s64[_f32]_m(svint64_t inactive, svbool_t pg, svfloat32_t op)
/// FCVTZS Ztied.D, Pg/M, Zop.S
/// svint64_t svcvt_s64[_f32]_x(svbool_t pg, svfloat32_t op)
/// FCVTZS Ztied.D, Pg/M, Ztied.S
/// svint64_t svcvt_s64[_f32]_z(svbool_t pg, svfloat32_t op)
/// </summary>
public static unsafe Vector<long> ConvertToInt64(Vector<float> value) { throw new PlatformNotSupportedException(); }

/// ConvertToUInt32 : Floating-point convert

/// <summary>
Expand All @@ -890,6 +910,27 @@ internal Arm64() { }
public static unsafe Vector<uint> ConvertToUInt32(Vector<float> value) { throw new PlatformNotSupportedException(); }


/// ConvertToUInt64 : Floating-point convert

/// <summary>
/// svuint64_t svcvt_u64[_f64]_m(svuint64_t inactive, svbool_t pg, svfloat64_t op)
/// FCVTZU Ztied.D, Pg/M, Zop.D
/// svuint64_t svcvt_u64[_f64]_x(svbool_t pg, svfloat64_t op)
/// FCVTZU Ztied.D, Pg/M, Ztied.D
/// svuint64_t svcvt_u64[_f64]_z(svbool_t pg, svfloat64_t op)
/// </summary>
public static unsafe Vector<ulong> ConvertToUInt64(Vector<double> value) { throw new PlatformNotSupportedException(); }

/// <summary>
/// svuint64_t svcvt_u64[_f32]_m(svuint64_t inactive, svbool_t pg, svfloat32_t op)
/// FCVTZU Ztied.D, Pg/M, Zop.S
/// svuint64_t svcvt_u64[_f32]_x(svbool_t pg, svfloat32_t op)
/// FCVTZU Ztied.D, Pg/M, Ztied.S
/// svuint64_t svcvt_u64[_f32]_z(svbool_t pg, svfloat32_t op)
/// </summary>
public static unsafe Vector<ulong> ConvertToUInt64(Vector<float> value) { throw new PlatformNotSupportedException(); }


/// Count16BitElements : Count the number of 16-bit elements in a vector

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -926,6 +926,27 @@ internal Arm64() { }
public static unsafe Vector<int> ConvertToInt32(Vector<float> value) => ConvertToInt32(value);


/// ConvertToInt64 : Floating-point convert

/// <summary>
/// svint64_t svcvt_s64[_f64]_m(svint64_t inactive, svbool_t pg, svfloat64_t op)
/// FCVTZS Ztied.D, Pg/M, Zop.D
/// svint64_t svcvt_s64[_f64]_x(svbool_t pg, svfloat64_t op)
/// FCVTZS Ztied.D, Pg/M, Ztied.D
/// svint64_t svcvt_s64[_f64]_z(svbool_t pg, svfloat64_t op)
/// </summary>
public static unsafe Vector<long> ConvertToInt64(Vector<double> value) => ConvertToInt64(value);

/// <summary>
/// svint64_t svcvt_s64[_f32]_m(svint64_t inactive, svbool_t pg, svfloat32_t op)
/// FCVTZS Ztied.D, Pg/M, Zop.S
/// svint64_t svcvt_s64[_f32]_x(svbool_t pg, svfloat32_t op)
/// FCVTZS Ztied.D, Pg/M, Ztied.S
/// svint64_t svcvt_s64[_f32]_z(svbool_t pg, svfloat32_t op)
/// </summary>
public static unsafe Vector<long> ConvertToInt64(Vector<float> value) => ConvertToInt64(value);


/// ConvertToUInt32 : Floating-point convert

/// <summary>
Expand All @@ -947,6 +968,27 @@ internal Arm64() { }
public static unsafe Vector<uint> ConvertToUInt32(Vector<float> value) => ConvertToUInt32(value);


/// ConvertToUInt64 : Floating-point convert

/// <summary>
/// svuint64_t svcvt_u64[_f64]_m(svuint64_t inactive, svbool_t pg, svfloat64_t op)
/// FCVTZU Ztied.D, Pg/M, Zop.D
/// svuint64_t svcvt_u64[_f64]_x(svbool_t pg, svfloat64_t op)
/// FCVTZU Ztied.D, Pg/M, Ztied.D
/// svuint64_t svcvt_u64[_f64]_z(svbool_t pg, svfloat64_t op)
/// </summary>
public static unsafe Vector<ulong> ConvertToUInt64(Vector<double> value) => ConvertToUInt64(value);

/// <summary>
/// svuint64_t svcvt_u64[_f32]_m(svuint64_t inactive, svbool_t pg, svfloat32_t op)
/// FCVTZU Ztied.D, Pg/M, Zop.S
/// svuint64_t svcvt_u64[_f32]_x(svbool_t pg, svfloat32_t op)
/// FCVTZU Ztied.D, Pg/M, Ztied.S
/// svuint64_t svcvt_u64[_f32]_z(svbool_t pg, svfloat32_t op)
/// </summary>
public static unsafe Vector<ulong> ConvertToUInt64(Vector<float> value) => ConvertToUInt64(value);


/// Count16BitElements : Count the number of 16-bit elements in a vector

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4317,8 +4317,12 @@ internal Arm64() { }

public static System.Numerics.Vector<int> ConvertToInt32(System.Numerics.Vector<double> value) { throw null; }
public static System.Numerics.Vector<int> ConvertToInt32(System.Numerics.Vector<float> value) { throw null; }
public static System.Numerics.Vector<long> ConvertToInt64(System.Numerics.Vector<double> value) { throw null; }
public static System.Numerics.Vector<long> ConvertToInt64(System.Numerics.Vector<float> value) { throw null; }
public static System.Numerics.Vector<uint> ConvertToUInt32(System.Numerics.Vector<double> value) { throw null; }
public static System.Numerics.Vector<uint> ConvertToUInt32(System.Numerics.Vector<float> value) { throw null; }
public static System.Numerics.Vector<ulong> ConvertToUInt64(System.Numerics.Vector<double> value) { throw null; }
public static System.Numerics.Vector<ulong> ConvertToUInt64(System.Numerics.Vector<float> value) { throw null; }

public static ulong Count16BitElements([ConstantExpected] SveMaskPattern pattern = SveMaskPattern.All) { throw null; }
public static ulong Count32BitElements([ConstantExpected] SveMaskPattern pattern = SveMaskPattern.All) { throw null; }
Expand Down
Loading
Loading