Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MetalPerformanceShadersGraph Bindings #14303

Merged
merged 59 commits into from
May 10, 2022
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
6b9b285
Scaffold MPSGraph
praeclarum Mar 2, 2022
f266883
Add graph options and placeholder tensors
praeclarum Mar 2, 2022
c6458fb
Finish MPSGraph.h
praeclarum Mar 2, 2022
592c55f
Update todos
praeclarum Mar 2, 2022
6e389eb
Add graph operations
praeclarum Mar 2, 2022
0ea620f
Fix bool
praeclarum Mar 2, 2022
03b2351
Add Sharpie definitions
praeclarum Mar 2, 2022
42a810e
Type erase MPSGraphType
praeclarum Mar 2, 2022
6bdcda1
Import Sharpie Graph Ops
praeclarum Mar 2, 2022
8a0f9f0
Improve Sharpie Op names
praeclarum Mar 2, 2022
057dcc9
Use int[] for NSNumber[] return types
praeclarum Mar 2, 2022
d313fa8
Use int[] for size parameters
praeclarum Mar 2, 2022
8cb1d9c
Add MacCatalyst (15,0)
praeclarum Mar 2, 2022
3823001
Move enums into correct file
praeclarum Mar 2, 2022
b047f75
Improve names
praeclarum Mar 3, 2022
5fa0e72
Make Options Flags
praeclarum Mar 3, 2022
690b25e
Use NSArray for image batches
praeclarum Mar 3, 2022
628bbc0
Add Variable and Constant convenience methods
praeclarum Mar 3, 2022
3e41079
Add convenience methods for reading and writing to MPSNDArray and MPS…
praeclarum Mar 3, 2022
7b4780f
Add tensor data create convenience taking individual images
praeclarum Mar 3, 2022
5e3ba5b
Apply suggestions from code review
praeclarum Mar 4, 2022
7867705
Update src/MetalPerformanceShadersGraph/MPSGraphExtensions.cs
praeclarum Mar 4, 2022
b5da6f9
Apply suggestions from code review
praeclarum Mar 4, 2022
3967104
Add MPSGraphType
praeclarum Mar 4, 2022
b5130b9
Merge branch 'main' into mpsgraph
dalexsoto Mar 4, 2022
ea9634a
Merge branch 'main' into mpsgraph
mandel-macaque Mar 12, 2022
74efed0
Apply suggestions from code review
praeclarum Mar 16, 2022
6be6c18
Add availability attributes
praeclarum Mar 16, 2022
948504a
Disable default MPSGraphVariableOp ctor
praeclarum Mar 16, 2022
974cbd6
Add MyMPSGraphApp MNIST test
praeclarum Mar 16, 2022
ce09180
Remove keyboard extension from MyMPSGraphApp
praeclarum Mar 16, 2022
91466fc
Merge remote-tracking branch 'origin/main' into mpsgraph
rolfbjarne Apr 6, 2022
814b412
[tests] Make the Mnist test a unit test.
rolfbjarne Apr 6, 2022
b6515e4
Merge branch 'main' into mpsgraph
rolfbjarne Apr 7, 2022
533868a
Merge branch 'main' into mpsgraph
dalexsoto Apr 8, 2022
915adb0
MPSGraphCompilationDescriptor conforms to NSCopying protocol
dalexsoto Apr 12, 2022
04293bb
Merge branch 'main' into mpsgraph
dalexsoto Apr 12, 2022
9b1f0ad
Merge branch 'main' into mpsgraph
dalexsoto Apr 13, 2022
4ad2e12
Merge branch 'main' into mpsgraph
mandel-macaque Apr 14, 2022
c354873
Merge branch 'main' into mpsgraph
dalexsoto Apr 18, 2022
560083b
Merge branch 'main' into mpsgraph
rolfbjarne Apr 19, 2022
9fa1b9e
Merge branch 'main' into mpsgraph
rolfbjarne Apr 25, 2022
3a31742
Merge branch 'main' into mpsgraph
dalexsoto Apr 27, 2022
38d6c6d
[introspection] Ignore copyWithZone: errors for MPSGraphCompilationDe…
rolfbjarne Apr 28, 2022
5af3279
[tests] Improve namespace for MnistTest.
rolfbjarne Apr 28, 2022
9f21b48
[tools] Don't skip linking with MetalPerformanceShadersGraph anymore.
rolfbjarne Apr 28, 2022
48cf912
[monotouch-test] Adjust MnistTest to only execute sometimes.
rolfbjarne Apr 28, 2022
46e593a
Update xtro.
rolfbjarne Apr 28, 2022
a109a8d
[MetalPerformanceShadersGraph] Add missing Release attribute to MPSGr…
rolfbjarne Apr 28, 2022
268b1b7
[introspection] Ignore MetalPerformanceShadersGraph on tvOS simulators.
rolfbjarne Apr 28, 2022
49b6ad8
Merge branch 'main' into mpsgraph
dalexsoto May 2, 2022
0500883
Don't run the new test in the simulator on bots.
rolfbjarne May 3, 2022
10b8600
[monotouch-test] Completely skip MnistTest on tvOS.
rolfbjarne May 4, 2022
6d9cbe2
Merge remote-tracking branch 'origin/main' into mpsgraph
rolfbjarne May 4, 2022
4d81997
Enable verbose mode and add reshape names to help diagnosing crash.
rolfbjarne May 5, 2022
6e54512
Merge remote-tracking branch 'origin/main' into mpsgraph
rolfbjarne May 5, 2022
eba108c
Revert "Enable verbose mode and add reshape names to help diagnosing …
rolfbjarne May 9, 2022
a9b1e57
Merge remote-tracking branch 'origin/main' into mpsgraph
rolfbjarne May 9, 2022
5323498
Require macOS 12+ for the new test.
rolfbjarne May 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,8 @@
/src/metalkit.cs
/src/MetalPerformanceShaders
/src/metalperformanceshaders.cs
/src/MetalPerformanceShadersGraph
/src/metalperformanceshadersgraph.cs
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
/src/MobileCoreServices
/src/mobilecoreservices.cs
/src/ModelIO
Expand Down
1 change: 1 addition & 0 deletions src/Foundation/NSObject.mac.cs
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ public partial class NSObject : INativeObject
static IntPtr ios = Dlfcn.dlopen (Constants.IOSurfaceLibrary, 1);
static IntPtr ex = Dlfcn.dlopen (Constants.ExternalAccessoryLibrary, 1);
static IntPtr ms = Dlfcn.dlopen (Constants.MetalPerformanceShadersLibrary, 1);
static IntPtr msg = Dlfcn.dlopen (Constants.MetalPerformanceShadersGraphLibrary, 1);
static IntPtr bc = Dlfcn.dlopen (Constants.BusinessChatLibrary, 1);
static IntPtr ad = Dlfcn.dlopen (Constants.AdSupportLibrary, 1);
static IntPtr nl = Dlfcn.dlopen (Constants.NaturalLanguageLibrary, 1);
Expand Down
42 changes: 42 additions & 0 deletions src/MetalPerformanceShaders/MPSNDArray.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,18 @@
namespace MetalPerformanceShaders {
public partial class MPSNDArray {

public static MPSNDArray Create (IMTLDevice device, ReadOnlySpan<float> values, params int[] shape)
{
var ushape = new nuint[shape.Length];
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
for (var i = 0; i < shape.Length; i++) {
ushape[i] = (nuint)shape[i];
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}
var desc = MPSNDArrayDescriptor.Create (MPSDataType.Float32, ushape);
var ndarray = new MPSNDArray (device, desc);
ndarray.Write (values);
return ndarray;
}

public void ExportData (IMTLCommandBuffer cmdBuf, IMTLBuffer buffer, MPSDataType sourceDataType, nuint offset)
{
ExportData (cmdBuf, buffer, sourceDataType, offset, IntPtr.Zero);
Expand Down Expand Up @@ -37,6 +49,21 @@ public unsafe void WriteBytes (IntPtr buffer, nint[] strideBytesPerDimension)
WriteBytes (buffer, (IntPtr)p);
}
}
public unsafe void Write (ReadOnlySpan<float> values)
{
if (DataType != MPSDataType.Float32)
throw new InvalidOperationException($"Attempted to write array data of type {DataType} to span of Float32s.");
nuint length = 1;
var ndims = NumberOfDimensions;
for (nuint i = 0; i < ndims; i++) {
length *= GetLength(i);
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}
if (length != (nuint)values.Length)
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
throw new ArgumentException($"The number of values ({values.Length}) does not match the shape length ({length})");
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
fixed (float* p = values) {
WriteBytes ((IntPtr)p, strideBytesPerDimension: IntPtr.Zero);
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}
}

public void ReadBytes (IntPtr buffer)
{
Expand All @@ -48,5 +75,20 @@ public unsafe void ReadBytes (IntPtr buffer, nint[] strideBytesPerDimension)
ReadBytes (buffer, (IntPtr)p);
}
}
public unsafe void Read (Span<float> values)
{
if (DataType != MPSDataType.Float32)
throw new InvalidOperationException($"Attempted to read array data of type {DataType} to span of Float32s.");
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
nuint length = 1;
var ndims = NumberOfDimensions;
for (nuint i = 0; i < ndims; i++) {
length *= GetLength(i);
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}
if (length != (nuint)values.Length)
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
throw new ArgumentException($"The number of values ({values.Length}) does not match the shape length ({length})");
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
fixed (float* p = values) {
ReadBytes ((IntPtr)p, strideBytesPerDimension: IntPtr.Zero);
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
}
117 changes: 117 additions & 0 deletions src/MetalPerformanceShadersGraph/MPSGraphEnums.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
using System;
using System.Runtime.InteropServices;

using Foundation;
using ObjCRuntime;
using Metal;

namespace MetalPerformanceShadersGraph
{
[Flags]
public enum MPSGraphOptions : ulong
{
None = 0,
SynchronizeResults = 1,
Verbose = 2,
Default = SynchronizeResults
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

[Native]
public enum MPSGraphTensorNamedDataLayout : ulong
{
Nchw = 0,
Nhwc = 1,
Oihw = 2,
Hwio = 3,
Chw = 4,
Hwc = 5,
Hw = 6
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

[Native]
public enum MPSGraphPaddingStyle : ulong
{
Explicit = 0,
Valid = 1,
Same = 2,
ExplicitOffset = 3
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

[Native]
public enum MPSGraphPaddingMode : long
{
Constant = 0,
Reflect = 1,
Symmetric = 2,
ClampToEdge = 3,
Zero = 4,
Periodic = 5,
AntiPeriodic = 6
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

[Native]
public enum MPSGraphReductionMode : ulong
{
Min = 0,
Max = 1,
Sum = 2,
Product = 3,
ArgumentMin = 4,
ArgumentMax = 5
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

[Native]
public enum MPSGraphResizeMode : ulong
{
Nearest = 0,
Bilinear = 1
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

[Native]
public enum MPSGraphScatterMode : long
{
Add = 0,
Sub = 1,
Mul = 2,
Div = 3,
Min = 4,
Max = 5,
Set = 6
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

public enum MPSGraphDeviceType : uint
{
MPSGraphDeviceTypeMetal = 0
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

public enum MPSGraphLossReductionType : ulong
{
Axis = 0,
Sum = 1,
Mean = 2
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

// For COO, indexTensor0 is x index and indexTensor1 is y index
// For CSC, indexTensor0 and indexTensor1 correspond to rowIndex and colStarts respectively.
// For CSR, indexTensor0 and indexTensor1 correspond to colIndex and rowStarts respectively.
public enum MPSGraphSparseStorageType : ulong
{
Coo = 0,
Csc = 1,
Csr = 2
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

public enum MPSGraphRandomDistribution : ulong
{
Uniform = 0,
Normal = 1,
TruncatedNormal = 2
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

public enum MPSGraphRandomNormalSamplingMethod : ulong
{
InvCdf = 0,
BoxMuller = 1
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

}
60 changes: 60 additions & 0 deletions src/MetalPerformanceShadersGraph/MPSGraphExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#nullable enable

using System;
using System.Buffers;
using System.Runtime.InteropServices;

using Foundation;
using ObjCRuntime;
using Metal;
using MetalPerformanceShaders;

namespace MetalPerformanceShadersGraph
{
public static partial class MPSGraph_MemoryOps
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
{
public static unsafe MPSGraphTensor Constant (this MPSGraph graph, float scalar)
{
return graph.Constant ((double)scalar, new[]{ 1 }, MPSDataType.Float32);
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
}

public static unsafe MPSGraphTensor Constant (this MPSGraph graph, ReadOnlySpan<float> values, int[] shape)
{
var length = 1;
for (var i = 0; i < shape.Length; i++)
length *= shape[i];
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
if (length != values.Length)
throw new ArgumentException ($"The number of values ({values.Length}) does not match the shape length ({length})");
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
fixed (float* p = values) {
using var data = NSData.FromBytesNoCopy ((IntPtr)p, (nuint)(values.Length * 4), freeWhenDone: false);
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
return graph.Constant (data, shape, MPSDataType.Float32);
rolfbjarne marked this conversation as resolved.
Show resolved Hide resolved
}
}

public static MPSGraphTensor Variable (this MPSGraph graph, float initialValue, int[] shape, string? name = null)
{
var length = 1;
for (var i = 0; i < shape.Length; i++)
length *= shape[i];
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
var pool = ArrayPool<float>.Shared;
var a = pool.Rent (length);
Array.Fill (a, initialValue);
var v = Variable (graph, a, shape, name);
pool.Return (a);
return v;
}

public static unsafe MPSGraphTensor Variable (this MPSGraph graph, ReadOnlySpan<float> initialValues, int[] shape, string? name = null)
{
var length = 1;
for (var i = 0; i < shape.Length; i++)
length *= shape[i];
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
if (length != initialValues.Length)
throw new ArgumentException ($"The number of initial values ({initialValues.Length}) does not match the shape length ({length})");
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
fixed (float* p = initialValues) {
using var data = NSData.FromBytesNoCopy ((IntPtr)p, (nuint)(initialValues.Length * 4), freeWhenDone: false);
praeclarum marked this conversation as resolved.
Show resolved Hide resolved
return graph.Variable (data, shape, MPSDataType.Float32, name);
}
}
}
}
33 changes: 33 additions & 0 deletions src/MetalPerformanceShadersGraph/MPSGraphTensorData.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#nullable enable

using System;
using System.Buffers;
using System.Runtime.InteropServices;

using Foundation;
using ObjCRuntime;
using Metal;
using MetalPerformanceShaders;

namespace MetalPerformanceShadersGraph
{
public partial class MPSGraphTensorData
{
public static MPSGraphTensorData Create (IMTLDevice device, ReadOnlySpan<float> values, params int[] shape)
{
var ndarray = MPSNDArray.Create (device, values, shape);
return new MPSGraphTensorData (ndarray);
}

public static MPSGraphTensorData Create (params MPSImage[] imageBatch)
{
return new MPSGraphTensorData (NSArray<MPSImage>.FromNSObjects (imageBatch));
}

public void Read (Span<float> values)
{
using var ndarray = this.MPSNDArray;
ndarray.Read (values);
}
}
}
13 changes: 13 additions & 0 deletions src/frameworks.sources
Original file line number Diff line number Diff line change
Expand Up @@ -1187,6 +1187,15 @@ METALPERFORMANCESHADERS_SOURCES = \
MetalPerformanceShaders/MPSStateBatch.cs \
MetalPerformanceShaders/MPSStateResourceList.cs \

# MetalPerformanceShadersGraph

METALPERFORMANCESHADERSGRAPH_API_SOURCES = \
MetalPerformanceShadersGraph/MPSGraphEnums.cs \

METALPERFORMANCESHADERSGRAPH_SOURCES = \
MetalPerformanceShadersGraph/MPSGraphExtensions.cs \
MetalPerformanceShadersGraph/MPSGraphTensorData.cs \

# MetricKit

METRICKIT_SOURCES = \
Expand Down Expand Up @@ -2017,6 +2026,7 @@ MACOS_FRAMEWORKS = \
Metal \
MetalKit \
MetalPerformanceShaders \
MetalPerformanceShadersGraph \
MetricKit \
MLCompute \
MobileCoreServices \
Expand Down Expand Up @@ -2125,6 +2135,7 @@ IOS_FRAMEWORKS = \
Metal \
MetalKit \
MetalPerformanceShaders \
MetalPerformanceShadersGraph \
MetricKit \
MLCompute \
MobileCoreServices \
Expand Down Expand Up @@ -2229,6 +2240,7 @@ TVOS_FRAMEWORKS = \
Metal \
MetalKit \
MetalPerformanceShaders \
MetalPerformanceShadersGraph \
MLCompute \
MobileCoreServices \
ModelIO \
Expand Down Expand Up @@ -2304,6 +2316,7 @@ MACCATALYST_FRAMEWORKS = \
Metal \
MetalKit \
MetalPerformanceShaders \
MetalPerformanceShadersGraph \
MetricKit \
MLCompute \
MobileCoreServices \
Expand Down
2 changes: 2 additions & 0 deletions src/generator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -790,6 +790,8 @@ public NamespaceManager (BindingTouch binding_touch, string customObjCRuntimeNS,
ImplicitNamespaces.Add ("ModelIO");
if (Frameworks.HaveMetal)
ImplicitNamespaces.Add ("Metal");
if (Frameworks.HaveMetalPerformanceShadersGraph)
ImplicitNamespaces.Add ("MetalPerformanceShadersGraph");

if (Frameworks.HaveCoreImage)
ImplicitNamespaces.Add ("CoreImage");
Expand Down
Loading