Add MetalPerformanceShadersGraph Bindings #14303

praeclarum · 2022-03-03T22:23:31Z

I'm very pleased to present full bindings to the MetalPerformanceShadersGraph framework!

I'm happy with how everything turned out with the exception of a few notes and questions below.

I re-implemented Apple's MNIST sample (from https://developer.apple.com/documentation/metalperformanceshadersgraph/training_a_neural_network_using_mps_graph) here:

https://gist.github.com/praeclarum/b8077771fb341a1f9c28240113e00425

Fixes #14286

Notes

Although the API says it works on macOS 11, it has bugs and crashes with errors even with Apple’s Swift examples. It’s better on macOS 12. iOS 14 and on is fine.
MPSGraphSparseStorageType has terrible names. They match Apple's but I wish they were better.
I added convenience methods to MPSNDArray and MPSGrapTensorData and the Variable and Constant operations to decrease the amount of unsafe code users have to write. I currently do this for 32-bit floats, the most common data type.

Questions

In Target.cs GatherFrameworks, Metal is removed from simulator builds. Doesn’t Metal work now?
Is the way I handle MPSGraphType OK? MPSGraphType was introduced in iOS 15 (macOS 12) and became the base class for MPSGraphShapedType which existed in iOS 14. In iOS 14, MPSGraphShapedType inherits directly from NSObject. To keep compatibility, MPSGraphType is type-erased to just NSObject and implementers have to add NSCopying.

…GraphTensorData

rolfbjarne · 2022-03-03T22:47:05Z

/azp run

azure-pipelines · 2022-03-03T22:47:24Z

Azure Pipelines successfully started running 1 pipeline(s).

dalexsoto

I'll continue reviewing/address questions tomorrow with a fresh brain heh, but so far it is looking great!

src/MetalPerformanceShaders/MPSNDArray.cs

src/MetalPerformanceShadersGraph/MPSGraphEnums.cs

src/MetalPerformanceShadersGraph/MPSGraphExtensions.cs

src/metalperformanceshadersgraph.cs

Co-authored-by: Alex Soto <[email protected]>

vs-mobiletools-engineering-service2 · 2022-05-06T01:31:29Z

🔥 Tests failed catastrophically on VSTS: simulator tests iOS (no summary found). 🔥

Result file D:\a\1\s\Reports\TestSummary-simulator\TestSummary.md not found.

Pipeline on Agent XAMBOT-1101.Monterey'
Merge 6e54512 into a896490

rolfbjarne · 2022-05-06T06:36:53Z

/azp run

azure-pipelines · 2022-05-06T06:37:17Z

Azure Pipelines successfully started running 1 pipeline(s).

praeclarum · 2022-05-06T16:58:39Z

@rolfbjarne I know I’ve seen this error before, but I frustratingly can’t remember the context..

I’m pretty sure the Apple sample produces this error too on old bad implementations of MPSGraph. I know I ran into it, got frustrated and moved on. I was working with the Swift sample at the time, but I can’t remember the OS.

I can’t tell from the tests whether this is working on Mac 12 (M1 and Intel). If it is, then I’m willing to chalk this up to old bugs in Mac 11. I know the Apple sample works on Mac 12.2 M1. If this does too, then I think we should just disable the test for older Mac versions.

rolfbjarne · 2022-05-09T08:38:55Z

@praeclarum ok, I'll just make the test require macOS 12+

…crash." This reverts commit 4d81997.

rolfbjarne · 2022-05-09T08:42:28Z

/azp run

azure-pipelines · 2022-05-09T08:42:51Z

Azure Pipelines successfully started running 1 pipeline(s).

vs-mobiletools-engineering-service2 · 2022-05-09T19:43:34Z

🔥 Tests failed catastrophically on VSTS: simulator tests iOS (no summary found). 🔥

Result file D:\a\1\s\Reports\TestSummary-simulator\TestSummary.md not found.

Pipeline on Agent XAMBOT-1109.Monterey'
Merge 5323498 into e9333ba

rolfbjarne · 2022-05-09T20:02:20Z

/azp run

azure-pipelines · 2022-05-09T20:02:45Z

Azure Pipelines successfully started running 1 pipeline(s).

vs-mobiletools-engineering-service2 · 2022-05-09T21:28:05Z

📚 [CI Build] Artifacts 📚

Artifacts were not provided.

Pipeline on Agent XAMBOT-1167.Monterey
Hash: 53234981def2e09aa0facd84996dec8c9760bda2

vs-mobiletools-engineering-service2 · 2022-05-09T21:42:33Z

📋 [PR Build] API Diff 📋

API Current PR diff

ℹ️ API Diff (from PR only) (please review changes)

View API diff

iOS vsdrops gist
macOS vsdrops gist
tvOS vsdrops gist
watchOS vsdrops gist

View dotnet API diff

dotnet-iOS vsdrops gist
dotnet-macOS vsdrops gist
dotnet-tvOS vsdrops gist
dotnet-MacCatalyst vsdrops gist

View dotnet legacy API diff

dotnet-legacy-iOS vsdrops gist
dotnet-legacy-macOS vsdrops gist
dotnet-legacy-tvOS vsdrops gist

View dotnet iOS-MacCatalayst API diff

dotnet-macCatiOS vsdrops gist

API diff

✅ API Diff from stable

View API diff

iOS vsdrops gist
macOS vsdrops gist
tvOS vsdrops gist
watchOS vsdrops gist

View dotnet API diff

dotnet-iOS vsdrops gist
dotnet-macOS vsdrops gist
dotnet-tvOS vsdrops gist
dotnet-MacCatalyst vsdrops gist

View dotnet legacy API diff

dotnet-legacy-iOS vsdrops gist
dotnet-legacy-macOS vsdrops gist
dotnet-legacy-tvOS vsdrops gist

View dotnet iOS-MacCatalayst API diff

dotnet-macCatiOS vsdrops gist

Generator diff

ℹ️ Generator Diff (please review changes)

Pipeline on Agent XAMBOT-1164.Monterey'
Hash: 53234981def2e09aa0facd84996dec8c9760bda2

vs-mobiletools-engineering-service2 · 2022-05-09T21:47:05Z

❌ [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) failed ❌

Failed tests are:

linksdk
linkall

Pipeline on Agent
Hash: 53234981def2e09aa0facd84996dec8c9760bda2

vs-mobiletools-engineering-service2 · 2022-05-09T22:18:04Z

💻 [PR Build] Tests on macOS Mac Catalina (10.15) passed 💻

✅ All tests on macOS Mac Catalina (10.15) passed.

Pipeline on Agent
Hash: 53234981def2e09aa0facd84996dec8c9760bda2

vs-mobiletools-engineering-service2 · 2022-05-10T05:30:24Z

❌ [CI Build] Tests failed on VSTS: simulator tests iOS ❌

Tests failed on VSTS: simulator tests iOS.

Test results

23 tests failed, 130 tests passed.

Failed tests

link sdk/Mac [dotnet]/Debug [dotnet]: Failed (Test run failed.
Tests run: 117 Passed: 107 Inconclusive: 0 Failed: 1 Ignored: 9)
link sdk/Mac [dotnet]/Release [dotnet]: Failed (Test run failed.
Tests run: 117 Passed: 106 Inconclusive: 0 Failed: 1 Ignored: 10)
link sdk/Mac Catalyst [dotnet]/Debug [dotnet]: Failed (Test run failed.
Tests run: 129 Passed: 119 Inconclusive: 0 Failed: 1 Ignored: 9)
link sdk/Mac Catalyst [dotnet]/Release [dotnet]: Failed (Test run failed.
Tests run: 129 Passed: 118 Inconclusive: 0 Failed: 1 Ignored: 10)
link sdk/Mac Modern/Debug: Failed (Test run failed.
Tests run: 9 Passed: 8 Inconclusive: 0 Failed: 1 Ignored: 0)
link sdk/Mac Modern/Release: Failed (Test run failed.
Tests run: 9 Passed: 8 Inconclusive: 0 Failed: 1 Ignored: 0)
link sdk/iOS Unified 64-bits - simulator/Debug [dotnet]: Failed
link sdk/iOS Unified 64-bits - simulator/Release [dotnet]: Failed
link sdk/iOS Unified 64-bits - simulator/Debug: Failed
link sdk/iOS Unified 64-bits - simulator/Release: Failed
link sdk/tvOS - simulator/Release [dotnet]: Failed
link sdk/tvOS - simulator/Release: Failed
trimmode link/Mac [dotnet]/Debug [dotnet]: Failed (Test run failed.
Tests run: 117 Passed: 107 Inconclusive: 0 Failed: 1 Ignored: 9)
trimmode link/Mac [dotnet]/Release [dotnet]: Failed (Test run failed.
Tests run: 117 Passed: 106 Inconclusive: 0 Failed: 1 Ignored: 10)
trimmode link/Mac Catalyst [dotnet]/Debug [dotnet]: Failed (Test run failed.
Tests run: 129 Passed: 119 Inconclusive: 0 Failed: 1 Ignored: 9)
trimmode link/Mac Catalyst [dotnet]/Release [dotnet]: Failed (Test run failed.
Tests run: 129 Passed: 118 Inconclusive: 0 Failed: 1 Ignored: 10)
trimmode link/iOS Unified 64-bits - simulator/Debug [dotnet]: Failed
trimmode link/iOS Unified 64-bits - simulator/Release [dotnet]: Failed
trimmode link/tvOS - simulator/Debug [dotnet]: Failed
trimmode link/tvOS - simulator/Release [dotnet]: Failed
link all/Mac Modern/Debug: Failed (Test run failed.
Tests run: 20 Passed: 18 Inconclusive: 0 Failed: 1 Ignored: 1)
link all/Mac Modern/Release: Failed (Test run failed.
Tests run: 20 Passed: 18 Inconclusive: 0 Failed: 1 Ignored: 1)
introspection/iOS Unified 64-bits - simulator/Debug: Crashed

Pipeline on Agent XAMBOT-1023.Monterey'
Merge 5323498 into 32c29c1

rolfbjarne

Looks like all test failures but one are network related (https://github.com/xamarin/maccore/issues/1067).

The remaining one seems to be a simulator hiccup.

praeclarum added 20 commits March 1, 2022 18:54

Scaffold MPSGraph

6b9b285

Add graph options and placeholder tensors

f266883

Finish MPSGraph.h

c6458fb

Update todos

592c55f

Add graph operations

6e389eb

Fix bool

0ea620f

Add Sharpie definitions

03b2351

Type erase MPSGraphType

42a810e

Import Sharpie Graph Ops

6bdcda1

Improve Sharpie Op names

8a0f9f0

Use int[] for NSNumber[] return types

057dcc9

Use int[] for size parameters

d313fa8

Add MacCatalyst (15,0)

8cb1d9c

Move enums into correct file

3823001

Improve names

b047f75

Make Options Flags

5fa0e72

Use NSArray for image batches

690b25e

Add Variable and Constant convenience methods

628bbc0

Add convenience methods for reading and writing to MPSNDArray and MPS…

3e41079

…GraphTensorData

Add tensor data create convenience taking individual images

7b4780f

praeclarum requested review from rolfbjarne, chamons and dalexsoto as code owners March 3, 2022 22:23

rolfbjarne added community Community contribution ❤ run-dotnet-tests Run all the .NET tests labels Mar 3, 2022

dalexsoto reviewed Mar 4, 2022

View reviewed changes

praeclarum and others added 2 commits March 3, 2022 19:52

Apply suggestions from code review

5e3ba5b

Co-authored-by: Alex Soto <[email protected]>

Update src/MetalPerformanceShadersGraph/MPSGraphExtensions.cs

7867705

Co-authored-by: Alex Soto <[email protected]>

This comment has been minimized.

Sign in to view

rolfbjarne added 3 commits May 9, 2022 10:39

Revert "Enable verbose mode and add reshape names to help diagnosing …

eba108c

…crash." This reverts commit 4d81997.

Merge remote-tracking branch 'origin/main' into mpsgraph

a9b1e57

Require macOS 12+ for the new test.

5323498

This comment has been minimized.

Sign in to view

rolfbjarne approved these changes May 10, 2022

View reviewed changes

rolfbjarne merged commit bd4fee0 into xamarin:main May 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MetalPerformanceShadersGraph Bindings #14303

Add MetalPerformanceShadersGraph Bindings #14303

praeclarum commented Mar 3, 2022 •

edited

Loading

rolfbjarne commented Mar 3, 2022

azure-pipelines bot commented Mar 3, 2022

dalexsoto left a comment

This comment has been minimized.

vs-mobiletools-engineering-service2 commented May 6, 2022

rolfbjarne commented May 6, 2022

azure-pipelines bot commented May 6, 2022

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

praeclarum commented May 6, 2022

rolfbjarne commented May 9, 2022

rolfbjarne commented May 9, 2022

azure-pipelines bot commented May 9, 2022

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented May 9, 2022

rolfbjarne commented May 9, 2022

azure-pipelines bot commented May 9, 2022

vs-mobiletools-engineering-service2 commented May 9, 2022

vs-mobiletools-engineering-service2 commented May 9, 2022

vs-mobiletools-engineering-service2 commented May 9, 2022

vs-mobiletools-engineering-service2 commented May 9, 2022

vs-mobiletools-engineering-service2 commented May 10, 2022

Failed tests

rolfbjarne left a comment

Add MetalPerformanceShadersGraph Bindings #14303

Add MetalPerformanceShadersGraph Bindings #14303

Conversation

praeclarum commented Mar 3, 2022 • edited Loading

Notes

Questions

rolfbjarne commented Mar 3, 2022

azure-pipelines bot commented Mar 3, 2022

dalexsoto left a comment

Choose a reason for hiding this comment

This comment has been minimized.

vs-mobiletools-engineering-service2 commented May 6, 2022

🔥 Tests failed catastrophically on VSTS: simulator tests iOS (no summary found). 🔥

rolfbjarne commented May 6, 2022

azure-pipelines bot commented May 6, 2022

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

praeclarum commented May 6, 2022

rolfbjarne commented May 9, 2022

rolfbjarne commented May 9, 2022

azure-pipelines bot commented May 9, 2022

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented May 9, 2022

🔥 Tests failed catastrophically on VSTS: simulator tests iOS (no summary found). 🔥

rolfbjarne commented May 9, 2022

azure-pipelines bot commented May 9, 2022

vs-mobiletools-engineering-service2 commented May 9, 2022

📚 [CI Build] Artifacts 📚

vs-mobiletools-engineering-service2 commented May 9, 2022

📋 [PR Build] API Diff 📋

API Current PR diff

API diff

Generator diff

vs-mobiletools-engineering-service2 commented May 9, 2022

❌ [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) failed ❌

vs-mobiletools-engineering-service2 commented May 9, 2022

💻 [PR Build] Tests on macOS Mac Catalina (10.15) passed 💻

vs-mobiletools-engineering-service2 commented May 10, 2022

❌ [CI Build] Tests failed on VSTS: simulator tests iOS ❌

Test results

Failed tests

rolfbjarne left a comment

Choose a reason for hiding this comment

praeclarum commented Mar 3, 2022 •

edited

Loading