Initial release of Avatar 3.0 Emulator v0.2

AvatarMasks/LyumaEmptyMask.mask

@@ -0,0 +1,11 @@
+%YAML 1.1
+%TAG !u! tag:unity3d.com,2011:
+--- !u!319 &31900000
+AvatarMask:
+  m_ObjectHideFlags: 0
+  m_CorrespondingSourceObject: {fileID: 0}
+  m_PrefabInstance: {fileID: 0}
+  m_PrefabAsset: {fileID: 0}
+  m_Name: LyumaEmptyMask
+  m_Mask: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
+  m_Elements: []

AvatarMasks/LyumaFullMask.mask

@@ -0,0 +1,13 @@
+%YAML 1.1
+%TAG !u! tag:unity3d.com,2011:
+--- !u!319 &31900000
+AvatarMask:
+  m_ObjectHideFlags: 0
+  m_CorrespondingSourceObject: {fileID: 0}
+  m_PrefabInstance: {fileID: 0}
+  m_PrefabAsset: {fileID: 0}
+  m_Name: LyumaFullMask
+  m_Mask: 01000000010000000100000001000000010000000100000001000000010000000100000001000000010000000100000001000000
+  m_Elements:
+  - m_Path: 
+    m_Weight: 1

AvatarMasks/LyumaNoTransformMask.mask

@@ -0,0 +1,129 @@
+%YAML 1.1
+%TAG !u! tag:unity3d.com,2011:
+--- !u!319 &31900000
+AvatarMask:
+  m_ObjectHideFlags: 0
+  m_CorrespondingSourceObject: {fileID: 0}
+  m_PrefabInstance: {fileID: 0}
+  m_PrefabAsset: {fileID: 0}
+  m_Name: LyumaNoTransformMask
+  m_Mask: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
+  m_Elements:
+  - m_Path: 
+    m_Weight: 1
+  - m_Path: Armature
+    m_Weight: 0
+  - m_Path: Armature/Hips
+    m_Weight: 0
+  - m_Path: Armature/Hips/LeftUpLeg
+    m_Weight: 0
+  - m_Path: Armature/Hips/LeftUpLeg/LeftLeg
+    m_Weight: 0
+  - m_Path: Armature/Hips/LeftUpLeg/LeftLeg/LeftFoot
+    m_Weight: 0
+  - m_Path: Armature/Hips/LeftUpLeg/LeftLeg/LeftFoot/LeftToe
+    m_Weight: 0
+  - m_Path: Armature/Hips/RightUpLeg
+    m_Weight: 0
+  - m_Path: Armature/Hips/RightUpLeg/RightLeg
+    m_Weight: 0
+  - m_Path: Armature/Hips/RightUpLeg/RightLeg/RightFoot
+    m_Weight: 0
+  - m_Path: Armature/Hips/RightUpLeg/RightLeg/RightFoot/RightToe
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandIndex1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandIndex1/LeftHandIndex2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandIndex1/LeftHandIndex2/LeftHandIndex3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandMiddle1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandMiddle1/LeftHandMiddle2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandMiddle1/LeftHandMiddle2/LeftHandMiddle3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandPinky1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandPinky1/LeftHandPinky2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandPinky1/LeftHandPinky2/LeftHandPinky3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandRing1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandRing1/LeftHandRing2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandRing1/LeftHandRing2/LeftHandRing3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandThumb1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandThumb1/LeftHandThumb2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/LeftHandThumb1/LeftHandThumb2/LeftHandThumb3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/LeftShoulder/LeftArm/LeftForeArm/LeftHand/lHandAttachmentPointL
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/Neck
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/Neck/Head
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/Neck/Head/HeadTop_End
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/Neck/Head/LeftEye
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/Neck/Head/RightEye
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/rHandAttachmentPointR
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandIndex1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandIndex1/RightHandIndex2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandIndex1/RightHandIndex2/RightHandIndex3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandMiddle1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandMiddle1/RightHandMiddle2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandMiddle1/RightHandMiddle2/RightHandMiddle3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandPinky1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandPinky1/RightHandPinky2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandPinky1/RightHandPinky2/RightHandPinky3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandRing1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandRing1/RightHandRing2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandRing1/RightHandRing2/RightHandRing3
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandThumb1
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandThumb1/RightHandThumb2
+    m_Weight: 0
+  - m_Path: Armature/Hips/Spine/Chest/RightShoulder/RightArm/RightForeArm/RightHand/RightHandThumb1/RightHandThumb2/RightHandThumb3
+    m_Weight: 0
+  - m_Path: Body
+    m_Weight: 0

Editor/LyumaAv3EditorSupport.cs

@@ -0,0 +1,110 @@
+/* Copyright (c) 2020 Lyuma <[email protected]>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE. */
+using System.Collections;
+using System.Collections.Generic;
+using UnityEngine;
+using UnityEditor;
+using UnityEngine.Animations;
+using UnityEditor.Animations;
+using UnityEditor.Playables;
+using UnityEngine.Playables;
+using VRC.SDK3.Components;
+
+[InitializeOnLoadAttribute]
+public static class LyumaAv3EditorSupport
+{
+    static Dictionary<VRCAvatarDescriptor.AnimLayerType, string> animLayerToDefaultFile = new Dictionary<VRCAvatarDescriptor.AnimLayerType, string> {
+        {VRCAvatarDescriptor.AnimLayerType.TPose, "vrc_AvatarV3UtilityTPose"},
+        {VRCAvatarDescriptor.AnimLayerType.IKPose, "vrc_AvatarV3UtilityIKPose"},
+        {VRCAvatarDescriptor.AnimLayerType.SpecialIK, "vrc_AvatarV3UtilityTPose"},
+        {VRCAvatarDescriptor.AnimLayerType.Base, "vrc_AvatarV3LocomotionLayer"},
+        {VRCAvatarDescriptor.AnimLayerType.Sitting, "vrc_AvatarV3SittingLayer"},
+        {VRCAvatarDescriptor.AnimLayerType.Additive, "vrc_AvatarV3IdleLayer"},
+        {VRCAvatarDescriptor.AnimLayerType.FX, "vrc_AvatarV3FaceLayer"},
+        {VRCAvatarDescriptor.AnimLayerType.Action, "vrc_AvatarV3ActionLayer"},
+        {VRCAvatarDescriptor.AnimLayerType.Gesture, "vrc_AvatarV3HandsLayer"},
+    };
+    static Dictionary<VRCAvatarDescriptor.AnimLayerType, string> animLayerToDefaultAvaMaskFile = new Dictionary<VRCAvatarDescriptor.AnimLayerType, string>
+    {
+        {VRCAvatarDescriptor.AnimLayerType.TPose, "vrc_MusclesOnly"},
+        {VRCAvatarDescriptor.AnimLayerType.IKPose, "vrc_MusclesOnly"},
+        {VRCAvatarDescriptor.AnimLayerType.SpecialIK, "vrc_MusclesOnly"},
+        {VRCAvatarDescriptor.AnimLayerType.Base, "LyumaFullMask"},
+        {VRCAvatarDescriptor.AnimLayerType.Sitting, "LyumaFullMask"},
+        {VRCAvatarDescriptor.AnimLayerType.Additive, "LyumaFullMask"},
+        {VRCAvatarDescriptor.AnimLayerType.FX, "LyumaEmptyMask"},
+        {VRCAvatarDescriptor.AnimLayerType.Action,"vrc_MusclesOnly"},
+        {VRCAvatarDescriptor.AnimLayerType.Gesture, "vrc_HandsOnly"},
+    };
+
+
+    static void InitDefaults() {
+        foreach (var kv in animLayerToDefaultFile) {
+            if (kv.Value == null) {
+                LyumaAv3Runtime.animLayerToDefaultController[kv.Key] = null;
+            } else
+            {
+                AnimatorController ac = null;
+                foreach (var guid in AssetDatabase.FindAssets(kv.Value))
+                {
+                    string path = AssetDatabase.GUIDToAssetPath(guid);
+                    ac = AssetDatabase.LoadAssetAtPath<AnimatorController>(path);
+                }
+                if (ac == null)
+                {
+                    Debug.LogWarning("Failed to resolve animator controller " + kv.Value + " for " + kv.Key);
+                    ac = null;
+                }
+                LyumaAv3Runtime.animLayerToDefaultController[kv.Key] = ac;
+            }
+        }
+        foreach (var kv in animLayerToDefaultAvaMaskFile) {
+            if (kv.Value == null) {
+                LyumaAv3Runtime.animLayerToDefaultAvaMask[kv.Key] = null;
+            } else
+            {
+                AvatarMask mask = null;
+                foreach (var guid in AssetDatabase.FindAssets(kv.Value))
+                {
+                    string path = AssetDatabase.GUIDToAssetPath(guid);
+                    mask = AssetDatabase.LoadAssetAtPath<AvatarMask>(path);
+                }
+                if (mask == null)
+                {
+                    Debug.LogWarning("Failed to resolve avatar mask " + kv.Value + " for " + kv.Key);
+                    mask = new AvatarMask();
+                }
+                LyumaAv3Runtime.animLayerToDefaultAvaMask[kv.Key] = mask;
+            }
+        }
+    }
+
+    // register an event handler when the class is initialized
+    static LyumaAv3EditorSupport()
+    {
+        InitDefaults();
+    }
+
+    [MenuItem("Tools/Enable Avatars 3.0 Emulator")]
+    public static void EnableAv3Testing() {
+        GameObject go = new GameObject("Avatars 3.0 Emulator Control");
+        go.AddComponent<LyumaAv3Emulator>();
+    }
+}

README.md

@@ -0,0 +1,46 @@
+# Avatar 3.0 Emulator
+This is an emulator for Avatars 3.0 reimplemented in the unity editor on top the the unity [PlayableGraph](https://docs.unity3d.com/Manual/Playables-Graph.html) API, using the [AnimationControllerPlayable](https://docs.unity3d.com/2018.4/Documentation/ScriptReference/Animations.AnimatorControllerPlayable.html) and [AnimationLayerMixerPlayable](https://docs.unity3d.com/2018.4/Documentation/ScriptReference/Animations.AnimationLayerMixerPlayable.html) APIs.
+
+## Features:
+* Should emulate most features of Avatar3.
+* Test non-local syncing by duplicating or clicking the "Create Non Local Clone" checkbox.
+* Supports viewing and editing float and int paramters. Alt-click the ▶Floats and ▶Ints headers at the bottom.
+* Supports live viewing of animator controller state. To use, click the avatar in the scene, then in project view, click the correct animator controller. This will be your own if overriding; or it will be one of the defaults in VRCSDK/Examples3/Animation/Controllers. If you did this right, the Animator window should "Auto Live Link" the controller state and allow you to observe what is happening.
+* Shows Tracking/Animation in the inspector.
+
+## Not implemented/todo:
+* Custom inspector
+* Custom Expression Menus
+* visualization of IK Tracking state when a limb is not in Animation mode.
+* Eye Tracking / Blinking support
+* Set View position is wrong.
+* Gesture left/right weight seems wrong
+
+## To use:
+Go to the **Tools** menu, and select **Avatar 3.0 Emulator**.
+This will add an object to your scene: you can always remove it if you don't want it to run.
+
+To emulate walking and movement, click the avatar and scroll down the inspector to the bottom section with Lyuma Av3 Runtime component. Here you can change stuff.
+
+It also supports live interacting with the animator controller. To use this, first click your avatar (even if it was already selected), and then open up Windows -> Animation -> Animator ; and find the locomotion or base controller from project. If you customize it, pick your customized version... otherwise, go to VRCSDK3/Examples3/Animation/Controllers and click `vrc_AvatarV3LocomotionLayer` (or whichever controller you want to debug). You can also change parameters from inside the controller, for example moving the red dot in the 2D Blend Tree for Standing. Crouch/Prone by changing the Upright slider; or test Sitting or AFK.
+
+If you wish to emulate walking, you can also do this by opening up the Locmotion controller with your avatar selected, and going to the Standing blendtree and dragging around the red dot.
+
+## NOTE: about viewing animator state from layers other than Base/locomotion:
+Only the Base layer will show parameters and layer weights in the unity Animator window. Other layers will show 0's for everything and every layer will have weight 0.
+
+You can still edit parameter values, just they show 0 when you finish editing. Additionally, while it is ok to open Blend Trees in the *inspector*, opening a BlendTree in the animation editor (such as double-clicking on it) will force the input values to 0. I believe this to be a Unity bug.
+
+A workaround is don't double-click blendtrees while playing, or if you want to test the state machine, you can put your FX layer into the base slot temporarily to test it, and tick reset Avatar in the emulator component. Another tool is the "PlayableGraph Visualizer" which can be found in the unity Package Manager (Advanced -> Show preview packages). It is hard to use, but does a good job of visualizing clip, layer, and playable weights.
+
+## Inputing custom stage params:
+
+For testing your own controls, alt-click the Floats and Ints sections at the bottom of the Lyuma Av3 Runtime script to expand them all, and change the values from there. Unfortunately the expressions menu is not emulated yet: you must change the values directly.
+
+## Other known issues:
+
+As mentioned above, a Unity bug prevents you from double-clicking blendtrees in the Animator window, or from *observing* parameter values or layer weights in the Animator window.
+
+The `proxy_` animations included in the SDK are incomplete. Unless you override them, do not expect your avatar to have a full walking cycle, and it is normal for backflip (VRCEmote=6) to stop halfway.
+
+Avoid changing parameter values too quickly (for example click-dragging on the inspector row): it can cause the avatar to glitch out.