Merge pull request #1306 from anatawa12/minimum-osc-support

Improve OSC Support in Avatar Optimizer

.docs/content/docs/developers/asset-description/index.en.md

@@ -8,6 +8,21 @@ Asset Description is the file to provide information of your assets for Avatar O
 
 ## Why Asset Description is needed
 
+Avatar Optimizer uses information from Asset Description to not excessively optimize user's avatar.
+
+Current Asset Description can provide the following information:
+
+- `Meaningless Components`\
+  Components that should be ignored by Avatar Optimizer.
+- `Parameters Read By External Tools`\
+  Parameters that can be read by external tools, especially for OSC Tools.
+- `Parameters Changed By External Tools`
+  Parameters that can be changed by external tools, especially for OSC Tools.
+
+We will describe why Asset Description is needed for each information below.
+
+### Meaningless Components {#why-meaningless-components}
+
 Avatar Optimizer has to know about all existing components in the Avatar to remove unnecessary ones.\
 Avatar Optimizer v1.6.0 added [document to make your components compatible with AAO][make-component-compatible] and API for it, but
 for in-place modification tools that do not process on build,
@@ -18,6 +33,24 @@ For non-destructive tools, we still recommend you to continue to remove componen
 
 [make-component-compatible]: ../make-your-components-compatible-with-aao
 
+### Parameters Read By Extenral Tools {#why-parameters-read-by-external-tools}
+
+Components such as PhysBone and Contact Receiver will make parameters that can be read by OSC tools.
+Such parameters are readable by the OSC tools without being defined on Animator Controller or Expression Parameter.\
+Therefore, Avatar Optimizer is unable to determine whether those parameters are simply unused or intended to be read by the OSC tools.
+Since undefined parameters are relatively rarely used by OSC Tools, components that make such parameters are removed if parameters are unused from anywhere on the avatar.
+
+To prevent Avatar Optimizer from removing those parameters and components, you can specify the parameters that are read by OSC Tools in Asset Description.
+
+### Parameters Changed By External Tools {#why-parameters-written-by-external-tools}
+
+Currently this information is not actually used by Avatar Optimizer, but it is planned to be used in the future.
+
+Avatar Optimizer is planned to optimize Animator Controller by analyzing unchanged parameters.
+However, if the parameters are changed by external tools, this optimization will break effects of the external tools.
+
+To prevent this, you can specify the parameters that are changed by external tools in Asset Description.
+
 ## Create Asset Description {#create-asset-description}
 
 To create Asset Description, select `Create/Avatar Optimizer/Asset Description` from right-click menu in the Project window.\
@@ -40,3 +73,15 @@ Avatar Optimizer ignores the component of the specified Script Asset type and it
 
 In Asset Description, as with the components in the Scene, types are stored in the form of guid and fileID of the Script Asset.\
 Therefore, even if the class name is changed, the specification in Asset Description will work without any problems as long as the components in the Scene are not broken.
+
+### Parameters Read By External Tools {#parameters-read-by-external-tools}
+
+Specify the parameters that are read by external tools.
+
+Please read [above](#why-parameters-read-by-external-tools) for more information.
+
+### Parameters Changed By External Tools {#parameters-changed-by-external-tools}
+
+Specify the parameters that are changed by external tools.
+
+Please read [above](#why-parameters-written-by-external-tools) for more information.

.docs/content/docs/developers/asset-description/index.ja.md

@@ -8,6 +8,21 @@ Asset DescriptionはAvatar Optimizerにアセットの情報を提供するた
 
 ## なぜAsset Descriptionが必要なのか {#why-asset-description-is-needed}
 
+Avatar Optimizerは、Asset Descriptionで提供された情報を用いて最適化の失敗を防ぎます。
+
+Asset Descriptionでは、以下の情報を提供することができます。
+
+- Meaningless Components\
+  Avatar Optimizerに無視されるべきコンポーネントを指定します。
+- Parameters Read By External Tools\
+  OSCツールなどの外部ツールから読み取られるパラメーターを指定します。
+- Parameters Changed By External Tools\
+  OSCツールなどの外部ツールから変更されるパラメーターを指定します。
+
+各項目が必要な理由については以下の説明を参照してください。
+
+### Meaningless Components {#why-meaningless-components}
+
 アバター上の不要な要素を削除するために、Avatar Optimizerはアバターに存在するすべてのコンポーネントのことを知る必要があります。\
 Avatar Optimizer v1.6.0で[コンポーネントにAAOとの互換性をもたせるためのドキュメント][make-component-compatible]とAPIが追加されましたが、
 非破壊ツールでなく、ビルド時に処理を行わないようなツールでは、`IVRCSDKPreprocessAvatarCallback`でコンポーネントを削除するのは少し面倒だろうと考えました。\
@@ -17,6 +32,24 @@ Avatar Optimizer v1.6.0で[コンポーネントにAAOとの互換性をもた
 
 [make-component-compatible]: ../make-your-components-compatible-with-aao
 
+### Parameters Read By Extenral Tools {#why-parameters-read-by-external-tools}
+
+PhysBoneやContact Receiverのようなコンポーネントは、OSCツールで読み取り可能なパラメーターを生成します。
+そのようなパラメーターは、Animator ControllerやExpression Parameterに登録しなくても、OSCツールから読み取り可能であることが知られています。\
+そのため、Avatar Optimizerはそれらのパラメーターが単に使用されていないものなのか、OSCツールで読み取るためのものなのかを判別することができません。
+登録されていないパラメーターがOSCツールで使用されていることは比較的少ないため、コンポーネントが生成するパラメーターがアバター上で全く使われていない場合にはコンポーネントが削除されることになります。
+
+これを防ぐために、OSCツールから読み取られる目的のパラメーターをAsset Descriptionで指定することができます。
+
+### Parameters Changed By External Tools {#why-parameters-written-by-external-tools}
+
+この情報は現在のAvatar Optimizerでは使用されていませんが、将来的に使用することが計画されています。
+
+Avatar Optimizerは、一度も変更されないパラメーターを検知してアニメーターを最適化することを計画しています。
+しかしながら、OSCツールのような外部ツールでパラメーターが変更される場合、この最適化はアバターの振る舞いを変えてしまうでしょう。
+
+これを防ぐために、OSCツールから変更されるパラメーターをAsset Descriptionで指定することができます。
+
 ## Asset Descriptionの作成 {#create-asset-description}
 
 Asset Descriptionを作成するには、Projectウィンドウの右クリックメニューから`Create/Avatar Optimizer/Asset Description`を選択してください。\
@@ -39,3 +72,15 @@ Meaningless ComponentsはAvatar Optimizerに無視してほしいコンポーネ
 
 Asset Descriptionでは実際のScene上のコンポーネントと同様に、Script AssetがguidとfileIDの形で保持されています。
 そのため、クラス名を変更したとしても、シーン上のコンポーネントが壊れていない限り、Asset Descriptionでの指定も問題なく機能します。
+
+### Parameters Read By External Tools {#parameters-read-by-external-tools}
+
+OSCツールなどの外部ツールから読み取られるパラメーターを指定します。
+
+詳細は[上のセクション](#why-parameters-read-by-external-tools)を参照してください。
+
+### Parameters Changed By External Tools {#parameters-changed-by-external-tools}
+
+OSCツールなどの外部ツールから変更されるパラメーターを指定します。
+
+詳細は[上のセクション](#why-parameters-written-by-external-tools)を参照してください。

CHANGELOG-PRERELEASE.md

@@ -25,6 +25,9 @@ The format is based on [Keep a Changelog].
   - However, in Unity 2020 and later, BlendShape manipulation load is mostly proportional to the number of moving vertices.
   - This means that increasing the number of vertices in a mesh which has BlendShapes does not increase the load of BlendShape manipulation much.
   - Therefore, we decided to automatically merge such meshes.
+- Improved OSC Gimmick Support `#1306`
+  - We added two information for OSC Gimmick in Asset Description.
+  - By defining parameters read / written by OSC Gimmick, your OSC Gimmick no longer breaks.
 
 ### Changed
 

CHANGELOG.md

@@ -66,6 +66,9 @@ The format is based on [Keep a Changelog].
   - However, in Unity 2020 and later, BlendShape manipulation load is mostly proportional to the number of moving vertices.
   - This means that increasing the number of vertices in a mesh which has BlendShapes does not increase the load of BlendShape manipulation much.
   - Therefore, we decided to automatically merge such meshes.
+- Improved OSC Gimmick Support `#1306`
+  - We added two information for OSC Gimmick in Asset Description.
+  - By defining parameters read / written by OSC Gimmick, your OSC Gimmick no longer breaks.
 
 ### Changed
 - Skip Enablement Mismatched Renderers is now disabled by default `#1169`

Editor/APIInternal/ComponentInfoRegistry.cs

@@ -106,20 +106,15 @@ internal static bool TryGetInformation(Type type, out ComponentInformation infor
             return false;
         }
 
-        private static HashSet<Type>? _meaninglessTypes;
-
-        public static void InvalidateCache() => _meaninglessTypes = null;
-
         private static bool IsMeaninglessType(Type type)
         {
-            if (_meaninglessTypes == null)
-                _meaninglessTypes = new HashSet<Type>(AssetDescription.GetMeaninglessComponents());
+            var meaninglessTypes = AssetDescription.GetMeaninglessComponents();
 
             // fast path: simple check
-            if (_meaninglessTypes.Contains(type)) return true;
+            if (meaninglessTypes.Contains(type)) return true;
             // slow path: check for parent class
             for (var current = type.BaseType; current != null; current = current.BaseType)
-                if (_meaninglessTypes.Contains(current))
+                if (meaninglessTypes.Contains(current))
                     return true;
 
             return false;

Editor/AssetDescription.cs

@@ -4,6 +4,7 @@
 using nadena.dev.ndmf.localization;
 using UnityEditor;
 using UnityEngine;
+using UnityEngine.Serialization;
 using Object = UnityEngine.Object;
 
 namespace Anatawa12.AvatarOptimizer
@@ -19,27 +20,83 @@ internal class AssetDescription : ScriptableObject
 #pragma warning restore CS0414
         [SerializeField]
         private ClassReference[] meaninglessComponents = Array.Empty<ClassReference>();
+        /// <summary>
+        /// <para>
+        /// The animator parameters that External Tools reads.
+        /// </para>
+        /// <para>
+        /// Avatar Optimizer will treat changing those parameters as side effects.
+        ///
+        /// As a part of optimization, Avatar Optimizer may remove PhysBones or Contact Receivers that are not used by the animator.
+        /// However, if those parameters are used by some External Tools like OSC Tools in VRChat, it may break the behavior of the avatar.
+        /// Registering parameters to this will prevent Avatar Optimizer from removing PhysBones or Contact Receivers that are used by OSC Tools.
+        /// </para>
+        /// </summary>
+        [SerializeField]
+        private OscParameter[] parametersReadByExternalTools = Array.Empty<OscParameter>();
+        /// <summary>
+        /// <para>
+        /// The animator parameters that OSC Tools changes.
+        /// </para>
+        /// <para>
+        /// Avatar Optimizer will assume those parameters might be changed by some external tools.
+        ///
+        /// Currently, this configuration is unused.
+        ///
+        /// Avatar Optimizer will implement optimizing Animator Controller by fixing non-animated parameters.
+        /// However, if those parameters are changed by some external tools like OSC Tools in VRChat, it may break the behavior of the avatar.
+        /// Therefore, registering parameters to this will prevent Avatar Optimizer from fixing non-animated parameters.
+        /// </para>
+        /// </summary>
+        [SerializeField]
+        private OscParameter[] parametersChangedByExternalTools = Array.Empty<OscParameter>();
 
         const int MonoScriptIdentifierType = 1;
 
+        private static AssetDescriptionData? _data;
+
+        private static AssetDescriptionData Data => _data ??= LoadData();
+
+        class AssetDescriptionData
+        {
+            public HashSet<Type> meaninglessComponents = new();
+            public HashSet<OscParameter> parametersReadByExternalTools = new();
+            public HashSet<OscParameter> parametersChangedByExternalTools = new();
+        }
+
+        static AssetDescriptionData LoadData()
+        {
+            var data = new AssetDescriptionData();
+            foreach (var description in GetAllAssetDescriptions())
+            {
+                foreach (var component in description.meaninglessComponents)
+                    if (GetMonoScriptFromGuid(component.guid, component.fileID) is MonoScript monoScript)
+                        data.meaninglessComponents.Add(monoScript.GetClass());
+
+                data.parametersReadByExternalTools.UnionWith(description.parametersReadByExternalTools);
+                data.parametersChangedByExternalTools.UnionWith(description.parametersChangedByExternalTools);
+            }
+
+            data.parametersReadByExternalTools.RemoveWhere(x => x.name == "");
+            data.parametersChangedByExternalTools.RemoveWhere(x => x.name == "");
+
+            return data;
+        }
+
         private static IEnumerable<AssetDescription> GetAllAssetDescriptions()
         {
             foreach (var findAsset in AssetDatabase.FindAssets("t:AssetDescription"))
             {
                 var path = AssetDatabase.GUIDToAssetPath(findAsset);
                 var asset = AssetDatabase.LoadAssetAtPath<AssetDescription>(path);
-                if (asset) yield return asset;
+                if (asset != null) yield return asset;
             }
         }
 
-        public static IEnumerable<Type> GetMeaninglessComponents()
-        {
-            return GetAllAssetDescriptions()
-                .SelectMany(description => description.meaninglessComponents)
-                .Select(component => GetMonoScriptFromGuid(component.guid, component.fileID) as MonoScript)
-                .Where(monoScript => monoScript != null)
-                .Select(monoScript => monoScript!.GetClass());
-        }
+        public static void Reload() => _data = LoadData();
+        public static HashSet<Type> GetMeaninglessComponents() => Data.meaninglessComponents;
+        public static HashSet<OscParameter> GetParametersReadByExternalTools() => Data.parametersReadByExternalTools;
+        public static HashSet<OscParameter> GetParametersChangedByExternalTools() => Data.parametersChangedByExternalTools;
 
         private static Object GetMonoScriptFromGuid(string guid, ulong fileid)
         {
@@ -53,11 +110,15 @@ internal class AssetDescriptionEditor : Editor
         {
             private SerializedProperty _comment = null!; // Initialized by OnEnable
             private SerializedProperty _meaninglessComponents = null!; // Initialized by OnEnable
+            private SerializedProperty _parametersExternalToolsReads = null!; // Initialized by OnEnable
+            private SerializedProperty _parametersExternalsToolsChanges = null!; // Initialized by OnEnable
 
             private void OnEnable()
             {
                 _comment = serializedObject.FindProperty("comment");
                 _meaninglessComponents = serializedObject.FindProperty("meaninglessComponents");
+                _parametersExternalToolsReads = serializedObject.FindProperty(nameof(parametersReadByExternalTools));
+                _parametersExternalsToolsChanges = serializedObject.FindProperty(nameof(parametersChangedByExternalTools));
             }
 
             public override void OnInspectorGUI()
@@ -76,6 +137,8 @@ public override void OnInspectorGUI()
                 EditorGUILayout.Space(20f);
                 EditorGUILayout.PropertyField(_comment);
                 EditorGUILayout.PropertyField(_meaninglessComponents);
+                EditorGUILayout.PropertyField(_parametersExternalToolsReads, true);
+                EditorGUILayout.PropertyField(_parametersExternalsToolsChanges, true);
 
                 serializedObject.ApplyModifiedProperties();
             }
@@ -170,5 +233,64 @@ GUIContent MissingScriptContent(string className) => EditorGUI.showMixedValue
                 ? Constants.MixedValueContent
                 : new GUIContent($"Missing: {className}");
         }
+
+        // This class describes the OSC parameter
+        [Serializable]
+        public struct OscParameter : IEquatable<OscParameter>
+        {
+            [SerializeField]
+            public string name;
+            [SerializeField]
+            public MatchMode matchMode;
+
+            public enum MatchMode
+            {
+                Exact,
+                Prefix,
+                Suffix,
+                Contains,
+            }
+
+            public bool Equals(OscParameter other) => name == other.name && matchMode == other.matchMode;
+            public override bool Equals(object? obj) => obj is OscParameter other && Equals(other);
+            public override int GetHashCode() => HashCode.Combine(name, (int)matchMode);
+            public static bool operator ==(OscParameter left, OscParameter right) => left.Equals(right);
+            public static bool operator !=(OscParameter left, OscParameter right) => !left.Equals(right);
+        }
+
+        [CustomPropertyDrawer(typeof(OscParameter))]
+        internal class OscParameterEditor : PropertyDrawer
+        {
+            public override float GetPropertyHeight(SerializedProperty property, GUIContent label) => 
+                EditorGUIUtility.singleLineHeight;
+
+            public override void OnGUI(Rect position, SerializedProperty property, GUIContent label)
+            {
+                var nameProperty = property.FindPropertyRelative("name");
+                var matchModeProperty = property.FindPropertyRelative("matchMode");
+
+                var rect = EditorGUI.PrefixLabel(position, label);
+                Rect text, popup;
+
+                float width;
+                if (rect.width > 200)
+                    width = 99.5f;
+                else
+                    width = rect.width / 2 - 0.5f;
+
+                text = rect with { width = rect.width - width - 1 };
+                popup = rect with { x = rect.xMax - width, width = width };
+
+                EditorGUI.BeginChangeCheck();
+                var newName = EditorGUI.TextField(text, nameProperty.stringValue);
+                if (EditorGUI.EndChangeCheck())
+                    nameProperty.stringValue = newName;
+
+                EditorGUI.BeginChangeCheck();
+                var newMatchMode = (OscParameter.MatchMode)EditorGUI.EnumPopup(popup, (OscParameter.MatchMode)matchModeProperty.enumValueIndex);
+                if (EditorGUI.EndChangeCheck())
+                    matchModeProperty.enumValueIndex = (int)newMatchMode;
+            }
+        }
     }
 }