Automatically Merge Blendshape

.docs/content/docs/basic-concept/index.ja.md

@@ -31,7 +31,7 @@ Avatar Optimizerは、アバターを非破壊的に最適化するためのツ
 BlendShapeを統合すると、そのBlendShapeに対するアニメーションによる動作が変わってしまうため、これはバグとして扱われます。\
 このバグを使用して、あるSkinned Mesh RendererのBlendShapeに対するアニメーションを、別のSkinned Mesh RendererのBlendShapeに対しても同時に作用させることができます。\
 しかし、これはサポートされていない動作であり、他のコンポーネントが動作を壊すかもしれません。\
-例えば、`AAO Trace and Optimize`コンポーネントの`BlendShapeを自動的に固定・除去する`は、このバグによって`AAO Merge Skinned Mesh`コンポーネントで統合されて動くようになる可能性があるBlendShapeであっても、固定・除去します。
+例えば、`AAO Trace and Optimize`コンポーネントの`BlendShapeを最適化する`は、このバグによって`AAO Merge Skinned Mesh`コンポーネントで統合されて動くようになる可能性があるBlendShapeであっても、固定・除去します。
 
 ## Avatar Optimizerの振る舞いは将来のバージョンでも安定していますか？ {#behavior-stability}
 

.docs/content/docs/basic-concept/index.md

@@ -33,7 +33,7 @@ For example, `AAO Merge Skinned Mesh` component in 1.7.x or older merges BlendSh
 This behavior is treated as a bug since this makes impossible to animate them separately.\
 You may use this bug to sync the BlendShape animation of an Skinned Mesh Renderer with one of another Skinned Mesh Renderer.\
 However, this is not supported behavior and some other components may break the behavior.\
-For example, `Automatically Freeze BlendShape` in `AAO Trace and Optimize` component will freezes the BlendShapes which might be animated with this buggy behavior by being merged by `AAO Merge Skinned Mesh` component.
+For example, `Optimize BlendShape` in `AAO Trace and Optimize` component will freezes the BlendShapes which might be animated with this buggy behavior by being merged by `AAO Merge Skinned Mesh` component.
 
 ## How is the behavior of Avatar Optimizer stable for future versions? {#behavior-stability}
 

.docs/content/docs/reference/merge-skinned-mesh/index.ja.md

@@ -33,7 +33,7 @@ BlendShapeは、頂点とBlendShapeの数に比例して負荷が大きくなる
 
 同様に、体や服のメッシュのBlendShapeは固定・除去することを推奨します。
 [Freeze BlendShape](../freeze-blendshape)コンポーネントを統合対象・統合先のSkinnedMeshRendererコンポーネントのいずれか(または両方)に追加して、BlendShapeを固定・除去することが出来ます。
-[Trace and Optimize](../trace-and-optimize)コンポーネントの`BlendShapeを自動的に固定・除去する`によっても同様の効果を得ることが出来ます。
+[Trace and Optimize](../trace-and-optimize)コンポーネントの`BlendShapeを最適化する`によっても同様の効果を得ることが出来ます。
 
 {{< hint info >}}
 

.docs/content/docs/reference/merge-skinned-mesh/index.md

@@ -35,7 +35,7 @@ That's why it's not good to merge face meshes.
 In addition, because of same reasons, you should freeze & remove unchanging BlendShapes for body / cloth meshes.
 You can freeze & remove BlendShape using [Freeze BlendShape](../freeze-blendshape) component.
 Add this component to both/either merge source SkinnedMeshRenderer and/or merged SkinnedMeshRenderer to freeze & remove BlendShapes.
-Also, you can use `Automatically Freeze BlendShape` of [Trace and Optimize](../trace-and-optimize) component to get the same benefits.
+Also, you can use `Optimize BlendShapes` of [Trace and Optimize](../trace-and-optimize) component to get the same benefits.
 
 {{< hint info >}}
 

.docs/content/docs/reference/trace-and-optimize/index.ja.md

@@ -23,8 +23,9 @@ Trace and Optimizeは「**見た目に絶対に影響させてはならない**
 {{< /hint >}}
 
 現在、以下の機能を使った自動最適化が可能です。
-- `BlendShapeを自動的に固定・除去する`\
-  アニメーションなどで使われていないか、常に同じ値になっているBlendShapeを自動的に固定・除去します。
+- `BlendShapeを最適化する`\
+  <small>以前は`BlendShapeを自動的に固定・除去する`という名前でしたが、機能が増えたため名前が変わりました。</small>\
+  アニメーションなどを走査して、BlendShapeを自動的に固定・除去・統合することでBlendShapeの数を削減します。
 - `使われていないObjectを自動的に削除する`\
   アニメーションなどを走査して、使われていないObject(GameObjectやコンポーネントなど)を自動的に削除します。\
   また、切り替えるものと一緒に使われていて、他の方法で使われていないPhysBooneコンポーネントを自動的に切り替えるようにします。

.docs/content/docs/reference/trace-and-optimize/index.md

@@ -24,8 +24,9 @@ We will fix it as much as we can.
 {{< /hint >}}
 
 Currently the following optimizations are applied automatically.
-- `Automatically Freeze BlendShape`\
-  Automatically freezes BlendShapes which are always the same value or unused in animation, etc.
+- `Optimize BlendShape`\
+  <small>Previously known as `Freeze BlendShapes` but renamed to add more functionality.</small>\
+  By scanning animation etc., remove, freeze, or merge BlendShapes automatically to reduce the number of BlendShapes.
 - `Remove unused Objects`\
   By scanning animation etc., automatically removes unused Objects (e.g. GameObjects, Components).\
   In addition, this will automatically toggle PhysBone Components if they are only used by toggled objects.

CHANGELOG-PRERELEASE.md

@@ -8,6 +8,12 @@ The format is based on [Keep a Changelog].
 
 ## [Unreleased]
 ### Added
+- Automatically Merge Blendshape `#1300`
+  - This is new automatic optimization in Trace and Optimize
+  - This is a part of "Optimize BlendShape" optimization.
+  - AAO 1.8.0 introduced BlendShape support for Merge Skinned Mesh, but new default mode "Rename to avoid conflicts" would increase number of BlendShape.
+  - This feature is added to relax this problem by automatically merging multiple BlendShapes of one Mesh.
+  - With this feature, you can use rename mode without performance loss.
 
 ### Changed
 

CHANGELOG.md

@@ -49,6 +49,12 @@ The format is based on [Keep a Changelog].
   - When you changed shader for an material, properties for previously used shaders might be remain
   - This may increase your avatar size by unexpectedly including unused textures
 - Right-click menu option to create a new GameObject with a specified component [`#1290`](https://github.com/anatawa12/AvatarOptimizer/pull/1290)
+- Automatically Merge Blendshape `#1300`
+  - This is new automatic optimization in Trace and Optimize
+  - This is a part of "Optimize BlendShape" optimization.
+  - AAO 1.8.0 introduced BlendShape support for Merge Skinned Mesh, but new default mode "Rename to avoid conflicts" would increase number of BlendShape.
+  - This feature is added to relax this problem by automatically merging multiple BlendShapes of one Mesh.
+  - With this feature, you can use rename mode without performance loss.
 
 ### Changed
 - Skip Enablement Mismatched Renderers is now disabled by default `#1169`
@@ -86,6 +92,7 @@ The format is based on [Keep a Changelog].
   - You now can successfully merge Meshes with BlendShape with Merge Skinned Mesh.
   - Actually, previous version does not have proper consideration for BlendShape.
   - This version introduces options to select BlendShape behavior in Merge Skinned Mesh.
+- Renamed "Automatically Freeze BlendShape" to "Optimize BlendShape" `#1300`
 
 ### Deprecated
 

Editor/Inspector/TraceAndOptimizeEditor.cs

@@ -22,7 +22,7 @@ internal class TraceAndOptimizeEditor : AvatarGlobalComponentEditorBase
 
         private void OnEnable()
         {
-            _freezeBlendShape = serializedObject.FindProperty(nameof(TraceAndOptimize.freezeBlendShape));
+            _freezeBlendShape = serializedObject.FindProperty(nameof(TraceAndOptimize.optimizeBlendShape));
             _removeUnusedObjects = serializedObject.FindProperty(nameof(TraceAndOptimize.removeUnusedObjects));
             _preserveEndBone = serializedObject.FindProperty(nameof(TraceAndOptimize.preserveEndBone));
             _removeZeroSizedPolygons = serializedObject.FindProperty(nameof(TraceAndOptimize.removeZeroSizedPolygons));

Editor/OptimizerPlugin.cs

@@ -78,6 +78,7 @@ protected override void Configure()
                         .Then.Run(Processors.TraceAndOptimizes.FindUnusedObjects.Instance)
                         .Then.Run(Processors.TraceAndOptimizes.ConfigureRemoveZeroSizedPolygon.Instance)
                         .Then.Run(Processors.TraceAndOptimizes.RemoveUnusedMaterialProperties.Instance)
+                        .Then.Run(Processors.TraceAndOptimizes.AutoMergeBlendShape.Instance)
                         .Then.Run(Processors.MergeBoneProcessor.Instance)
                         .Then.Run(Processors.RemoveZeroSizedPolygonProcessor.Instance)
                         .Then.Run(Processors.TraceAndOptimizes.OptimizeTexture.Instance)

Editor/Processors/TraceAndOptimize/AutoFreezeBlendShape.cs

@@ -18,7 +18,7 @@ internal class AutoFreezeBlendShape : TraceAndOptimizePass<AutoFreezeBlendShape>
 
         protected override void Execute(BuildContext context, TraceAndOptimizeState state)
         {
-            if (!state.FreezeBlendShape) return;
+            if (!state.OptimizeBlendShape) return;
 
             if (!state.SkipFreezingNonAnimatedBlendShape)
                 FreezeNonAnimatedBlendShapes(context, state);

Editor/Processors/TraceAndOptimize/AutoMergeBlendShape.cs

@@ -0,0 +1,175 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using Anatawa12.AvatarOptimizer.AnimatorParsersV2;
+using Anatawa12.AvatarOptimizer.Processors.SkinnedMeshes;
+using nadena.dev.ndmf;
+using Unity.Collections;
+using Unity.Jobs;
+using UnityEngine;
+
+namespace Anatawa12.AvatarOptimizer.Processors.TraceAndOptimizes;
+
+class AutoMergeBlendShape: TraceAndOptimizePass<AutoMergeBlendShape>
+{
+    public override string DisplayName => "T&O: Auto Merge Blend Shape";
+
+    protected override void Execute(BuildContext context, TraceAndOptimizeState state)
+    {
+        if (!state.OptimizeBlendShape) return;
+        if (state.SkipAutoMergeBlendShape) return;
+
+        foreach (var skinnedMeshRenderer in context.GetComponents<SkinnedMeshRenderer>())
+        {
+            if (state.Exclusions.Contains(skinnedMeshRenderer.gameObject)) continue; // manual exclusion
+
+            ErrorReport.WithContextObject(skinnedMeshRenderer, () => DoAutoMerge(context.GetMeshInfoFor(skinnedMeshRenderer), context));
+        }
+    }
+
+    private static void DoAutoMerge(MeshInfo2 meshInfo2, BuildContext context)
+    {
+        var animationComponent = context.GetAnimationComponent(meshInfo2.SourceRenderer);
+        var groups = new Dictionary<MergeKey, List<string>>();
+
+        foreach (var (name, weight) in meshInfo2.BlendShapes)
+        {
+            if (MergeKey.Create(weight, name, animationComponent) is { } key)
+            {
+                if (!groups.TryGetValue(key, out var list))
+                    groups.Add(key, list = new List<string>());
+                list.Add(name);
+            }
+        }
+
+        // nothing to merge
+        if (groups.Values.All(x => x.Count <= 1)) return;
+
+        // prepare merge
+        var buffers = meshInfo2.Vertices.Select(x => x.BlendShapeBuffer).ToArray();
+
+        // bulk remove to optimize removing blendshape process
+        var removeNames = new HashSet<string>();
+
+        var i = 0;
+        // there is thing to merge
+        foreach (var (key, names) in groups)
+        {
+            // validate the blendShapes are simple enough to merge
+            // if not, skip
+            foreach (var buffer in buffers)
+            {
+                float? commonFrameWeight = null;
+
+                foreach (var name in names)
+                {
+                    if (buffer.Shapes.TryGetValue(name, out var shapeShape))
+                    {
+                        if (shapeShape.FrameCount != 1) goto next_shape;
+                        var frameWeight = shapeShape.Frames[0].Weight;
+                        if (commonFrameWeight is { } common)
+                        {
+                            if (!common.Equals(frameWeight)) goto next_shape;
+                        }
+                        else
+                        {
+                            commonFrameWeight = frameWeight;
+                        }
+                    }
+                }
+            }
+
+            // validation passed, merge
+            var newName = $"AAO_Merged_{string.Join("_", names)}_{i++}";
+
+            // process meshInfo2
+            meshInfo2.BlendShapes.Add((newName, key.defaultWeight));
+            removeNames.UnionWith(names);
+
+            // actually merge data
+            foreach (var buffer in buffers)
+            {
+                BlendShapeShape? newShapeShape = null;
+
+                foreach (var name in names)
+                {
+                    if (buffer.Shapes.Remove(name, out var shapeShape))
+                    {
+                        if (newShapeShape == null)
+                        {
+                            newShapeShape = shapeShape;
+                        }
+                        else
+                        {
+                            var vertices = new MergeBlendShapeJob
+                            {
+                                vertices = buffer.DeltaVertices[newShapeShape.Frames[0].BufferIndex],
+                                toMerge = buffer.DeltaVertices[shapeShape.Frames[0].BufferIndex],
+                            }.Schedule(buffer.VertexCount, 64);
+                            var normals = new MergeBlendShapeJob
+                            {
+                                vertices = buffer.DeltaNormals[newShapeShape.Frames[0].BufferIndex],
+                                toMerge = buffer.DeltaNormals[shapeShape.Frames[0].BufferIndex],
+                            }.Schedule(buffer.VertexCount, 64);
+                            var tangents = new MergeBlendShapeJob
+                            {
+                                vertices = buffer.DeltaTangents[newShapeShape.Frames[0].BufferIndex],
+                                toMerge = buffer.DeltaTangents[shapeShape.Frames[0].BufferIndex],
+                            }.Schedule(buffer.VertexCount, 64);
+                            JobHandle.CombineDependencies(vertices, normals, tangents).Complete();
+                        }
+                    }
+                }
+
+                // null means there are no shapes to merge for this buffer
+                if (newShapeShape != null)
+                {
+                    buffer.Shapes.Add(newName, newShapeShape);
+                }
+            }
+
+            next_shape: ;
+        }
+
+        // remove merged blendShapes
+        meshInfo2.BlendShapes.RemoveAll(x => removeNames.Contains(x.name));
+    }
+
+    readonly struct MergeKey : IEquatable<MergeKey>
+    {
+        public readonly float defaultWeight;
+        public readonly EqualsHashSet<AnimationLocation> animationLocations;
+
+        public MergeKey(float defaultWeight, EqualsHashSet<AnimationLocation> animationLocations)
+        {
+            this.defaultWeight = defaultWeight;
+            this.animationLocations = animationLocations;
+        }
+
+        public static MergeKey? Create(float defaultWeight, string name, AnimationComponentInfo<PropertyInfo> animationComponent)
+        {
+            var node = animationComponent.GetFloatNode($"blendShape.{name}");
+            // to merge, all nodes must be AnimatorPropModNode
+            if (!node.ComponentNodes.All(x => x is AnimatorPropModNode<FloatValueInfo>)) return null;
+            var animationLocations = AnimationLocation.CollectAnimationLocation(node).ToEqualsHashSet();
+            return new MergeKey(defaultWeight, animationLocations);
+        }
+
+        public bool Equals(MergeKey other) =>
+            defaultWeight.Equals(other.defaultWeight) &&
+            animationLocations.Equals(other.animationLocations);
+
+        public override bool Equals(object? obj) => obj is MergeKey other && Equals(other);
+        public override int GetHashCode() => HashCode.Combine(defaultWeight, animationLocations);
+        public static bool operator ==(MergeKey left, MergeKey right) => left.Equals(right);
+        public static bool operator !=(MergeKey left, MergeKey right) => !left.Equals(right);
+    }
+
+    struct MergeBlendShapeJob : IJobParallelFor
+    {
+        public NativeArray<Vector3> vertices;
+        public NativeArray<Vector3> toMerge;
+
+        public void Execute(int index) => vertices[index] += toMerge[index];
+    }
+}

Internal/TraceAndOptimizeBase/TraceAndOptimizeProcessor.cs

@@ -7,7 +7,7 @@ namespace Anatawa12.AvatarOptimizer.Processors.TraceAndOptimizes
     public class TraceAndOptimizeState
     {
         public bool Enabled;
-        public bool FreezeBlendShape;
+        public bool OptimizeBlendShape;
         public bool RemoveUnusedObjects;
         public bool RemoveZeroSizedPolygon;
         public bool OptimizePhysBone;
@@ -37,13 +37,14 @@ public class TraceAndOptimizeState
         public bool SkipRemoveEmptySubMesh;
         public bool SkipAnyStateToEntryExit;
         public bool SkipRemoveMaterialUnusedProperties;
+        public bool SkipAutoMergeBlendShape;
 
         public Dictionary<SkinnedMeshRenderer, HashSet<string>> PreserveBlendShapes =
             new Dictionary<SkinnedMeshRenderer, HashSet<string>>();
 
         internal void Initialize(TraceAndOptimize config)
         {
-            FreezeBlendShape = config.freezeBlendShape;
+            OptimizeBlendShape = config.optimizeBlendShape;
             RemoveUnusedObjects = config.removeUnusedObjects;
             RemoveZeroSizedPolygon = config.removeZeroSizedPolygons;
             OptimizePhysBone = config.optimizePhysBone;
@@ -74,6 +75,7 @@ internal void Initialize(TraceAndOptimize config)
             SkipRemoveEmptySubMesh = config.debugOptions.skipRemoveEmptySubMesh;
             SkipAnyStateToEntryExit = config.debugOptions.skipAnyStateToEntryExit;
             SkipRemoveMaterialUnusedProperties = config.debugOptions.skipRemoveMaterialUnusedProperties;
+            SkipAutoMergeBlendShape = config.debugOptions.skipAutoMergeBlendShape;
 
             Enabled = true;
         }

Localization/en-us.po

@@ -700,8 +700,8 @@ msgstr "Migrating to Trace and Optimize is finished!"
 msgid "TraceAndOptimize:description"
 msgstr "When you added this component to your avatar, AvatarOptimizer will trace your avatar and optimize automatically."
 
-msgid "TraceAndOptimize:prop:freezeBlendShape"
-msgstr "Automatically Freeze BlendShape"
+msgid "TraceAndOptimize:prop:optimizeBlendShape"
+msgstr "Optimize BlendShape"
 
 msgid "TraceAndOptimize:prop:removeUnusedObjects"
 msgstr "Remove unused Objects Automatically"

Localization/ja-jp.po

@@ -611,8 +611,8 @@ msgstr "Trace and Optimizeへの移行が完了しました！"
 msgid "TraceAndOptimize:description"
 msgstr "このコンポーネントをアバターにつけると、アバターを走査して自動的にできる限りの最適化を行います。"
 
-msgid "TraceAndOptimize:prop:freezeBlendShape"
-msgstr "BlendShapeを自動的に固定・除去する"
+msgid "TraceAndOptimize:prop:optimizeBlendShape"
+msgstr "BlendShapeを最適化する"
 
 msgid "TraceAndOptimize:prop:removeUnusedObjects"
 msgstr "使われていないObjectを自動的に削除する"

Runtime/TraceAndOptimize.cs

@@ -23,10 +23,11 @@ internal TraceAndOptimize()
         }
 
         [NotKeyable]
-        [AAOLocalized("TraceAndOptimize:prop:freezeBlendShape")]
+        [AAOLocalized("TraceAndOptimize:prop:optimizeBlendShape")]
         [ToggleLeft]
         [SerializeField]
-        internal bool freezeBlendShape = true;
+        [FormerlySerializedAs("freezeBlendShape")] // renamed in 1.8.0
+        internal bool optimizeBlendShape = true;
         [NotKeyable]
         [AAOLocalized("TraceAndOptimize:prop:removeUnusedObjects")]
         [ToggleLeft]
@@ -140,6 +141,8 @@ internal struct DebugOptions
             public bool skipAnyStateToEntryExit;
             [ToggleLeft]
             public bool skipRemoveMaterialUnusedProperties;
+            [ToggleLeft]
+            public bool skipAutoMergeBlendShape;
         }
     }
 }