Dispatcher Flush UI sends "Delete" Requests One Node at a Time

[eberggren343]

Required Information

• AEM Version, including Service Packs, Cumulative Fix Packs, etc: 6.5.15
• ACS AEM Commons Version: 5.5.2
• Reproducible on Latest? Yes

Expected Behavior

Within Dispatcher Flush UI, clicking "Flush Paths on Dispatcher" with the Flush Action set as "Delete" would delete files in the Dispatcher cache as configured via the "Paths to Flush" parameters. This process was almost instantaneous, allowing quick turn-around when needing to fully clear the cache.

Actual Behavior

Now, when initiating the Flush, the page spins. When checking replication logs, it appears that each individual node under the declared paths is cleared one at a time. For large repositories, this is very slow and can take hours to complete.

This seems to have started happening after we upgraded to 6.5.15, though not certain.

Steps to Reproduce

1. Install SP 6.5.15
2. Install ACS AEM Commons 5.5.2
3. Create and configure Dispatcher Flush UI page to "Delete" various paths (such as "/content/dam/x", "/content/pages/y")
4. Initiate flush

Links

Links to related assets, e.g. content packages containing test components

[sudheersundalam1]

Hi ACS Team, The dispatcher Flush UI is our most used ACS Commons feature other than Error page handler. This issue basically defeating the advantage of doing builds using Cloud Manager without involving the CSE support. Due to this issue, every time we do the production deployment, we need to request the CSE to manually clear the dispatcher cache.

I have attached the two log files to highlight the difference between Dispatcher Flush on the AEM 6.5.10 vs AEM 6.5.15.

Dispatcher-Flush-Log-AEM.6.5.10.log
Dispatcher-Flush-Log-AEM.6.5.15.log

Also, I did some digging to figure out the internal difference between AEM 6.5.10 code vs AEM 6.5.15. Below are my findings:
In AEM 6.5.10, the ReplicatorImpl.java class is building single instance of ReplicationAction by passing userID, replicationType and aggregateLocations(this aggregateLocations consists of configured path in the flush UI+ it's children)

Where as in AEM 6.5.15, the ReplicatorImpl.java class is building a List of ReplicationAction by passing the type, path, userID and some other params. I don't the understand need for building a List of ReplicationActions when the replication type is same for all the paths.
The list of ReplicationActions is lopped through further in the code to perform the replication action.

With this update in the Replicator implementation code, I don't see any straight forward way to get original functionality back any time soon.

[davidjgonzalez]

@sudheersundalam1 thanks for confirming. I reviews the ACS Code and we just pas in the specified path (whatever you pass in) to the to the the AEM Replication API -- so the change in internals explains this change in behavior.

Have you been able to log an issue with Adobe Customer Care to see if they can re-introduce the previous functionality? Maybe if they revert the existing functionality, and then put the new functionality behind a new parameter/action type? (like DELETE_RECURSIVELY or something?)

Im not sure what we can do in this project, since we obviously cant re-write the Replication API. This seems like a breaking change to the AEM Replication API though - im not sure what the policy is on that, but its worth asking.

[sudheersundalam1]

@davidjgonzalez , I have raised the Adobe support ticket explaining the breaking changes to the replication API. I will keep you posted with the updates from Adobe support.

[royteeuwen]

+1, same issue here

[royteeuwen]

@davidjgonzalez would you be open to changing the com.adobe.acs.commons.replication.dispatcher.DispatcherFlusher to ConsumerType? That way in AEM on-premises we can just implement it ourselves and opt-in to not use the replicator but instead do a direct http call (already doing this in a few locations, don't mind contributing this)

[davidjgonzalez]

@royteeuwen yeh - that works. wanna make a quick PR? I can release a build release this week.

[helms-charity]

@royteeuwen what version of ACS commons did you test with? Has anyone tested with 5.7.0 yet?

[ben-newman-hp]

Just curious if there's been any response from Adobe. This would certainly impact our ability to use the latest service pack.

[johnb4]

Adobe bug filed as ASSETS-21447.
A work-around that can be used until this Adobe bug is resolved is to write your own AggregateHander and change the DispatcherFlusherImpl to use that new AggregateHandler.
The custom AggregateHandler returns only the passed-in node, not all of the passed-in node's descendants. I've attached an example implementation. This example was written using Commons v5.2.0.
work-around-issue-3045.zip

[kwin]

Fixed in 6.0.8