Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Url Asset Ingestor tries to create folder that was created several steps ago #1583

Closed
3 tasks
cylinder-y opened this issue Nov 26, 2018 · 6 comments
Closed
3 tasks

Url Asset Ingestor tries to create folder that was created several steps ago #1583

cylinder-y opened this issue Nov 26, 2018 · 6 comments
Milestone
4.0.0

Comments

@cylinder-y
Copy link
Contributor

Url Asset Ingestor tries to create folder that was created several steps ago

Required Information

  • AEM Version, including Service Packs, Cumulative Fix Packs, etc: 6.4.1, 6.4.2
  • ACS AEM Commons Version: 3.19.1
  • Reproducible on Latest? yes

Expected Behavior

When I have the following content in excel file used for migration:

Source Destination
/some-assets/somefolder/dam-test/myproject/images/2017/10/destination/file1.jpg /some-assets/somefolder/dam-test/myproject/images/2017/10/destination
/some-assets/somefolder/dam-test/myproject/images/2017/10/destination/file2.jpg /some-assets/somefolder/dam-test/myproject/images/2017/10/destination
/some-assets/somefolder/dam-test/myproject/images/2017/10/destination/file3.jpg /some-assets/somefolder/dam-test/myproject/images/2017/10/destination
...... .......

Non-existing folders should be created once:

/content/dam/some-assets/
/content/dam/some-assets/somefolder/
..........

Actual Behavior

The following folders are generated in report:

/content/dam/some-assets
/content/dam/some-assets/somefolder
/content/dam/some-assets
/content/dam/some-assets/somefolder
/content/dam/some-assets/somefolder/dam-test
/content/dam/some-assets/somefolder/dam-test
/content/dam/some-assets/somefolder/dam-test/myproject
/content/dam/some-assets/somefolder/dam-test
/content/dam/some-assets/somefolder/dam-test/myproject/images
/content/dam/some-assets/somefolder/dam-test/myproject/images/2017
/content/dam/some-assets/somefolder/dam-test/myproject/images/2017/10
/content/dam/some-assets/somefolder/dam-test/myproject/images/2017/10/destination
/content/dam/some-assets/somefolder/dam-test/myproject
/content/dam/some-assets/somefolder/dam-test/myproject/images/2017/10/destination/File2.jpg
/content/dam/some-assets/somefolder/dam-test/myproject/images/2017/10/destination/file1.jpg

log error example:

26.11.2018 18:23:31.260 INFO [pool-28-thread-4] com.adobe.acs.commons.fam.actions.Actions Error commit, retry count is 10
javax.jcr.ItemExistsException: dam-test
at org.apache.jackrabbit.oak.jcr.session.NodeImpl$5.perform(NodeImpl.java:287) [org.apache.jackrabbit.oak-jcr:1.8.3]
at org.apache.jackrabbit.oak.jcr.session.NodeImpl$5.perform(NodeImpl.java:264) [org.apache.jackrabbit.oak-jcr:1.8.3]
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:208) [org.apache.jackrabbit.oak-jcr:1.8.3]
at org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112) [org.apache.jackrabbit.oak-jcr:1.8.3]
at org.apache.jackrabbit.oak.jcr.session.NodeImpl.addNode(NodeImpl.java:264) [org.apache.jackrabbit.oak-jcr:1.8.3]
at com.adobe.acs.commons.mcp.impl.processes.asset.AssetIngestor.createFolderNode(AssetIngestor.java:333) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.mcp.impl.processes.asset.AssetIngestor.createFolderNode(AssetIngestor.java:327) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.mcp.impl.processes.asset.AssetIngestor.createFolderNode(AssetIngestor.java:327) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.mcp.impl.processes.asset.AssetIngestor.createFolderNode(AssetIngestor.java:327) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.mcp.impl.processes.asset.AssetIngestor.createFolderNode(AssetIngestor.java:327) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.mcp.impl.processes.asset.UrlAssetImport.lambda$null$7(UrlAssetImport.java:226) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.fam.actions.Actions.lambda$retry$2(Actions.java:141) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.fam.impl.ActionManagerImpl.withResolver(ActionManagerImpl.java:200) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.fam.impl.ActionManagerImpl.runActionAndLogErrors(ActionManagerImpl.java:167) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.fam.impl.ActionManagerImpl.lambda$deferredWithResolver$0(ActionManagerImpl.java:159) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at com.adobe.acs.commons.fam.impl.TimedRunnable.run(TimedRunnable.java:88) [com.adobe.acs.acs-aem-commons-bundle:3.19.1.SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Steps to Reproduce

  1. Create new Url Asset Import process
  2. Upload excel mapping with non existing folders in target column
  3. Use file:/ or sftp:/ prefix
  4. Use /content/dam path Target JCR folder field
  5. Press Start
  6. See report
  7. See logs
@badvision
Copy link
Contributor

Let me know if PR #1584 is sufficient. It does try to prevent doing duplicate work in most places, but the fact that folder creates are run in parallel does allow for some (non-fatal) race conditions to occur. This patch should minimize the effect of that.

@badvision badvision added this to the 4.0.0 milestone Nov 26, 2018
@cylinder-y
Copy link
Contributor Author

Hi @badvision, I would try to verify it tomorrow

@badvision
Copy link
Contributor

No rush. Thanks!

@cylinder-y
Copy link
Contributor Author

cylinder-y commented Nov 27, 2018 •
edited
Loading

Looks good, closing issue

@cylinder-y cylinder-y reopened this Nov 27, 2018
@cylinder-y
Copy link
Contributor Author

Let's close it when pull request will be merged

badvision added a commit that referenced this issue Nov 28, 2018
@badvision
Fix for #1583 - Asset Ingestor may try to create asset folders when t…
4c6af07 
…hey already exist (#1584)

* Fix for 1583 to prevent race conditions while creating folders
@badvision
Copy link
Contributor

Thanks @cylinder-y for verifying! 👍

davidjgonzalez pushed a commit to davidjgonzalez/acs-aem-commons that referenced this issue Jan 27, 2019
@badvision @davidjgonzalez
Fix for Adobe-Consulting-Services#1583 - Asset Ingestor may try to cr…
8f35600 
…eate asset folders when they already exist (Adobe-Consulting-Services#1584)

* Fix for 1583 to prevent race conditions while creating folders
justinedelson pushed a commit that referenced this issue Feb 6, 2019
@badvision @justinedelson
Fix for #1583 - Asset Ingestor may try to create asset folders when t…
2ffaa0a 
…hey already exist (#1584)

* Fix for 1583 to prevent race conditions while creating folders
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
4.0.0
Development

No branches or pull requests
2 participants
@badvision @cylinder-y