Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Huge memory usage when importing large quantities of products #10914

Closed
diwipl opened this issue Sep 17, 2017 · 2 comments
Closed

Huge memory usage when importing large quantities of products #10914

diwipl opened this issue Sep 17, 2017 · 2 comments
Assignees
@diwipl
Labels
Component: ImportExport Issue: Clear Description Gate 2 Passed. Manual verification of the issue description passed Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed Issue: Format is valid Gate 1 Passed. Automatic verification of issue format passed Issue: Ready for Work Gate 4. Acknowledged. Issue is added to backlog and ready for development Reproduced on 2.1.x The issue has been reproduced on latest 2.1 release Reproduced on 2.2.x The issue has been reproduced on latest 2.2 release

Comments

@diwipl
Copy link

diwipl commented Sep 17, 2017

When importing huge amounts of products Magento 2 is loading contents of importexport_importdata to memory and then iterate it in memory which results in huge memory usage

Preconditions

  1. Magento 2.1.7
  2. CSV with 1 500 000 products

Steps to reproduce

  1. Validate 1 500 000 products
  2. Run import

Expected result

  1. Memory usage for iterating trough import data should be constant

Actual result

  1. Whole table contents is loaded to memory
@magento-engcom-team magento-engcom-team added the Issue: Format is valid Gate 1 Passed. Automatic verification of issue format passed label Sep 17, 2017
@diwipl diwipl mentioned this issue Sep 17, 2017
Closed
4 tasks
@magento-engcom-team magento-engcom-team added Issue: Clear Description Gate 2 Passed. Manual verification of the issue description passed Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed labels Sep 19, 2017
@magento-engcom-team
Copy link
Contributor

@diwipl, thank you for your report.
We've created internal ticket(s) MAGETWO-84592 to track progress on the issue.

@magento-engcom-team magento-engcom-team added 2.1.x Issue: Ready for Work Gate 4. Acknowledged. Issue is added to backlog and ready for development Reproduced on 2.1.x The issue has been reproduced on latest 2.1 release Reproduced on 2.2.x The issue has been reproduced on latest 2.2 release labels Nov 27, 2017
@piotrekkaminski piotrekkaminski mentioned this issue Jan 3, 2018
Open
@piotrekkaminski
Copy link
Contributor

The issue was moved to: magento-engcom/import-export-improvements#92

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@diwipl diwipl

Labels
Component: ImportExport Issue: Clear Description Gate 2 Passed. Manual verification of the issue description passed Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed Issue: Format is valid Gate 1 Passed. Automatic verification of issue format passed Issue: Ready for Work Gate 4. Acknowledged. Issue is added to backlog and ready for development Reproduced on 2.1.x The issue has been reproduced on latest 2.1 release Reproduced on 2.2.x The issue has been reproduced on latest 2.2 release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@dmanners @piotrekkaminski @diwipl @okorshenko @magento-engcom-team