Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High memory usage #467

Closed
sschizas opened this issue Mar 16, 2017 · 11 comments
Closed

High memory usage #467

sschizas opened this issue Mar 16, 2017 · 11 comments

Comments

@sschizas
Copy link

Hello i am using bull with node v6.9.1 and pm2 v.2.4.2. When i upgraded to latest bull version (2.2.3) i saw that memory usage went to 1.5Gb in a development server which most of the time is idle. I reverted bull to version 2.1.2 and the problem disappeared.

Can you please let me know what you need from my side to find out why this happens?

@manast
Copy link
Member

manast commented Mar 16, 2017

1.5Gb sounds like quite a lot. We had a memory leak that was fixed in 2.2.3. see this issue: #426
But this was a leak making the memory usage grow over time. How is your scenario? does the memory usage grow immediately when the queue starts? can you make a reproducible test for this?

@sschizas
Copy link
Author

sschizas commented Mar 17, 2017

@manast the memory grows over time in nodejs to 1.5Gb. The reason i want to go my code to latest version of bull is fix for #426.

I tested with v2.2.4 and had the same results:

┌───────────┬────┬─────────┬───────┬────────┬─────────┬────────┬─────┬──────────┬──────────┐
│ App name │ id │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ watching │
├───────────┼────┼─────────┼───────┼────────┼─────────┼────────┼─────┼──────────┼──────────┤
│ wover-dev │ 0 │ cluster │ 17609 │ online │ 0 │ 14m │ 0% │ 1.2 GB │ disabled │
│ wover-dev │ 1 │ cluster │ 17619 │ online │ 0 │ 14m │ 0% │ 1.2 GB │ disabled │
└───────────┴────┴─────────┴───────┴────────┴─────────┴────────┴─────┴──────────┴──────────┘

also looking at PM2 logs i saw this one:

You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
1|wover-de | ReplyError: NOSCRIPT No matching script. Please use EVAL.
1|wover-de |     at JavascriptReplyParser.returnError (/home/azureuser/wover-backend/node_modules/ioredis/lib/redis/parser.js:25:25)
1|wover-de |     at JavascriptReplyParser.run (/home/azureuser/wover-backend/node_modules/ioredis/node_modules/redis-parser/lib/javascript.js:135:18)
1|wover-de |     at JavascriptReplyParser.execute (/home/azureuser/wover-backend/node_modules/ioredis/node_modules/redis-parser/lib/javascript.js:112:10)
1|wover-de |     at Socket.<anonymous> (/home/azureuser/wover-backend/node_modules/ioredis/lib/redis/event_handler.js:107:22)
1|wover-de |     at emitOne (events.js:96:13)
1|wover-de |     at Socket.emit (events.js:188:7)
1|wover-de |     at readableAddChunk (_stream_readable.js:176:18)
1|wover-de |     at Socket.Readable.push (_stream_readable.js:134:10)
1|wover-de |     at TCP.onread (net.js:548:20)
1|wover-de | From previous event:
1|wover-de |     at Command.initPromise (/home/azureuser/wover-backend/node_modules/ioredis/lib/command.js:63:18)
1|wover-de |     at new Command (/home/azureuser/wover-backend/node_modules/ioredis/lib/command.js:51:8)
1|wover-de |     at Script.execute (/home/azureuser/wover-backend/node_modules/ioredis/lib/script.js:22:17)
1|wover-de |     at Redis.addJob (/home/azureuser/wover-backend/node_modules/ioredis/lib/commander.js:164:20)
1|wover-de |     at Object.execScript (/home/azureuser/wover-backend/node_modules/bull/lib/scripts.js:24:24)
1|wover-de |     at Object.addJob (/home/azureuser/wover-backend/node_modules/bull/lib/scripts.js:161:23)
1|wover-de |     at addJob (/home/azureuser/wover-backend/node_modules/bull/lib/job.js:40:18)
1|wover-de |     at Function.Job.create (/home/azureuser/wover-backend/node_modules/bull/lib/job.js:50:10)
1|wover-de |     at Queue.add (/home/azureuser/wover-backend/node_modules/bull/lib/queue.js:417:14)
1|wover-de |     at Object.exports.createSecret (/home/azureuser/wover-backend/boot/queues/keyvault_queue.js:151:23)
1|wover-de |     at /home/azureuser/wover-backend/core/keyVault.js:23:32
1|wover-de |     at Promise.wrappedExecutorCaller [as _execute] (/home/azureuser/wover-backend/node_modules/newrelic/lib/instrumentation/promise.js:204:18)

@manast
Copy link
Member

manast commented Mar 17, 2017

can you write a minimal piece of code that reproduces this so that I can check it locally?

@sschizas
Copy link
Author

@manast it will try so as i am not expert in nodejs. One interesting thing, the issue appears on v6.10.0 nodejs. I installed v7 nodejs and i think the problem disappeared.

@manast
Copy link
Member

manast commented Mar 17, 2017

@n3trino that is really strange actually...

@sschizas
Copy link
Author

sschizas commented Mar 19, 2017

@manast I managed to get memory heap snapshots. I couldn't figure out what's wrong. You can download them here and maybe you can figure out.

@sschizas
Copy link
Author

@manast did have a chance to see the memory heapshots? if so, did you figure out what wrong?

@manast
Copy link
Member

manast commented Mar 21, 2017

I checked the snapshots but I could not get any valuable information. Best would be to get a minimal test case that reproduces the issue, so I can try to find where the root of the leak is.

@sschizas
Copy link
Author

@manast i figure out that running my node app + bull in development environment it creates this high memory/cpu situation. In production environment it works like a charm. Is there anything special happening in Bull when it runs at development environment?

@manast
Copy link
Member

manast commented Mar 28, 2017

There is nothing different that I am aware of, at least in bull's code.

@manast
Copy link
Member

manast commented May 24, 2017

As far as I can see, there are no memory issues with latest version of bull (3.0.0). so I will close this issue for now. Reopen if needed.

@manast manast closed this as completed May 24, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants