Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major performance issues with bytea performance #2240

Closed
regevbr opened this issue Jun 17, 2020 · 1 comment · Fixed by #2241
Closed

Major performance issues with bytea performance #2240

regevbr opened this issue Jun 17, 2020 · 1 comment · Fixed by #2241
Labels

Comments

@regevbr
Copy link
Contributor

regevbr commented Jun 17, 2020

Hi,

I encountered the exact same issue as #1286 wheres a simple query to retrieve a 100mb bytea column from a one table, one row DB takes forever and causes the cpu to spike at 100%, rendering my node instance unresponsive.

After investigating, I think I found the cause in the pg-protocol package

combinedBuffer = Buffer.allocUnsafe(this.remainingBuffer.byteLength + buffer.byteLength)
this.remainingBuffer.copy(combinedBuffer)
buffer.copy(combinedBuffer, this.remainingBuffer.byteLength)

Which suffers from the same thing as the original issue - too many allocations and buffer copies going around. I believe the fix introduced back then in brianc/node-packet-reader#3 should solve the problem here.

@charmander charmander added the bug label Jun 18, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jun 18, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jun 18, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jun 18, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jun 19, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jun 19, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jun 20, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jul 3, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jul 3, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jul 3, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jul 3, 2020
regevbr added a commit to PruvoNet/node-postgres that referenced this issue Jul 3, 2020
brianc added a commit that referenced this issue Jul 7, 2020
fix: major performance issues with bytea performance #2240
@lognaturel
Copy link

Just started running into this and feeling very grateful to see it’s been fixed for a while now already! Thank you, @regevbr, @brianc. 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants