bug: GC Seg Faults with Websockets Consumer

[ryanrussell]

Version

0.1.2

Platform

5.13.0-52-generic #59~20.04.1-Ubuntu

What steps will reproduce the bug?

1. Create a websockets server/producer. I tested 10k-100k Events per second with 100-200 bytes per message.

2. Create a websockets consumer in bun and connect it to the server.

3. Wait a few seconds for the GC crash

There are a plethora of websockets load testing tools to emulate this behavior.

I can also pair down a producer and consumer script if it helps... Did not seem like this was necessary to understand the cause based on feedback in discord.

How often does it reproduce? Is there a required condition?

It will crash within a few seconds every time.

It will have different run longevity(sometimes crashes with as few as 100-200 records, highest I've gotten to is 120k messages before crashing).

It also has different crash/exit messages. Seg Fault's, sometimes it says 'Crash', sometimes I get a silent exit with no error.

What is the expected behavior?

Consume websockets at high throughput without GC crash.

What do you see instead?

3 sample crashes:

SegmentationFault at 0


–––– bun meta ––––
Bun v0.1.2 Linux x64 #59~20.04.1-Ubuntu SMP Thu Jun 16 21:21:28 UTC 2022
AutoCommand: dotenv 
Elapsed: 411ms | User: 293ms | Sys: 117ms
RSS: 69.21MB | Peak: 94.71MB | Commit: 69.21MB | Faults: 0
–––– bun meta ––––

...

Block 0x7fb7a4078000: marks not empty!
Block lock is held: false
Marking version of block: 2
Marking version of heap: 2

Crash at 4294970678855


–––– bun meta ––––
Bun v0.1.2 Linux x64 #59~20.04.1-Ubuntu SMP Thu Jun 16 21:21:28 UTC 2022
AutoCommand: dotenv 
Elapsed: 3723ms | User: 3064ms | Sys: 1292ms
RSS: 69.21MB | Peak: 0.15GB | Commit: 69.21MB | Faults: 0
–––– bun meta ––––


...


SegmentationFault at 55351


–––– bun meta ––––
Bun v0.1.2 Linux x64 #59~20.04.1-Ubuntu SMP Thu Jun 16 21:21:28 UTC 2022
AutoCommand: dotenv 
Elapsed: 396ms | User: 289ms | Sys: 101ms
RSS: 69.21MB | Peak: 93.73MB | Commit: 69.21MB | Faults: 0
–––– bun meta ––––

Additional information

Was confirmed as a GC bug by Jarred in Discord.

From the @Jarred-Sumner on discord: "This is a GC bug for sure. I have a pretty good idea of the cause."

This is a placeholder to check off when it is solved.

I am happy to test a nightly release candidate if/when it may help debug.

[ryanrussell]

Also confirmed this is still present on 0.1.3 via recommendation in discord.

SegmentationFault at 53876069808464


–––– bun meta ––––
Bun v0.1.3 Linux x64 #59~20.04.1-Ubuntu SMP Thu Jun 16 21:21:28 UTC 2022
AutoCommand: dotenv 
Elapsed: 452ms | User: 351ms | Sys: 118ms
RSS: 69.21MB | Peak: 95.39MB | Commit: 69.21MB | Faults: 0
–––– bun meta ––––

Ask for #help in https://bun.sh/discord or go to https://bun.sh/issues

[ryanrussell]

Update: Marked improvement in 0.1.5

I ran the same load test on my end for 15 minutes consuming 64,952,603 websockets messages with no errors.

➜ bun --version
0.1.5

This represents successfully using bun to sustain an average of ~72k 100-200 byte websockets events per second on a single connection from a single producer over a 15 minute span.

I will run this for a longer duration and report back; I also see new SegFault work since the 1.5 version bump as well...

Are there contributions that you need from me that would help ensure this does not regress in future versions?

FANTASTIC WORK @Jarred-Sumner

[ryanrussell]

Feels like this is stabilized and only improving as of 0.1.10

Closing as housekeeping, re-open if necessary.

Thanks again!