PubSub Emulator Memory Leak: Node Processes Persist After Function Execution

[kdawgwilk]

[REQUIRED] Environment info

firebase-tools: 13.29.1
Platform: macOS
Node.js Version: v20.18.1

[REQUIRED] Description

The Firebase PubSub emulator creates a new Node process for each PubSub event, but these processes are not being terminated after function execution completes. This leads to memory exhaustion as more events are processed, making local development difficult and potentially causing system stability issues.

[REQUIRED] Test case

Reproduction repository: https://github.com/kdawgwilk/pubsub-emulator-bug-repro

[REQUIRED] Steps to reproduce

1. Clone the reproduction repository
2. Install dependencies:

   cd functions
   npm install

3. Start the Firebase emulators:

   npm run serve -- --project <firebase-project-id>

4. In a separate terminal, run the test script that publishes 100 messages:

   GCLOUD_PROJECT=<firebase-project-id> \
   FIREBASE_AUTH_EMULATOR_HOST=localhost:9099 \
   FIRESTORE_EMULATOR_HOST=localhost:8080 \
   PUBSUB_EMULATOR_HOST=localhost:8085 \
   npx tsx ./functions/scripts/test_pubsub.ts

5. Open Activity Monitor (macOS) and observe:
   • Multiple Node processes being created
   • Processes remain active even after function execution
   • Memory usage increases with each new message
   • Processes do not terminate automatically

[REQUIRED] Expected behavior

• Node processes should be created when handling PubSub events
• Processes should terminate automatically after function execution completes
• System resources should be properly cleaned up
• Memory usage should remain stable across multiple events

[REQUIRED] Actual behavior

• New Node processes are spawned for each PubSub event
• Processes persist indefinitely after function completion
• Memory usage grows continuously as new messages are processed
• No automatic cleanup of completed function processes
• System resources are exhausted if running for extended periods

Impact

This issue significantly impacts local development by:

• Consuming system resources unnecessarily
• Requiring manual intervention to clean up processes
• Potentially causing system slowdown or crashes during extended development sessions
• Making it difficult to run long-term tests or process multiple messages

[joehan]

Thanks for reporting this! This appears to be an issue inside the Pub/Sub emulator, so I've passed this along to the Google team responsible for maintaining it. Internal bug reference b/389997173

[joehan]

To assist with debugging this - if you have them, could you also include any firebase-debug.log or pubsub-debug.log that are generated during a run?

[kdawgwilk]

This is the only debug logs that were created from the run

pubsub-debug.log

This is the Google Pub/Sub fake.
Implementation may be incomplete or differ from the real system.
Jan 16, 2025 1:38:18 AM com.google.cloud.pubsub.testing.v1.Main main
INFO: IAM integration is disabled. IAM policy methods and ACL checks are not supported
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Jan 16, 2025 1:38:19 AM com.google.cloud.pubsub.testing.v1.Main main
INFO: Server started, listening on 8085
Jan 16, 2025 1:38:24 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected HTTP/2 connection.
Jan 16, 2025 1:39:05 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected HTTP/2 connection.
*** shutting down gRPC server since JVM is shutting down
*** server shut down