Inbuilt support for p-queue (Bun.PQueue)

[vikasg603]

What is the problem this feature would solve?

Bun currently lacks a native solution for managing promises with controlled concurrency and priority. While JavaScript’s Promise.all and Promise.allSettled methods are useful for running multiple promises in parallel, they fall short when dealing with thousands of promises that require concurrent execution with a specified cap on the number of promises running at any given time.

For example, when managing a large set of tasks where a maximum of 100 concurrent promises is desired, Promise.all and Promise.allSettled necessitate waiting for an entire batch of 100 promises to complete before initiating the next batch. This “batching” behavior is inefficient for scenarios where promises complete at different times, leading to idle periods between batch completions.

This issue is prevalent in many real-world applications, such as:

• Sending Emails Concurrently: Managing the dispatch of a large volume of emails where only a certain number should be sent simultaneously to avoid overwhelming the email server or hitting rate limits.
• Web Scraping: Scraping data from multiple websites concurrently while respecting the target servers’ capacity and avoiding IP bans.
• Job Processing Systems: Handling background jobs where tasks have different priority levels and resource constraints.
• API Rate Limiting: Managing API requests to third-party services that enforce rate limits, ensuring that the application does not exceed the allowed number of concurrent requests.

Currently, developers must rely on third-party libraries like p-queue to handle such scenarios. This reliance introduces additional dependencies, increases project size, and may impact performance. A native solution like Bun.PQueue would streamline development, enhance performance, and reduce dependency management complexities.

What is the feature you are proposing to solve the problem?

I propose adding a native priority queue implementation, Bun.PQueue, as a built-in module in Bun. Bun.PQueue would enable developers to manage promise concurrency efficiently with continuous task processing and prioritization. Key functionalities should include:

• Priority Levels: Allow tasks to be enqueued with assigned priority levels, ensuring that higher-priority tasks are executed before lower-priority ones.
• Concurrency Control: Enable developers to set the maximum number of concurrent promises, ensuring that a new promise starts immediately when another finishes, maintaining a consistent level of concurrency.
• Continuous Execution: Facilitate smooth, continuous execution without idle periods by automatically queuing the next available task as soon as a running task completes.
• Task Management Options: Provide settings for task timeouts, retry mechanisms, and robust error handling to enhance reliability and flexibility.
• Lightweight API: Design the API to be intuitive and similar to popular third-party libraries like p-queue, making migration straightforward and encouraging adoption among existing Bun users.
• Integration with Bun’s Ecosystem: Leverage Bun’s performance optimizations to ensure that Bun.PQueue operates efficiently within the Bun runtime environment.

Having an inbuilt Bun.PQueue would empower developers to implement efficient and prioritized task management without the need for external dependencies, thereby improving application performance and maintainability.

What alternatives have you considered?

The primary alternative currently available is using third-party libraries such as p-queue. While p-queue effectively manages promise concurrency and prioritization, it introduces additional dependencies, which can lead to increased project size and potential performance overhead. Moreover, relying on external libraries requires maintaining compatibility and managing updates separately from Bun’s core updates.

A native Bun.PQueue would eliminate the need for such third-party dependencies, simplifying dependency management and potentially offering better performance through direct integration with Bun’s optimized runtime. Additionally, a built-in solution would ensure consistent support and updates aligned with Bun’s release cycle, providing a more reliable and maintainable tool for developers.

[bakkot]

You might be interested in this nascent TC39 proposal (cc @michaelficarra).

It doesn't give you quite the same interface as p-queue, but it's very similar for the most common use case:

let governor = new CountingGovernor(concurrencyN);

await governor.with(async () => whatever);

[vikasg603]

You might be interested in this nascent TC39 proposal (cc @michaelficarra).

It doesn't give you quite the same interface as p-queue, but it's very similar for the most common use case:

let governor = new CountingGovernor(concurrencyN);

await governor.with(async () => whatever);

Hey @bakkot, This really looks promising.
But still lacks a priority.