Support multi-worker Durable Object RPC #7098

RamIdeas · 2024-10-25T15:52:24Z

Fixes #5918

Support Durable Object RPC across multiple wrangler instances

Only supports RPC method calls right now. RPC property access is coming shortly (in this PR)

The entire implementation is in src/dev/miniflare.ts

Tests
- TODO (before merge)
- Tests included
- Tests not necessary because:
E2E Tests CI Job required? (Use "e2e" label or ask maintainer to run separately)
- I don't know
- Required
- Not required because:
Public documentation
- TODO (before merge)
- Cloudflare docs PR(s):
- Documentation not necessary because:

changeset-bot · 2024-10-25T15:52:29Z

⚠️ No Changeset found

Latest commit: 1fe06a5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2024-10-25T15:55:53Z

A wrangler prerelease is available for testing. You can install this latest build in your project with:

npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-wrangler-7098

You can reference the automatically updated head of this PR with:

npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/prs/7098/npm-package-wrangler-7098

Or you can use npx with this latest build directly:

npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-wrangler-7098 dev path/to/script.js

Additional artifacts:

npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-create-cloudflare-7098 --no-auto-update

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-cloudflare-kv-asset-handler-7098

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-miniflare-7098

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-cloudflare-pages-shared-7098

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-cloudflare-vitest-pool-workers-7098

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-cloudflare-workers-editor-shared-7098

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-cloudflare-workers-shared-7098

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/11521424958/npm-package-cloudflare-workflows-shared-7098

Note that these links will no longer work once the GitHub Actions artifact expires.

[email protected] includes the following runtime dependencies:

Package	Constraint	Resolved
`miniflare`	workspace:*	3.20241022.0
`workerd`	1.20241022.0	1.20241022.0
`workerd --version`	1.20241022.0	2024-10-22

Please ensure constraints are pinned, and miniflare/workerd minor versions match.

RamIdeas · 2024-10-25T15:59:50Z

packages/wrangler/src/dev/miniflare.ts

+const EXTERNAL_SERVICE_RECEIVER_SCRIPT = `
+\n;; ${/* this script needs to be concatenated with the user's script :( -- use this to avoid ASI syntax errors*/ ""}
+import { WorkerEntrypoint as __Wrangler__LocalOnly__WorkerEntrypoint } from "cloudflare:workers";
+export class ${EXTERNAL_SERVICE_RECEIVER_NAME} extends __Wrangler__LocalOnly__WorkerEntrypoint {
+    async proxyMethod({ bindingName, doId, method, args }) {
+        const ns = this.env[bindingName];
+        const stub = ns.get(ns.idFromString(doId));
+        return stub[method](...args);
 	}
 }
 `;


This script is added to the user's script – not a separate service as the previous implementation did – so we can bind to it as a regular external entrypoint

This may sound scary but I've taken care to use long+clear names to avoid clashing with user variables in scope and that explain their purpose so if a user ever does discover them they will understand its: (a) injected by wrangler and (b) in local code only, not deployed code

We could avoid this if we could introduce a concept to the dev registry of an entrypoint within a worker[1] within a worker[2]

[1] miniflare subworker
[2] dev registry worker (user worker)

RamIdeas · 2024-10-25T16:00:58Z

packages/wrangler/src/dev/miniflare.ts

+			// redeclare internal durable object bindings for use by __Wrangler__LocalOnly__ExternalDurableObjectsReceiver
+			...internalObjects.map(({ class_name, script_name }) => {
+				const useSQLite = classNameToUseSQLite.get(class_name);
+				const bindingName = getIdentifier(
+					`__Wrangler__LocalOnly__DurableObject__${class_name}`
+				);
+				return [
+					bindingName,
+					{
+						className: class_name,
+						useSQLite,
+						script_name,
 					},
 				];
 			}),


These bindings are also added to the user's worker and on their env object but, again, I've taken care to make long+clear names for the same reasons as above

petebacondarwin · 2024-10-26T14:21:42Z

vitest.shared.ts

@@ -19,6 +19,6 @@ export default defineConfig({
 		restoreMocks: true,
 		// A lot of the fixture tests are extremely flaky because of the dev registry
 		// Retry tests by default so that only real errors are reported
-		retry: 2,
+		retry: 0,


Is this temporary? If not then perhaps update the comment?

Temporary. Will undo

top-level method calls, working top-level property access, working chained promise property access, wip

penalosa · 2024-11-20T13:52:18Z

Closing in favour of #7251

RamIdeas added 4 commits October 24, 2024 17:14

remove miniflare DO special casing

0f1a924

fixed!!!!!

ef7970f

fixup notFoundServices

9591c8f

add fixture so we can test this manually

06fef31

RamIdeas commented Oct 25, 2024

View reviewed changes

petebacondarwin reviewed Oct 26, 2024

View reviewed changes

petebacondarwin mentioned this pull request Oct 27, 2024

🚀 Feature Request:RPC Support for Durable Objects in wrangler dev #5918

Open

emily-shen and others added 3 commits October 28, 2024 11:27

make fetch happen

67949ed

temp

51d7d1d

wip

1fe06a5

top-level method calls, working top-level property access, working chained promise property access, wip

penalosa closed this Nov 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support multi-worker Durable Object RPC #7098

Support multi-worker Durable Object RPC #7098

RamIdeas commented Oct 25, 2024 •

edited

Loading

changeset-bot bot commented Oct 25, 2024 •

edited

Loading

github-actions bot commented Oct 25, 2024

RamIdeas Oct 25, 2024

RamIdeas Oct 25, 2024

RamIdeas Oct 25, 2024

petebacondarwin Oct 26, 2024

RamIdeas Oct 26, 2024

penalosa commented Nov 20, 2024

Support multi-worker Durable Object RPC #7098

Support multi-worker Durable Object RPC #7098

Conversation

RamIdeas commented Oct 25, 2024 • edited Loading

changeset-bot bot commented Oct 25, 2024 • edited Loading

⚠️ No Changeset found

github-actions bot commented Oct 25, 2024

RamIdeas Oct 25, 2024

Choose a reason for hiding this comment

RamIdeas Oct 25, 2024

Choose a reason for hiding this comment

RamIdeas Oct 25, 2024

Choose a reason for hiding this comment

petebacondarwin Oct 26, 2024

Choose a reason for hiding this comment

RamIdeas Oct 26, 2024

Choose a reason for hiding this comment

penalosa commented Nov 20, 2024

RamIdeas commented Oct 25, 2024 •

edited

Loading

changeset-bot bot commented Oct 25, 2024 •

edited

Loading