Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 BUG: Miniflare KV persistence not working in Wrangler2 --local mode #766

Closed
hkochniss opened this issue Apr 6, 2022 · 6 comments · Fixed by #767 or #721
Closed

🐛 BUG: Miniflare KV persistence not working in Wrangler2 --local mode #766

hkochniss opened this issue Apr 6, 2022 · 6 comments · Fixed by #767 or #721

Comments

@hkochniss
Copy link

hkochniss commented Apr 6, 2022

What version of Wrangler are you using?

0.0.24

What operating system are you using?

Mac OS (ARM64)

Describe the Bug

It seems as Wrangler2 uses Miniflare under the hood in --local mode, many Miniflare features should work, especially persistence around KV etc. But for some reason persistence doesn't kick in when using

[miniflare]
kv_persist = true

in wrangler.toml. Also other kv_persist options don't work, KV namespace are just empty on each change of a worker file. This really hurts productivity, I'm assuming this is unintended and should work?

Using typescript and modules, can provide a minimal reproduction case once this is confirmed as a bug.

@hkochniss hkochniss added the bug label Apr 6, 2022
@threepointone
Copy link
Contributor

Try this cli flag: --experimental-enable-local-persistence

@hkochniss
Copy link
Author

hkochniss commented Apr 6, 2022

A bit better, but unfortunately only works for keeping KV stuff between worker changes in watch mode, does not persist to disk apparently

So once I stop wrangler 2 manually (gently via hitting X) and start again, the KV data is lost. Also I can't find anything written to disk visible, not in ./mf/kv or anywhere else, so I assume this is in-memory "persistence" for now (or I'm overlooking a secret folder somewhere, still new to the Mac ecosystem)

How is this intended to work right now? Is there any roadmap/plans around offline persistence? I feel there isn't much missing feature wise (only let Miniflare do its thing), but maybe I'm missing something

@threepointone
Copy link
Contributor

We'd only just started working on this, hence the experimental flag. What the flag does, is enable persistence for kv/durable_objects/cache in miniflare https://github.com/cloudflare/wrangler2/blob/ed4f2ef00f86374ac7d1ac6ff072d939c80b6c18/packages/wrangler/src/dev/local.tsx#L163-L170
This does persist to disk. However, we use a fresh temp directory where the actual worker is bundled, and that temp directory disappears when you close wrangler dev. Instead, we should be persisting to a path relative to the working directory, so that it sticks around in a known location (and let users like you decide what to do with the generated files). I'll send a fix for this soon.

The workaround for now if you want to test persistence, is to use regular "remote" mode. But we'll fix it very soon!

@hkochniss
Copy link
Author

hkochniss commented Apr 6, 2022

Really looking forward to this, that sounds like a big improvement. Also props for the late reply, knowing your timezone is London ;)

threepointone added a commit that referenced this issue Apr 7, 2022
This sets up `--experiment-enable-local-persistence` to explicitly use `process.cwd() + wrangler-local-state` as a path to store values. Without it, local mode uses the temp dir that we use to bundle the worker, which gets wiped out on ending wrangler dev. In the future, based on usage, we may want to make the path configurable as well.

Fixes #766
threepointone added a commit that referenced this issue Apr 7, 2022
This sets up `--experiment-enable-local-persistence` to explicitly use `process.cwd() + wrangler-local-state` as a path to store values. Without it, local mode uses the temp dir that we use to bundle the worker, which gets wiped out on ending wrangler dev. In the future, based on usage, we may want to make the path configurable as well.

Fixes #766
threepointone added a commit that referenced this issue Apr 7, 2022
This sets up `--experiment-enable-local-persistence` to explicitly use `process.cwd() + wrangler-local-state` as a path to store values. Without it, local mode uses the temp dir that we use to bundle the worker, which gets wiped out on ending wrangler dev. In the future, based on usage, we may want to make the path configurable as well.

Fixes #766
@threepointone
Copy link
Contributor

We have a fix in place, you can try it with wrangler@alpha. We will revisit it and make it better, but it should hopefully unblock it for you now. We will do a beta release soon too. Thank you for reporting!

@hkochniss
Copy link
Author

works like a charm, thanks again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants