Don't rely on symlinks in pub

[DartBot]

This issue was originally filed by [email protected]

---

I am using dropbox to sync all my projects that I am working on. Unfortunately the symbolic packages link causes some severe confusion on the dropbox side because they are pointing outside the dropbox folder. And so it happens that dropbox syncs non-sense and erases pub-cache data.
Then two different computers are running that have a slightly different version set of dart, the sync process can cause the pub cache on both computers to become corrupt. This is highly annoying.
There is an option called selective sync, but when you create a new folder in any dart project that is open in the IDE, a new package symlink is created and the havoc continues.

I would thus recommend the following:
Create the option to make the packages folder in the project root a regular folder and COPY all files there.
Make symlinks from all sub package folders point to this package folder.
A pub get updates the package folder like if it were the pub-cache folder.

But having the option to not use symlinks in favor of code duplication in general would be awesome.

[sethladd]

Removed Type-Defect label.
Added Type-Enhancement, Area-Pub, Triaged labels.

[nex3]

We have a broader plan to stop using symlinks entirely in pub, in favor of "pub serve" and "pub build" for web-apps and the as-yet-unimplemented "pub run" command for command-line projects. I don't think we're likely to add a mode that copies files rather than symlinking them, but I'll leave this issue open to track the removal of symlinks, since it looks like we don't have an issue for that yet.

---

Changed the title to: "Don't rely on symlinks in pub".

[DartBot]

This comment was originally written by [email protected]

---

While in general I think getting rid of the symlink altogether is a good idea, relying on pub serve alone is not the best idea. Or at least it would need to get a bit more powerful.
Right now I have the problem that I dynamically generate Dart code that I need to serve and spawn. Unfortunately the origin policy applies to that very stricly. I thus can not spawn from localhost:9998 when my file is served from localhost:1030. The way I worked around that is that serve the static files from localhost:9998. This works because the packages files are there. If you would simply remove the symlink this would break.
So it would be pretty neat to allow pub serve to proxy a certain path to another server. For example /api/* would proxy to localhost:9998/api

Otherwise I am cheering in joy thinking about getting rid of the symlinks :)

[nex3]

If you're integrating pub serve with a custom server, we suggest you proxy requests from your server to the pub serve server.

[DartBot]

This comment was originally written by [email protected]

---

I would rather prefer it the other way around as pub serve is not meant for production, so having a proxy there does not matter. But integrating proxy code into a server that is deployed into production is something I would rather want to avoid. Although certainly not critical. It is more of a nice to have feature.

[munificent]

Marked this as blocking #8037.

[munificent]

Marked this as blocking #9418.

[jbdeboer]

Could you share a design doc on these broader plans? We have processes that are dependant on the current packages/ layout and don't want to be broken unexpectedly.

[munificent]

Removed Priority-Unassigned label.
Added Priority-Medium label.

[munificent]

Issue #16494 has been merged into this issue.

---

cc @nex3.

[DartBot]

This comment was originally written by [email protected]

---

I would like to see this done. I am using package webstart as a dart webserver and having non symlinked packages would be great for this.

[nex3]

Issue #17050 has been merged into this issue.

[nex3]

Marked this as being blocked by #19742.

[DartBot]

This comment was originally written by @kaendfinger

---

If I am correct, npm supports this, or I think it might not symlink at all. Anyways, I think this would be an amazing addition.

[DartBot]

This comment was originally written by @zoechi

---

You could try using dart2dart to get rid of the symlinks
https://code.google.com/p/dart/issues/detail?id=19978#c1

[DartBot]

This comment was originally written by @PaulAnnekov

---

There is one more problem appears when using symlinks. It's the problem with VirtualBox and its shares. Vagrant uses VirtualBox so it's relevant to developing process.

VB shares does not support symlinks from guest filesystem to shared folders if host OS is Windows. Bug ticket: https://www.virtualbox.org/ticket/10085

This leads to great discomfort when trying to launch console app in VB. I have solved this problem using pub build --all --mode=debug + adding the following transformation:

  transformers:
  - $dart2js:
      commandLineOptions: ['--output-type=dart']

But this solution is not clean. It's workaround. I want release app but I have to set mode=debug :(.

And one more thing. Even if I use pub build or pub run the error message due to symlinks still displayed (Cannot create link, path = '/home/user/packages/args' (OS Error: Protocol error, errno = 71)). But it displayed only if packages is not downloaded yet. I think that it tries to make symlinks.
Do I need to create new issue about this bug? pub run should not create symlinks.

[nex3]

"pub build" and "pub run" need to run "pub get" if your lockfile is out-of-date, and "pub get" needs to create symlinks (at least until this issue is fixed).

[DartBot]

This comment was originally written by @PaulAnnekov

---

Yep, that's what I'm talking about. I can't build or run project without errors in VirtualBox.

You should modify pub get so it had opportunity to just download packages without creating links (maybe using some argument). Then pub run can call it with this parameter and error won't happen.
Or pub should stop using symlinks.

[nex3]

We can't get rid of symlinks entirely until issue #19742 is fixed. Until then, you can pass "--package-symlinks=false" to any pub process and it won't create them. Note that this option is unsupported, though; it may change or disappear in the future without any notice.

[DartBot]

This issue has been moved to dart-lang/pub#727.