[local-cli] Support symlinks-to-symlinks #9792

aleclarson · 2016-09-07T16:13:31Z

I could be wrong here, but I think Watchman can't handle symlinks, so we need to make sure symlinks-to-symlinks are handled up front.

edit: Here's a summary of the changes made by this commit:

Squash symlink resolution into a single forEach
Resolve symlinks recursively
Throw an error for circular symlinks
Use lookupFolder to resolve folder (instead of process.cwd())
Resolve relative symlinks correctly
Simplify isSubPathOfPath function (and rename to isDescendant)
Rename isSubPathOfPaths function to rootExists
Rename existingSearchPaths argument to ignoredRoots
Pass both args.projectRoots and args.root to findSymlinksPaths

ghost · 2016-09-07T16:13:59Z

By analyzing the blame information on this pull request, we identified @Kureev to be a potential reviewer.

mkonicek · 2016-09-08T13:39:38Z

Thanks for the PR. I'll let @Kureev take a look as he's added the symlink support (thank you for doing that!).

Kureev · 2016-09-09T07:58:53Z

So, recursive lookup for symlinks? Nice idea!

offtopic: haven't seen do...while for a errrm while already

Kureev · 2016-09-09T08:07:41Z

IMHO,

do {
  symlink = path.resolve(process.cwd(), fs.readlinkSync(symlink));
} while (fs.lstatSync(symlink).isSymbolicLink());

is a bit harder to grasp than

var recursiveSymlink;
while (fs.lstatSync(symlink).isSymbolicLink()) {
  recursiveSymlink = path.resolve(process.cwd(), fs.readlinkSync(symlink))
};

this version has two improvements from my perspective:

we don't override iterator argument
while {} seems to be more popular construction then do {} while

Also, if you want to solve recursive symlink problem, you probably have to think about loops:
A -> B -> A. I propose to store a linked list of "visits" and check if current symlink path is not in it, otherwise throw.

What do you think, guys? Does it make any sense to you?

ghost · 2016-09-09T13:41:03Z

@aleclarson updated the pull request - view changes

aleclarson · 2016-09-09T13:43:51Z

local-cli/server/findSymlinksPaths.js

+
+        prevSymlink = nextSymlink;
+        nextSymlink = path.resolve(
+          path.dirname(prevSymlink),


Note: Symlinks are now resolved relative to their parent directory. This is the correct behavior, because symlinks aren't aware of process.cwd() and thus could be resolved into the wrong path.

aleclarson · 2016-09-09T13:46:37Z

local-cli/server/findSymlinksPaths.js

+      const symlinkCache = new Set();
+      let prevSymlink;
+      let nextSymlink = symlink;
+      while (true) {


while (true) is used here because we know for a fact that symlink is a symlink.
At the end of each loop, !fs.lstatSync(nextSymlink).isSymbolicLink() determines when to stop.

Why not

while (fs.lstatSync(nextSymlink).isSymbolicLink()) {...}

?

Seems redundant, no? We already do the filter call to make sure only symlinks are present.

aleclarson · 2016-09-09T13:53:38Z

@Kureev Let me know what you think of the new changes. 😄

I'll squash the commits once everything is okay'd.

Kureev · 2016-09-09T14:40:18Z

local-cli/server/findSymlinksPaths.js

+      let nextSymlink = symlink;
+      while (true) {
+        if (symlinkCache.has(nextSymlink)) {
+          throw Error(`Symlink recursion detected:\n  ${nextSymlink}`);


Will be really great if we can throw an error with the whole sequence that leads to the recursion. Like "Symlink recursion detected: A -> B -> C -> B"

Truth. I'll just switch to using an Array and then use slice to get the relevant sequence.

Though it could get a little messy if the sequence is long.

Kureev · 2016-09-09T14:43:21Z

In general, looks really good. I think we can merge it in once we agree on some minor details

facebook-github-bot · 2016-09-09T16:07:02Z

@aleclarson updated the pull request - view changes

aleclarson · 2016-09-09T16:09:59Z

local-cli/server/findSymlinksPaths.js

+      );
+    }
+
+    if (visited.length) {


Since we no longer call folders.filter to guarantee each item is a symlink, we need to protect against while (fs.lstatSync(symlink).isSymbolicLink()) doing zero loops.

aleclarson · 2016-09-09T16:10:53Z

local-cli/server/findSymlinksPaths.js

+      if (index !== -1) {
+        throw Error(
+          `Infinite symlink recursion detected:\n  ` +
+            visited.slice(index).join(`\n  `)


Note: This prints the symlink sequence in order, as opposed to being reversed.
Not sure which way is optimal, though.

Kureev · 2016-09-12T09:06:35Z

@facebook-github-bot shipit

Kureev · 2016-09-12T09:07:42Z

Thank you for your contribution, @aleclarson! I think it's good to go now 👍

Kureev · 2016-09-13T19:48:58Z

@facebook-github-bot shipit

Kureev · 2016-09-13T19:49:23Z

Could you be so kind to take a look why import fails, @bestander?

ghost · 2016-09-13T19:49:39Z

Thanks for importing. If you are an FB employee go to Phabricator to review internal test results.

bestander · 2016-09-13T21:59:15Z

Will do

ghost · 2016-09-14T08:48:18Z

I tried to merge this pull request into the Facebook internal repo but some checks failed. To unblock yourself please check the following: Does this pull request pass all open source tests on GitHub? If not please fix those. Does the code still apply cleanly on top of GitHub master? If not can please rebase. In all other cases this means some internal test failed, for example a part of a fb app won't work with this pull request. I've added the Import Failed label to this pull request so it is easy for someone at fb to find the pull request and check what failed. If you don't see anyone comment in a few days feel free to comment mentioning one of the core contributors to the project so they get a notification.

bestander · 2016-09-14T09:08:44Z

This file was changed in #9819.
Could you rebase?

- Throw an error for circular symlinks - Rename (and simplify) `isSubPathOfPath` to `isDescendant` - Rename `isSubPathOfPaths` to `rootExists` - Rename var `existingSearchPaths` to `ignoredRoots` - Squash symlink resolution into a single `forEach` - Pass both `args.projectRoots` and `args.root` to `findSymlinksPaths`

ghost · 2016-09-14T10:04:13Z

@aleclarson updated the pull request - view changes - changes since last import

ghost · 2016-09-14T10:05:16Z

@aleclarson updated the pull request - view changes - changes since last import

Kureev

LGTM! 👍

Kureev · 2016-09-15T08:13:52Z

@facebook-github-bot shipit

ghost · 2016-09-15T08:15:58Z

Thanks for importing. If you are an FB employee go to Phabricator to review internal test results.

ekryski · 2016-09-22T15:56:50Z

@aleclarson nice work! Sorry I was MIA for a bit on my other PR but then this came along 😄 . Stoked!

ericvicenti · 2016-12-13T00:11:39Z

I see this is closed, but symlinks are still not working for me. Did we have a regression?

Summary: In response to [this comment](facebook/react-native#9009 (comment)). I could be wrong here, but I think Watchman can't handle symlinks, so we need to make sure symlinks-to-symlinks are handled up front. Closes facebook/react-native#9792 Differential Revision: D3858349 Pulled By: bestander fbshipit-source-id: f3a34dae90ed9a7004a03158288db5e1932bfc69

ghost added GH Review: review-needed CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. labels Sep 7, 2016

mkonicek assigned Kureev Sep 8, 2016

ghost added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 8, 2016

aleclarson reviewed Sep 9, 2016
View reviewed changes

ghost added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 9, 2016

aleclarson reviewed Sep 9, 2016
View reviewed changes

ghost added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 9, 2016

Kureev reviewed Sep 9, 2016
View reviewed changes

aleclarson reviewed Sep 9, 2016
View reviewed changes

ghost added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 9, 2016

aleclarson reviewed Sep 9, 2016
View reviewed changes

aleclarson force-pushed the symlinks-to-symlinks branch from e27139a to 165f1ba Compare September 12, 2016 09:55

ghost added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 12, 2016

ghost added the Import Started This pull request has been imported. This does not imply the PR has been approved. label Sep 13, 2016

ghost added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 13, 2016

ghost added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. and removed Import Started This pull request has been imported. This does not imply the PR has been approved. labels Sep 14, 2016

aleclarson force-pushed the symlinks-to-symlinks branch from 165f1ba to 817f7f8 Compare September 14, 2016 10:03

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 14, 2016

aleclarson mentioned this pull request Sep 14, 2016

packager: dedupe symlinks if they point to already covered paths #9819

Closed

Kureev approved these changes Sep 15, 2016

View reviewed changes

facebook-github-bot added the Import Started This pull request has been imported. This does not imply the PR has been approved. label Sep 15, 2016

ghost added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 15, 2016

ghost closed this in fa6191f Sep 15, 2016

ghost added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 15, 2016

aleclarson deleted the symlinks-to-symlinks branch September 16, 2016 17:35

aleclarson mentioned this pull request Sep 20, 2016

Symlink support for packager #9009

Closed

ekryski mentioned this pull request Sep 22, 2016

use lookup folder instead of cwd for symlink path resolution. Refs #9009 #9574

Closed

xdmx mentioned this pull request Oct 11, 2016

[Packager] doesn't resolve modules that are symlinks #637

Closed

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[local-cli] Support symlinks-to-symlinks #9792

[local-cli] Support symlinks-to-symlinks #9792

aleclarson commented Sep 7, 2016 •

edited

Loading

ghost commented Sep 7, 2016

mkonicek commented Sep 8, 2016 •

edited

Loading

Kureev commented Sep 9, 2016

Kureev commented Sep 9, 2016 •

edited

Loading

ghost commented Sep 9, 2016

aleclarson Sep 9, 2016 •

edited

Loading

aleclarson Sep 9, 2016

Kureev Sep 9, 2016

aleclarson Sep 9, 2016

aleclarson commented Sep 9, 2016 •

edited

Loading

Kureev Sep 9, 2016

aleclarson Sep 9, 2016

Kureev commented Sep 9, 2016

facebook-github-bot commented Sep 9, 2016

aleclarson Sep 9, 2016

aleclarson Sep 9, 2016 •

edited

Loading

Kureev commented Sep 12, 2016

Kureev commented Sep 12, 2016

Kureev commented Sep 13, 2016

Kureev commented Sep 13, 2016 •

edited

Loading

ghost commented Sep 13, 2016

bestander commented Sep 13, 2016

ghost commented Sep 14, 2016

bestander commented Sep 14, 2016

ghost commented Sep 14, 2016

ghost commented Sep 14, 2016

Kureev left a comment

Kureev commented Sep 15, 2016

ghost commented Sep 15, 2016

ekryski commented Sep 22, 2016

ericvicenti commented Dec 13, 2016

[local-cli] Support symlinks-to-symlinks #9792

[local-cli] Support symlinks-to-symlinks #9792

Conversation

aleclarson commented Sep 7, 2016 • edited Loading

ghost commented Sep 7, 2016

mkonicek commented Sep 8, 2016 • edited Loading

Kureev commented Sep 9, 2016

Kureev commented Sep 9, 2016 • edited Loading

ghost commented Sep 9, 2016

aleclarson Sep 9, 2016 • edited Loading

Choose a reason for hiding this comment

aleclarson Sep 9, 2016

Choose a reason for hiding this comment

Kureev Sep 9, 2016

Choose a reason for hiding this comment

aleclarson Sep 9, 2016

Choose a reason for hiding this comment

aleclarson commented Sep 9, 2016 • edited Loading

Kureev Sep 9, 2016

Choose a reason for hiding this comment

aleclarson Sep 9, 2016

Choose a reason for hiding this comment

Kureev commented Sep 9, 2016

facebook-github-bot commented Sep 9, 2016

aleclarson Sep 9, 2016

Choose a reason for hiding this comment

aleclarson Sep 9, 2016 • edited Loading

Choose a reason for hiding this comment

Kureev commented Sep 12, 2016

Kureev commented Sep 12, 2016

Kureev commented Sep 13, 2016

Kureev commented Sep 13, 2016 • edited Loading

ghost commented Sep 13, 2016

bestander commented Sep 13, 2016

ghost commented Sep 14, 2016

bestander commented Sep 14, 2016

ghost commented Sep 14, 2016

ghost commented Sep 14, 2016

Kureev left a comment

Choose a reason for hiding this comment

Kureev commented Sep 15, 2016

ghost commented Sep 15, 2016

ekryski commented Sep 22, 2016

ericvicenti commented Dec 13, 2016

aleclarson commented Sep 7, 2016 •

edited

Loading

mkonicek commented Sep 8, 2016 •

edited

Loading

Kureev commented Sep 9, 2016 •

edited

Loading

aleclarson Sep 9, 2016 •

edited

Loading

aleclarson commented Sep 9, 2016 •

edited

Loading

aleclarson Sep 9, 2016 •

edited

Loading

Kureev commented Sep 13, 2016 •

edited

Loading