Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OS Error: File name too long, errno = 63 with ios build tasks #222

Closed
ragingsquirrel3 opened this issue Jan 11, 2022 · 8 comments · Fixed by #227
Closed

OS Error: File name too long, errno = 63 with ios build tasks #222

ragingsquirrel3 opened this issue Jan 11, 2022 · 8 comments · Fixed by #227

Comments

@ragingsquirrel3
Copy link

I have been getting this error with melos version 1.1.2 but cannot repro when I downgrade to 0.4.11. When running a series of of melos commands (that happen to build example apps and run integration tests on ios simulator) I get the following error:

FileSystemException: Directory listing failed, path = '/Users/tshepp/amplify_src/amplify-flutter/packages/amplify_api/example/ios/.symlinks/plugins/amplify_api/example/ios/.symlinks/plugins/amplify_storage_s3/example/ios/.symlinks/plugins/amplify_api/example/ios/.symlinks/plugins/amplify_flutter/example/ios/.symlinks/plugins/amplify_api/example/ios/.symlinks/plugins/amplify_datastore/example/ios/.symlinks/plugins/amplify_api/example/ios/.symlinks/plugins/amplify_auth_cognito/example/ios/.symlinks/plugins/amplify_storage_s3/example/ios/.symlinks/plugins/amplify_storage_s3/example/ios/.symlinks/plugins/amplify_flutter/example/ios/.symlinks/plugins/amplify_storage_s3/example/ios/.symlinks/plugins/amplify_datastore/example/ios/.symlinks/plugins/amplify_storage_s3/example/ios/.symlinks/plugins/amplify_auth_cognito/example/ios/.symlinks/plugins/amplify_flutter/example/ios/.symlinks/plugins/amplify_flutter/example/ios/Pods/AmplifyPlugins/AmplifyPlugins/DataStore/AWSDataStoreCategoryPlugin/Sync/SubscriptionSync/ReconcileAndLocalSave/ReconcileAndLocalSaveOperation+Action' (OS Error: File name too long, errno = 63)

The same tasks do not fail when running similar tasks for android build, maybe because these file name are specific to ios? Not sure, however, this makes it hard to use latest melos in build so figured I would file here in case there is a workaround or if others have the same issue.

Here is a github action run where I repro:

https://github.com/ragingsquirrel3/amplify-flutter/actions/runs/1684037448

and the same workflow passing when only specifying melos 0.4.11

https://github.com/ragingsquirrel3/amplify-flutter/actions/runs/1684157840

@blaugold
Copy link
Collaborator

blaugold commented Jan 12, 2022

Could you try version 1.0.0?

There was a change in 1.1.0 which could be related.

@Salakar
Copy link
Contributor

Salakar commented Jan 12, 2022

Could you try version 1.0.0?

There was a change in 1.1.0 which could be related.

Feel like that change is going to haunt me 🙈 I think you are right in that it's related to this, though not sure how I could fix this since it's coming directly from calling Directory(workspacePath).list(recursive: true) since the '**/.symlinks/plugins/**' glob should already be excluding this path from further interaction in Melos

Perhaps @Quijx may have some suggestions here?

@Quijx
Copy link
Contributor

Quijx commented Jan 12, 2022

The problem here is that we are only excluding .symlinks/plugins during the iteration, but all the paths including it are still iterated over. I have some ideas on how to fix this. Hopefully I will find some time after work today to take a further look.

@Salakar
Copy link
Contributor

Salakar commented Jan 12, 2022

Setting followLinks to false could be a solution, then optionally following them inside the iteration if it's a symlink we should care about, since list still returns them just as a Link instance instead if followLinks is false

@ragingsquirrel3
Copy link
Author

Could you try version 1.0.0?

There was a change in 1.1.0 which could be related.

I just tried 1.0.0 and can confirm it works https://github.com/ragingsquirrel3/amplify-flutter/actions/runs/1688807556

@Quijx
Copy link
Contributor

Quijx commented Jan 12, 2022

Ok. So my rough plan for now is to define:

extension DirectoryUtils on Directory {
  Stream<FileSystemEntity> listConditionallyRecursive({
    required bool Function(Directory directory) recurseCondition,
    bool followLinks = true,
  }) {
    Stream<FileSystemEntity> recurse(
      Directory directory,
      Set<String> visitedLinks,
    ) async* {
      await for (final entity in directory.list(followLinks: false)) {
        if (entity is File) {
          yield entity;
        } else if (entity is Directory) {
          yield entity;
          if (recurseCondition(entity)) {
            yield* recurse(entity, visitedLinks);
          }
        } else if (entity is Link) {
          if (!followLinks ||
              visitedLinks.contains(await entity.resolveSymbolicLinks())) {
            yield entity;
            break;
          }
          // ignore: exhaustive_cases
          switch (FileSystemEntity.typeSync(entity.path)) {
            case FileSystemEntityType.directory:
              final directory = Directory(entity.path);
              yield directory;
              if (recurseCondition(directory)) {
                yield* recurse(
                  directory,
                  {...visitedLinks, await entity.resolveSymbolicLinks()},
                );
              }
              break;
            case FileSystemEntityType.file:
              yield File(entity.path);
              break;
            case FileSystemEntityType.notFound:
              yield entity;
              break;
          }
        }
      }
    }

    return recurse(this, {});
  }
}

and then do

await for (final entity
        in Directory(workspacePath).listConditionallyRecursive(
      recurseCondition: (dir) {
        final path = dir.path;
        return !fvmGlob.matches(path) &&
            !dartToolGlob.matches(path) &&
            !symlinksPluginsGlob.matches(path);
      },
    )) {

instead of this

await for (final entity in Directory(workspacePath).list(recursive: true)) {

(and of course remove the glob checks from inside the loops).

It seems a bit overkill to me, so anyone got any better ideas?

@Quijx
Copy link
Contributor

Quijx commented Jan 14, 2022

I have made a PR that should fix this issue: #227
I have tested it locally for one project of mine that uses a linked package, but since I don't use iOS I can't really test this specific case. @ragingsquirrel3 you could test this by cloning my fork https://github.com/Quijx/melos/tree/fix-file-name-too-long-when-following-links on the branch fix-file-name-too-long-when-following-links and run dart pub global activate -s path . in the packages/melos directory. (Sadly pub global activate -s git can't handle branches: dart-lang/pub#29)

@ragingsquirrel3
Copy link
Author

I have made a PR that should fix this issue: #227 I have tested it locally for one project of mine that uses a linked package, but since I don't use iOS I can't really test this specific case. @ragingsquirrel3 you could test this by cloning my fork https://github.com/Quijx/melos/tree/fix-file-name-too-long-when-following-links on the branch fix-file-name-too-long-when-following-links and run dart pub global activate -s path . in the packages/melos directory. (Sadly pub global activate -s git can't handle branches: dart-lang/pub#29)

Will try to do in the next few days. Thanks for working on a fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants