ts_library does not work with Bazel 1.0.0+ on Windows

[gregmagolan]

Updating to Bazel 1.0.0 on Windows ts_library fails with

---8<---8<--- Start of log, file at C:/users/circleci/_bazel_circleci/u4uoan2j/bazel-workers/worker-2-TypeScriptCompile.log ---8<---8<---
Error: Cannot find module 'npm/node_modules/@bazel/typescript/internal/tsc_wrapped/tsc_wrapped.js'. Please verify that the package.json has a valid "main" entry
    at Function.module.constructor._resolveFilename (C:\users\circleci\_bazel_circleci\u4uoan2j\execroot\angular\bazel-out\host\bin\external\npm\@bazel\typescript\bin\tsc_wrapped_loader.js:485:17)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Object.<anonymous> (C:\users\circleci\_bazel_circleci\u4uoan2j\execroot\angular\bazel-out\host\bin\external\npm\@bazel\typescript\bin\tsc_wrapped_loader.js:524:24)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
    at startup (internal/bootstrap/node.js:283:19)
---8<---8<--- End of log ---8<---8<---

Originally seen on angular WIndows CI after angular/angular#33367 landed: https://app.circleci.com/jobs/github/angular/angular/501770

I can repro on my local Windows machine.

/cc @josephperrott @alexeagle

This is blocking the Angular update to Bazel 1.0.0

[gregmagolan]

Looks flaky. I saw it only once on initial build after clone and can't reproduce a 2nd time.

[gregmagolan]

Characterized this some more. Its not a worker problem. Something with bazel test behaving differently than bazel build.

C:\Users\gmago\google\angular>bazel test //packages:types --strategy=TypeScriptCompile=local
INFO: Writing tracer profile to 'C:/users/gmago/_bazel_greg/t3jlnqqf/command.profile.gz'
INFO: Invocation ID: 3e30134d-3b8d-4650-a3d1-97f03ee1e0eb
INFO: Analyzed target //packages:types (0 packages loaded, 0 targets configured).
INFO: Found 1 target and 0 test targets...
ERROR: C:/users/gmago/google/angular/packages/BUILD.bazel:11:1: Compiling TypeScript (devmode) //packages:types failed (Exit 1)
Error: Cannot find module 'npm/node_modules/@bazel/typescript/internal/tsc_wrapped/tsc_wrapped.js'. Please verify that the package.json has a valid "main" entry
    at Function.module.constructor._resolveFilename (C:\users\gmago\_bazel_greg\t3jlnqqf\execroot\angular\bazel-out\host\bin\external\npm\@bazel\typescript\bin\tsc_wrapped_loader.js:485:17)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Object.<anonymous> (C:\users\gmago\_bazel_greg\t3jlnqqf\execroot\angular\bazel-out\host\bin\external\npm\@bazel\typescript\bin\tsc_wrapped_loader.js:524:24)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
    at startup (internal/bootstrap/node.js:283:19)
Target //packages:types failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 3.600s, Critical Path: 2.18s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

C:\Users\gmago\google\angular>bazel build //packages:types --strategy=TypeScriptCompile=local
INFO: Writing tracer profile to 'C:/users/gmago/_bazel_greg/t3jlnqqf/command.profile.gz'
INFO: Invocation ID: 4d37ca1d-6a85-4e0b-9ce2-9f2c1e730380
INFO: Build option --legacy_external_runfiles has changed, discarding analysis cache.
INFO: Analyzed target //packages:types (0 packages loaded, 1039 targets configured).
INFO: Found 1 target...
Target //packages:types up-to-date:
  dist/bin/packages/empty.d.ts
INFO: Elapsed time: 5.125s, Critical Path: 3.61s
INFO: 1 process: 1 local.
INFO: Build completed successfully, 4 total actions

[gregmagolan]

Ok. Figured it out and I have a repro on rules_nodejs and a fix.

When running bazel test bazel will add a {workspace}/.runfile entry with no real path to the MANIFEST if and only if there is no other {workspace} entry in the runfiles (i.e., all runfiles files are from external workspaces). It seems like this got introduced in Bazel 1.0.0 as this issue was not observed with Bazel 0.29.0. This change exposed a bug in the node loader resolver logic.

Relevant code in the bazel repo is:

/**
     * Returns the manifest, adding the workspaceName directory if it is not already present.
     */
    public Map<PathFragment, Artifact> build() {
      if (!sawWorkspaceName) {
        // If we haven't seen it and we have seen other files, add the workspace name directory.
        // It might not be there if all of the runfiles are from other repos (and then running from
        // x.runfiles/ws will fail, because ws won't exist). We can't tell Runfiles to create a
        // directory, so instead this creates a hidden file inside the desired directory.
        manifest.put(workspaceName.getRelative(".runfile"), null);
      }
      return manifest;
    }

In rules_nodejs I was able to repo this with bazel 1.0.0 & 1.1.0 in the bazel-in-bazel tests only as ts_library targets in the root repo don't meet the external runfiles files only case.

https://buildkite.com/bazel/rules-nodejs-nodejs/builds/4678#1e469c69-7e89-4066-a2c2-0dbb0e8a48e5 && https://buildkite.com/bazel/rules-nodejs-nodejs/builds/4679#093fd1ba-1d6b-40ac-be71-ef8869c4eb34:

==================== Test output for //e2e:e2e_typescript_3_0_x:
--
  | [test_runner.js] running 'bazel version'
  | $TEST_TMPDIR defined: output root default is 'd:\b\w5q2lxao\execroot\build_bazel_rules_nodejs\_tmp\5f5f25bf78b3d80c5bfc0d73d83d3d53' and max_idle_secs default is '15'.
  | Extracting Bazel installation...
  | Starting local Bazel server and connecting to it...
  | Build label: 1.0.0
  | Build target: bazel-out/x64_windows-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
  | Build time: Thu Oct 10 10:17:16 2019 (1570702636)
  | Build timestamp: 1570702636
  | Build timestamp as int: 1570702636
  | [test_runner.js] running 'bazel test ... --test_tag_filters=-no-bazelci,-no-bazelci-windows,-fix-windows,-manual --local_resources=13288,1.0,1.0'
  | $TEST_TMPDIR defined: output root default is 'd:\b\w5q2lxao\execroot\build_bazel_rules_nodejs\_tmp\5f5f25bf78b3d80c5bfc0d73d83d3d53' and max_idle_secs default is '15'.
  | INFO: Writing tracer profile to 'D:/b/w5q2lxao/execroot/build_bazel_rules_nodejs/_tmp/5f5f25bf78b3d80c5bfc0d73d83d3d53/_bazel_b/p6eutxls/command.profile.gz'
  | Loading:
  | Loading: 0 packages loaded
  | INFO: SHA256 (file:D:/b/w5q2lxao/execroot/build_bazel_rules_nodejs/bazel-out/x64_windows-fastbuild/bin/release.tar.gz) = 14c2c4c306c35b761d2db84c49f2d99b1851fdb3fda4351dada8f226e35ebdca
  | DEBUG: Rule 'build_bazel_rules_nodejs' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "14c2c4c306c35b761d2db84c49f2d99b1851fdb3fda4351dada8f226e35ebdca"
  | DEBUG: Call stack for the definition of repository 'build_bazel_rules_nodejs' which is a http_archive (rule definition at D:/b/w5q2lxao/execroot/build_bazel_rules_nodejs/_tmp/5f5f25bf78b3d80c5bfc0d73d83d3d53/_bazel_b/p6eutxls/external/bazel_tools/tools/build_defs/repo/http.bzl:262:16):
  | - D:/temp/tmp-5404atp63qzcpfs5/WORKSPACE:23:1
  | Loading: 0 packages loaded
  | Loading: 0 packages loaded
  | Loading: 0 packages loaded
  | Loading: 0 packages loaded
  | Loading: 0 packages loaded
  | Loading: 0 packages loaded
  | Analyzing: 5 targets (1 packages loaded, 0 targets configured)
  | Analyzing: 5 targets (160 packages loaded, 2983 targets configured)
  | Analyzing: 5 targets (163 packages loaded, 3026 targets configured)
  | Analyzing: 5 targets (164 packages loaded, 3055 targets configured)
  | Analyzing: 5 targets (164 packages loaded, 3055 targets configured)
  | INFO: Analyzed 5 targets (165 packages loaded, 3189 targets configured).
  | INFO: Found 4 targets and 1 test target...
  | [0 / 10] [Prepa] Writing script test.bat ... (4 actions, 0 running)
  | ERROR: D:/temp/tmp-5404atp63qzcpfs5/BUILD.bazel:18:1: Compiling TypeScript (devmode) //:lib failed: Worker process quit or closed its stdin stream when we tried to send a WorkRequest:
  |  
  | ---8<---8<--- Exception details ---8<---8<---
  | java.io.IOException: ERROR: src/main/native/windows/processes-jni.cc(356): NativeProcess:WriteStdin(2156): The pipe is being closed.
  |  
  | at com.google.devtools.build.lib.windows.WindowsSubprocess.writeStream(WindowsSubprocess.java:263)
  | at com.google.devtools.build.lib.windows.WindowsSubprocess.access$000(WindowsSubprocess.java:33)
  | at com.google.devtools.build.lib.windows.WindowsSubprocess$ProcessOutputStream.write(WindowsSubprocess.java:57)
  | at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.doFlush(CodedOutputStream.java:3003)
  | at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.write(CodedOutputStream.java:2935)
  | at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeLazy(CodedOutputStream.java:2954)
  | at com.google.protobuf.ByteString$LiteralByteString.writeTo(ByteString.java:1331)
  | at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeBytesNoTag(CodedOutputStream.java:2751)
  | at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeBytes(CodedOutputStream.java:2724)
  | at com.google.protobuf.GeneratedMessageV3.writeString(GeneratedMessageV3.java:2721)
  | at com.google.devtools.build.lib.worker.WorkerProtocol$Input.writeTo(WorkerProtocol.java:220)
  | at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeMessageNoTag(CodedOutputStream.java:2800)
  | at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeMessage(CodedOutputStream.java:2775)
  | at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.writeTo(WorkerProtocol.java:1009)
  | at com.google.protobuf.AbstractMessageLite.writeDelimitedTo(AbstractMessageLite.java:98)
  | at com.google.devtools.build.lib.worker.WorkerSpawnRunner.execInWorker(WorkerSpawnRunner.java:348)
  | at com.google.devtools.build.lib.worker.WorkerSpawnRunner.actuallyExec(WorkerSpawnRunner.java:190)
  | at com.google.devtools.build.lib.worker.WorkerSpawnRunner.exec(WorkerSpawnRunner.java:129)
  | at com.google.devtools.build.lib.exec.SpawnRunner.execAsync(SpawnRunner.java:225)
  | at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:123)
  | at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:88)
  | at com.google.devtools.build.lib.actions.SpawnActionContext.beginExecution(SpawnActionContext.java:41)
  | at com.google.devtools.build.lib.exec.ProxySpawnActionContext.beginExecution(ProxySpawnActionContext.java:56)
  | at com.google.devtools.build.lib.actions.SpawnContinuation$1.execute(SpawnContinuation.java:80)
  | at com.google.devtools.build.lib.analysis.actions.SpawnAction$SpawnActionContinuation.execute(SpawnAction.java:1342)
  | at com.google.devtools.build.lib.analysis.actions.SpawnAction.beginExecution(SpawnAction.java:314)
  | at com.google.devtools.build.lib.actions.Action.execute(Action.java:123)
  | at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$4.execute(SkyframeActionExecutor.java:851)
  | at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.continueAction(SkyframeActionExecutor.java:985)
  | at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.run(SkyframeActionExecutor.java:957)
  | at com.google.devtools.build.lib.skyframe.ActionExecutionState.runStateMachine(ActionExecutionState.java:116)
  | at com.google.devtools.build.lib.skyframe.ActionExecutionState.getResultOrDependOnFuture(ActionExecutionState.java:77)
  | at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:577)
  | at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:760)
  | at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:275)
  | at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:454)
  | at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)
  | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | ---8<---8<--- End of exception details ---8<---8<---
  |  
  | ---8<---8<--- Start of log, file at D:/b/w5q2lxao/execroot/build_bazel_rules_nodejs/_tmp/5f5f25bf78b3d80c5bfc0d73d83d3d53/_bazel_b/p6eutxls/bazel-workers/worker-0-TypeScriptCompile.log ---8<---8<---
  | Error: Cannot find module 'npm/node_modules/@bazel/typescript/internal/tsc_wrapped/tsc_wrapped.js'. Please verify that the package.json has a valid "main" entry
  | at Function.module.constructor._resolveFilename (D:\b\w5q2lxao\execroot\build_bazel_rules_nodejs\_tmp\5f5f25bf78b3d80c5bfc0d73d83d3d53\_bazel_b\p6eutxls\execroot\e2e_typescript\bazel-out\host\bin\external\npm\@bazel\typescript\bin\tsc_wrapped_loader.js:485:17)
  | at Function.Module._load (internal/modules/cjs/loader.js:562:25)
  | at Object.<anonymous> (D:\b\w5q2lxao\execroot\build_bazel_rules_nodejs\_tmp\5f5f25bf78b3d80c5bfc0d73d83d3d53\_bazel_b\p6eutxls\execroot\e2e_typescript\bazel-out\host\bin\external\npm\@bazel\typescript\bin\tsc_wrapped_loader.js:524:24)
  | at Module._compile (internal/modules/cjs/loader.js:776:30)
  | at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
  | at Module.load (internal/modules/cjs/loader.js:653:32)
  | at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
  | at Function.Module._load (internal/modules/cjs/loader.js:585:3)
  | at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
  | at startup (internal/bootstrap/node.js:283:19)
  | ---8<---8<--- End of log ---8<---8<---
  | INFO: Elapsed time: 19.465s, Critical Path: 0.57s
  | INFO: 0 processes.
  | FAILED: Build did NOT complete successfully
  | //:test                                                               NO STATUS

[gregmagolan]

Fix is a one-liner in the loader