Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Detect binary files by checking first 256 code units for 0xFFFD #57008

Merged
merged 7 commits into from
Jan 17, 2024
Merged

Detect binary files by checking first 256 code units for 0xFFFD #57008

merged 7 commits into from
Jan 17, 2024
+31 −8

Conversation

jakebailey
Copy link
Member

@jakebailey jakebailey commented Jan 10, 2024 •
edited
Loading

Fixes #56516
Fixes #21136

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jan 10, 2024
@jakebailey
Copy link
Member Author

@typescript-bot test top200
@typescript-bot user test this
@typescript-bot run dt

@typescript-bot perf test this
@typescript-bot perf test public
@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 10, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the tarball bundle task on this PR at 4e80546. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 10, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at 4e80546. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 10, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the parallelized Definitely Typed test suite on this PR at 4e80546. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 10, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at 4e80546. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 10, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the public perf test suite on this PR at 4e80546. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 10, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based top-repos suite on this PR at 4e80546. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 10, 2024 •
edited
Loading

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/159404/artifacts?artifactName=tgz&fileId=2232A0CF2DEC9DFC433B675454EF705C0B418417C9FF1B0B4275551E4DA3176902&fileName=/typescript-5.4.0-insiders.20240110.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

@jakebailey jakebailey mentioned this pull request Jan 10, 2024
Closed
@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/57008/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Package install failed"

Otherwise...

Something interesting changed - please have a look.

Details

puppeteer

packages/browsers/test/src/tsconfig.json

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
mui-docs - node (v20.5.1, x64)
Memory used 1,735,237k (± 0.00%) 1,735,225k (± 0.00%) ~ 1,735,192k 1,735,242k p=0.378 n=6
Parse Time 6.83s (± 0.29%) 6.79s (± 0.61%) ~ 6.73s 6.86s p=0.063 n=6
Bind Time 2.43s (± 0.97%) 2.42s (± 1.76%) ~ 2.34s 2.46s p=0.872 n=6
Check Time 52.65s (± 0.95%) 52.53s (± 0.65%) ~ 52.19s 52.96s p=0.471 n=6
Emit Time 0.15s (± 0.00%) 0.15s (± 0.00%) ~ 0.15s 0.15s p=1.000 n=6
Total Time 62.06s (± 0.75%) 61.89s (± 0.53%) ~ 61.53s 62.32s p=0.378 n=6
self-build-src - node (v20.5.1, x64)
Memory used 2,658,883k (± 5.36%) 2,708,278k (± 7.08%) ~ 2,576,500k 3,071,562k p=0.378 n=6
Parse Time 5.03s (± 0.47%) 5.05s (± 1.14%) ~ 4.94s 5.10s p=0.149 n=6
Bind Time 1.99s (± 0.88%) 2.00s (± 1.04%) ~ 1.97s 2.02s p=1.000 n=6
Check Time 32.12s (± 0.39%) 32.03s (± 0.55%) ~ 31.81s 32.27s p=0.422 n=6
Emit Time 2.84s (± 1.94%) 2.81s (± 2.36%) ~ 2.71s 2.91s p=0.470 n=6
Total Time 42.00s (± 0.34%) 41.90s (± 0.60%) ~ 41.57s 42.28s p=0.230 n=6
self-compiler - node (v20.5.1, x64)
Memory used 419,054k (± 0.02%) 419,066k (± 0.02%) ~ 418,944k 419,181k p=0.810 n=6
Parse Time 2.88s (± 0.38%) 2.90s (± 0.74%) ~ 2.88s 2.93s p=0.150 n=6
Bind Time 1.13s (± 0.46%) 1.13s (± 0.56%) ~ 1.12s 1.14s p=0.386 n=6
Check Time 14.11s (± 0.33%) 14.07s (± 0.46%) ~ 14.02s 14.19s p=0.173 n=6
Emit Time 1.05s (± 1.70%) 1.05s (± 1.20%) ~ 1.04s 1.07s p=0.934 n=6
Total Time 19.18s (± 0.21%) 19.14s (± 0.38%) ~ 19.10s 19.29s p=0.092 n=6
vscode - node (v20.5.1, x64)
Memory used 2,828,075k (± 0.00%) 2,828,087k (± 0.00%) ~ 2,828,031k 2,828,134k p=0.470 n=6
Parse Time 10.75s (± 0.11%) 10.72s (± 0.34%) ~ 10.66s 10.75s p=0.061 n=6
Bind Time 3.43s (± 0.43%) 3.43s (± 0.90%) ~ 3.39s 3.48s p=0.569 n=6
Check Time 56.08s (± 0.51%) 56.15s (± 0.30%) ~ 55.87s 56.32s p=0.748 n=6
Emit Time 16.24s (± 0.54%) 16.19s (± 0.61%) ~ 16.03s 16.29s p=0.226 n=6
Total Time 86.50s (± 0.29%) 86.49s (± 0.22%) ~ 86.31s 86.73s p=0.936 n=6
webpack - node (v20.5.1, x64)
Memory used 390,798k (± 0.02%) 390,769k (± 0.01%) ~ 390,712k 390,833k p=0.689 n=6
Parse Time 3.30s (± 0.56%) 3.31s (± 0.47%) ~ 3.29s 3.32s p=0.548 n=6
Bind Time 1.42s (± 0.59%) 1.42s (± 1.03%) ~ 1.40s 1.44s p=0.797 n=6
Check Time 12.80s (± 0.17%) 12.79s (± 0.34%) ~ 12.73s 12.85s p=0.571 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 17.53s (± 0.23%) 17.52s (± 0.33%) ~ 17.45s 17.60s p=0.809 n=6
System info unknown
Hosts
  • node (v20.5.1, x64)
Scenarios
  • mui-docs - node (v20.5.1, x64)
  • self-build-src - node (v20.5.1, x64)
  • self-compiler - node (v20.5.1, x64)
  • vscode - node (v20.5.1, x64)
  • webpack - node (v20.5.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,538k (± 0.01%) 295,492k (± 0.01%) -46k (- 0.02%) 295,424k 295,537k p=0.031 n=6
Parse Time 2.65s (± 0.15%) 2.65s (± 0.21%) ~ 2.64s 2.65s p=0.282 n=6
Bind Time 0.82s (± 0.00%) 0.83s (± 0.66%) +0.02s (+ 1.83%) 0.83s 0.84s p=0.002 n=6
Check Time 8.16s (± 0.30%) 8.14s (± 0.21%) ~ 8.11s 8.16s p=0.195 n=6
Emit Time 7.11s (± 0.26%) 7.11s (± 0.30%) ~ 7.09s 7.14s p=0.934 n=6
Total Time 18.74s (± 0.22%) 18.73s (± 0.17%) ~ 18.68s 18.77s p=0.936 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 191,964k (± 0.55%) 192,937k (± 1.25%) ~ 191,520k 197,397k p=0.748 n=6
Parse Time 1.37s (± 1.42%) 1.36s (± 0.55%) ~ 1.35s 1.37s p=0.454 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.35s (± 0.35%) 9.33s (± 0.28%) ~ 9.30s 9.37s p=0.295 n=6
Emit Time 2.62s (± 0.92%) 2.63s (± 0.51%) ~ 2.61s 2.64s p=0.677 n=6
Total Time 14.06s (± 0.32%) 14.04s (± 0.17%) ~ 14.00s 14.07s p=0.334 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,423k (± 0.00%) 347,427k (± 0.00%) ~ 347,414k 347,440k p=0.689 n=6
Parse Time 2.46s (± 0.63%) 2.46s (± 0.56%) ~ 2.44s 2.48s p=0.459 n=6
Bind Time 0.92s (± 0.56%) 0.92s (± 0.59%) ~ 0.92s 0.93s p=0.640 n=6
Check Time 6.87s (± 0.44%) 6.89s (± 0.18%) +0.03s (+ 0.39%) 6.87s 6.90s p=0.038 n=6
Emit Time 4.05s (± 0.34%) 4.05s (± 0.54%) ~ 4.01s 4.07s p=0.514 n=6
Total Time 14.30s (± 0.22%) 14.33s (± 0.25%) ~ 14.27s 14.37s p=0.102 n=6
TFS - node (v18.15.0, x64)
Memory used 302,767k (± 0.00%) 302,776k (± 0.01%) ~ 302,750k 302,800k p=0.378 n=6
Parse Time 2.01s (± 1.03%) 2.02s (± 1.12%) ~ 1.99s 2.04s p=0.625 n=6
Bind Time 1.00s (± 1.03%) 1.00s (± 1.03%) ~ 0.99s 1.02s p=1.000 n=6
Check Time 6.29s (± 0.35%) 6.30s (± 0.13%) ~ 6.29s 6.31s p=0.411 n=6
Emit Time 3.58s (± 0.29%) 3.58s (± 0.50%) ~ 3.56s 3.61s p=1.000 n=6
Total Time 12.88s (± 0.21%) 12.90s (± 0.26%) ~ 12.87s 12.96s p=0.418 n=6
material-ui - node (v18.15.0, x64)
Memory used 508,264k (± 0.00%) 508,270k (± 0.01%) ~ 508,238k 508,309k p=0.748 n=6
Parse Time 2.60s (± 0.40%) 2.58s (± 0.32%) ~ 2.58s 2.60s p=0.075 n=6
Bind Time 0.99s (± 0.76%) 0.99s (± 1.18%) ~ 0.98s 1.01s p=0.734 n=6
Check Time 17.13s (± 0.14%) 17.12s (± 0.58%) ~ 16.99s 17.21s p=0.747 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.72s (± 0.16%) 20.69s (± 0.40%) ~ 20.59s 20.77s p=1.000 n=6
xstate - node (v18.15.0, x64)
Memory used 513,012k (± 0.02%) 513,024k (± 0.01%) ~ 512,960k 513,113k p=1.000 n=6
Parse Time 3.28s (± 0.37%) 3.29s (± 0.23%) +0.02s (+ 0.51%) 3.28s 3.30s p=0.024 n=6
Bind Time 1.54s (± 0.53%) 1.53s (± 0.34%) ~ 1.53s 1.54s p=0.523 n=6
Check Time 2.84s (± 0.52%) 2.84s (± 0.94%) ~ 2.80s 2.88s p=0.936 n=6
Emit Time 0.07s (± 7.03%) 0.07s (± 0.00%) ~ 0.07s 0.07s p=0.174 n=6
Total Time 7.72s (± 0.29%) 7.74s (± 0.33%) ~ 7.70s 7.77s p=0.331 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,344ms (± 0.46%) 2,341ms (± 0.56%) ~ 2,329ms 2,365ms p=0.872 n=6
Req 2 - geterr 5,488ms (± 1.91%) 5,463ms (± 1.68%) ~ 5,395ms 5,584ms p=0.936 n=6
Req 3 - references 328ms (± 1.23%) 323ms (± 0.41%) -5ms (- 1.42%) 321ms 324ms p=0.033 n=6
Req 4 - navto 274ms (± 1.34%) 275ms (± 1.23%) ~ 270ms 278ms p=0.934 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 87ms (± 7.99%) 88ms (± 7.39%) ~ 79ms 95ms p=0.459 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,477ms (± 0.61%) 2,496ms (± 1.21%) ~ 2,465ms 2,540ms p=0.378 n=6
Req 2 - geterr 4,157ms (± 1.96%) 4,165ms (± 1.96%) ~ 4,076ms 4,250ms p=0.689 n=6
Req 3 - references 337ms (± 1.40%) 338ms (± 1.41%) ~ 332ms 342ms p=0.935 n=6
Req 4 - navto 285ms (± 0.95%) 286ms (± 1.10%) ~ 283ms 292ms p=0.625 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 86ms (± 6.88%) 84ms (± 8.30%) ~ 77ms 90ms p=0.933 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,603ms (± 0.43%) 2,614ms (± 0.34%) ~ 2,603ms 2,622ms p=0.228 n=6
Req 2 - geterr 1,714ms (± 2.18%) 1,718ms (± 2.30%) ~ 1,676ms 1,754ms p=0.575 n=6
Req 3 - references 123ms (± 6.67%) 116ms (± 9.45%) ~ 105ms 127ms p=0.560 n=6
Req 4 - navto 365ms (± 0.91%) 365ms (± 0.82%) ~ 359ms 368ms p=0.678 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 307ms (± 1.06%) 309ms (± 1.25%) ~ 302ms 313ms p=0.464 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 154.76ms (± 0.21%) 154.50ms (± 0.18%) -0.26ms (- 0.17%) 153.45ms 157.88ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 229.40ms (± 0.15%) 229.35ms (± 0.18%) -0.05ms (- 0.02%) 227.82ms 233.42ms p=0.010 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 230.98ms (± 0.19%) 231.02ms (± 0.19%) ~ 229.28ms 236.77ms p=0.191 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 230.88ms (± 0.18%) 231.01ms (± 0.21%) +0.13ms (+ 0.06%) 229.23ms 237.59ms p=0.002 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

Hey @jakebailey, the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/57008/merge:

Something interesting changed - please have a look.

Details

chakra-ui/chakra-ui

4 of 28 projects failed to build with the old tsc and were ignored

packages/components/tsconfig.build.json

  • error TS5056: Cannot write file '/mnt/ts_downloads/chakra-ui/packages/components/dist/types/menu/menu.stories.d.ts' because it would be overwritten by multiple input files.
    • Project Scope

sandersn
sandersn approved these changes Jan 11, 2024
View reviewed changes
Copy link
Member

@sandersn sandersn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now do cryptocurrency .ts files.

@jakebailey
Copy link
Member Author

jakebailey commented Jan 11, 2024 •
edited
Loading

One thing I'm wondering is if we should just search the first 80 characters or something and say it's binary if any of them are replacementCharacter. Theoretically any garbage could be in a ts file and break things, and it's not clear to me that replacementCharacter would ever legally appear in any reasonable file.

I'm trying to find how other tools detect that things are binary (e.g. vim and ripgrep can both tell).

@fatcerberus
Copy link

fatcerberus commented Jan 11, 2024 •
edited
Loading

One thing I'm wondering is if we should just search the first 80 characters or something and say it's binary if any of them are replacementCharacter. Theoretically any garbage could be in a ts file and break things, and it's not clear to me that replacementCharacter would ever legally appear in any reasonable file.

That's what I (indirectly) suggested here: #56516 (comment)

I'm pretty sure the "first 80 characters" method is the way it's typically done (e.g. in text editors, to determine if they should open a hex view). The Windows API even has a function for it IIRC.

@fatcerberus
Copy link

Now do cryptocurrency .ts files.

NFT types when

@jakebailey
Copy link
Member Author

That or I could just check for /\ufffd/. A replacement character won't happen in good files, right?

@fatcerberus
Copy link

Yeah, check the first 80 chars and if any is a replacement char, treat as binary

DanielRosenwasser
DanielRosenwasser reviewed Jan 12, 2024
View reviewed changes
src/compiler/scanner.ts Outdated
@@ -1799,7 +1799,21 @@ export function createScanner(languageVersion: ScriptTarget, skipTrivia: boolean
// position 0 because UTF-8 decode will fail and produce U+FFFD.
// If that happens, just issue one error and refuse to try to scan further;
// this is likely a binary file that cannot be parsed
if (ch === CharacterCodes.replacementCharacter) {
let isBinary = ch === CharacterCodes.replacementCharacter;
// See if this is an MPEG Transport Stream, where every 188th byte is "G" and the rest is garbage.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is an MPEG transport stream guaranteed to be a multiple of 188 bytes long?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, unless it's malformed. But, I actually think I should dump this PR and instead just check the first 80 or 256 characters for replacementCharacter instead. Curious what people think.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably fine, but my concern is really more the case that at any arbitrary position, when scan is called, you will look ahead 188 characters.

If you had the byte position and the raw content length, you could avoid doing any of this, but because we're operating on something parsed into UTF-16, I don't think you have that option.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only happens at pos == 0.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missed that, thanks!

@typescript-bot
Copy link
Collaborator

Heya @jakebailey, I'm starting to run the public perf test suite on this PR at 4ee2758. Hold tight - I'll update this comment with the log link once the build has been queued.

@jakebailey
Copy link
Member Author

Bot is a liar

@typescript-bot perf test this
@typescript-bot perf test public

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 16, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the public perf test suite on this PR at 4ee2758. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 16, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at 4ee2758. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
mui-docs - node (v20.5.1, x64)
Memory used 1,730,058k (± 0.01%) 1,730,098k (± 0.00%) ~ 1,730,073k 1,730,117k p=1.000 n=6
Parse Time 6.79s (± 0.43%) 6.83s (± 0.57%) ~ 6.80s 6.91s p=0.052 n=6
Bind Time 2.40s (± 0.90%) 2.39s (± 1.54%) ~ 2.33s 2.44s p=0.936 n=6
Check Time 52.96s (± 3.50%) 52.34s (± 0.55%) ~ 51.97s 52.72s p=0.936 n=6
Emit Time 0.15s (± 3.36%) 0.15s (± 0.00%) ~ 0.15s 0.15s p=0.174 n=6
Total Time 62.30s (± 2.99%) 61.70s (± 0.45%) ~ 61.38s 62.09s p=0.748 n=6
self-build-src - node (v20.5.1, x64)
Memory used 2,685,007k (± 5.14%) 2,718,637k (± 6.26%) ~ 2,577,760k 2,926,055k p=0.689 n=6
Parse Time 5.05s (± 1.41%) 5.06s (± 1.12%) ~ 5.00s 5.15s p=0.810 n=6
Bind Time 1.97s (± 0.69%) 1.98s (± 0.83%) ~ 1.96s 2.00s p=1.000 n=6
Check Time 32.15s (± 0.53%) 32.21s (± 0.22%) ~ 32.12s 32.32s p=0.199 n=6
Emit Time 2.81s (± 2.14%) 2.76s (± 2.20%) ~ 2.67s 2.84s p=0.229 n=6
Total Time 41.98s (± 0.45%) 42.02s (± 0.13%) ~ 41.93s 42.07s p=0.810 n=6
self-compiler - node (v20.5.1, x64)
Memory used 419,813k (± 0.30%) 419,784k (± 0.30%) ~ 419,191k 422,366k p=0.810 n=6
Parse Time 2.88s (± 0.36%) 2.92s (± 1.03%) ~ 2.87s 2.95s p=0.105 n=6
Bind Time 1.13s (± 0.92%) 1.13s (± 0.46%) ~ 1.13s 1.14s p=0.794 n=6
Check Time 14.13s (± 0.48%) 14.13s (± 0.35%) ~ 14.06s 14.20s p=1.000 n=6
Emit Time 1.03s (± 1.06%) 1.03s (± 1.32%) ~ 1.01s 1.05s p=0.611 n=6
Total Time 19.18s (± 0.40%) 19.20s (± 0.47%) ~ 19.07s 19.32s p=0.575 n=6
vscode - node (v20.5.1, x64)
Memory used 2,840,879k (± 0.00%) 2,840,899k (± 0.00%) ~ 2,840,839k 2,840,979k p=0.423 n=6
Parse Time 10.78s (± 0.29%) 10.84s (± 0.24%) +0.06s (+ 0.53%) 10.80s 10.87s p=0.024 n=6
Bind Time 3.45s (± 0.50%) 3.45s (± 0.47%) ~ 3.43s 3.47s p=0.623 n=6
Check Time 56.69s (± 0.33%) 56.79s (± 0.30%) ~ 56.52s 56.97s p=0.378 n=6
Emit Time 16.24s (± 0.43%) 16.26s (± 0.55%) ~ 16.09s 16.34s p=0.629 n=6
Total Time 87.17s (± 0.25%) 87.34s (± 0.25%) ~ 87.10s 87.56s p=0.173 n=6
webpack - node (v20.5.1, x64)
Memory used 395,231k (± 0.01%) 395,215k (± 0.00%) ~ 395,182k 395,231k p=0.092 n=6
Parse Time 3.31s (± 0.56%) 3.32s (± 0.61%) ~ 3.30s 3.34s p=0.807 n=6
Bind Time 1.44s (± 0.57%) 1.44s (± 0.68%) ~ 1.43s 1.45s p=0.394 n=6
Check Time 12.96s (± 0.27%) 12.97s (± 0.22%) ~ 12.93s 13.01s p=0.806 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 17.70s (± 0.25%) 17.72s (± 0.26%) ~ 17.66s 17.79s p=0.520 n=6
System info unknown
Hosts
  • node (v20.5.1, x64)
Scenarios
  • mui-docs - node (v20.5.1, x64)
  • self-build-src - node (v20.5.1, x64)
  • self-compiler - node (v20.5.1, x64)
  • vscode - node (v20.5.1, x64)
  • webpack - node (v20.5.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,520k (± 0.01%) 295,498k (± 0.01%) ~ 295,469k 295,519k p=0.104 n=6
Parse Time 2.65s (± 0.15%) 2.67s (± 0.28%) +0.02s (+ 0.88%) 2.66s 2.68s p=0.003 n=6
Bind Time 0.82s (± 0.50%) 0.82s (± 0.00%) ~ 0.82s 0.82s p=0.405 n=6
Check Time 8.15s (± 0.19%) 8.16s (± 0.39%) ~ 8.13s 8.21s p=0.800 n=6
Emit Time 7.08s (± 0.17%) 7.10s (± 0.33%) ~ 7.06s 7.12s p=0.188 n=6
Total Time 18.70s (± 0.16%) 18.75s (± 0.13%) +0.04s (+ 0.23%) 18.72s 18.78s p=0.034 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,513k (± 1.58%) 194,495k (± 1.65%) ~ 191,530k 197,463k p=0.298 n=6
Parse Time 1.35s (± 0.47%) 1.36s (± 1.04%) ~ 1.34s 1.38s p=0.180 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.38s (± 0.35%) 9.37s (± 0.45%) ~ 9.32s 9.42s p=0.810 n=6
Emit Time 2.62s (± 0.52%) 2.63s (± 0.79%) ~ 2.59s 2.65s p=0.459 n=6
Total Time 14.07s (± 0.25%) 14.07s (± 0.49%) ~ 13.97s 14.15s p=0.872 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,382k (± 0.01%) 347,405k (± 0.00%) ~ 347,393k 347,422k p=0.378 n=6
Parse Time 2.47s (± 0.36%) 2.51s (± 0.92%) +0.04s (+ 1.69%) 2.48s 2.55s p=0.007 n=6
Bind Time 0.93s (± 0.56%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=0.174 n=6
Check Time 6.87s (± 0.49%) 6.85s (± 0.41%) ~ 6.82s 6.89s p=0.418 n=6
Emit Time 4.04s (± 0.30%) 4.06s (± 0.37%) ~ 4.04s 4.08s p=0.276 n=6
Total Time 14.31s (± 0.20%) 14.35s (± 0.28%) ~ 14.29s 14.39s p=0.105 n=6
TFS - node (v18.15.0, x64)
Memory used 302,783k (± 0.00%) 302,812k (± 0.01%) +30k (+ 0.01%) 302,773k 302,861k p=0.045 n=6
Parse Time 2.00s (± 1.04%) 2.03s (± 1.75%) ~ 1.98s 2.09s p=0.052 n=6
Bind Time 1.00s (± 1.03%) 1.00s (± 1.03%) ~ 0.99s 1.02s p=1.000 n=6
Check Time 6.29s (± 0.16%) 6.31s (± 0.19%) ~ 6.29s 6.32s p=0.084 n=6
Emit Time 3.58s (± 0.23%) 3.58s (± 0.51%) ~ 3.56s 3.61s p=0.244 n=6
Total Time 12.88s (± 0.18%) 12.92s (± 0.26%) ~ 12.88s 12.96s p=0.089 n=6
material-ui - node (v18.15.0, x64)
Memory used 511,016k (± 0.01%) 511,004k (± 0.00%) ~ 510,966k 511,034k p=0.521 n=6
Parse Time 2.63s (± 0.44%) 2.65s (± 0.78%) ~ 2.63s 2.68s p=0.119 n=6
Bind Time 0.99s (± 0.64%) 0.99s (± 0.76%) ~ 0.98s 1.00s p=0.718 n=6
Check Time 17.09s (± 0.48%) 17.20s (± 0.34%) ~ 17.13s 17.30s p=0.053 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.71s (± 0.38%) 20.84s (± 0.28%) +0.13s (+ 0.63%) 20.77s 20.94s p=0.016 n=6
xstate - node (v18.15.0, x64)
Memory used 513,016k (± 0.01%) 513,010k (± 0.02%) ~ 512,870k 513,136k p=1.000 n=6
Parse Time 3.28s (± 0.51%) 3.31s (± 0.35%) +0.03s (+ 0.86%) 3.29s 3.32s p=0.024 n=6
Bind Time 1.54s (± 0.36%) 1.53s (± 0.34%) ~ 1.53s 1.54s p=0.640 n=6
Check Time 2.83s (± 0.27%) 2.83s (± 0.53%) ~ 2.82s 2.86s p=1.000 n=6
Emit Time 0.08s (± 4.99%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=1.000 n=6
Total Time 7.73s (± 0.25%) 7.75s (± 0.29%) ~ 7.73s 7.79s p=0.103 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,356ms (± 0.89%) 2,379ms (± 0.35%) +24ms (+ 1.01%) 2,372ms 2,395ms p=0.024 n=6
Req 2 - geterr 5,493ms (± 0.99%) 5,477ms (± 1.35%) ~ 5,431ms 5,625ms p=0.173 n=6
Req 3 - references 324ms (± 1.36%) 331ms (± 1.30%) +8ms (+ 2.37%) 325ms 335ms p=0.019 n=6
Req 4 - navto 276ms (± 1.14%) 275ms (± 0.30%) ~ 274ms 276ms p=0.120 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 84ms (± 6.10%) 81ms (± 7.02%) ~ 79ms 93ms p=0.206 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,484ms (± 0.53%) 2,503ms (± 0.77%) ~ 2,479ms 2,534ms p=0.109 n=6
Req 2 - geterr 4,163ms (± 1.70%) 4,136ms (± 1.44%) ~ 4,102ms 4,257ms p=0.470 n=6
Req 3 - references 332ms (± 0.73%) 341ms (± 1.52%) +9ms (+ 2.81%) 331ms 345ms p=0.024 n=6
Req 4 - navto 284ms (± 0.62%) 285ms (± 0.26%) ~ 284ms 286ms p=0.117 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 75ms (± 3.26%) 88ms (± 7.03%) 🔻+13ms (+16.67%) 75ms 91ms p=0.012 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,612ms (± 0.39%) 2,625ms (± 0.29%) +14ms (+ 0.52%) 2,617ms 2,634ms p=0.016 n=6
Req 2 - geterr 1,730ms (± 2.55%) 1,724ms (± 2.59%) ~ 1,653ms 1,765ms p=0.689 n=6
Req 3 - references 109ms (± 7.50%) 109ms (± 7.58%) ~ 105ms 126ms p=0.868 n=6
Req 4 - navto 365ms (± 0.17%) 364ms (± 1.12%) ~ 356ms 368ms p=0.720 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 308ms (± 2.08%) 314ms (± 4.97%) ~ 303ms 345ms p=1.000 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 154.31ms (± 0.19%) 154.25ms (± 0.21%) -0.06ms (- 0.04%) 153.09ms 158.71ms p=0.006 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 229.77ms (± 0.16%) 229.46ms (± 0.19%) -0.31ms (- 0.13%) 228.16ms 236.71ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 231.11ms (± 0.17%) 231.17ms (± 0.18%) ~ 229.61ms 237.12ms p=0.585 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 231.18ms (± 0.21%) 230.93ms (± 0.17%) -0.25ms (- 0.11%) 229.25ms 234.51ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

@typescript-bot perf test public
@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 16, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the tarball bundle task on this PR at eb2dcc7. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 16, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the public perf test suite on this PR at eb2dcc7. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 16, 2024 •
edited
Loading

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/159525/artifacts?artifactName=tgz&fileId=BC4AFE7CDEBD38C65F350E410F48C9F99C20778EDBCB53D18A5AB4D911BA805502&fileName=/typescript-5.4.0-insiders.20240116.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
mui-docs - node (v20.5.1, x64)
Memory used 1,730,278k (± 0.00%) 1,730,277k (± 0.00%) ~ 1,730,260k 1,730,298k p=0.936 n=6
Parse Time 6.80s (± 0.24%) 6.79s (± 0.33%) ~ 6.77s 6.83s p=0.512 n=6
Bind Time 2.35s (± 6.38%) 2.39s (± 1.58%) ~ 2.31s 2.41s p=0.519 n=6
Check Time 52.39s (± 0.72%) 52.28s (± 0.47%) ~ 52.09s 52.71s p=0.575 n=6
Emit Time 0.15s (± 0.00%) 0.15s (± 3.36%) ~ 0.15s 0.16s p=0.174 n=6
Total Time 61.68s (± 0.80%) 61.61s (± 0.41%) ~ 61.41s 62.08s p=0.810 n=6
self-build-src - node (v20.5.1, x64)
Memory used 2,635,874k (± 5.38%) 2,660,339k (± 5.36%) ~ 2,576,755k 2,925,229k p=0.810 n=6
Parse Time 5.02s (± 0.70%) 5.03s (± 0.90%) ~ 4.98s 5.09s p=0.936 n=6
Bind Time 1.98s (± 0.42%) 1.97s (± 0.41%) ~ 1.96s 1.98s p=0.125 n=6
Check Time 32.26s (± 0.26%) 32.16s (± 0.31%) ~ 32.04s 32.29s p=0.173 n=6
Emit Time 2.79s (± 1.61%) 2.78s (± 1.82%) ~ 2.72s 2.86s p=1.000 n=6
Total Time 42.06s (± 0.22%) 41.93s (± 0.31%) ~ 41.76s 42.08s p=0.199 n=6
self-compiler - node (v20.5.1, x64)
Memory used 419,308k (± 0.02%) 419,248k (± 0.02%) ~ 419,151k 419,339k p=0.092 n=6
Parse Time 2.88s (± 0.79%) 2.90s (± 0.36%) ~ 2.88s 2.91s p=0.217 n=6
Bind Time 1.13s (± 0.46%) 1.13s (± 0.74%) ~ 1.12s 1.14s p=0.533 n=6
Check Time 14.14s (± 0.31%) 14.13s (± 0.20%) ~ 14.10s 14.18s p=0.873 n=6
Emit Time 1.03s (± 0.73%) 1.04s (± 1.59%) ~ 1.01s 1.06s p=0.672 n=6
Total Time 19.18s (± 0.23%) 19.21s (± 0.25%) ~ 19.15s 19.28s p=0.630 n=6
vscode - node (v20.5.1, x64)
Memory used 2,841,494k (± 0.00%) 2,841,480k (± 0.00%) ~ 2,841,437k 2,841,569k p=0.810 n=6
Parse Time 10.79s (± 0.46%) 10.77s (± 0.31%) ~ 10.72s 10.82s p=0.624 n=6
Bind Time 3.43s (± 0.66%) 3.42s (± 0.47%) ~ 3.41s 3.45s p=0.565 n=6
Check Time 56.66s (± 0.32%) 56.56s (± 0.27%) ~ 56.35s 56.76s p=0.297 n=6
Emit Time 16.34s (± 0.75%) 16.23s (± 0.27%) -0.10s (- 0.63%) 16.20s 16.32s p=0.030 n=6
Total Time 87.22s (± 0.27%) 86.99s (± 0.20%) ~ 86.80s 87.24s p=0.127 n=6
webpack - node (v20.5.1, x64)
Memory used 395,415k (± 0.01%) 395,420k (± 0.02%) ~ 395,337k 395,513k p=0.936 n=6
Parse Time 3.31s (± 0.53%) 3.31s (± 0.40%) ~ 3.30s 3.33s p=0.737 n=6
Bind Time 1.44s (± 0.81%) 1.44s (± 0.57%) ~ 1.43s 1.45s p=0.498 n=6
Check Time 12.95s (± 0.19%) 12.94s (± 0.13%) ~ 12.92s 12.96s p=0.290 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 17.70s (± 0.16%) 17.69s (± 0.15%) ~ 17.65s 17.73s p=0.747 n=6
System info unknown
Hosts
  • node (v20.5.1, x64)
Scenarios
  • mui-docs - node (v20.5.1, x64)
  • self-build-src - node (v20.5.1, x64)
  • self-compiler - node (v20.5.1, x64)
  • vscode - node (v20.5.1, x64)
  • webpack - node (v20.5.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Oops, wrong comment

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 16, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at eb2dcc7. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,621k (± 0.01%) 295,627k (± 0.01%) ~ 295,608k 295,661k p=0.689 n=6
Parse Time 2.66s (± 0.31%) 2.65s (± 0.00%) ~ 2.65s 2.65s p=0.121 n=6
Bind Time 0.83s (± 0.49%) 0.83s (± 0.49%) ~ 0.82s 0.83s p=1.000 n=6
Check Time 8.16s (± 0.50%) 8.16s (± 0.28%) ~ 8.14s 8.19s p=0.809 n=6
Emit Time 7.08s (± 0.27%) 7.07s (± 0.19%) ~ 7.05s 7.08s p=0.285 n=6
Total Time 18.72s (± 0.21%) 18.71s (± 0.15%) ~ 18.68s 18.76s p=0.418 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 191,511k (± 0.01%) 194,445k (± 1.66%) ~ 191,477k 197,470k p=0.298 n=6
Parse Time 1.36s (± 0.93%) 1.36s (± 0.55%) ~ 1.35s 1.37s p=1.000 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.39s (± 0.30%) 9.36s (± 0.65%) ~ 9.29s 9.45s p=0.294 n=6
Emit Time 2.62s (± 0.39%) 2.61s (± 0.40%) ~ 2.60s 2.63s p=0.801 n=6
Total Time 14.09s (± 0.18%) 14.05s (± 0.47%) ~ 13.98s 14.15s p=0.422 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,387k (± 0.00%) 347,395k (± 0.01%) ~ 347,354k 347,427k p=0.470 n=6
Parse Time 2.47s (± 0.44%) 2.47s (± 0.54%) ~ 2.45s 2.48s p=0.604 n=6
Bind Time 0.93s (± 0.56%) 0.92s (± 0.59%) ~ 0.92s 0.93s p=0.640 n=6
Check Time 6.88s (± 0.75%) 6.90s (± 0.51%) ~ 6.85s 6.94s p=0.332 n=6
Emit Time 4.05s (± 0.59%) 4.06s (± 0.24%) ~ 4.04s 4.07s p=0.438 n=6
Total Time 14.32s (± 0.40%) 14.35s (± 0.34%) ~ 14.27s 14.40s p=0.294 n=6
TFS - node (v18.15.0, x64)
Memory used 302,797k (± 0.00%) 302,778k (± 0.01%) ~ 302,751k 302,797k p=0.066 n=6
Parse Time 1.99s (± 0.82%) 2.00s (± 0.41%) ~ 1.99s 2.01s p=0.186 n=6
Bind Time 1.01s (± 1.15%) 1.00s (± 1.26%) ~ 0.99s 1.02s p=0.140 n=6
Check Time 6.31s (± 0.55%) 6.30s (± 0.14%) ~ 6.29s 6.31s p=0.934 n=6
Emit Time 3.57s (± 0.29%) 3.58s (± 0.34%) ~ 3.57s 3.60s p=0.180 n=6
Total Time 12.89s (± 0.37%) 12.89s (± 0.13%) ~ 12.87s 12.91s p=0.255 n=6
material-ui - node (v18.15.0, x64)
Memory used 511,185k (± 0.01%) 511,173k (± 0.01%) ~ 511,120k 511,210k p=0.630 n=6
Parse Time 2.64s (± 0.62%) 2.64s (± 0.71%) ~ 2.60s 2.65s p=0.732 n=6
Bind Time 1.00s (± 1.04%) 1.00s (± 1.04%) ~ 0.98s 1.01s p=1.000 n=6
Check Time 17.15s (± 0.29%) 17.10s (± 0.28%) ~ 17.02s 17.15s p=0.090 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.79s (± 0.25%) 20.73s (± 0.21%) -0.06s (- 0.28%) 20.66s 20.77s p=0.037 n=6
xstate - node (v18.15.0, x64)
Memory used 513,392k (± 0.01%) 513,407k (± 0.01%) ~ 513,357k 513,495k p=0.471 n=6
Parse Time 3.28s (± 0.25%) 3.29s (± 0.27%) +0.01s (+ 0.41%) 3.28s 3.30s p=0.038 n=6
Bind Time 1.54s (± 0.41%) 1.54s (± 0.49%) ~ 1.53s 1.55s p=0.718 n=6
Check Time 2.84s (± 0.61%) 2.85s (± 0.69%) ~ 2.83s 2.88s p=0.933 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 7.74s (± 0.19%) 7.75s (± 0.30%) ~ 7.71s 7.78s p=0.332 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,348ms (± 0.46%) 2,341ms (± 0.65%) ~ 2,327ms 2,364ms p=0.471 n=6
Req 2 - geterr 5,479ms (± 1.11%) 5,526ms (± 1.51%) ~ 5,427ms 5,614ms p=0.336 n=6
Req 3 - references 323ms (± 0.95%) 323ms (± 0.53%) ~ 320ms 325ms p=1.000 n=6
Req 4 - navto 276ms (± 1.01%) 274ms (± 1.41%) ~ 270ms 278ms p=0.287 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 84ms (± 6.12%) 88ms (± 7.34%) ~ 79ms 94ms p=0.325 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,464ms (± 0.97%) 2,461ms (± 0.73%) ~ 2,432ms 2,482ms p=0.872 n=6
Req 2 - geterr 4,216ms (± 1.78%) 4,139ms (± 1.47%) ~ 4,092ms 4,260ms p=0.172 n=6
Req 3 - references 337ms (± 1.52%) 336ms (± 1.72%) ~ 328ms 343ms p=0.872 n=6
Req 4 - navto 285ms (± 1.12%) 285ms (± 0.42%) ~ 283ms 286ms p=1.000 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 83ms (± 6.81%) 82ms (±10.31%) ~ 74ms 90ms p=0.745 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,607ms (± 0.92%) 2,609ms (± 0.60%) ~ 2,580ms 2,625ms p=0.810 n=6
Req 2 - geterr 1,722ms (± 1.36%) 1,724ms (± 1.80%) ~ 1,661ms 1,743ms p=0.257 n=6
Req 3 - references 114ms (± 8.64%) 112ms (± 9.04%) ~ 104ms 125ms p=0.329 n=6
Req 4 - navto 367ms (± 1.28%) 365ms (± 0.22%) ~ 364ms 366ms p=0.412 n=6
Req 5 - completionInfo count 2,078 (± 0.00%) 2,078 (± 0.00%) ~ 2,078 2,078 p=1.000 n=6
Req 5 - completionInfo 310ms (± 1.07%) 312ms (± 1.79%) ~ 301ms 316ms p=0.170 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 153.75ms (± 0.21%) 153.70ms (± 0.20%) ~ 152.59ms 157.61ms p=0.172 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 229.78ms (± 0.17%) 229.54ms (± 0.15%) -0.23ms (- 0.10%) 228.29ms 233.27ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 231.18ms (± 0.20%) 231.13ms (± 0.18%) ~ 229.50ms 234.86ms p=0.784 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 232.15ms (± 0.20%) 232.18ms (± 0.19%) ~ 230.50ms 238.81ms p=0.228 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot typescript-bot added For Backlog Bug PRs that fix a backlog bug and removed For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jan 17, 2024
@jakebailey jakebailey changed the title Detect binary files by checking first 256 code points for 0xFFFD Detect binary files by checking first 256 code units for 0xFFFD Jan 17, 2024
@jakebailey jakebailey merged commit ad2f465 into microsoft:main Jan 17, 2024
19 checks passed
@jakebailey jakebailey deleted the fix-56516 branch January 17, 2024 00:15
@RyanCavanaugh RyanCavanaugh mentioned this pull request Feb 20, 2024
Closed
@erik-krogh erik-krogh mentioned this pull request Mar 12, 2024
Merged
@wooorm
Copy link

wooorm commented Mar 25, 2024

@jakebailey this seems to affect JavaScript files that include a replacement character. Example: https://github.com/micromark/micromark/blob/8b08d16891f4d6801eb0c0e990bd7b87836c0826/test/io/misc/zero.js#L9.

Is this intentional? Or a bug? Should I create an issue?

Cross-reference micromark/micromark#166

@wooorm wooorm mentioned this pull request Mar 25, 2024
Closed
5 tasks
@jakebailey
Copy link
Member Author

Ugh, not super thrilled that people are quite literally putting the replacement character into their files instead of escaping it; that's going to be indistinguishable from actually broken files.

I'm not sure if it's "intentional" because I really did not think people would be doing that. So far the linked issues appear to be cases where it's possible to just use \uFFFD (which is what I expected everyone do be doing).

I guess it could deserve a dedicated issue rather than discussing on the PR. Not doing this means we have to come up with some other, new way to handle this :(

@wooorm
Copy link

wooorm commented Mar 25, 2024 •
edited
Loading

it's possible to just use \uFFFD (which is what I expected everyone do be doing).

Well, you can always use an escape for all unicode characters.
But like. I have to test the handling of markdown. Which says that some unicode characters result in another unicode character meant for “cannot display this nicely”.
So I didn’t expect linters or TS or so to not like them.

Many algos in the HTML spec also result in REPLACEMENT CHARACTER. I don’t expect most people to write them in files but there are APIs that generate them, so there will be test files.

In my experience with GH internals, they have some heuristic with control characters and lone surrogates to infer whether something is binary. Could that be viable?

@wooorm wooorm mentioned this pull request Mar 25, 2024
Closed
@RyanCavanaugh
Copy link
Member

I don't want to make TS slower for every other file in the world, or forego doing other more impactful work, for the sake of a preference in the small number of scenarios where people need to write text encoding testcases.

We struggle with this too -- people are always asking us to normalize \r\n in our repo, etc, but we can't do that because TS needs to be able to successfully parse both \r\n and \n in files. At the point you're working with the byte structure of text instead of the semantic interpretation of it, things are going to be slightly less convenient compared to all other text content. Them's the breaks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sheetalkamat sheetalkamat sheetalkamat approved these changes

@RyanCavanaugh RyanCavanaugh RyanCavanaugh approved these changes

@sandersn sandersn Awaiting requested review from sandersn

@DanielRosenwasser DanielRosenwasser Awaiting requested review from DanielRosenwasser

Assignees

@jakebailey jakebailey

Labels
Author: Team For Backlog Bug PRs that fix a backlog bug
Projects
None yet
Milestone
No milestone
8 participants
@jakebailey @typescript-bot @fatcerberus @DanielRosenwasser @wooorm @RyanCavanaugh @sandersn @sheetalkamat