Add TS projects for src/plugins & x-pack/plugins #78440

mshustov · 2020-09-24T15:25:22Z

Summary

Add TS projects for:

kibana_utils
kibana_react
x-pack/licensing

I'm going to document migration path in the following PR.
I'd appreciate if @elastic/kibana-app-arch tested the changes and how they impact IDE responsiveness when working inside kibana_utils & kibana_react plugins.

Results

OSS project

node --max-old-space-size=6144 ./node_modules/.bin/tsc -p tsconfig.json --extendedDiagnostics --noEmit
branch

Files:                         5196
Lines:                       693734
Nodes:                      2281721
Identifiers:                 776729

master

Files:                        5278
Lines:                      713209
Nodes:                     2354541
Identifiers:                801842

x-pack project

node --max-old-space-size=6144 ./node_modules/.bin/tsc -p x-pack/tsconfig.json --extendedDiagnostics --noEmit
branch

Files:                        17378
Lines:                      2021622
Nodes:                      7108433
Identifiers:                2367775

master

Files:                        17403
Lines:                      2024480
Nodes:                      7120542
Identifiers:                2371851

mshustov · 2020-09-24T16:21:05Z

src/dev/typescript/build_refs.ts

-    log.info('Building TypeScript projects refs...');
-    await execa(require.resolve('typescript/bin/tsc'), ['-b', 'tsconfig.refs.json']);
+    log.debug(`Building TypeScript projects refs for ${projectPath}...`);
+    await execa(require.resolve('typescript/bin/tsc'), ['-b', projectPath]);


to avoid hardcoding x-pack/tsconfig.refs.json

mshustov · 2020-09-24T16:29:30Z

src/dev/typescript/run_type_check_cli.ts

@@ -80,7 +80,8 @@ export async function runTypeCheckCli() {
    process.exit();
  }

-  await buildRefs(log);
+  await buildRefs(log, 'tsconfig.refs.json');
+  await buildRefs(log, 'x-pack/tsconfig.refs.json');


I hardcoded the path here since this script is meant to be used in dev only. @elastic/kibana-operations any problems with this?

mshustov · 2020-09-24T16:31:34Z

examples/bfetch_explorer/tsconfig.json

@@ -13,6 +13,7 @@
  ],
  "exclude": [],
  "references": [
-    { "path": "../../src/core/tsconfig.json" }
+    { "path": "../../src/core/tsconfig.json" },
+    { "path": "../../src/plugins/kibana_react/tsconfig.json" },


it's not necessary, as we don't build the project with tsc, but it reduces the number of processed files:
with refs

Files: 884 Lines: 219774 Nodes: 789226 Identifiers: 286286

without refs

Files: 1214 Lines: 271616 Nodes: 928343 Identifiers: 333268

What project did you build for these stats?

There are a lot of other example projects that import kibana_react types, should we add references from their tsconfig.json files too?

It might become hard to maintain all the references if we start having many small projects. Maybe we can set composite: true in ./tsconfig.refs.json and then all other projects that might need these references just contain a single reference to ./tsconfig.refs.json instead of a reference for each project.

I didn't test this, but my understanding is that you can create such hierarchies of projects and TS will still be clever about what files it includes in the build instead of building the whole parent project.

I didn't test this, but my understanding is that you can create such hierarchies of projects and TS will still be clever about what files it includes in the build instead of building the whole parent project.

Tested just to make sure it doesn't work: TS cannot build a DAG since a project references itself and other project dependent on it.

It might become hard to maintain all the references if we start having many small projects.

I declared the reference to kibana_react only for projects with an explicit dependency on kibanaReact plugin in kibana.json. Indeed, kibana_react might me compiled as a transitive dependency (you can run tsc with --traceResolution flag to learn how a file was included). But we will resolve the over-compilation problem when switching all the plugins to TypeScript project refs, so I think it's okay for now.

elasticmachine · 2020-09-24T16:38:09Z

Pinging @elastic/kibana-platform (Team:Platform)

walterra

ML/Transform changes LGTM

Dosant

I pulled and played with kibana_react, didn't notice any issues. Rebuild on changes works, node scrtips/type_check works, IDE is responsive with suggestions and errors. Hard to assess responsiveness improvements during such testing.

mistic

LGTM. Just a suggestion @restrry : I would probably create a buildAllRefs function on src/dev/typescript/build_refs.ts which builds all the refs projects (even if hardcoded) and use it at src/dev/typescript/run_type_check_cli.ts instead

mshustov · 2020-09-29T15:28:37Z

@mistic like in this commit 7c6a40e ?

mshustov · 2020-09-30T09:55:45Z

@elasticmachine merge upstream

kibanamachine · 2020-09-30T11:30:11Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 3e26554

Metrics [docs]

async chunks size

id	value	diff	baseline
`enterpriseSearch`	430.1KB	+29.0B	430.1KB

distributable file count

id	value	diff	baseline
`default`	45806	+21	45785

page load bundle size

id	value	diff	baseline
`upgradeAssistant`	64.8KB	+29.0B	64.8KB

History

💚 Build #77916 succeeded 7c6a40e
💚 Build #77298 succeeded 5b89668
💛 Build #76991 was flaky eaa507f
💔 Build #76984 failed 8d41c8a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

mistic · 2020-09-30T12:10:12Z

@restrry yes, I think it's more manageable like this! We do something similar for kbn/pm projects

* bump query-string version to remove manual type definitions * remove manual type declaration * add kibana_utils tsconfig * add refs to kibana_utils tsconfig * add kibana_utils to the project list * add kibana_react project * add support for x-pack/tsconfig.refs.json * add ts project for x-pack licensing plugins * add glob for ts projects in src/plugins & x-pack/plugins * add refs to projects in examples * fix ref paths in x-pack/test * address mistic comments Co-authored-by: Elastic Machine <[email protected]> # Conflicts: # src/plugins/kibana_utils/kibana.json

mshustov added 12 commits September 22, 2020 15:25

bump query-string version to remove manual type definitions

7463e15

Merge branch 'master' into bump-query-string

4ab436d

remove manual type declaration

e187e76

add kibana_utils tsconfig

23bb02f

add refs to kibana_utils tsconfig

9be6fe2

add kibana_utils to the project list

7c9398b

add kibana_react project

7d4a0bf

add support for x-pack/tsconfig.refs.json

40d95fd

add ts project for x-pack licensing plugins

b0d92f4

add glob for ts projects in src/plugins & x-pack/plugins

9df9ddf

add refs to projects in examples

014c4b4

Merge branch 'master' into plugins-ts-projects

8d41c8a

mshustov added chore Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.10.0 labels Sep 24, 2020

mshustov mentioned this pull request Sep 24, 2020

Switch to typescript project references and incremental builds #46773

Closed

7 tasks

fix ref paths in x-pack/test

eaa507f

mshustov commented Sep 24, 2020

View reviewed changes

mshustov marked this pull request as ready for review September 24, 2020 16:38

mshustov requested review from a team as code owners September 24, 2020 16:38

walterra approved these changes Sep 28, 2020

View reviewed changes

Merge branch 'master' into plugins-ts-projects

5b89668

rudolf approved these changes Sep 28, 2020

View reviewed changes

Dosant approved these changes Sep 28, 2020

View reviewed changes

mistic approved these changes Sep 29, 2020

View reviewed changes

address mistic comments

7c6a40e

Merge branch 'master' into plugins-ts-projects

3e26554

mshustov merged commit ae8f8e1 into elastic:master Sep 30, 2020

mshustov deleted the plugins-ts-projects branch September 30, 2020 13:02

mshustov mentioned this pull request Sep 30, 2020

[7.x] Add TS projects for src/plugins & x-pack/plugins (#78440) #78946

Merged

mshustov mentioned this pull request Oct 5, 2020

build all ts refs in a single kbn:bootstrap #79438

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add TS projects for src/plugins & x-pack/plugins #78440

Add TS projects for src/plugins & x-pack/plugins #78440

mshustov commented Sep 24, 2020 •

edited

Loading

mshustov Sep 24, 2020 •

edited

Loading

mshustov Sep 24, 2020

mshustov Sep 24, 2020 •

edited

Loading

rudolf Sep 25, 2020

rudolf Sep 25, 2020

mshustov Sep 28, 2020

elasticmachine commented Sep 24, 2020

walterra left a comment

Dosant left a comment

mistic left a comment

mshustov commented Sep 29, 2020

mshustov commented Sep 30, 2020

kibanamachine commented Sep 30, 2020

mistic commented Sep 30, 2020 •

edited

Loading

Add TS projects for src/plugins & x-pack/plugins #78440

Add TS projects for src/plugins & x-pack/plugins #78440

Conversation

mshustov commented Sep 24, 2020 • edited Loading

Summary

Results

OSS project

x-pack project

mshustov Sep 24, 2020 • edited Loading

Choose a reason for hiding this comment

mshustov Sep 24, 2020

Choose a reason for hiding this comment

mshustov Sep 24, 2020 • edited Loading

Choose a reason for hiding this comment

rudolf Sep 25, 2020

Choose a reason for hiding this comment

rudolf Sep 25, 2020

Choose a reason for hiding this comment

mshustov Sep 28, 2020

Choose a reason for hiding this comment

elasticmachine commented Sep 24, 2020

walterra left a comment

Choose a reason for hiding this comment

Dosant left a comment

Choose a reason for hiding this comment

mistic left a comment

Choose a reason for hiding this comment

mshustov commented Sep 29, 2020

mshustov commented Sep 30, 2020

kibanamachine commented Sep 30, 2020

💚 Build Succeeded

Metrics [docs]

async chunks size

distributable file count

page load bundle size

History

mistic commented Sep 30, 2020 • edited Loading

mshustov commented Sep 24, 2020 •

edited

Loading

mshustov Sep 24, 2020 •

edited

Loading

mshustov Sep 24, 2020 •

edited

Loading

mistic commented Sep 30, 2020 •

edited

Loading