Build the frontend by esbuild instead of webpack #1063

baurine · 2021-11-16T10:29:22Z

What did:

Build the whole frontend by esbuild instead of webpack, and keep the functions and style exact same as before.

Details:

Upgrade antd from 4.8 to 4.16 to eliminate tons of compilation errors
Upgarde dayjs to dismiss the type check errors
Hack the esbuild-plugin-postcss2
- Fix the bug that can't find .scss files when compiling fluent-ui
- Fix the bug that can't find bulma/dist/bulma.css
- Add the cache to reduce incremental build time from more than 4s to near 600ms
- Log the process time
Fix broken styles
Fix broken BaseSelect component
Fix runtime crash: sentry, formatSQL, path, process.cwd
Support diagnoseReport proxy
Remove webpack-only methods to handle i18n
...

Result:

Speedup the compilation time nearly by 9x. In a high-performance desktop machine (CPU: Intel Core i5-9400F @ 6x 4.1GHz, Memory: 64GB), the full build costs near 70s by webpack while only near 8s by esbuild, and only near 5s to build js and css.

In the dev mode, after changing js or CSS code, esbuild needs near 600ms to make an incremental build.

	production	dev incremental
webpack	70.68s	fast by hrm
esbuild	7.74s	600ms

Full build time with webpack (1.13 min, aka 67.8s for webpack:build stage):

[09:57:43] Finished 'swagger:generate_client' after 1.51 s
[09:57:43] Finished 'swagger:generate' after 1.82 s
[09:57:43] Starting 'webpack:build'...
[09:58:51] Finished 'webpack:build' after 1.13 min
[09:58:51] Finished 'build' after 1.15 min
Done in 70.68s.

Full build time with esbuild (5.26s for esbuild:build stage):

[22:19:27] Finished 'swagger:generate_client' after 1.58 s
[22:19:27] Finished 'swagger:generate' after 1.9 s
[22:19:27] Starting 'esbuild:build'...
Build started
...
Build ended: 4795ms
[22:19:32] Finished 'esbuild:build' after 5.26 s
[22:19:32] Finished 'build' after 7.16 s
Done in 7.74s.

In dev mode, esbuild cost near 600ms to make incremental build:

Build ended: 4633ms
Serving "build" at http://127.0.0.1:3001
Ready for changes
...
CSS change detected build/dashboard.css
Change detected build/diagnoseReport.css.map
Build ended: 598ms

CI

In CI, the Build UI job cost time is reduced from 2m 8s to 19s.

Demo

Kapture.2021-12-02.at.22.18.40.mp4

Kapture.2021-12-02.at.22.13.52.mp4

Test

Tab 1:

$ tiup playground nightly

You need to install tiup first.

Tab 2:

$ make dev
$ make run

Tab 3:

$ cd ui
$ yarn
$ yarn start

Feedback to upstream

issues or PR reported to upstream:

- init by `npx esbuild-create-react-app ui-esbuild`

ti-chi-bot · 2021-11-16T10:29:23Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

breeswish

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

baurine · 2021-11-16T10:30:59Z

step 1: compile less/module less/scss
step 2: compile antd with customized CSS

ui-esbuild/builder.js

baurine · 2021-11-16T23:56:17Z

step3: compile fluentui (need modify esbuild-plugin-postcss2 code)

breezewish · 2022-01-20T15:22:06Z

Hope we can get this merged before the Spring festival! It would be a giant step.

baurine · 2022-01-21T11:00:43Z

ui/dashboardApp/layout/main/Sider/index.tsx

@@ -129,6 +132,9 @@ function Sider({
    // NOTE: Don't remove above comment line, it is a placeholder for code generator
    debugSubMenu,
  ]
+  if (topSQLSupport) {
+    menuItems.splice(2, 0, topSQLMenu)


Insert topSQL menu.

baurine · 2022-01-21T11:10:12Z

hi @YiniXu9506 , can you help have a look why the e2e test always fails, thanks!

YiniXu9506 · 2022-01-24T01:53:21Z

hi @YiniXu9506 , can you help have a look why the e2e test always fails, thanks!

Sure!

baurine · 2022-01-24T09:26:26Z

All CI jobs are passed, PTAL, @breeswish , thanks!

breezewish · 2022-01-24T10:40:47Z

.github/workflows/test.yaml

-          fail_ci_if_error: true
-          flags: e2e_test
-          verbose: true
+      # - name: Upload coverage to Codecov


What's wrong with this block?

/cc @YiniXu9506

The introduced esbuild causes the plugin in .babelrc does not work. I am working on this.

pkg/apiserver/profiling/model.go

pkg/apiserver/profiling/service.go

ui/.env.production

breezewish · 2022-01-24T10:42:54Z

ui/dashboardApp/layout/main/Sider/index.module.less

@@ -19,6 +19,10 @@
      display: flex;
      flex-direction: column;
      padding-bottom: 20px;
+
+      .ant-menu.ant-menu-inline-collapsed {
+        width: @menu-collapsed-width; // 80px


Is this an ant-design bug? If so, maybe you'd better propose a minimal reproducible example and file an issue.

breezewish · 2022-01-24T11:15:09Z

ui/lib/utils/sqlFormatter/index.ts

+//   return mySqlFormatter.format(sql || '')
+// }
+
+import { format } from 'sql-formatter'


What's the impact of not using a TiDB SQL formatter? I believe something may get wrong.

It will crash when using the forked formatter, and the upstream still keep updated.

But as we are specially introducing a TiDB formatter, will there be any feature breaks when it is not used?

I did some manual tests, it looks the same as before, but I'm not 100% confident, I think we should add some unit tests for this method, I will book a todo.

breezewish · 2022-01-24T14:29:16Z

ui/lib/apps/TopSQL/pages/List/List.tsx

  ])

  // fetch data when instance id / time window size update
  useEffect(() => {
    handleUpdateTopSQLData()
+    // eslint-disable-next-line react-hooks/exhaustive-deps


Do we have some safer way to do this magic? I mean, something that will not be reported by the lint while simple enough. I'm not an expert of React hook. How does other React hook users deal with this kind of case? @shhdgit

OK, this is not a good scenario for the use of side effects. According to this article: https://stackoverflow.com/questions/58866796/understanding-the-react-hooks-exhaustive-deps-lint-rule

In the similar case, I should call the function directly when the event is triggered. Oh my react.

ui/lib/apps/KeyViz/components/KeyViz.tsx

ui/lib/apps/KeyViz/heatmap/index.tsx

ui/gulpfile.esm.js

breezewish

The rest LGTM. I think we can fix trivial issues after the PR is merged.

Co-authored-by: Yini Xu <[email protected]> Co-authored-by: Suhaha <[email protected]>

baurine added 12 commits November 15, 2021 16:58

install esbuild

6bf9856

install live-server and chokidar

c4f0493

init ui-esbuild folder

e38de03

- init by `npx esbuild-create-react-app ui-esbuild`

test compile css

95d254e

compile less

4acb3e9

test build prod

7ac21dc

compile module less

c5e60e5

refine build

cc6c615

test less variable

bc8295e

compile scss

b53c605

rename

5623843

compile antd

2f148fc

baurine marked this pull request as draft November 16, 2021 10:29

breezewish reviewed Nov 16, 2021

View reviewed changes

ui-esbuild/builder.js Outdated Show resolved Hide resolved

compile fluentui

c3ac7c5

baurine added 12 commits November 17, 2021 09:04

wip

238ec57

wip

7b4f2a1

test DatePicker

eced827

compile @ant-design/icons

e8863c4

compile svg

d3808ee

refine

450bb89

unify builder.js

40f6167

compile yaml

3fef299

hack postcss2

cb9c4e8

test alias paths

c61ed0d

compile basic dynamic import

588fd61

compile lazy load component

fd9d8d9

baurine added 4 commits January 21, 2022 17:39

config eslint

c4745fd

eslint watch

c567a10

fix lint

ce12e39

Merge remote-tracking branch 'origin/master' into esbuild

8c27ecf

baurine force-pushed the esbuild branch from 439e1dd to 8c27ecf Compare January 21, 2022 10:50

baurine commented Jan 21, 2022

View reviewed changes

YiniXu9506 and others added 5 commits January 24, 2022 12:28

fix: update cy get selector (#5)

0510692

fix(topsql): react-hooks/exhaustive-deps lint (#4)

472b469

fix lint

823d13d

fix broken style

090c16b

update cypress selector and hide code coverage temporary (#6)

aa9f33b

breezewish reviewed Jan 24, 2022

View reviewed changes

ti-chi-bot requested a review from YiniXu9506 January 25, 2022 00:44

baurine added 5 commits January 25, 2022 09:30

do tsc and eslint check for production build

b9cb4b7

remove unused code

7087880

revert unnecessary changes

e33782f

Merge branch 'esbuild' of github.com:baurine/tidb-dashboard into esbuild

f6a26ba

fix production build

fc9e482

breezewish approved these changes Jan 25, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 label Jan 25, 2022

baurine merged commit db6fb66 into pingcap:master Jan 25, 2022

baurine mentioned this pull request Jan 25, 2022

Add unit test for formatSql() method #1157

Closed

shhdgit added a commit to shhdgit/tidb-dashboard that referenced this pull request Mar 16, 2022

chore: build the frontend by esbuild instead of webpack (pingcap#1063)

67ae2e3

Co-authored-by: Yini Xu <[email protected]> Co-authored-by: Suhaha <[email protected]>

shhdgit mentioned this pull request Mar 16, 2022

Release v2022.03.16.1 #1199

Merged

shhdgit added a commit to shhdgit/tidb-dashboard that referenced this pull request Mar 16, 2022

chore: build the frontend by esbuild instead of webpack (pingcap#1063)

6295d25

Co-authored-by: Yini Xu <[email protected]> Co-authored-by: Suhaha <[email protected]>

baurine deleted the esbuild branch April 15, 2022 08:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build the frontend by esbuild instead of webpack #1063

Build the frontend by esbuild instead of webpack #1063

baurine commented Nov 16, 2021 •

edited

Loading

ti-chi-bot commented Nov 16, 2021 •

edited

Loading

baurine commented Nov 16, 2021

baurine commented Nov 16, 2021 •

edited

Loading

breezewish commented Jan 20, 2022

baurine Jan 21, 2022

baurine commented Jan 21, 2022

YiniXu9506 commented Jan 24, 2022

baurine commented Jan 24, 2022

breezewish Jan 24, 2022

baurine Jan 25, 2022

YiniXu9506 Jan 25, 2022

breezewish Jan 24, 2022

breezewish Jan 24, 2022 •

edited

Loading

baurine Jan 25, 2022

breezewish Jan 25, 2022

baurine Jan 25, 2022

breezewish Jan 24, 2022

shhdgit Jan 25, 2022

breezewish left a comment

Build the frontend by esbuild instead of webpack #1063

Build the frontend by esbuild instead of webpack #1063

Conversation

baurine commented Nov 16, 2021 • edited Loading

What did:

Details:

Result:

Demo

Test

Feedback to upstream

ti-chi-bot commented Nov 16, 2021 • edited Loading

baurine commented Nov 16, 2021

baurine commented Nov 16, 2021 • edited Loading

breezewish commented Jan 20, 2022

Choose a reason for hiding this comment

baurine commented Jan 21, 2022

YiniXu9506 commented Jan 24, 2022

baurine commented Jan 24, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

breezewish Jan 24, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

breezewish left a comment

Choose a reason for hiding this comment

baurine commented Nov 16, 2021 •

edited

Loading

ti-chi-bot commented Nov 16, 2021 •

edited

Loading

baurine commented Nov 16, 2021 •

edited

Loading

breezewish Jan 24, 2022 •

edited

Loading