Docker for development and preview of DApps in container environment #848
Conversation
ltfschoen
requested review from
AhmedKorim,
AtelyPham and
devpavan04
as code owners
ltfschoen
changed the title
|
Thanks for the contribution @ltfschoen ! |
There was a problem hiding this comment.
Can you add a .env.example file? Something like this in the relayer.
Thanks for your contribution @ltfschoen.
i've added .env.example here. incase the .env file isn't populated by the user the variable values in .env.example are used instead here |
|
@AtelyPham in addition to what's in this PR i tried to run webb-ui-components from the Docker container and have it exposed on port 4400 using this code ltfschoen@8e989b5, but even though it compiled successfully i wasn't able to access anything in a browser, and even when i ran |
| WORKDIR ${PATH_PROJECT} | ||
|
|
||
| COPY package.json . | ||
|
|
There was a problem hiding this comment.
Consider adding this line to install to improve the time to install the npm packages.
COPY yarn.lock .There was a problem hiding this comment.
i've pushed commit adding this ae65b67
without yarn.lock copied it took time ./docker-dev.sh duration real 11m6.119s
then with yarn.lock copied it took time ./docker-dev.sh duration real 5m14.703s
although it now shows this when building
[1/4] Resolving packages...
warning Lockfile has incorrect entry for "@polkadot/[email protected]". Ignoring it.
warning Lockfile has incorrect entry for "@polkadot/[email protected]". Ignoring it.
warning Lockfile has incorrect entry for "@polkadot/[email protected]". Ignoring it.
warning Lockfile has incorrect entry for "@polkadot/[email protected]". Ignoring it.
warning Lockfile has incorrect entry for "@polkadot/[email protected]". Ignoring it.
warning Lockfile has incorrect entry for "@polkadot/[email protected]". Ignoring it.
warning Lockfile has incorrect entry for "@polkadot/[email protected]". Ignoring it.
warning Lockfile has incorrect entry for "typescript@~4.8.2". Ignoring it.
There was a problem hiding this comment.
Sorry for the late response @ltfschoen. Added a comment and I hit this error while trying to run ./docker-dev.sh locally.
Error
error /app/webb-monorepo/node_modules/utf-8-validate: Command failed.
Exit code: 1
Command: node-gyp-build
Arguments:
Directory: /app/webb-monorepo/node_modules/utf-8-validate
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | arm64
gyp ERR! find Python
gyp ERR! find Python Python is not set from command line or npm configuration
gyp ERR! find Python Python is not set from environment variable PYTHON
gyp ERR! find Python checking if "python3" can be used
gyp ERR! find Python - "python3" is not in PATH or produced an error
gyp ERR! find Python checking if "python" can be used
gyp ERR! find Python - "python" is not in PATH or produced an error
gyp ERR! find Python
gyp ERR! find Python **********************************************************
gyp ERR! find Python You need to install the latest version of Python.
gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
gyp ERR! find Python you can try one of the following options:
gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
gyp ERR! find Python (accepted by both node-gyp and npm)
gyp ERR! find Python - Set the environment variable PYTHON
gyp ERR! find Python - Set the npm configuration variable python:
gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
gyp ERR! find Python For more information consult the documentation at:
gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
gyp ERR! find Python **********************************************************
gyp ERR! find Python
gyp ERR! configure error
gyp ERR! stack Error: Could not find any Python installation to use
gyp ERR! stack at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:330:47)
gyp ERR! stack at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:159:21)
gyp ERR! stack at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:202:16)
gyp ERR! stack at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:294:16)
gyp ERR! stack at exithandler (node:child_process:410:5)
gyp ERR! stack at ChildProcess.errorhandler (node:child_process:422:5)
gyp ERR! stack at ChildProcess.emit (node:events:513:28)
gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! stack at onErrorNT (node:internal/child_process:485:16)
gyp ERR! stack at processTicksAndRejections (node:internal/process/task_queues:83:21)
gyp ERR! System Linux 5.10.104-linuxkit
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /app/webb-monorepo/node_modules/utf-8-validate
gyp ERR! node -v v16.19.0
gyp ERR! node-gyp -v v9.1.0
gyp ERR! not ok
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
1 error occurred:
* Status: The command '/bin/sh -c yarn' returned a non-zero code: 1, Code: 1
Screenshot
I added below line to fix the error
# ...
RUN apk update && apk add --update git
RUN apk add --update python3 make g++\
&& rm -rf /var/cache/apk/*
RUN yarn
# ...
|
Another thing is after running the docker successfully, there are lots of
|
with the but we're using i think the best solution for this would be to use this approach https://stackoverflow.com/a/55082473/3208553, so the changes could be something like: Dockerfile.dev Then either provide the Then can prune them after the final image and container is created: |
|
another consideration is that this PR only supports users that are using the latest |
i've pushed a commit ae65b67 adding this |
i tried adding the labelling using i found that those images are dangling images, they don't appear when you run even if i try building with |
|
after building with running error```bash ERROR in ./src/index.tsx Module build failed (from ../../node_modules/thread-loader/dist/cjs.js): Thread Loader (Worker 0) Unknown option: .assumptions. Check out https://babeljs.io/docs/en/babel-core/#options for more information about options. webpack 5.75.0 compiled with 1 error in 1842 ms |
|
and, error```bash [webpack-dev-server] Project is running at: [webpack-dev-server] Loopback: http://localhost:3001/ [webpack-dev-server] On Your Network (IPv4): http://172.18.0.2:3001/ [webpack-dev-server] Content not from webpack is served from '/app/webb-monorepo/public' directory Listening on port: 3001 [BABEL] Note: The code generator has deoptimised the styling of /app/webb-monorepo/apps/stats-dapp/src/generated/graphql.tsx as it exceeds the max of 500KB. assets by path *.js 22.8 MiB assets by chunk 18.1 MiB (id hint: vendors) 4 assets + 53 assets assets by path static/*.png 43.8 KiB asset static/star.67b49168.png 34.6 KiB [emitted] [immutable] [from: ../../libs/webb-ui-components/src/assets/star.png] (auxiliary name: main) asset static/star-dark.96780024.png 9.16 KiB [emitted] [immutable] [from: ../../libs/webb-ui-components/src/assets/star-dark.png] (auxiliary name: main) assets by path *.png 32.4 KiB asset favicon.png 16.2 KiB [emitted] [from: src/public/favicon.png] [copied] asset logo.png 16.2 KiB [emitted] [from: src/public/logo.png] [copied] assets by path *.json 390 bytes asset manifest.json 312 bytes [emitted] [from: src/public/manifest.json] [copied] asset i18n.json 78 bytes [emitted] [from: src/public/i18n.json] [copied] asset adcaf6a1ffc740d7.module.wasm 1.35 MiB [emitted] [immutable] [big] (auxiliary id hint: vendors) asset ./index.html 930 bytes [emitted] asset robots.txt 57 bytes [emitted] [from: src/public/robots.txt] [copied] Entrypoint main [big] 22 MiB (17.7 MiB) = vendors-node_modules_babel_polyfill_lib_index_js-node_modules_ngneat_falso_index_esm_js-node_-e26667.6e46a2c9.js 17.7 MiB main.85ab18fd.js 4.3 MiB 4 auxiliary assets orphan modules 7.72 MiB [orphan] 2049 modules runtime modules 44.8 KiB 26 modules modules by path ../../ 17.2 MiB (javascript) 1.35 MiB (webassembly) 43.8 KiB (asset) modules by path ../../node_modules/ 15.2 MiB (javascript) 1.35 MiB (webassembly) 2646 modules modules by path ../../libs/ 1.96 MiB (javascript) 43.8 KiB (asset) 401 modules modules by path ./ 813 KiB modules by path ./src/ 810 KiB 63 modules modules by path ./*.css 2.82 KiB ./styles.css 2.78 KiB [built] [code generated] ../../node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].use[1]!../../node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[1].use[2]!../../node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[1].use[3]!./styles.css 39 bytes [built] [code generated] [1 error] modules by path ^/.// 320 bytes ../../libs/icons/src/ lazy !!../../node_modules/@svgr/webpack/lib/index.js?+svgo,+titleProp,+ref! ^\.\/.*\.svg$ namespace object 160 bytes [built] [code generated] ../../libs/icons/src/ lazy !!../../node_modules/@svgr/webpack/lib/index.js?+svgo,+titleProp,+ref! ^\.\/.*s\/default\.svg$ namespace object 160 bytes [built] [code generated] + 28 modules ERROR in ./src/containers/StackedAreaChartContainer/StackedAreaChartContainer.tsx
ERROR in ./src/hooks/useDonutColor.ts
ERROR in ./src/pages/AuthoritiesHistory.tsx
ERROR in ../../libs/webb-ui-components/src/components/Accordion/Accordion.tsx 10:0-64 ERROR in ../../libs/webb-ui-components/src/components/Accordion/AccordionButton.tsx 10:0-64 ERROR in ../../libs/webb-ui-components/src/components/Accordion/AccordionContent.tsx 10:0-64 ERROR in ../../libs/webb-ui-components/src/components/Accordion/AccordionItem.tsx 10:0-64 ERROR in ../../libs/webb-ui-components/src/components/Popover/Popover.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/Popover/PopoverArrow.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/Popover/PopoverClose.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/Popover/PopoverContent.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/Popover/PopoverTrigger.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/ScrollArea/ScrollArea.tsx 13:0-67 ERROR in ../../libs/webb-ui-components/src/containers/TransactionProgressCard/TransactionProgressCard.tsx 17:0-34 ERROR in ./styles.css (../../node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].use[1]!../../node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[1].use[2]!../../node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[1].use[3]!./styles.css)
ERROR in ../../libs/webb-ui-components/src/tailwind.css (../../node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].use[1]!../../node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[1].use[2]!../../node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[1].use[3]!../../libs/webb-ui-components/src/tailwind.css)
11 errors have detailed information that is not shown. webpack 5.75.0 compiled with 16 errors in 38649 ms
ERROR in ./src/containers/StackedAreaChartContainer/StackedAreaChartContainer.tsx
ERROR in ./src/hooks/useDonutColor.ts
ERROR in ./src/pages/AuthoritiesHistory.tsx
ERROR in ../../libs/webb-ui-components/src/components/Accordion/Accordion.tsx 10:0-64 ERROR in ../../libs/webb-ui-components/src/components/Accordion/AccordionButton.tsx 10:0-64 ERROR in ../../libs/webb-ui-components/src/components/Accordion/AccordionContent.tsx 10:0-64 ERROR in ../../libs/webb-ui-components/src/components/Accordion/AccordionItem.tsx 10:0-64 ERROR in ../../libs/webb-ui-components/src/components/Popover/Popover.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/Popover/PopoverArrow.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/Popover/PopoverClose.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/Popover/PopoverContent.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/Popover/PopoverTrigger.tsx 8:0-60 ERROR in ../../libs/webb-ui-components/src/components/ScrollArea/ScrollArea.tsx 13:0-67 ERROR in ../../libs/webb-ui-components/src/containers/TransactionProgressCard/TransactionProgressCard.tsx 17:0-34 ERROR in ./styles.css (../../node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].use[1]!../../node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[1].use[2]!../../node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[1].use[3]!./styles.css)
ERROR in ../../libs/webb-ui-components/src/tailwind.css (../../node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].use[1]!../../node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[1].use[2]!../../node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[1].use[3]!../../libs/webb-ui-components/src/tailwind.css)
11 errors have detailed information that is not shown. webpack 5.75.0 compiled with 16 errors in 38649 ms |
|
FYI, i tried to use previous new |
| ```bash | ||
| ./docker-dev.sh | ||
| ``` | ||
| Note: The script requires `jq` command to be installed (i.e. Ubuntu `apt-get install -y jq`) |
There was a problem hiding this comment.
Consider adding wget here, because wget does not exist by default in some OS.
There was a problem hiding this comment.
I don't think its a requirement as you have fallback namespaces in place, no?
There was a problem hiding this comment.
Consider adding
wgethere, becausewgetdoes not exist by default in some OS.
yes, good idea. i think it would be most seamless to embed a script like this in the docker-dev.sh script https://github.com/paritytech/scripts/blob/master/get-substrate.sh, where it installs the necessary dependencies depending on what operating system it detects.
There was a problem hiding this comment.
I don't think its a requirement as you have fallback namespaces in place, no?
i just tried a fresh install on my local macOS machine where i don't have wget and haven't created the .env file. when i run docker-dev.sh it outputs the following. so yes it uses the existing fallbacks defined in the docker-dev.sh file so it works even if wget command not found, but probably should detect their OS and try to install it.
./docker-dev.sh: line 7: wget: command not found
./docker-dev.sh: line 15: ./.env: No such file or directory
...
*** Started building Docker container.
*** Please wait...
***open /Users/me/ltfschoen/webb-dapp/.env: no such file or directory
i was also concerned because we don't have fallbacks for the ports PORT_BRIDGE_DAPP and PORT_STATS_DAPP, but it doesn't get any further than trying to run docker compose ... command where expects an .env file to continue, since in docker-compose-dev.yml it has:
services:
dev:
...
env_file:
- .env
i think we'd only be concerned about fallbacks if the user created an empty .env file but didn't follow the README.md instructions where it says to create it based on the .env.example with cp .env.example .env
dutterbutter
requested review from
dutterbutter
and removed request for
AhmedKorim
|
@ltfschoen thanks for the contribution! Consider:
In addition, I was able to build the docker container fine with the provided instructions but failed to serve both the bridge-dapp and stats-dapp. Since I ran this on macos the fallback namespaces were used so perhaps that might be a potential cause but not entirely sure? Error logs for each provided below. bridge-dapp logServing bridge-dapp error logStats-dapp logServing stats-dapp error log |
yes, then it would be consistent with how its done here https://github.com/paritytech/substrate/tree/master/docker i just had a look at your errors and tried it on my macOS too and i was able to replicate your error. unfortunately it takes a lot longer on my local macOS machine that has old Catalina and its only an i5 dual-core and i also have a slow internet connection (40 Mbps, much slower than the 40Gbps i get using cloud provider connections where i've been testing it on Ubuntu) and its necessary to build these large anyway i found that on my machine i had permissions issues and xcode command line tools were missing so i had to: the changes i made are here https://github.com/ltfschoen/webb-dapp/pull/1/files i think the only feasible way i'd be able to further investigate this issue is if i found a cloud provider that allows use of a macOS instance. unfortunately it just takes far too long on my local machine and it crashed it today and i'd need to setup an external ssd specifically to store the docker images as i'm low on space. i wonder if i could use Golem to use someone else's cheaper |
|
@ltfschoen I am going to close this since its become stale. Please feel free to open back up if you pick it up again. Thanks! |
|
i just got a new macOS computer, so i'm curious to try webb again and try an attempt this issue again if you think it'd provide some value. from memory the problem with this PR was this comment was not yet resolved when @dutterbutter tried it on macOS #848 (comment) |
Summary of changes
Code Checklist