Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade latest puppeteer (includes M1 Mac support) #699

Merged
merged 3 commits into from
Sep 10, 2021

Conversation

thescientist13
Copy link
Member

@thescientist13 thescientist13 commented Aug 13, 2021

Related Issue

related to #675 + #611 / #630

(my guess is due to one of the issues below, the build github action will likely time out and fail)

Summary of Changes

Was testing out upgrading puppeteer in the #611 branch but was seeing an issue where I could build index.html but anything more than that, like just index.html + pages/about would hang and never complete all the pre-rendering. 😕

Found a solution!

```sh % yarn clean && yarn build yarn run v1.22.5 $ rimraf ./**/.greenwood/** && rimraf ./**/public/** && rimraf ./coverage ✨ Done in 7.90s. yarn run v1.22.5 $ cross-env __GWD_ROLLUP_MODE__=strict node . build ------------------------------------------------------- Welcome to Greenwood (v0.15.2) ♻️ ------------------------------------------------------- Running Greenwood with the build command. Initializing project config Initializing project workspace contexts Generating graph of workspace files... Started local development server at localhost:1984 GraphQLServer started at http://localhost:4000/ Prerendering pages at http://127.0.0.1:1984 pages to render www/pages/about/community.md www/pages/about/features.md www/pages/about/goals.md www/pages/about/how-it-works.md www/pages/about/index.md www/pages/docs/component-model.md www/pages/docs/configuration.md www/pages/docs/css-and-images.md www/pages/docs/data.md www/pages/docs/front-matter.md www/pages/docs/index.md www/pages/docs/layouts.md www/pages/docs/markdown.md www/pages/docs/menus.md www/pages/docs/tech-stack.md www/pages/getting-started/branding.md www/pages/getting-started/build-and-deploy.md www/pages/getting-started/creating-content.md www/pages/getting-started/index.md www/pages/getting-started/key-concepts.md www/pages/getting-started/next-steps.md www/pages/getting-started/project-setup.md www/pages/getting-started/quick-start.md www/pages/guides/cloudflare-workers-deployment.md www/pages/guides/firebase.md www/pages/guides/github-pages.md www/pages/guides/index.md www/pages/guides/netlify-cms.md www/pages/guides/netlify-deploy.md www/pages/guides/now.md www/pages/guides/s3-cloudfront.md www/pages/guides/theme-packs.md www/pages/index.html www/pages/plugins/context.md www/pages/plugins/custom-plugins.md www/pages/plugins/index.md www/pages/plugins/resource.md www/pages/plugins/rollup.md www/pages/plugins/server.md prerendering page... /about/community/ prerendering page... /about/features/ prerendering page... /about/goals/ prerendering page... /about/how-it-works/ prerendering page... /about/ prerendering page... /docs/component-model/ prerendering page... /docs/configuration/ prerendering page... /docs/css-and-images/ prerendering page... /docs/data/ prerendering page... /docs/front-matter/ prerendering page... /docs/ prerendering page... /docs/layouts/ prerendering page... /docs/markdown/ prerendering page... /docs/menus/ prerendering page... /docs/tech-stack/ prerendering page... /getting-started/branding/ prerendering page... /getting-started/build-and-deploy/ prerendering page... /getting-started/creating-content/ prerendering page... /getting-started/ prerendering page... /getting-started/key-concepts/ prerendering page... /getting-started/next-steps/ prerendering page... /getting-started/project-setup/ prerendering page... /getting-started/quick-start/ prerendering page... /guides/cloudflare-workers-deployment/ prerendering page... /guides/firebase/ prerendering page... /guides/github-pages/ prerendering page... /guides/ prerendering page... /guides/netlify-cms/ prerendering page... /guides/netlify-deploy/ prerendering page... /guides/now/ prerendering page... /guides/s3-cloudfront/ prerendering page... /guides/theme-packs/ prerendering page... / prerendering page... /plugins/context/ prerendering page... /plugins/custom-plugins/ prerendering page... /plugins/ prerendering page... /plugins/resource/ prerendering page... /plugins/rollup/ prerendering page... /plugins/server/ prerendering complete for page /docs/front-matter/. prerendering complete for page /docs/data/. prerendering complete for page /docs/markdown/. prerendering complete for page /docs/layouts/. prerendering complete for page /getting-started/build-and-deploy/. prerendering complete for page /guides/firebase/. prerendering complete for page /getting-started/key-concepts/. prerendering complete for page /docs/menus/. prerendering complete for page /getting-started/. prerendering complete for page /guides/github-pages/. prerendering complete for page /getting-started/branding/. prerendering complete for page /guides/. prerendering complete for page /guides/theme-packs/. prerendering complete for page /getting-started/quick-start/. prerendering complete for page /docs/tech-stack/. prerendering complete for page /getting-started/next-steps/. prerendering complete for page /about/features/. prerendering complete for page /docs/component-model/. prerendering complete for page /plugins/resource/. prerendering complete for page /plugins/custom-plugins/. prerendering complete for page /guides/netlify-deploy/. prerendering complete for page /plugins/context/. prerendering complete for page /plugins/. prerendering complete for page /guides/now/. prerendering complete for page /. prerendering complete for page /plugins/rollup/. prerendering complete for page /guides/s3-cloudfront/. prerendering complete for page /plugins/server/. ```

(I was able to get the issue in #675 somewhat resolved by seemingly attributing it to <app-banner> but my results are still mostly inconclusive)

So decided to test it against master and still the same issue with not being able to build more than a few pages.

So aside from that, things work (obviously no menu if no pages 🤷‍♂️ )


Thoughts / TODOs

  1. we may want to throttle the [number of pages open at any one time] - enable throttling of max number of pages that can be prerendered at once #720. Although we are preserving the same browser instance, we are opening a page / tab for every page in the app at once. Perhaps this is overwhelming the newer version of chromium?
    • probably we should throttle so should make an, but surely it should at least be able to handle 10-12 pages? If so, then this as a root cause wouldn't explain why by just trying to build index.html + about/ pages, it still hangs 😠
  2. Analyze actual shadow DOM support for when we get there, some good links around polyfilling and shimming in support on our own, see if it's the actual cause of Lit@2 Shady DOM styles not present when pre-rendering with puppeteer #675 ?
  3. Tested to see if we still needed our little work arounds in browser.js and plugin-standard-html and we do
  4. From what I can tell, none of the breaking changes in puppeteer seem like they would be relevant / impactful to us...
  5. I might also take a side journey on experimenting with actual Lit + SSR ( Server Side Rendering and Templating (puppeteer alternatives) #576 ) since that might provide us a more programmatic to maintaining this sort of functionality and not having to depend on a real browser so much, though we might have to roll our own a bit as we might have to roll our own if sticking to and / or supporting puppeteer going forward.
  6. FWIW, looks like v6 brings with it support for M1 macs, so might be worth testing Greenwood out there using the branch to confirm
    • so @ls-lint is still waiting to get there re: M1 support, and there are some breaking changes in later versions of NodeJS (>= 15.5.x) that fail our tests but was able to get most tests to pass and the build to complete, so from a user's perspective, it should work! 💥
    Screen Shot 2021-09-07 at 13 23 14

@thescientist13 thescientist13 added bug Something isn't working chore unit testing, maintenance, etc website Tasks related to the projects website / documentation P0 Critical issue that should get addressed ASAP SSR labels Aug 13, 2021
@thescientist13 thescientist13 self-assigned this Aug 13, 2021
@thescientist13
Copy link
Member Author

PHEW! Think I found the issue! 🥳

When running in non-headless mode, I saw that all though the pages loaded fine, they were hanging seemingly due to waiting on the cache
Screen Shot 2021-09-07 at 12 48 57 PM

Which led me to this SO post, and validated in puppeteer/puppeteer#1556 / puppeteer/puppeteer#1609!

Let me just do a little more testing.

@thescientist13 thescientist13 linked an issue Sep 7, 2021 that may be closed by this pull request
12 tasks
@thescientist13 thescientist13 force-pushed the bug/issue-675-puppeteer-upgrade-test branch from cd6e0a6 to fce72cf Compare September 7, 2021 17:03
@thescientist13 thescientist13 removed bug Something isn't working website Tasks related to the projects website / documentation labels Sep 7, 2021
@thescientist13 thescientist13 marked this pull request as ready for review September 7, 2021 17:12
@thescientist13 thescientist13 changed the title upgrade latest puppeteer upgrade latest puppeteer (includes M1 Mac support) Sep 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alpha.0 chore unit testing, maintenance, etc P0 Critical issue that should get addressed ASAP SSR v0.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

routine maintenance and semver major upgrades of key dependencies
1 participant