[v3] Template Extensibility 💦 (#1224)

* committing early progress, local npm repo requires it

* commit for local npm

* getting local npm working

* Cannot get property `main` of undefined + can't follow glob pattern `app/static` fixes

* Working now (side-loading from node_modules/retail-react-app) next step: overrides working

* fix broken WIP bit

* Working example of overrides

* remove system dependent lookup

* WIP, working server side now, broken client side (bundle/development/vendor.js throws an error)

* WIP, fix broken webpack dev server 404ing on `mobify/bundle/development/**/*` files

* fix broken header menu state

* cleanup

* make paths non-absolute as interim solution (so others can check out / run the POC)

* fix webpack module pathing

* fix `react-intl` webpack loading failure, restore more of baseline template now that things are working

* remove product detail page override

* Make overrideables (files like `_app-config`, `_app` etc. dynamic) + reduce filesystem lookups for performance

* commit WIP to share

* WIP algorithim for `file/path(.)(/index)(js|jsx|ts|tsx)

* clean up / simplify

* working now! hardcoded for just icons/index.jsx override

* cleanup

* Icons working with magic local (*) and base template (^) directives

* Remove dependency on hardcoded `icons/index.jsx` path

* minor cleanup

* YES! freaking works... fixed imports so that magic characters work properly

* remove most of the unnecessary overrides

* minor cleanup

* progress on debugging double-request processing in webpack

* fix helper function that was returning `undefined` as a value in regex

* Early WIP toward consolidating 3 plugins into one. Regex check returning at line 317, but shouldn't be

* add regex of path where filename is NOT included

* forgot to commit plugin file

* comment out `resource.createData.resource` which seems unnecessary

* add social icons to overrides

* fix bad icon pathing (svg spritemaps), removed the unneeded `template-retail-react-app` path pushed to regex map, add WIP hashmap for optimizing lookups from _og_overrides, added `(svg|jpg|jpeg)` extensions to regex

* Fix `pwa-kit-dev` paths that didn't know about `overrides` in build pipeline

* add the whole `template-retail-react-app` under `app` dir to stress test performance with high number of file overrides

* fix bad upstream merge changes

* fix bad upstream merge resolutions

* Add missing template files

* Add back missing `scripts` in package.json after bad merge

* more fixes for out of sync package / lockfiles

* add missing pwa-kit packages

* Fix regex matching patterns (#1057)

* resolve package lock errors

* clean console logs and code

* more cleanup, remove duplicate items in overrideregex

* upgrade minimatch version, match all file extensions, match with no file extension

* fix dependency dupes

* Remove all console logs / cleanup / unnecessary code

* fix some problems with final cleanup

* merge

* remove stress test files post-merge

* added pdp page override with small customization

* add constants file to spike project and tweak overrides values

* Fix bug where `<path>/dir/index.(js|jsx|etc)` was being picked up by template extensibility, but `<path>/dir/non-index-name.(js|jsx|etc)` was not

* fix package.json scripts

* fix falsy value defaulting causing errors in `path.join` fix broken `npm run push`

* delete `build/loadable-stat.json` from git, gitignore should work from there

* fix some file pathing issues in how SDK looks up default / universal / project components

* Perf enhancements, don't use regex all (.*) for module replacement, reduce conditional logic execution

* minor code style tweak

* cleanup

* fix conflicts

* fix package lockfiles that were out of sync

* Revert "fix package lockfiles that were out of sync"

This reverts commit 7f3c364.

* fix out of sync lockfiles

* merge conflict

* resolve merge conflict

* update spike project's engines / regen v3 lockfiles

* add missing node engines

* add missing useToast (bad merge)

* updates from underlying template-retail-react-app

* drop unnecessary pages/product-detail override

* migrate to new directory

* remove unnecessary directory nesting, remove unnecessary files for "minimum file project"

* add example of extending routes non-destructively, add example of quickly toggling some base constant values (e.g. categories shown on home page)

* Update example route

* add some demo examples of overrides

* commit package name change

* fix bad merge conflict

* bring back deleted package lockfiles

* regen package lockfiles

* remove demo extensible app in light of soon-to-be-merged PR from @bendvc

* lockfiles from reaact18 / chakra2

* Feature/template extensibility (#1162)

* initial work on moving code to use resolver

* fix getoverridepath and break on ^ imports

* add todos and comments

* rewrite requestcontext.request for path with caret

* perform overrideshashmap lookup for relative import

* progress

* progress

* regenerate lock files

* fix dupe useToast

* progress on making `charles-solution` work

* initial pass at migration to `resolver.getHook('resolve').tapAsync` working, TODO next... cleanup

* restore icons file and add extensible text to home

* add some comments and construct hashmap in new plugin

* working glob sync lookup??

* resolve lookup when there is no extension

* add component slot for testing

* fix some lookup errors

* cleanup

* rewrite requests coming from underlying template

* cleanup

* rename plugin file

* remove unnecessary build files

* rename to overrides

* fix lookup errors

* fix some broken upstream changes, eslint fixes

* Bring back the deleted package.json

* Install new eslint plugin

* No more relative imports in retail-react-app

* Create .prettierrc.yaml

* restore other deleted package lockfiles

* fix some eslint issues

* remove `^` imports and begin migrating logic away from previous API. TODO: get webpack to resolve `retail-react-app` base import path from within `template-retail-react-app`

* short circuit plugin temporarily, get webpack `alias` working for template to resolve self-referencing `retail-react-app` imports

* fix unreachable condition

* wow... freaking works 🚀

* cleanup, get `template-retail-react-app` working again

* cleanup

* Create .eslintrc.js

* fix routes.jsx which no longer pulled relative template

* make routes.jsx more concise

* update lockfiles

* add jsconfig.json for IDEs to have magic `retail-react-app` be discoverable

* more cleanup

* change package.json key from `mobify` => `ccExtensibility`

* cleanup

* Final cleanup, remove console.logs, make `extendable` dynamic for internal file resolution in `template-retail-react-app` (and future projects), remove some non-required files,

* fix straggler non-dynamic filepath

* cleanup

* remove code that shouldn't be in v3+

* sync package.json deps in extended example project

* cleanup / pr feedback

* pr feedback, great catch @kevinxh

* fix problem with bad copy from underlying template file

* pr feedback / cleanup

* cleanup

Co-authored-by: Kevin He <[email protected]>

* Update package-lock.json

* WIP

* drop unneeded complexity

* lockfiles

* moving toward an array of `alias`es

* Refactor template extensibility functionality to use `resolve.alias` + greatly reduce code footprint + make leading slash optional in `ccExtensibility.overridesDir`

* add brand-logo with no clash for server side viewBox

* add `request-processor.js` (required for worker.js to load) and fix bad extensibility path loading in `build-dev-server.js`

* lint fix

* fix failing pwa-kit-react-sdk tests

* cleanup

* lint:fix

* fix failing tests in CI re: `TextDecoder is not defined`

* take 2 attempting to fix TextDecoder global in jest

* fix jest module mapper

* fix jest moduleMapper config

* REALLY fix jest moduleNameMapper config for new pathing + fix node `util` global TextDecoder is not defined in jest

* add brand-logo

* remove previously added `AbovePDP` Template Hook component

* replay #1128 from bad merge

* eslint:fix for `retail-react-app` base paths

* de-duplicate deps in bundle

* fix path defaulting, don't pass nullish value to path.resolve

* improve pathing for overridesDir in pwa-kit-dev webapack config + add missing worker/main.js

* fix pathing

* fix pathing issue that was mysteriously UNFIXED

* fix `worker.js` path

* fix app/static pathing in non-extensible builds

* [Extensibility] Remove Einstein proxy and send Einstein events directly from app (#1202)

* Remove Einstein proxy and send Einstein events directly from app

* Update ssrParameters

* Replace ExtendsCircularImportsPlugin with OverridesResolverPlugin (#1211)

* Replace ExtendsCircularImportsPlugin with OverridesResolverPlugin

* fix filepath problem

---------

Co-authored-by: Brian Feister <[email protected]>

* remove irrelevant bundlesize check to pass CI

* Fix `vendor.js` inclusion of `retail-react-app` files (#1198)

* fix duplicate lilbs in bundle

* fix app/static pathing for non-extensible builds

* oof, fix extensible app/static filepaths

* Resolve conflicting npm deps via `resolve.alias`

* add missing worker/main.js

* cleanup unused imports

* WIP, add list of original overridable deps

* prevent template extensibility vendor.js from treating baseline route files as deps in vendor.js

* fix irrelevant bundlesize declaration to pass CI

* drop test:max-file-size test from my-extended-retail-app

* replace relative with absolute path fetchTranslations (#1215)

Co-authored-by: Brian Feister <[email protected]>

* Refactor for unit testing, fix pathing issue from old version of file parsing algoritihim

* fix merge conflict resolution

* remove template extensible demo in prep for @bendvc's pr for generator work

* regen lockfiles for react 18 changes

* fix linting

---------

Co-authored-by: yunakim714 <[email protected]>
Co-authored-by: Vincent Marta <[email protected]>
Co-authored-by: Kevin He <[email protected]>
Co-authored-by: vcua-mobify <[email protected]>
Co-authored-by: vcua-mobify <[email protected]>
Co-authored-by: Alex Vuong <[email protected]>
Co-authored-by: Alex Vuong <[email protected]>

* support windows file paths

* fix failing tests

* fix merge resolution

---------

Co-authored-by: Ben Chypak <[email protected]>
Co-authored-by: yunakim714 <[email protected]>
Co-authored-by: Yuna Kim <[email protected]>
Co-authored-by: Vincent Marta <[email protected]>
Co-authored-by: Kevin He <[email protected]>
Co-authored-by: vcua-mobify <[email protected]>
Co-authored-by: vcua-mobify <[email protected]>
Co-authored-by: Alex Vuong <[email protected]>
Co-authored-by: Alex Vuong <[email protected]>

packages/commerce-sdk-react/setup-jest.js

@@ -25,6 +25,9 @@ class LocalStorageMock {
     }
 }
 
+/* eslint-disable @typescript-eslint/no-var-requires */
+global.TextDecoder = require('util').TextDecoder
+
 const localStorageMock = new LocalStorageMock()
 
 Object.defineProperty(window, 'localStorage', {

packages/pwa-kit-dev/bin/pwa-kit-dev.js

@@ -13,6 +13,7 @@ const WebSocket = require('ws')
 const program = require('commander')
 const validator = require('validator')
 const {execSync: _execSync} = require('child_process')
+const projectPkg = require(process.cwd() + '/package.json')
 const {getConfig} = require('pwa-kit-runtime/utils/ssr-config')
 
 // Scripts in ./bin have never gone through babel, so we
@@ -177,6 +178,20 @@ const main = async () => {
             }
         })
 
+    const appSSRpath = p.join(process.cwd(), 'app', 'ssr.js')
+    const appSSRjs = fse.pathExistsSync(appSSRpath)
+    const overrideSSRpath = p.join(
+        process.cwd(),
+        typeof projectPkg?.ccExtensibility?.overridesDir === 'string' &&
+            !projectPkg?.ccExtensibility?.overridesDir?.startsWith(p.sep)
+            ? p.sep + projectPkg?.ccExtensibility?.overridesDir
+            : projectPkg?.ccExtensibility?.overridesDir ?? '',
+        'app',
+        'ssr.js'
+    )
+    const overrideSSRjs = fse.pathExistsSync(overrideSSRpath)
+    const resolvedSSRPath = appSSRjs ? appSSRpath : overrideSSRjs ? overrideSSRpath : null
+
     program
         .command('start')
         .description(`develop your app locally`)
@@ -185,15 +200,12 @@ const main = async () => {
         )
         .addOption(new program.Option('--noHMR', 'disable the client-side hot module replacement'))
         .action(async ({inspect, noHMR}) => {
-            execSync(
-                `node${inspect ? ' --inspect' : ''} ${p.join(process.cwd(), 'app', 'ssr.js')}`,
-                {
-                    env: {
-                        ...process.env,
-                        ...(noHMR ? {HMR: 'false'} : {})
-                    }
+            execSync(`node${inspect ? ' --inspect' : ''} ${resolvedSSRPath}`, {
+                env: {
+                    ...process.env,
+                    ...(noHMR ? {HMR: 'false'} : {})
                 }
-            )
+            })
         })
 
     program