From 1543a6a980941ddfea744c8400561ad194787a10 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Mon, 19 Sep 2022 15:36:21 -0700 Subject: [PATCH 01/23] feat: checkpoint work on new create package --- packages/create/.eslintignore | 1 + packages/create/.eslintrc.cjs | 3 + packages/create/.gitignore | 2 + packages/create/.prettierignore | 1 + packages/create/CHANGELOG.md | 159 +++++++++ packages/create/LICENSE | 21 ++ packages/create/README.md | 20 ++ packages/create/bin/create-sde.js | 14 + packages/create/package.json | 54 +++ packages/create/src/index.ts | 328 ++++++++++++++++++ packages/create/tests/create-dirs.spec.ts | 116 +++++++ .../non-empty-dir/packages/empty-file.txt | 0 packages/create/tsconfig-base.json | 17 + packages/create/tsconfig-build.json | 6 + packages/create/tsconfig-test.json | 5 + packages/create/tsconfig.json | 6 + packages/create/tsup.config.ts | 11 + pnpm-lock.yaml | 277 ++++++++++++++- 18 files changed, 1036 insertions(+), 5 deletions(-) create mode 100644 packages/create/.eslintignore create mode 100644 packages/create/.eslintrc.cjs create mode 100644 packages/create/.gitignore create mode 100644 packages/create/.prettierignore create mode 100644 packages/create/CHANGELOG.md create mode 100644 packages/create/LICENSE create mode 100644 packages/create/README.md create mode 100755 packages/create/bin/create-sde.js create mode 100644 packages/create/package.json create mode 100644 packages/create/src/index.ts create mode 100644 packages/create/tests/create-dirs.spec.ts create mode 100644 packages/create/tests/fixtures/non-empty-dir/packages/empty-file.txt create mode 100644 packages/create/tsconfig-base.json create mode 100644 packages/create/tsconfig-build.json create mode 100644 packages/create/tsconfig-test.json create mode 100644 packages/create/tsconfig.json create mode 100644 packages/create/tsup.config.ts diff --git a/packages/create/.eslintignore b/packages/create/.eslintignore new file mode 100644 index 00000000..1521c8b7 --- /dev/null +++ b/packages/create/.eslintignore @@ -0,0 +1 @@ +dist diff --git a/packages/create/.eslintrc.cjs b/packages/create/.eslintrc.cjs new file mode 100644 index 00000000..3a98c61d --- /dev/null +++ b/packages/create/.eslintrc.cjs @@ -0,0 +1,3 @@ +module.exports = { + extends: ['../../.eslintrc-ts-common.cjs'] +} diff --git a/packages/create/.gitignore b/packages/create/.gitignore new file mode 100644 index 00000000..c319855b --- /dev/null +++ b/packages/create/.gitignore @@ -0,0 +1,2 @@ +dist +tests/fixtures/empty-dir diff --git a/packages/create/.prettierignore b/packages/create/.prettierignore new file mode 100644 index 00000000..1b763b1b --- /dev/null +++ b/packages/create/.prettierignore @@ -0,0 +1 @@ +CHANGELOG.md diff --git a/packages/create/CHANGELOG.md b/packages/create/CHANGELOG.md new file mode 100644 index 00000000..e2f856e0 --- /dev/null +++ b/packages/create/CHANGELOG.md @@ -0,0 +1,159 @@ +# Changelog + +## [0.7.1](https://github.com/climateinteractive/SDEverywhere/compare/cli-v0.7.0...cli-v0.7.1) (2022-08-06) + + +### Bug Fixes + +* correct handling of paths on Windows and those containing spaces ([#223](https://github.com/climateinteractive/SDEverywhere/issues/223)) ([c783e81](https://github.com/climateinteractive/SDEverywhere/commit/c783e811a43331e4c563438b8fa441792bdcfe28)), closes [#222](https://github.com/climateinteractive/SDEverywhere/issues/222) + + +### Dependencies + +* The following workspace dependencies were updated + * dependencies + * @sdeverywhere/build bumped from ^0.1.0 to ^0.1.1 + +## [0.7.0](https://github.com/climateinteractive/SDEverywhere/compare/cli-v0.6.0...cli-v0.7.0) (2022-06-28) + + +### ⚠ BREAKING CHANGES + +* The `sdeverywhere` package is deprecated and effectively replaced by `@sdeverywhere/cli`. Additionally, the `sde generate --genhtml` command and supporting code has been removed and will be replaced with a different solution in the near future. + +### Features + +* add build and plugin-{check,vite,wasm,worker} packages ([#206](https://github.com/climateinteractive/SDEverywhere/issues/206)) ([dd34cbf](https://github.com/climateinteractive/SDEverywhere/commit/dd34cbfcc0b8b3fb1655c8aa64fb919f9757b8be)), closes [#203](https://github.com/climateinteractive/SDEverywhere/issues/203) +* refactor into monorepo with compile and cli packages ([#192](https://github.com/climateinteractive/SDEverywhere/issues/192)) ([8946f18](https://github.com/climateinteractive/SDEverywhere/commit/8946f1854a116f7e9935d5e93d4485865d06d114)), closes [#191](https://github.com/climateinteractive/SDEverywhere/issues/191) + +## [0.6.0](https://github.com/climateinteractive/SDEverywhere/compare/0.5.3...sdeverywhere-v0.6.0) (2022-06-04) + + +### Features + +* add `runModelWithBuffers` entry point that takes pre-allocated input/output buffers ([#50](https://github.com/climateinteractive/SDEverywhere/issues/50)) ([083109a](https://github.com/climateinteractive/SDEverywhere/commit/083109a1f03d9648c6a30efbdfbd3d1543d81cfd)), closes [#49](https://github.com/climateinteractive/SDEverywhere/issues/49) +* add basic support for GET DATA BETWEEN TIMES function ([#42](https://github.com/climateinteractive/SDEverywhere/issues/42)) ([8a294aa](https://github.com/climateinteractive/SDEverywhere/commit/8a294aa5f65c1ffc76a397c77257c02687988770)), closes [#33](https://github.com/climateinteractive/SDEverywhere/issues/33) +* add sde command for flattening parent+submodels ([#59](https://github.com/climateinteractive/SDEverywhere/issues/59)) ([50e11ec](https://github.com/climateinteractive/SDEverywhere/commit/50e11ec69ff0c09d84b72a9b16a8ae3d7d3c1433)), closes [#58](https://github.com/climateinteractive/SDEverywhere/issues/58) +* add support for external data variables that use subscript or dimension ([#41](https://github.com/climateinteractive/SDEverywhere/issues/41)) ([035ab5c](https://github.com/climateinteractive/SDEverywhere/commit/035ab5cb88a7e1ff79423cccacde3b827ce8d0dd)), closes [#32](https://github.com/climateinteractive/SDEverywhere/issues/32) +* add support for external data variables with > 2 dimensions ([#47](https://github.com/climateinteractive/SDEverywhere/issues/47)) ([6683e63](https://github.com/climateinteractive/SDEverywhere/commit/6683e63a90cc45fe5109a39c5918ae6d05c878cd)), closes [#45](https://github.com/climateinteractive/SDEverywhere/issues/45) +* allow for specifying I/O variables using Vensim names in spec file ([#61](https://github.com/climateinteractive/SDEverywhere/issues/61)) ([a6bc98d](https://github.com/climateinteractive/SDEverywhere/commit/a6bc98d48f267cbc05244a0749e1108be680f3f7)), closes [#60](https://github.com/climateinteractive/SDEverywhere/issues/60) +* implement ALLOCATE AVAILABLE function ([#106](https://github.com/climateinteractive/SDEverywhere/issues/106)) ([ebd0311](https://github.com/climateinteractive/SDEverywhere/commit/ebd031160d80e52faa7b98f6e1f1d73fcbca1e15)) +* implement DELAY FIXED function ([#108](https://github.com/climateinteractive/SDEverywhere/issues/108)) ([bd3b3e8](https://github.com/climateinteractive/SDEverywhere/commit/bd3b3e8163bc95dc6957442277311e31b5c4bef9)), closes [#29](https://github.com/climateinteractive/SDEverywhere/issues/29) +* implement GET DIRECT CONSTANTS for CSV ([#86](https://github.com/climateinteractive/SDEverywhere/issues/86)) ([beedd4f](https://github.com/climateinteractive/SDEverywhere/commit/beedd4f1efec31490076badc910fd20003829044)) +* implement GET DIRECT DATA for CSV ([#82](https://github.com/climateinteractive/SDEverywhere/issues/82)) ([b40e738](https://github.com/climateinteractive/SDEverywhere/commit/b40e738389f315984206bac9f9d28f6555549180)), closes [#81](https://github.com/climateinteractive/SDEverywhere/issues/81) +* implement GET DIRECT LOOKUPS including a test model ([#99](https://github.com/climateinteractive/SDEverywhere/issues/99)) ([47f6fe5](https://github.com/climateinteractive/SDEverywhere/commit/47f6fe5d4188b3d14703ff7730d5c874c296df60)), closes [#98](https://github.com/climateinteractive/SDEverywhere/issues/98) +* implement GET DIRECT SUBSCRIPT for CSV ([#79](https://github.com/climateinteractive/SDEverywhere/issues/79)) ([51691e7](https://github.com/climateinteractive/SDEverywhere/commit/51691e7a209cb01911756ff54037dd756a5d439e)) +* implement NPV function ([#95](https://github.com/climateinteractive/SDEverywhere/issues/95)) ([9fa17eb](https://github.com/climateinteractive/SDEverywhere/commit/9fa17eb0f2b2a7ef63608771cb8234eaa3c2b35e)), closes [#94](https://github.com/climateinteractive/SDEverywhere/issues/94) +* implement QUANTUM function ([#89](https://github.com/climateinteractive/SDEverywhere/issues/89)) ([42225f9](https://github.com/climateinteractive/SDEverywhere/commit/42225f9508337cd6dbb0e31c2e5f73269f4b526a)), closes [#87](https://github.com/climateinteractive/SDEverywhere/issues/87) +* implement the <-> subscript alias operator ([#80](https://github.com/climateinteractive/SDEverywhere/issues/80)) ([a43917f](https://github.com/climateinteractive/SDEverywhere/commit/a43917f63e0830530c2faf2b76bc65dc90da7533)), closes [#78](https://github.com/climateinteractive/SDEverywhere/issues/78) +* increase the number of dimension and array loop index vars ([#138](https://github.com/climateinteractive/SDEverywhere/issues/138)) ([4c66470](https://github.com/climateinteractive/SDEverywhere/commit/4c6647069582632e89845b2f08b73348c5cc63e6)), closes [#137](https://github.com/climateinteractive/SDEverywhere/issues/137) +* remove inline comments in the preprocessor ([#74](https://github.com/climateinteractive/SDEverywhere/issues/74)) ([d23b1c3](https://github.com/climateinteractive/SDEverywhere/commit/d23b1c355c8b936d704f7ed4affa012b0eb27356)) +* sort equations alphabetically when preprocessing mdl file ([#56](https://github.com/climateinteractive/SDEverywhere/issues/56)) ([bb968f7](https://github.com/climateinteractive/SDEverywhere/commit/bb968f79348401ceb9c75930fb66ff32ac7c453f)), closes [#55](https://github.com/climateinteractive/SDEverywhere/issues/55) + + +### Bug Fixes + +* abort code generation on finding a lookup of size zero ([#162](https://github.com/climateinteractive/SDEverywhere/issues/162)) ([44c1202](https://github.com/climateinteractive/SDEverywhere/commit/44c1202aaa113505132ab86a49d8ad7f3bee8673)), closes [#161](https://github.com/climateinteractive/SDEverywhere/issues/161) +* add async and await to some chained cli functions ([#37](https://github.com/climateinteractive/SDEverywhere/issues/37)) ([afdbb77](https://github.com/climateinteractive/SDEverywhere/commit/afdbb773db6fdfb8f7ba8d35782254307389f895)), closes [#35](https://github.com/climateinteractive/SDEverywhere/issues/35) +* allow > 2 dimensions when generating Vensim array names ([#155](https://github.com/climateinteractive/SDEverywhere/issues/155)) ([6575ea9](https://github.com/climateinteractive/SDEverywhere/commit/6575ea9c3d8b72e4d172c4add00bf7f1d2cf33bf)), closes [#154](https://github.com/climateinteractive/SDEverywhere/issues/154) +* allow extra index subscripts in 2D const lists ([#110](https://github.com/climateinteractive/SDEverywhere/issues/110)) ([f5494af](https://github.com/climateinteractive/SDEverywhere/commit/f5494afd1837aebbea5d1cdb329447aa5904d264)), closes [#109](https://github.com/climateinteractive/SDEverywhere/issues/109) +* allow GET DIRECT CONSTANTS to use 2 subscripts in the same family ([#144](https://github.com/climateinteractive/SDEverywhere/issues/144)) ([e53d876](https://github.com/climateinteractive/SDEverywhere/commit/e53d876571a6b89705c007f5d2ebf0de366b09e3)), closes [#143](https://github.com/climateinteractive/SDEverywhere/issues/143) +* correct declarations when subscripted variable is initialized from data and constants ([#116](https://github.com/climateinteractive/SDEverywhere/issues/116)) ([7c51641](https://github.com/climateinteractive/SDEverywhere/commit/7c51641223b869d5cff7e5b28692344134d5e4ee)), closes [#115](https://github.com/climateinteractive/SDEverywhere/issues/115) +* correct GET DIRECT DATA with mapped dim by directly comparing indices ([#146](https://github.com/climateinteractive/SDEverywhere/issues/146)) ([fa2097b](https://github.com/climateinteractive/SDEverywhere/commit/fa2097b4b7a0e1ae6580334f93936f70fde19a36)), closes [#145](https://github.com/climateinteractive/SDEverywhere/issues/145) +* correct initialization of 2D arrays to allow dimensions with matching subscript names ([#101](https://github.com/climateinteractive/SDEverywhere/issues/101)) ([2ed7e42](https://github.com/climateinteractive/SDEverywhere/commit/2ed7e427d6af956a9c6b9c03ef38845700cb3ff3)), closes [#84](https://github.com/climateinteractive/SDEverywhere/issues/84) +* emit any expression for the offset arg of VECTOR ELM MAP ([#129](https://github.com/climateinteractive/SDEverywhere/issues/129)) ([bd0a724](https://github.com/climateinteractive/SDEverywhere/commit/bd0a724462aaab09e205294b98cb450180f619c9)), closes [#128](https://github.com/climateinteractive/SDEverywhere/issues/128) +* exclude data vars from initLevels ([#127](https://github.com/climateinteractive/SDEverywhere/issues/127)) ([93a49cf](https://github.com/climateinteractive/SDEverywhere/commit/93a49cf1e7562987a9836dab580a6fad4428d25d)), closes [#126](https://github.com/climateinteractive/SDEverywhere/issues/126) +* expand references to vars with any number of separated dimensions ([#112](https://github.com/climateinteractive/SDEverywhere/issues/112)) ([0d0d40e](https://github.com/climateinteractive/SDEverywhere/commit/0d0d40e1b5b39560ae67f440172e5ded90a6cd3a)), closes [#111](https://github.com/climateinteractive/SDEverywhere/issues/111) +* expand variables allowing for any number of indices in EXCEPT clause ([#118](https://github.com/climateinteractive/SDEverywhere/issues/118)) ([d92343e](https://github.com/climateinteractive/SDEverywhere/commit/d92343e0806f228537f905956d053777accf9e97)), closes [#117](https://github.com/climateinteractive/SDEverywhere/issues/117) +* fix LOOKUP FORWARD to correctly handle fractional inputs ([#38](https://github.com/climateinteractive/SDEverywhere/issues/38)) ([c1f9580](https://github.com/climateinteractive/SDEverywhere/commit/c1f95804533b70036b14fa1caee64d402aeacfeb)), closes [#36](https://github.com/climateinteractive/SDEverywhere/issues/36) +* generate correct references for the ALLOCATE AVAILABLE priority profile ([#136](https://github.com/climateinteractive/SDEverywhere/issues/136)) ([b1d8ae2](https://github.com/climateinteractive/SDEverywhere/commit/b1d8ae2c19b4717df49a4a3e001625ce9568a3ac)), closes [#135](https://github.com/climateinteractive/SDEverywhere/issues/135) +* get direct data offset from the separated dimension ([#114](https://github.com/climateinteractive/SDEverywhere/issues/114)) ([ebbaa01](https://github.com/climateinteractive/SDEverywhere/commit/ebbaa0161dd0d5272f43f5323c0658413d21da47)), closes [#113](https://github.com/climateinteractive/SDEverywhere/issues/113) +* handle subdimensions correctly for GET DIRECT CONSTANTS and fix EXCEPT handling ([#125](https://github.com/climateinteractive/SDEverywhere/issues/125)) ([2fdfb34](https://github.com/climateinteractive/SDEverywhere/commit/2fdfb343dad047b1583a321ed174fa8813107a6c)), closes [#124](https://github.com/climateinteractive/SDEverywhere/issues/124) [#134](https://github.com/climateinteractive/SDEverywhere/issues/134) +* handle subscripts correctly when nested in expr within array function call ([#48](https://github.com/climateinteractive/SDEverywhere/issues/48)) ([b2458ab](https://github.com/climateinteractive/SDEverywhere/commit/b2458ab7e764b4ec33bebf1ee9ac3a80de9b474b)), closes [#46](https://github.com/climateinteractive/SDEverywhere/issues/46) +* make `sde log` wait for DAT file to be fully written and improve error handling ([#123](https://github.com/climateinteractive/SDEverywhere/issues/123)) ([34f25f8](https://github.com/climateinteractive/SDEverywhere/commit/34f25f8b97a8f8d2d0c949b6b81ca655a5e048d1)), closes [#122](https://github.com/climateinteractive/SDEverywhere/issues/122) +* make browserify an optional dependency ([#53](https://github.com/climateinteractive/SDEverywhere/issues/53)) ([e9bbbc6](https://github.com/climateinteractive/SDEverywhere/commit/e9bbbc66dcb59d629b6053a51faeee83a347147d)), closes [#52](https://github.com/climateinteractive/SDEverywhere/issues/52) +* make flatten command work when equations don't include continuation backslash ([#173](https://github.com/climateinteractive/SDEverywhere/issues/173)) ([ac7c027](https://github.com/climateinteractive/SDEverywhere/commit/ac7c0277c3676f09222f4c36fc2abb23894fad26)) +* prevent memory leaks in fixed delay initialization ([#160](https://github.com/climateinteractive/SDEverywhere/issues/160)) ([e158b2f](https://github.com/climateinteractive/SDEverywhere/commit/e158b2f3abe0a1419fb3211fc5952e60b1daaac0)), closes [#159](https://github.com/climateinteractive/SDEverywhere/issues/159) +* record variants of a subscripted variable in removeUnusedVariables ([#65](https://github.com/climateinteractive/SDEverywhere/issues/65)) ([f6d7035](https://github.com/climateinteractive/SDEverywhere/commit/f6d70356f814d3936a5e8c13296088ea241a268c)), closes [#64](https://github.com/climateinteractive/SDEverywhere/issues/64) +* refine the ALLOCATE AVAILABLE search algorithm ([#141](https://github.com/climateinteractive/SDEverywhere/issues/141)) ([bca43a0](https://github.com/climateinteractive/SDEverywhere/commit/bca43a049f666289a2c536cc4174718d43efa4d3)), closes [#139](https://github.com/climateinteractive/SDEverywhere/issues/139) +* remove fcmp library and rewrite expressions without using its macros ([#131](https://github.com/climateinteractive/SDEverywhere/issues/131)) ([df76872](https://github.com/climateinteractive/SDEverywhere/commit/df768724c36ded22677ecdf54c0a3019f9cbab8f)), closes [#107](https://github.com/climateinteractive/SDEverywhere/issues/107) +* remove unnecessary memcpy loop in lookup data initialization ([#166](https://github.com/climateinteractive/SDEverywhere/issues/166)) ([b94d9c4](https://github.com/climateinteractive/SDEverywhere/commit/b94d9c4c2c14a08440fc2f769a6e08e33a165f2a)), closes [#165](https://github.com/climateinteractive/SDEverywhere/issues/165) +* set model directory in the sde causes command ([#142](https://github.com/climateinteractive/SDEverywhere/issues/142)) ([44d326e](https://github.com/climateinteractive/SDEverywhere/commit/44d326e3d684f25ad76d9bca3ca7bd2bb7379768)), closes [#140](https://github.com/climateinteractive/SDEverywhere/issues/140) +* take DELAY FIXED value from input when delay time = 0 ([#148](https://github.com/climateinteractive/SDEverywhere/issues/148)) ([328d050](https://github.com/climateinteractive/SDEverywhere/commit/328d05097aa741c09f8be057eed18c05350ce486)), closes [#147](https://github.com/climateinteractive/SDEverywhere/issues/147) +* terminate generated equations with ~~| ([#120](https://github.com/climateinteractive/SDEverywhere/issues/120)) ([44d1c2a](https://github.com/climateinteractive/SDEverywhere/commit/44d1c2a5fe36e11c57ac9828959f77303819c90e)), closes [#119](https://github.com/climateinteractive/SDEverywhere/issues/119) +* use a global replace to join multiple line Vensim equations in comments ([#175](https://github.com/climateinteractive/SDEverywhere/issues/175)) ([678f2bb](https://github.com/climateinteractive/SDEverywhere/commit/678f2bb4aa8f0a1bd5fdbe8e05c355bc4db5d517)) +* use case-insensitive sort and remove trailing whitespace in preprocessor ([#57](https://github.com/climateinteractive/SDEverywhere/issues/57)) ([d58390a](https://github.com/climateinteractive/SDEverywhere/commit/d58390ae9754562b38e54aeb7917605aab54b894)), closes [#55](https://github.com/climateinteractive/SDEverywhere/issues/55) +* use chunkedFunction to break up initLookups into smaller functions ([#133](https://github.com/climateinteractive/SDEverywhere/issues/133)) ([bad4580](https://github.com/climateinteractive/SDEverywhere/commit/bad4580d6d98c8ea66082bbe984542ce84ba9164)), closes [#132](https://github.com/climateinteractive/SDEverywhere/issues/132) +* use correct subdimension index for delay aux vars ([#92](https://github.com/climateinteractive/SDEverywhere/issues/92)) ([7158b0f](https://github.com/climateinteractive/SDEverywhere/commit/7158b0fb45af1cfb9396d22010a41a3df7c917f4)), closes [#91](https://github.com/climateinteractive/SDEverywhere/issues/91) +* use subdim indices for GET DIRECT DATA ([#164](https://github.com/climateinteractive/SDEverywhere/issues/164)) ([d7b46c6](https://github.com/climateinteractive/SDEverywhere/commit/d7b46c6b0720b716d466b40ef975103338e5398c)), closes [#163](https://github.com/climateinteractive/SDEverywhere/issues/163) +* wrap conditional branch expression in parentheses when optimizing IF THEN ELSE ([#153](https://github.com/climateinteractive/SDEverywhere/issues/153)) ([bd42d54](https://github.com/climateinteractive/SDEverywhere/commit/bd42d540e6ef573ce7f713e429eaad61e87398ce)), closes [#152](https://github.com/climateinteractive/SDEverywhere/issues/152) + + +### Performance Improvements + +* cache last input and last accessed index for faster lookups ([#43](https://github.com/climateinteractive/SDEverywhere/issues/43)) ([0933a89](https://github.com/climateinteractive/SDEverywhere/commit/0933a89819cf91000354f45459556fcb212312f3)), closes [#34](https://github.com/climateinteractive/SDEverywhere/issues/34) +* cache parsed CSV file data and replace Array with Set to improve code gen performance ([#168](https://github.com/climateinteractive/SDEverywhere/issues/168)) ([58a45ba](https://github.com/climateinteractive/SDEverywhere/commit/58a45bafda4ccec754b5e60f4fa0045d60b5dcab)), closes [#167](https://github.com/climateinteractive/SDEverywhere/issues/167) +* improve code gen performance by avoiding linear searches ([#63](https://github.com/climateinteractive/SDEverywhere/issues/63)) ([d4bf555](https://github.com/climateinteractive/SDEverywhere/commit/d4bf555bd8c568af8c837eccd135e79a490d5c0f)), closes [#62](https://github.com/climateinteractive/SDEverywhere/issues/62) +* optimize IF THEN ELSE for cases where condition expression resolves to a constant ([#103](https://github.com/climateinteractive/SDEverywhere/issues/103)) ([f9ef675](https://github.com/climateinteractive/SDEverywhere/commit/f9ef67539938ed949a17022df05707a2c06c558a)), closes [#102](https://github.com/climateinteractive/SDEverywhere/issues/102) +* remove variables that are not referenced by input or output variables ([#44](https://github.com/climateinteractive/SDEverywhere/issues/44)) ([6c80c59](https://github.com/climateinteractive/SDEverywhere/commit/6c80c5919d94b9d66c2df3d27f181989ac864000)), closes [#1](https://github.com/climateinteractive/SDEverywhere/issues/1) + +## 0.5.3 (2020-07-29) + +- improved performance of LOOKUP +- optimized dimension name references to avoid array accesses +- changed lookup initialization to use static arrays for improved Wasm performance +- replaced wrapper functions with C macros to reduce function call overhead +- split large functions reduce stack frame size (improves Wasm memory use and performance) + +## 0.5.2 (2020-06-03) + +- includes fixes that more fully automate conversion of complicated MDL model files +- moved tools to Python 3 +- use the updated ANTLR-Version package +- improved support for two-dimensional arrays +- added handling of 2D constant arrays with subscripts in any order +- added support for dimension name references +- added support for ELMCOUNT +- added support for PULSE TRAIN +- updated documentation +- updated npm package dependencies + +## 0.5.1 (2019-09-27) + +- support multiple chartDatfiles delimited by semicolons in app.csv +- override generated app styles in an optional custom.css file in the config folder +- add optional varname prefix to readDat + +## 0.5.0 (2019-07-24) + +- web app generation uses simpler CSV configuration instead of YAML +- three-dimensional arrays +- :EXCEPT: subscripts +- two-dimensional const arrays +- GET DIRECT DATA for Excel at code generation time +- read output variables from DAT files to WITH LOOKUP variables +- generate variable documention in text and YAML formats +- allow all special characters in variable names +- improved coverage of subrange and mapping edge cases + +## 0.4.1 (2018-03-11) + +- enable a blank cell in the HTML input panel with an empty value in `sliders` +- add `sde causes` command to print model variable dependencies +- fixed HTML generation on Linux +- corrected instructions for building from the repo + +## 0.4.0 (2018-02-05) + +- updated web app generation to use an improved template +- added new app.yaml web app specification file +- generate complete web app with the `sde generate --genhtml` command +- removed the Vensim grammar to an independent package +- removed the lotka sample model +- added the SIR sample model +- optimized performance by making high-precision floating point comparisons optional +- added support for generating code to run the model interactively +- removed unnecessary glib2 dependency +- added a warning message when an input or output variable does not exist in the model +- fill in all ref ids for a generated variable that is expanded over a non-apply-to-all array +- implement Vensim data variables in DAT files with lookups diff --git a/packages/create/LICENSE b/packages/create/LICENSE new file mode 100644 index 00000000..29bed4e9 --- /dev/null +++ b/packages/create/LICENSE @@ -0,0 +1,21 @@ +The MIT License (MIT) + +Copyright (c) 2022 Climate Interactive / New Venture Fund + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/packages/create/README.md b/packages/create/README.md new file mode 100644 index 00000000..2c8842be --- /dev/null +++ b/packages/create/README.md @@ -0,0 +1,20 @@ +# @sdeverywhere/create + +Create a new SDEverywhere project with minimal configuration. + +## Quick Start + +```sh +# with npm +npm create @sdeverywhere + +# with pnpm +pnpm create @sdeverywhere + +# with yarn +yarn create @sdeverywhere +``` + +## License + +SDEverywhere is distributed under the MIT license. See `LICENSE` for more details. diff --git a/packages/create/bin/create-sde.js b/packages/create/bin/create-sde.js new file mode 100755 index 00000000..20b69186 --- /dev/null +++ b/packages/create/bin/create-sde.js @@ -0,0 +1,14 @@ +#!/usr/bin/env node +'use strict' + +const currentVersion = process.versions.node +const requiredMajorVersion = parseInt(currentVersion.split('.')[0], 10) +const minimumMajorVersion = 14 + +if (requiredMajorVersion < minimumMajorVersion) { + console.error(`Node.js v${currentVersion} is not supported by SDEverywhere.`) + console.error(`Please use Node.js v${minimumMajorVersion} or higher.`) + process.exit(1) +} + +import('../dist/index.js').then(({ main }) => main()) diff --git a/packages/create/package.json b/packages/create/package.json new file mode 100644 index 00000000..65991276 --- /dev/null +++ b/packages/create/package.json @@ -0,0 +1,54 @@ +{ + "name": "@sdeverywhere/create", + "version": "0.1.0", + "description": "Create a new SDEverywhere project with minimal configuration", + "type": "module", + "files": [ + "dist/**", + "!.DS_Store" + ], + "bin": { + "create-sde": "bin/create-sde.js" + }, + "scripts": { + "clean": "rm -rf dist", + "lint": "eslint src --ext .ts --max-warnings 0", + "prettier:check": "prettier --check .", + "prettier:fix": "prettier --write .", + "precommit": "../../scripts/precommit", + "test": "vitest run", + "test:watch": "vitest", + "test:ci": "vitest run", + "type-check": "tsc --noEmit -p tsconfig-build.json", + "build": "tsup", + "start": "./bin/create-sde.js", + "ci:build": "run-s clean lint prettier:check type-check build test:ci" + }, + "dependencies": { + "degit": "^2.8.4", + "execa": "^6.1.0", + "kleur": "^4.1.5", + "ora": "^6.1.2", + "prompts": "^2.4.2", + "which-pm-runs": "^1.1.0", + "yargs-parser": "^21.1.1" + }, + "devDependencies": { + "@types/degit": "^2.8.3", + "@types/node": "^16.11.7", + "@types/prompts": "^2.0.14", + "@types/which-pm-runs": "^1.0.0", + "@types/yargs-parser": "^21.0.0" + }, + "author": "Climate Interactive", + "license": "MIT", + "homepage": "https://sdeverywhere.org", + "repository": { + "type": "git", + "url": "https://github.com/climateinteractive/SDEverywhere.git", + "directory": "packages/create" + }, + "bugs": { + "url": "https://github.com/climateinteractive/SDEverywhere/issues" + } +} diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts new file mode 100644 index 00000000..f9b4cdc5 --- /dev/null +++ b/packages/create/src/index.ts @@ -0,0 +1,328 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { existsSync } from 'fs' +import { relative, resolve as resolvePath } from 'path' + +import degit from 'degit' +import { execa, execaCommand } from 'execa' +import { bgCyan, black, bold, cyan, dim, green, red, reset, yellow } from 'kleur/colors' +import type { Ora } from 'ora' +import ora from 'ora' +import prompts from 'prompts' +import detectPackageManager from 'which-pm-runs' +import yargs from 'yargs-parser' + +export const TEMPLATES = [ + { + title: 'Default project', + description: 'Includes recommended structure with config files, app, core library, model-check, etc', + value: 'template-default' + }, + { + title: 'Minimal project', + description: 'Includes simple config for model-check', + value: 'template-minimal' + } +] + +// Detect the package manager +const pkgManager = detectPackageManager()?.name || 'npm' + +// Parse command line arguments +const args = yargs(process.argv) +prompts.override(args) + +async function chooseProjectDir(): Promise { + /** + * Return true if the given directory does not exist, or it does not contain important files + * like `package.json`. + */ + function isValidDir(dir: string): boolean { + const packageJson = resolvePath(dir, 'package.json') + const packagesDir = resolvePath(dir, 'packages') + return !existsSync(dir) || (!existsSync(packageJson) && !existsSync(packagesDir)) + } + + const showValidDirMsg = (dir: string) => { + const ack = ora({ + color: 'green', + text: `Using "${bold(dir)}" as project directory.` + }) + ack.succeed() + } + + const showInvalidDirMsg = (dir: string) => { + const reject = ora({ + color: 'red', + text: `"${bold(dir)}" contains existing 'package.json' and/or 'packages' directory, stopping.` + }) + reject.fail() + } + + // See if project directory is provided on command line + let projDir = args['_'][2] as string + if (projDir) { + // Directory was provided, see if it is valid + if (isValidDir(projDir)) { + // The provided directory is valid, so proceed + showValidDirMsg(projDir) + } else { + // The provided directory is not valid, so show error message and exit + showInvalidDirMsg(projDir) + process.exit(1) + } + } else { + // Directory was not provided, so prompt the user + const dirResponse = await prompts( + { + type: 'text', + name: 'directory', + message: 'Where would you like to create your new project?', + initial: '' + // validate(value) { + // if (value === '') { + // value = process.cwd() + // } + // if (!isValidDir(value)) { + // return notValidMsg(value) + // } + // return true + // } + }, + { onCancel: () => ora().info(dim('Operation cancelled.')) } + ) + projDir = dirResponse.directory + if (projDir === '') { + projDir = process.cwd() + } + if (isValidDir(projDir)) { + showValidDirMsg(projDir) + } else { + showInvalidDirMsg(projDir) + process.exit(1) + } + } + + return projDir +} + +// XXX: This is mostly copied from Astro's create package: +// https://github.com/withastro/astro/blob/main/packages/create-astro/src/index.ts +// It contains workarounds for degit issues that may or may not be relevant for SDE, +// so this should be re-evaluated later. +async function runDegit(templateTarget: string, hash: string, dstDir: string, spinner: Ora): Promise { + const emitter = degit(`${templateTarget}${hash}`, { + cache: false, + force: true + }) + + try { + // emitter.on('info', info => { + // logger.debug(info.message) + // }) + await emitter.clone(dstDir) + + // degit does not return an error when an invalid template is provided, as such we + // need to handle this manually + // if (isEmpty(cwd)) { + // fs.rmdirSync(cwd) + // throw new Error(`Error: The provided template (${cyan(options.template)}) does not exist`) + // } + } catch (e) { + spinner.fail() + + // degit is compiled, so the stacktrace is pretty noisy. Only report the stacktrace when using verbose mode. + // logger.debug(err) + console.error(red(e.message)) + + // TODO: Handle common degit issues like below; for now, just log the error and exit + console.error(yellow('There was a problem copying the template.')) + console.error( + yellow( + 'Please file a new issue with the command output here: https://github.com/climateinteractive/sdeverywhere/issues' + ) + ) + process.exit(1) + + // // Warning for issue #655 and other corrupted cache issue + // if (e.message === 'zlib: unexpected end of file' || e.message === 'TAR_BAD_ARCHIVE: Unrecognized archive format') { + // console.log( + // yellow( + // // 'Local degit cache seems to be corrupted.' + // // 'For more information check out this issue: https://github.com/withastro/astro/issues/655.' + // ) + // ) + // const cacheIssueResponse = await prompts({ + // type: 'confirm', + // name: 'cache', + // message: 'Would you like us to clear the cache and try again?', + // initial: true + // }) + // if (cacheIssueResponse.cache) { + // const homeDirectory = os.homedir() + // const cacheDir = joinPath(homeDirectory, '.degit', 'github', '@sdeverywhere') + // rmSync(cacheDir, { recursive: true, force: true, maxRetries: 3 }) + // spinner = ora('Copying project files...').start() + // try { + // await emitter.clone(dstDir) + // } catch (e) { + // // logger.debug(e) + // console.error(red(e.message)) + // } + // } else { + // console.log( + // "Okay, no worries! To fix this manually, remove the folder '~/.degit/github/withastro' and rerun the command." + // ) + // } + // } + + // // Helpful message when encountering the "could not find commit hash for ..." error + // if (e.code === 'MISSING_REF') { + // console.log( + // yellow( + // "This seems to be an issue with degit. Please check if you have 'git' installed on your system, and if you don't, go here to install: https://git-scm.com" + // ) + // ) + // console.log( + // yellow( + // "If you do have 'git' installed, please file a new issue with the command output here: https://github.com/climateinteractive/sdeverywhere/issues" + // ) + // ) + // } + + // process.exit(1) + } +} + +async function chooseTemplate(projDir: string): Promise { + // Prompt the user + const options = await prompts( + [ + { + type: 'select', + name: 'template', + message: 'Which template would you like to use?', + choices: TEMPLATES + } + ], + { onCancel: () => ora().info(dim('Operation cancelled.')) } + ) + if (!options.template) { + process.exit(1) + } + + // Handle response + const templateSpinner = ora('Copying project files...').start() + const templateTarget = `climateinteractive/sdeverywhere/examples/${options.template}` + // TODO: Fix this before branch is merged + const hash = '#chris/228-create-package' + + // Copy the template files to the project directory + if (!args.dryRun) { + await runDegit(templateTarget, hash, projDir, templateSpinner) + } + + templateSpinner.text = green('Template copied!') + templateSpinner.succeed() +} + +async function chooseInstall(projDir: string): Promise { + // Prompt the user + const installResponse = await prompts( + { + type: 'confirm', + name: 'install', + message: `Would you like to install ${pkgManager} dependencies? ${reset(dim('(recommended)'))}`, + initial: true + }, + { + onCancel: () => { + ora().info( + dim('Operation cancelled. Your project folder has been created, but no dependencies have been installed.') + ) + process.exit(1) + } + } + ) + + // Handle response + if (args.dryRun) { + ora().info(dim(`--dry-run enabled, skipping.`)) + } else if (installResponse.install) { + const installExec = execa(pkgManager, ['install'], { cwd: projDir }) + const installingPackagesMsg = 'Installing packages...' + const installSpinner = ora(installingPackagesMsg).start() + await new Promise((resolve, reject) => { + installExec.stdout?.on('data', function (data) { + installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` + }) + installExec.on('error', error => reject(error)) + installExec.on('close', () => resolve()) + }) + installSpinner.text = green('Packages installed!') + installSpinner.succeed() + } else { + ora().info(dim(`No problem! Remember to install dependencies after setup.`)) + } +} + +async function chooseGitInit(projDir: string): Promise { + // Prompt the user + const gitResponse = await prompts( + { + type: 'confirm', + name: 'git', + message: `Would you like to initialize a new git repository? ${reset(dim('(optional)'))}`, + initial: true + }, + { + onCancel: () => { + ora().info(dim('Operation cancelled. Your project folder has already been created.')) + process.exit(1) + } + } + ) + if (args.dryRun) { + ora().info(dim(`--dry-run enabled, skipping.`)) + } else if (gitResponse.git) { + await execaCommand('git init', { cwd: projDir }) + ora().succeed('Git repository created!') + } else { + ora().info(dim(`No problem! You can come back and run ${cyan(`git init`)} later.`)) + } +} + +export async function main(): Promise { + // Display welcome message + console.log(`\n${bold('Welcome to SDEverywhere!')}`) + console.log(`Let's create a new SDEverywhere project for your model.\n`) + + // Prompt the user to select a project directory + const projDir = await chooseProjectDir() + + // Prompt the user to select a template + await chooseTemplate(projDir) + + // Prompt the user to install dependencies + await chooseInstall(projDir) + + // Prompt the user to initialize a git repo + await chooseGitInit(projDir) + + // TODO: See if project contains an mdl file, if not, use hello-world + // TODO: Set up sde.config.js file to use selected mdl file + + ora({ text: green('Setup complete!') }).succeed() + + console.log(`\n${bgCyan(black(' Next steps '))}\n`) + + const relProjDir = relative(process.cwd(), projDir) + const devCmd = pkgManager === 'npm' ? 'npm run dev' : `${pkgManager} dev` + + // If the project dir is the current dir, no need to tell users to cd + if (relProjDir !== '') { + console.log(`You can now ${bold(cyan('cd'))} into the ${bold(cyan(relProjDir))} project directory.`) + } + console.log(`Run ${bold(cyan(devCmd))} to start the local dev server. ${bold(cyan('CTRL-C'))} to close.`) + console.log('') +} diff --git a/packages/create/tests/create-dirs.spec.ts b/packages/create/tests/create-dirs.spec.ts new file mode 100644 index 00000000..a55a9b54 --- /dev/null +++ b/packages/create/tests/create-dirs.spec.ts @@ -0,0 +1,116 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { existsSync } from 'fs' +import { mkdir } from 'fs/promises' +import { dirname, resolve as resolvePath } from 'path' +import { fileURLToPath } from 'url' + +import { execa } from 'execa' +import { describe, it } from 'vitest' + +const testsDir = dirname(fileURLToPath(import.meta.url)) +const dirs = { + nonExistent: './fixtures/non-existent-dir', + nonEmpty: './fixtures/non-empty-dir', + empty: './fixtures/empty-dir' +} + +const promptMessages = { + directory: 'Where would you like to create your new project?', + template: 'Which template would you like to use?' +} + +function runCreate(args: string[] = []) { + const { stdout, stdin } = execa('../bin/create-sde.js', [...args, '--dryrun'], { cwd: testsDir }) + return { + stdin, + stdout + } +} + +describe('create directories', () => { + // it('should stop if directory provided on command line contains important files', () => { + // // TODO + // }) + + it('should proceed if directory provided on command line is empty', async () => { + const emptyDir = resolvePath(testsDir, dirs.empty) + if (!existsSync(emptyDir)) { + await mkdir(emptyDir) + } + return new Promise(resolve => { + const { stdout } = runCreate([dirs.empty]) + stdout?.on('data', chunk => { + if (chunk.includes(promptMessages.template)) { + resolve(undefined) + } + }) + }) + }) + + it('should proceed if directory provided on command line does not exist', () => { + return new Promise(resolve => { + const { stdout } = runCreate([dirs.nonExistent]) + stdout?.on('data', chunk => { + if (chunk.includes(promptMessages.template)) { + resolve(undefined) + } + }) + }) + }) + + it('should prompt for directory when none is provided on command line', () => { + return new Promise(resolve => { + const { stdout } = runCreate() + stdout?.on('data', chunk => { + if (chunk.includes(promptMessages.directory)) { + resolve(undefined) + } + }) + }) + }) + + // TODO: The child process should exit in this case; need to check exit code + // it('should stop if directory provided at prompt contains important files', () => { + // return new Promise(resolve => { + // const { stdout, stdin } = runCreate() + // stdout?.on('data', chunk => { + // console.log(chunk.toString()) + // if (chunk.includes('contains existing')) { + // resolve(undefined) + // } + // if (chunk.includes(promptMessages.directory)) { + // stdin?.write(`${dirs.nonEmpty}\x0D`) + // } + // }) + // }) + // }) + + it('should proceed if directory provided at prompt is empty', async () => { + return new Promise(resolve => { + const { stdout, stdin } = runCreate() + stdout?.on('data', chunk => { + if (chunk.includes(promptMessages.template)) { + resolve(undefined) + } + if (chunk.includes(promptMessages.directory)) { + stdin?.write(`${dirs.empty}\x0D`) + } + }) + }) + }) + + it('should proceed if directory provided at prompt does not exist', () => { + return new Promise(resolve => { + const { stdout, stdin } = runCreate() + stdout?.on('data', chunk => { + if (chunk.includes(promptMessages.template)) { + resolve(undefined) + } + if (chunk.includes(promptMessages.directory)) { + stdin?.write(`${dirs.nonExistent}\x0D`) + } + }) + }) + }) +}) diff --git a/packages/create/tests/fixtures/non-empty-dir/packages/empty-file.txt b/packages/create/tests/fixtures/non-empty-dir/packages/empty-file.txt new file mode 100644 index 00000000..e69de29b diff --git a/packages/create/tsconfig-base.json b/packages/create/tsconfig-base.json new file mode 100644 index 00000000..01a7d112 --- /dev/null +++ b/packages/create/tsconfig-base.json @@ -0,0 +1,17 @@ +// This contains the TypeScript configuration that is shared between +// testing (`tsconfig-test.json`) and production builds (`tsconfig-build.json`). +{ + "extends": "../../tsconfig-common.json", + "compilerOptions": { + "outDir": "./dist", + // Use "es2021" because this is the ES version for Node 16 + "target": "es2021", + // Use "es2020" because this is a Node module (using ESM) and we need to + // use dynamic imports + "module": "es2020", + "noImplicitAny": true, + "noUnusedLocals": true, + "noUnusedParameters": true, + "types": ["node"] + } +} diff --git a/packages/create/tsconfig-build.json b/packages/create/tsconfig-build.json new file mode 100644 index 00000000..c16db5ed --- /dev/null +++ b/packages/create/tsconfig-build.json @@ -0,0 +1,6 @@ +// This contains the TypeScript configuration for production builds. +{ + "extends": "./tsconfig-base.json", + "include": ["src/**/*"], + "exclude": ["src/**/_mocks/**/*", "**/*.spec.ts"] +} diff --git a/packages/create/tsconfig-test.json b/packages/create/tsconfig-test.json new file mode 100644 index 00000000..7bd4cad8 --- /dev/null +++ b/packages/create/tsconfig-test.json @@ -0,0 +1,5 @@ +// This contains the TypeScript configuration for testing. +{ + "extends": "./tsconfig-base.json", + "include": ["src/**/*"] +} diff --git a/packages/create/tsconfig.json b/packages/create/tsconfig.json new file mode 100644 index 00000000..fc8b16a2 --- /dev/null +++ b/packages/create/tsconfig.json @@ -0,0 +1,6 @@ +// This contains the TypeScript configuration for local development and testing. +// It is used as the TypeScript config for tools like VSCode that look for +// `tsconfig.json` by default. +{ + "extends": "./tsconfig-test.json" +} diff --git a/packages/create/tsup.config.ts b/packages/create/tsup.config.ts new file mode 100644 index 00000000..7b62ef95 --- /dev/null +++ b/packages/create/tsup.config.ts @@ -0,0 +1,11 @@ +import { defineConfig } from 'tsup' + +export default defineConfig({ + tsconfig: 'tsconfig-build.json', + entry: ['src/index.ts'], + format: ['esm', 'cjs'], + dts: true, + splitting: false, + sourcemap: true, + clean: true +}) diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 57cdb985..23260001 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -246,6 +246,35 @@ importers: split-string: 6.1.0 xlsx: 0.17.5 + packages/create: + specifiers: + '@types/degit': ^2.8.3 + '@types/node': ^16.11.7 + '@types/prompts': ^2.0.14 + '@types/which-pm-runs': ^1.0.0 + '@types/yargs-parser': ^21.0.0 + degit: ^2.8.4 + execa: ^6.1.0 + kleur: ^4.1.5 + ora: ^6.1.2 + prompts: ^2.4.2 + which-pm-runs: ^1.1.0 + yargs-parser: ^21.1.1 + dependencies: + degit: 2.8.4 + execa: 6.1.0 + kleur: 4.1.5 + ora: 6.1.2 + prompts: 2.4.2 + which-pm-runs: 1.1.0 + yargs-parser: 21.1.1 + devDependencies: + '@types/degit': 2.8.3 + '@types/node': 16.11.40 + '@types/prompts': 2.0.14 + '@types/which-pm-runs': 1.0.0 + '@types/yargs-parser': 21.0.0 + packages/plugin-check: specifiers: '@rollup/plugin-node-resolve': ^13.3.0 @@ -594,6 +623,10 @@ packages: resolution: {integrity: sha512-EGlKlgMhnLt/cM4DbUSafFdrkeJoC9Mvnj0PUCU7tFmTjMjNRT957kXCx0wYm3JuEq4o4ZsS5vG+NlkM2DMd2A==} dev: true + /@types/degit/2.8.3: + resolution: {integrity: sha512-CL7y71j2zaDmtPLD5Xq5S1Gv2dFoHl0/GBZm6s39Mj/ls28L3NzAOqf7H4H0/2TNVMgMjMVf9CAFYSjmXhi3bw==} + dev: true + /@types/estree/0.0.39: resolution: {integrity: sha512-EYNwp3bU+98cpU4lAWYYL7Zz+2gryWH1qbdDTidVd6hkiR6weksdbMadyXKXNPEkQFhXM+hVO9ZygomHXp+AIw==} dev: false @@ -621,6 +654,12 @@ packages: /@types/node/17.0.42: resolution: {integrity: sha512-Q5BPGyGKcvQgAMbsr7qEGN/kIPN6zZecYYABeTDBizOsau+2NMdSVTar9UQw21A2+JyA2KRNDYaYrPB0Rpk2oQ==} + /@types/prompts/2.0.14: + resolution: {integrity: sha512-HZBd99fKxRWpYCErtm2/yxUZv6/PBI9J7N4TNFffl5JbrYMHBwF25DjQGTW3b3jmXq+9P6/8fCIb2ee57BFfYA==} + dependencies: + '@types/node': 17.0.42 + dev: true + /@types/pug/2.0.6: resolution: {integrity: sha512-SnHmG9wN1UVmagJOnyo/qkk0Z7gejYxOYYmaAwr5u2yFYfsupN3sg10kyzN8Hep/2zbHxCnsumxOoRIRMBwKCg==} dev: true @@ -649,6 +688,14 @@ packages: '@types/node': 17.0.42 dev: true + /@types/which-pm-runs/1.0.0: + resolution: {integrity: sha512-BXfdlYLWvRhngJbih4N57DjO+63Z7AxiFiip8yq3rD46U7V4I2W538gngPvBsZiMehhD8sfGf4xLI6k7TgXvNw==} + dev: true + + /@types/yargs-parser/21.0.0: + resolution: {integrity: sha512-iO9ZQHkZxHn4mSakYV0vFHAVDyEOIJQrV2uZ06HxEPcx+mt8swXoZHIbaaJ2crJYFfErySgktuTZ3BeLz+XmFA==} + dev: true + /@typescript-eslint/eslint-plugin/5.27.1_aq7uryhocdbvbqum33pitcm3y4: resolution: {integrity: sha512-6dM5NKT57ZduNnJfpY81Phe9nc9wolnMCnknb1im6brWi1RYv84nbMS3olJa27B6+irUVV1X/Wb+Am0FjJdGFw==} engines: {node: ^12.22.0 || ^14.17.0 || >=16.0.0} @@ -839,6 +886,11 @@ packages: resolution: {integrity: sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==} engines: {node: '>=8'} + /ansi-regex/6.0.1: + resolution: {integrity: sha512-n5M855fKb2SsfMIiFFoVrABHJC8QtHwVx+mHWP3QcEqBHYienj5dHSgjbxtC0WEZXYt4wcD6zrQElDPhFuZgfA==} + engines: {node: '>=12'} + dev: false + /ansi-styles/3.2.1: resolution: {integrity: sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==} engines: {node: '>=4'} @@ -910,10 +962,22 @@ packages: /balanced-match/1.0.2: resolution: {integrity: sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==} + /base64-js/1.5.1: + resolution: {integrity: sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==} + dev: false + /binary-extensions/2.2.0: resolution: {integrity: sha512-jDctJ/IVQbZoJykoeHbhXpOlNBqGNcwXJKJog42E5HDPUwQTSdjCHdihjj0DlnheQ7blbT6dHOafNAiS8ooQKA==} engines: {node: '>=8'} + /bl/5.0.0: + resolution: {integrity: sha512-8vxFNZ0pflFfi0WXA3WQXlj6CaMEwsmh63I1CNp0q+wWv8sD0ARx1KovSQd0l2GkwrMIOyedq0EF1FxI+RCZLQ==} + dependencies: + buffer: 6.0.3 + inherits: 2.0.4 + readable-stream: 3.6.0 + dev: false + /bootstrap-slider/10.6.2: resolution: {integrity: sha512-8JTPZB9QVOdrGzYF3YgC3YW6ssfPeBvBwZnXffiZ7YH/zz1D0EKlZvmQsm/w3N0XjVNYQEoQ0ax+jHrErV4K1Q==} dev: false @@ -939,6 +1003,13 @@ packages: resolution: {integrity: sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ==} dev: true + /buffer/6.0.3: + resolution: {integrity: sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==} + dependencies: + base64-js: 1.5.1 + ieee754: 1.2.1 + dev: false + /bufx/1.0.5: resolution: {integrity: sha512-AzOd+vXDVhRAIR4k0ZopOLef+XjqLU6h3buAqVXTUrZ5IYWmoPqLtIoITeye174Uq5qiDS+83Rx9U9ItXgNE+A==} dependencies: @@ -1022,6 +1093,11 @@ packages: supports-color: 7.2.0 dev: true + /chalk/5.0.1: + resolution: {integrity: sha512-Fo07WOYGqMfCWHOzSXOt2CxDbC6skS/jO9ynEcmpANMoPrD+W1r1K6Vx7iNm+AQmETU1Xr2t+n8nzkV9t6xh3w==} + engines: {node: ^12.17.0 || ^14.13 || >=16.0.0} + dev: false + /character-parser/2.2.0: resolution: {integrity: sha512-+UqJQjFEFaTAs3bNsF2j2kEN1baG/zghZbdqoYEDxGZtJo9LBzl1A+m0D4n3qKx8N2FNv8/Xp6yV9mQmBuptaw==} dependencies: @@ -1066,6 +1142,18 @@ packages: optionalDependencies: fsevents: 2.3.2 + /cli-cursor/4.0.0: + resolution: {integrity: sha512-VGtlMu3x/4DOtIUwEkRezxUZ2lBacNJCHash0N0WeZDBS+7Ux1dm3XWAgWYxLJFMMdOeXMHXorshEFhbMSGelg==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + dependencies: + restore-cursor: 4.0.0 + dev: false + + /cli-spinners/2.7.0: + resolution: {integrity: sha512-qu3pN8Y3qHNgE2AFweciB1IfMnmZ/fsNTEE+NOFjmGB2F/7rLhnhzppvpCnN4FovtP26k8lHyy9ptEbNwWFLzw==} + engines: {node: '>=6'} + dev: false + /cliui/7.0.4: resolution: {integrity: sha512-OcRE68cOsVMXp1Yvonl/fzkQOyjLSu/8bhPDfQt0e0/Eb283TKP20Fs2MqoPsr9SwA595rRCA+QMzYc9nBP+JQ==} dependencies: @@ -1074,6 +1162,11 @@ packages: wrap-ansi: 7.0.0 dev: false + /clone/1.0.4: + resolution: {integrity: sha512-JQHZ2QMW6l3aH/j6xCqQThY/9OH4D/9ls34cgkUBiEeocRTU04tHfKPBsUK1PqZCUQM7GiA0IIXJSuXHI64Kbg==} + engines: {node: '>=0.8'} + dev: false + /codepage/1.15.0: resolution: {integrity: sha512-3g6NUTPd/YtuuGrhMnOMRjFc+LJw/bnMp3+0r/Wcz3IXUuCosKRJvMphm5+Q+bvTVGcJJuRvVLuYba+WojaFaA==} engines: {node: '>=0.8'} @@ -1181,6 +1274,12 @@ packages: engines: {node: '>=0.10.0'} dev: false + /defaults/1.0.3: + resolution: {integrity: sha512-s82itHOnYrN0Ib8r+z7laQz3sdE+4FP3d9Q7VLO7U+KRT+CR0GsWuyHxzdAY82I7cXv0G/twrqomTJLOssO5HA==} + dependencies: + clone: 1.0.4 + dev: false + /define-properties/1.1.4: resolution: {integrity: sha512-uckOqKcfaVvtBdsVkdPv3XjveQJsNQqmhXgRi8uhvWWuPYZCNlzT8qAyblUgNoXdHdjMTzAqeGjAoli8f+bzPA==} engines: {node: '>= 0.4'} @@ -1189,6 +1288,12 @@ packages: object-keys: 1.1.1 dev: true + /degit/2.8.4: + resolution: {integrity: sha512-vqYuzmSA5I50J882jd+AbAhQtgK6bdKUJIex1JNfEUPENCgYsxugzKVZlFyMwV4i06MmnV47/Iqi5Io86zf3Ng==} + engines: {node: '>=8.0.0'} + hasBin: true + dev: false + /detect-indent/6.1.0: resolution: {integrity: sha512-reYkTUJAZb9gUuZ2RvVCNhVHdg62RHnJ7WJl8ftMi4diZ6NWlciOzQN88pUhSELEwflJht4oQDv0F0BMlwaYtA==} engines: {node: '>=8'} @@ -1687,6 +1792,21 @@ packages: strip-final-newline: 2.0.0 dev: true + /execa/6.1.0: + resolution: {integrity: sha512-QVWlX2e50heYJcCPG0iWtf8r0xjEYfz/OYLGDYH+IyjWezzPNxz63qNFOu0l4YftGWuizFVZHHs8PrLU5p2IDA==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + dependencies: + cross-spawn: 7.0.3 + get-stream: 6.0.1 + human-signals: 3.0.1 + is-stream: 3.0.0 + merge-stream: 2.0.0 + npm-run-path: 5.1.0 + onetime: 6.0.0 + signal-exit: 3.0.7 + strip-final-newline: 3.0.0 + dev: false + /exit-on-epipe/1.0.1: resolution: {integrity: sha512-h2z5mrROTxce56S+pnvAV890uu7ls7f1kEvVGJbw1OlFH3/mlJ5bkXu0KRyW94v37zzHPiUd55iLn3DA7TjWpw==} engines: {node: '>=0.8'} @@ -1828,7 +1948,6 @@ packages: /get-stream/6.0.1: resolution: {integrity: sha512-ts6Wi+2j3jQjqi70w5AlN8DFnkSwC+MqmxEzdEALB2qXZYV3X/b1CTfgPLGJNMeAWxdPfU8FO1ms3NUfaHCPYg==} engines: {node: '>=10'} - dev: true /get-symbol-description/1.0.0: resolution: {integrity: sha512-2EmdH1YvIQiZpltCNgkuiUnyukzxM/R6NDJX31Ke3BG1Nq5b0S2PhX59UKi9vZpPDQVdqn+1IcaAwnzTT5vCjw==} @@ -1981,6 +2100,15 @@ packages: engines: {node: '>=10.17.0'} dev: true + /human-signals/3.0.1: + resolution: {integrity: sha512-rQLskxnM/5OCldHo+wNXbpVgDn5A17CUoKX+7Sokwaknlq7CdSnphy0W39GU8dw59XiCXmFXDg4fRuckQRKewQ==} + engines: {node: '>=12.20.0'} + dev: false + + /ieee754/1.2.1: + resolution: {integrity: sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==} + dev: false + /ignore/5.2.0: resolution: {integrity: sha512-CmxgYGiEPCLhfLnpPp1MoRmifwEIOgjcHXxOBjv7mY96c+eWScsOP9c112ZyLdWHi0FxHjI+4uVhKYp/gcdRmQ==} engines: {node: '>= 4'} @@ -2096,6 +2224,11 @@ packages: dependencies: is-extglob: 2.1.1 + /is-interactive/2.0.0: + resolution: {integrity: sha512-qP1vozQRI+BMOPcjFzrjXuQvdak2pHNUMZoeG2eRbiSqyvbEf/wQtEOTOX1guk6E3t36RkaqiSt8A/6YElNxLQ==} + engines: {node: '>=12'} + dev: false + /is-module/1.0.0: resolution: {integrity: sha512-51ypPSPCoTEIN9dy5Oy+h4pShgJmPCygKfyRCISBI+JoWT/2oJvK8QPxmwv7b/p239jXrm9M1mlQbyKJ5A152g==} dev: false @@ -2149,6 +2282,11 @@ packages: engines: {node: '>=8'} dev: true + /is-stream/3.0.0: + resolution: {integrity: sha512-LnQR4bZ9IADDRSkvpqMGvt/tEJWclzklNgSw48V5EAaAeDd6qGvN8ei6k5p0tvxSR171VmGyHuTiAOfxAbr8kA==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + dev: false + /is-string/1.0.7: resolution: {integrity: sha512-tE2UXzivje6ofPW7l23cjDOMa09gb7xlAqG6jG5ej6uPV32TlWP3NKPigtaGeHNu9fohccRYvIiZMfOOnOYUtg==} engines: {node: '>= 0.4'} @@ -2163,6 +2301,11 @@ packages: has-symbols: 1.0.3 dev: true + /is-unicode-supported/1.3.0: + resolution: {integrity: sha512-43r2mRvz+8JRIKnWJ+3j8JtjRKZ6GmjzfaE/qiBJnikNnYv/6bagRJ1kUhNk8R5EX/GkobD+r+sfxCPJsiKBLQ==} + engines: {node: '>=12'} + dev: false + /is-weakref/1.0.2: resolution: {integrity: sha512-qctsuLZmIQ0+vSSMfoVvyFe2+GSEvnmZ2ezTup1SBse9+twCCeial6EEi3Nc2KFcf6+qz2FBPnjXsk8xhKSaPQ==} dependencies: @@ -2227,10 +2370,20 @@ packages: promise: 7.3.1 dev: true + /kleur/3.0.3: + resolution: {integrity: sha512-eTIzlVOSUR+JxdDFepEYcBMtZ9Qqdef+rnzWdRZuMbOywu5tO2w2N7rqjoANZ5k9vywhL6Br1VRjUIgTQx4E8w==} + engines: {node: '>=6'} + dev: false + /kleur/4.1.4: resolution: {integrity: sha512-8QADVssbrFjivHWQU7KkMgptGTl6WAcSdlbBPY4uNF+mWr6DGcKrvY2w4FQJoXch7+fKMjj0dRrL75vk3k23OA==} engines: {node: '>=6'} + /kleur/4.1.5: + resolution: {integrity: sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ==} + engines: {node: '>=6'} + dev: false + /levn/0.4.1: resolution: {integrity: sha512-+bT2uH4E5LGE7h/n3evcS/sQlJXCpIp6ym8OWJ5eV6+67Dsql/LaaT7qJBAt2rzfoa/5QBGBhxDix1dMt2kQKQ==} engines: {node: '>= 0.8.0'} @@ -2288,6 +2441,14 @@ packages: resolution: {integrity: sha512-HDWXG8isMntAyRF5vZ7xKuEvOhT4AhlRt/3czTSjvGUxjYCBVRQY48ViDHyfYz9VIoBkW4TMGQNapx+l3RUwdA==} dev: true + /log-symbols/5.1.0: + resolution: {integrity: sha512-l0x2DvrW294C9uDCoQe1VSU4gf529FkSZ6leBl4TiqZH/e+0R7hSfHQBNut2mNygDgHwvYHfFLn6Oxb3VWj2rA==} + engines: {node: '>=12'} + dependencies: + chalk: 5.0.1 + is-unicode-supported: 1.3.0 + dev: false + /loupe/2.3.4: resolution: {integrity: sha512-OvKfgCC2Ndby6aSTREl5aCCPTNIzlDfQZvZxNUrBrihDhL3xcrYegTblhmEiCrg2kKQz4XsFIaemE5BF4ybSaQ==} dependencies: @@ -2329,7 +2490,6 @@ packages: /merge-stream/2.0.0: resolution: {integrity: sha512-abv/qOcuPfk3URPfDzmZU1LKmuw8kT+0nIHvKrKgFrwifol/doWcdA4ZqsWQ8ENrFKkd67Mfpo/LovbIUsbt3w==} - dev: true /merge2/1.4.1: resolution: {integrity: sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg==} @@ -2345,7 +2505,11 @@ packages: /mimic-fn/2.1.0: resolution: {integrity: sha512-OqbOk5oEQeAZ8WXWydlu9HJjz9WVdEIvamMCcXmuqUYjTknH/sqsWvhQ3vgwKFRR1HpjvNBKQ37nbJgYzGqGcg==} engines: {node: '>=6'} - dev: true + + /mimic-fn/4.0.0: + resolution: {integrity: sha512-vqiC06CuhBTUdZH+RYl8sFrL096vA45Ok5ISO6sE/Mr1jRbGH4Csnhi8f3wKVl7x8mO4Au7Ir9D3Oyv1VYMFJw==} + engines: {node: '>=12'} + dev: false /min-indent/1.0.1: resolution: {integrity: sha512-I9jwMn07Sy/IwOj3zVkVik2JTvgpaykDZEigL6Rx6N9LbMywwUSMtxET+7lVoDLLd3O3IXwJwvuuns8UB/HeAg==} @@ -2466,6 +2630,13 @@ packages: path-key: 3.1.1 dev: true + /npm-run-path/5.1.0: + resolution: {integrity: sha512-sJOdmRGrY2sjNTRMbSvluQqg+8X7ZK61yvzBEIDhz4f8z1TZFYABsqjjCBd/0PUNE9M6QDgHJXQkGUEm7Q+l9Q==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + dependencies: + path-key: 4.0.0 + dev: false + /object-assign/4.1.1: resolution: {integrity: sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==} engines: {node: '>=0.10.0'} @@ -2504,7 +2675,13 @@ packages: engines: {node: '>=6'} dependencies: mimic-fn: 2.1.0 - dev: true + + /onetime/6.0.0: + resolution: {integrity: sha512-1FlR+gjXK7X+AsAHso35MnyN5KqGwJRi/31ft6x0M194ht7S+rWAvd7PHss9xSKMzE0asv1pyIHaJYq+BbacAQ==} + engines: {node: '>=12'} + dependencies: + mimic-fn: 4.0.0 + dev: false /optionator/0.9.1: resolution: {integrity: sha512-74RlY5FCnhq4jRxVUPKDaRwrVNXMqsGsiW6AJw4XK8hmtm10wC0ypZBLw5IIp85NZMr91+qd1RvvENwg7jjRFw==} @@ -2518,6 +2695,21 @@ packages: word-wrap: 1.2.3 dev: true + /ora/6.1.2: + resolution: {integrity: sha512-EJQ3NiP5Xo94wJXIzAyOtSb0QEIAUu7m8t6UZ9krbz0vAJqr92JpcK/lEXg91q6B9pEGqrykkd2EQplnifDSBw==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + dependencies: + bl: 5.0.0 + chalk: 5.0.1 + cli-cursor: 4.0.0 + cli-spinners: 2.7.0 + is-interactive: 2.0.0 + is-unicode-supported: 1.3.0 + log-symbols: 5.1.0 + strip-ansi: 7.0.1 + wcwidth: 1.0.1 + dev: false + /p-limit/4.0.0: resolution: {integrity: sha512-5b0R4txpzjPWVw/cXXUResoD4hb6U/x9BH08L7nw+GN1sezDzPdxeRvpc9c433fZhBan/wusjbCsqwqm4EIBIQ==} engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} @@ -2569,6 +2761,11 @@ packages: resolution: {integrity: sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==} engines: {node: '>=8'} + /path-key/4.0.0: + resolution: {integrity: sha512-haREypq7xkM7ErfgIyA0z+Bj4AGKlMSdlQE2jvJo6huWD1EdkKYV+G/T4nq0YEF2vgTT8kqMFKo1uHn950r4SQ==} + engines: {node: '>=12'} + dev: false + /path-parse/1.0.7: resolution: {integrity: sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==} @@ -2657,6 +2854,14 @@ packages: asap: 2.0.6 dev: true + /prompts/2.4.2: + resolution: {integrity: sha512-NxNv/kLguCA7p3jE8oL2aEBsrJWgAakBpgmgK6lpPWV+WuOmY6r2/zbAVnP+T8bQlA0nzHXSJSJW0Hq7ylaD2Q==} + engines: {node: '>= 6'} + dependencies: + kleur: 3.0.3 + sisteransi: 1.0.5 + dev: false + /pug-attrs/3.0.0: resolution: {integrity: sha512-azINV9dUtzPMFQktvTXciNAfAuVh/L/JCl0vtPCwvOA21uZrC08K/UnmrL+SXGEVc1FwzjW62+xw5S/uaLj6cA==} dependencies: @@ -2768,6 +2973,15 @@ packages: path-type: 3.0.0 dev: true + /readable-stream/3.6.0: + resolution: {integrity: sha512-BViHy7LKeTz4oNnkcLJ+lVSL6vpiFeX6/d3oSH8zCW7UxP2onchk+vTGB143xuFjHS3deTgkKoXXymXqymiIdA==} + engines: {node: '>= 6'} + dependencies: + inherits: 2.0.4 + string_decoder: 1.3.0 + util-deprecate: 1.0.2 + dev: false + /readdirp/3.6.0: resolution: {integrity: sha512-hOS089on8RduqdbhvQ5Z37A0ESjsqz6qnRcffsMU3495FuTdqSm+7bhJ29JvIOsBDEEnan5DPu9t3To9VRlMzA==} engines: {node: '>=8.10.0'} @@ -2823,6 +3037,14 @@ packages: path-parse: 1.0.7 supports-preserve-symlinks-flag: 1.0.0 + /restore-cursor/4.0.0: + resolution: {integrity: sha512-I9fPXU9geO9bHOt9pHHOhOkYerIMsmVaWB0rA2AI9ERh/+x/i7MV5HKBNrg+ljO5eoPVgCcnFuRjJ9uH6I/3eg==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + dependencies: + onetime: 5.1.2 + signal-exit: 3.0.7 + dev: false + /reusify/1.0.4: resolution: {integrity: sha512-U9nH88a3fc/ekCF1l0/UP1IosiuIjyTh7hBvXVMHYgVcfGvt897Xguj2UOLDeI5BG2m7/uwyaLVT6fbtCwTyzw==} engines: {iojs: '>=1.0.0', node: '>=0.10.0'} @@ -2874,6 +3096,10 @@ packages: dependencies: mri: 1.2.0 + /safe-buffer/5.2.1: + resolution: {integrity: sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==} + dev: false + /sander/0.5.1: resolution: {integrity: sha512-3lVqBir7WuKDHGrKRDn/1Ye3kwpXaDOMsiRP1wd6wpZW56gJhsbp5RqQpA6JG/P+pkXizygnr1dKR8vzWaVsfA==} dependencies: @@ -2976,7 +3202,6 @@ packages: /signal-exit/3.0.7: resolution: {integrity: sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ==} - dev: true /sirv-cli/2.0.2: resolution: {integrity: sha512-OtSJDwxsF1NWHc7ps3Sa0s+dPtP15iQNJzfKVz+MxkEo3z72mCD+yu30ct79rPr0CaV1HXSOBp+MIY5uIhHZ1A==} @@ -3002,6 +3227,10 @@ packages: totalist: 3.0.0 dev: false + /sisteransi/1.0.5: + resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==} + dev: false + /slash/3.0.0: resolution: {integrity: sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==} engines: {node: '>=8'} @@ -3108,12 +3337,25 @@ packages: es-abstract: 1.20.1 dev: true + /string_decoder/1.3.0: + resolution: {integrity: sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==} + dependencies: + safe-buffer: 5.2.1 + dev: false + /strip-ansi/6.0.1: resolution: {integrity: sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==} engines: {node: '>=8'} dependencies: ansi-regex: 5.0.1 + /strip-ansi/7.0.1: + resolution: {integrity: sha512-cXNxvT8dFNRVfhVME3JAe98mkXDYN2O1l7jmcwMnOslDeESg1rF/OZMtK0nRAhiari1unG5cD4jG3rapUAkLbw==} + engines: {node: '>=12'} + dependencies: + ansi-regex: 6.0.1 + dev: false + /strip-bom/3.0.0: resolution: {integrity: sha1-IzTBjpx1n3vdVv3vfprj1YjmjtM=} engines: {node: '>=4'} @@ -3129,6 +3371,11 @@ packages: engines: {node: '>=6'} dev: true + /strip-final-newline/3.0.0: + resolution: {integrity: sha512-dOESqjYr96iWYylGObzd39EuNTa5VJxyvVAEm5Jnh7KGo75V43Hk1odPQkNDyXNmUR6k+gEiDVXnjB8HJ3crXw==} + engines: {node: '>=12'} + dev: false + /strip-indent/3.0.0: resolution: {integrity: sha512-laJTa3Jb+VQpaC6DseHhF7dXVqHTfJPCRDaEbid/drOhgitgYku/letMUqOXFoWV0zIIUbjpdH2t+tYj4bQMRQ==} engines: {node: '>=8'} @@ -3566,6 +3813,10 @@ packages: dependencies: punycode: 2.1.1 + /util-deprecate/1.0.2: + resolution: {integrity: sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==} + dev: false + /v8-compile-cache/2.3.0: resolution: {integrity: sha512-l8lCEmLcLYZh4nbunNZvQCJc5pv7+RCwa8q/LdUx8u7lsWvPDKmpodJAJNwkAhJC//dFY48KuIEmjtd4RViDrA==} dev: true @@ -3682,6 +3933,12 @@ packages: resolution: {integrity: sha512-Uw5ooOQxRASHgu6C7GVvUxisKXfSgW4oFlO+aa+PAkgmH89O3CXxEEzNRNtHSqtXFTl0nAC1uYj0GMSH27uwtQ==} dev: true + /wcwidth/1.0.1: + resolution: {integrity: sha512-XHPEwS0q6TaxcvG85+8EYkbiCux2XtWG2mkc47Ng2A77BQu9+DqIOJldST4HgPkuea7dvKSj5VgX3P1d4rW8Tg==} + dependencies: + defaults: 1.0.3 + dev: false + /webidl-conversions/4.0.2: resolution: {integrity: sha512-YQ+BmxuTgd6UXZW3+ICGfyqRyHXVlD5GtQr5+qjiNW7bF0cqrzX500HVXPBOvgXb5YnzDd+h0zqyv61KUD7+Sg==} dev: true @@ -3704,6 +3961,11 @@ packages: is-symbol: 1.0.4 dev: true + /which-pm-runs/1.1.0: + resolution: {integrity: sha512-n1brCuqClxfFfq/Rb0ICg9giSZqCS+pLtccdag6C2HyufBrh3fBOiy9nb6ggRMvWOVH5GrdJskj5iGTZNxd7SA==} + engines: {node: '>=4'} + dev: false + /which/1.3.1: resolution: {integrity: sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ==} hasBin: true @@ -3797,6 +4059,11 @@ packages: engines: {node: '>=12'} dev: false + /yargs-parser/21.1.1: + resolution: {integrity: sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw==} + engines: {node: '>=12'} + dev: false + /yargs/17.5.1: resolution: {integrity: sha512-t6YAJcxDkNX7NFYiVtKvWUz8l+PaKTLiL63mJYWR2GnHq2gjEWISzsLp9wg3aY36dY1j+gfIEL3pIF+XlJJfbA==} engines: {node: '>=12'} From a801e8338a2550c0230748f5643e7251fb67ce59 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Mon, 19 Sep 2022 16:08:53 -0700 Subject: [PATCH 02/23] feat: add template-default using sources from sir example project --- examples/sir/README.md | 41 +++++++++++++++++++ examples/template-default/.gitignore | 4 ++ examples/template-default/README.md | 24 +++++++++++ examples/template-default/config/colors.csv | 3 ++ examples/template-default/config/graphs.csv | 1 + examples/template-default/config/inputs.csv | 1 + examples/template-default/config/model.csv | 2 + examples/template-default/config/outputs.csv | 1 + examples/template-default/config/strings.csv | 2 + .../{sir => template-default}/package.json | 5 ++- .../packages/app}/.gitignore | 0 .../packages/app}/index.html | 0 .../packages/app}/package.json | 5 +-- .../packages/app}/src/dev-overlay.js | 0 .../packages/app}/src/graph-view.ts | 0 .../packages/app}/src/index.css | 0 .../packages/app}/src/index.js | 0 .../packages/app}/tsconfig.json | 4 +- .../packages/app}/vite.config.js | 8 ++-- .../packages/core}/.gitignore | 0 .../packages/core}/package.json | 2 +- .../packages/core}/src/config/config.ts | 0 .../packages/core}/src/index.ts | 0 .../packages/core}/src/model/inputs.ts | 0 .../packages/core}/src/model/model.ts | 0 .../packages/core}/tsconfig.json | 0 .../packages/core}/vite.config.js | 0 .../{sir => template-default}/sde.config.js | 9 ++-- packages/create/src/index.ts | 5 ++- pnpm-workspace.yaml | 3 +- 30 files changed, 99 insertions(+), 21 deletions(-) create mode 100644 examples/sir/README.md create mode 100644 examples/template-default/.gitignore create mode 100644 examples/template-default/README.md create mode 100644 examples/template-default/config/colors.csv create mode 100644 examples/template-default/config/graphs.csv create mode 100644 examples/template-default/config/inputs.csv create mode 100644 examples/template-default/config/model.csv create mode 100644 examples/template-default/config/outputs.csv create mode 100644 examples/template-default/config/strings.csv rename examples/{sir => template-default}/package.json (85%) rename examples/{sir/packages/sir-app => template-default/packages/app}/.gitignore (100%) rename examples/{sir/packages/sir-app => template-default/packages/app}/index.html (100%) rename examples/{sir/packages/sir-app => template-default/packages/app}/package.json (87%) rename examples/{sir/packages/sir-app => template-default/packages/app}/src/dev-overlay.js (100%) rename examples/{sir/packages/sir-app => template-default/packages/app}/src/graph-view.ts (100%) rename examples/{sir/packages/sir-app => template-default/packages/app}/src/index.css (100%) rename examples/{sir/packages/sir-app => template-default/packages/app}/src/index.js (100%) rename examples/{sir/packages/sir-app => template-default/packages/app}/tsconfig.json (91%) rename examples/{sir/packages/sir-app => template-default/packages/app}/vite.config.js (89%) rename examples/{sir/packages/sir-core => template-default/packages/core}/.gitignore (100%) rename examples/{sir/packages/sir-core => template-default/packages/core}/package.json (95%) rename examples/{sir/packages/sir-core => template-default/packages/core}/src/config/config.ts (100%) rename examples/{sir/packages/sir-core => template-default/packages/core}/src/index.ts (100%) rename examples/{sir/packages/sir-core => template-default/packages/core}/src/model/inputs.ts (100%) rename examples/{sir/packages/sir-core => template-default/packages/core}/src/model/model.ts (100%) rename examples/{sir/packages/sir-core => template-default/packages/core}/tsconfig.json (100%) rename examples/{sir/packages/sir-core => template-default/packages/core}/vite.config.js (100%) rename examples/{sir => template-default}/sde.config.js (88%) diff --git a/examples/sir/README.md b/examples/sir/README.md new file mode 100644 index 00000000..d690b416 --- /dev/null +++ b/examples/sir/README.md @@ -0,0 +1,41 @@ +# sir + +This example directory contains the class SIR (Susceptible-Infectious-Recovered) model of +infectious disease. +It is intended to demonstrate the use of the `@sdeverywhere/create` package to quickly +set up a new project that uses the provided config files to generate a simple web application. + +## Quick Start + +The quickest way to get started using the `sir` example project is to copy +it into a separate directory (outside of the `SDEverywhere` working copy). +This will allow you to install the `@sdeverywhere/*` packages using your +package manager of choice (npm, yarn, or pnpm). + +```sh +# Change to the parent of your SDEverywhere working copy +cd + +# Copy the example to a separate directory +cp -rf SDEverywhere/examples/sir . + +# Change to the copied directory +cd ./sir + +# Create a new project (you can also use yarn or pnpm here, if preferred). +# Be sure to choose the "Default" template, which will make use of the +# existing files in the `config` directory. +npm create @sdeverywhere + +# Enter development mode for the sample model. This will start a live +# development environment that will build a WebAssembly version of the +# sample model and run checks on it any time you make changes to: +# - the config files +# - the Vensim model file (sir.mdl) +# - the checks file (sir.check.yaml) +npm run dev +``` + +## License + +SDEverywhere is distributed under the MIT license. See `LICENSE` for more details. diff --git a/examples/template-default/.gitignore b/examples/template-default/.gitignore new file mode 100644 index 00000000..f5c47214 --- /dev/null +++ b/examples/template-default/.gitignore @@ -0,0 +1,4 @@ +sde-prep +*.vdf +*.vdfx +*.3vmfx diff --git a/examples/template-default/README.md b/examples/template-default/README.md new file mode 100644 index 00000000..8f3e4da5 --- /dev/null +++ b/examples/template-default/README.md @@ -0,0 +1,24 @@ +# template-default + +This is a template that is used by the `@sdeverywhere/create` package to generate a +new project that uses SDEverywhere. + +## Quick Start + +```sh +# Create a new project (you can also use yarn or pnpm here, if preferred). +# Be sure to choose the "Default" template. +npm create @sdeverywhere + +# Enter development mode for your model. This will start a live +# development environment that will build a WebAssembly version of the +# model and run checks on it any time you make changes to: +# - the config files +# - the Vensim model file (.mdl) +# - the checks file (.check.yaml) +npm run dev +``` + +## License + +SDEverywhere is distributed under the MIT license. See `LICENSE` for more details. diff --git a/examples/template-default/config/colors.csv b/examples/template-default/config/colors.csv new file mode 100644 index 00000000..84c72c7c --- /dev/null +++ b/examples/template-default/config/colors.csv @@ -0,0 +1,3 @@ +id,hex code,name,comment +black,#000000,, +blue,#0072b2,, diff --git a/examples/template-default/config/graphs.csv b/examples/template-default/config/graphs.csv new file mode 100644 index 00000000..1d20bde6 --- /dev/null +++ b/examples/template-default/config/graphs.csv @@ -0,0 +1 @@ +id,side,parent menu,graph title,menu title,mini title,vensim graph,kind,modes,units,alternate,unused 1,unused 2,unused 3,x axis min,x axis max,x axis label,unused 4,unused 5,y axis min,y axis max,y axis soft max,y axis label,y axis format,unused 6,unused 7,plot 1 variable,plot 1 source,plot 1 style,plot 1 label,plot 1 color,plot 1 unused 1,plot 1 unused 2,plot 2 variable,plot 2 source,plot 2 style,plot 2 label,plot 2 color,plot 2 unused 1,plot 2 unused 2,plot 3 variable,plot 3 source,plot 3 style,plot 3 label,plot 3 color,plot 3 unused 1,plot 3 unused 2,plot 4 variable,plot 4 source,plot 4 style,plot 4 label,plot 4 color,plot 4 unused 1,plot 4 unused 2,plot 5 variable,plot 5 source,plot 5 style,plot 5 label,plot 5 color,plot 5 unused 1,plot 5 unused 2,plot 6 variable,plot 6 source,plot 6 style,plot 6 label,plot 6 color,plot 6 unused 1,plot 6 unused 2,plot 7 variable,plot 7 source,plot 7 style,plot 7 label,plot 7 color,plot 7 unused 1,plot 7 unused 2,plot 8 variable,plot 8 source,plot 8 style,plot 8 label,plot 8 color,plot 8 unused 1,plot 8 unused 2,plot 9 variable,plot 9 source,plot 9 style,plot 9 label,plot 9 color,plot 9 unused 1,plot 9 unused 2,plot 10 variable,plot 10 source,plot 10 style,plot 10 label,plot 10 color,plot 10 unused 1,plot 10 unused 2,plot 11 variable,plot 11 source,plot 11 style,plot 11 label,plot 11 color,plot 11 unused 1,plot 11 unused 2,plot 12 variable,plot 12 source,plot 12 style,plot 12 label,plot 12 color,plot 12 unused 1,plot 12 unused 2,plot 13 variable,plot 13 source,plot 13 style,plot 13 label,plot 13 color,plot 13 unused 1,plot 13 unused 2,plot 14 variable,plot 14 source,plot 14 style,plot 14 label,plot 14 color,plot 14 unused 1,plot 14 unused 2,plot 15 variable,plot 15 source,plot 15 style,plot 15 label,plot 15 color,plot 15 unused 1,plot 15 unused 2 diff --git a/examples/template-default/config/inputs.csv b/examples/template-default/config/inputs.csv new file mode 100644 index 00000000..c64c5986 --- /dev/null +++ b/examples/template-default/config/inputs.csv @@ -0,0 +1 @@ +id,input type,viewid,varname,label,view level,group name,slider min,slider max,slider/switch default,slider step,units,format,reversed,range 2 start,range 3 start,range 4 start,range 5 start,range 1 label,range 2 label,range 3 label,range 4 label,range 5 label,enabled value,disabled value,controlled input ids,listing label,description diff --git a/examples/template-default/config/model.csv b/examples/template-default/config/model.csv new file mode 100644 index 00000000..b7160fc1 --- /dev/null +++ b/examples/template-default/config/model.csv @@ -0,0 +1,2 @@ +model start time,model end time,graph default min time,graph default max time,model dat files +0,100,0,100, diff --git a/examples/template-default/config/outputs.csv b/examples/template-default/config/outputs.csv new file mode 100644 index 00000000..72a4edf6 --- /dev/null +++ b/examples/template-default/config/outputs.csv @@ -0,0 +1 @@ +variable name diff --git a/examples/template-default/config/strings.csv b/examples/template-default/config/strings.csv new file mode 100644 index 00000000..e3f45d0c --- /dev/null +++ b/examples/template-default/config/strings.csv @@ -0,0 +1,2 @@ +id,string +__model_name,My Model diff --git a/examples/sir/package.json b/examples/template-default/package.json similarity index 85% rename from examples/sir/package.json rename to examples/template-default/package.json index 0c55dbd8..f980e43e 100644 --- a/examples/sir/package.json +++ b/examples/template-default/package.json @@ -1,11 +1,12 @@ { - "name": "sir", + "name": "project", "version": "1.0.0", "private": true, "type": "module", "scripts": { "build": "sde bundle", - "dev": "sde dev" + "dev": "sde dev", + "start": "sde dev" }, "dependencies": { "@sdeverywhere/cli": "^0.7.0", diff --git a/examples/sir/packages/sir-app/.gitignore b/examples/template-default/packages/app/.gitignore similarity index 100% rename from examples/sir/packages/sir-app/.gitignore rename to examples/template-default/packages/app/.gitignore diff --git a/examples/sir/packages/sir-app/index.html b/examples/template-default/packages/app/index.html similarity index 100% rename from examples/sir/packages/sir-app/index.html rename to examples/template-default/packages/app/index.html diff --git a/examples/sir/packages/sir-app/package.json b/examples/template-default/packages/app/package.json similarity index 87% rename from examples/sir/packages/sir-app/package.json rename to examples/template-default/packages/app/package.json index abb5e224..63c2b4e9 100644 --- a/examples/sir/packages/sir-app/package.json +++ b/examples/template-default/packages/app/package.json @@ -1,5 +1,5 @@ { - "name": "sir-app", + "name": "app", "version": "1.0.0", "private": true, "type": "module", @@ -15,8 +15,7 @@ "dependencies": { "bootstrap-slider": "10.6.2", "chart.js": "^2.9.4", - "jquery": "^3.5.1", - "sir-core": "^1.0.0" + "jquery": "^3.5.1" }, "devDependencies": { "@types/chart.js": "^2.9.34", diff --git a/examples/sir/packages/sir-app/src/dev-overlay.js b/examples/template-default/packages/app/src/dev-overlay.js similarity index 100% rename from examples/sir/packages/sir-app/src/dev-overlay.js rename to examples/template-default/packages/app/src/dev-overlay.js diff --git a/examples/sir/packages/sir-app/src/graph-view.ts b/examples/template-default/packages/app/src/graph-view.ts similarity index 100% rename from examples/sir/packages/sir-app/src/graph-view.ts rename to examples/template-default/packages/app/src/graph-view.ts diff --git a/examples/sir/packages/sir-app/src/index.css b/examples/template-default/packages/app/src/index.css similarity index 100% rename from examples/sir/packages/sir-app/src/index.css rename to examples/template-default/packages/app/src/index.css diff --git a/examples/sir/packages/sir-app/src/index.js b/examples/template-default/packages/app/src/index.js similarity index 100% rename from examples/sir/packages/sir-app/src/index.js rename to examples/template-default/packages/app/src/index.js diff --git a/examples/sir/packages/sir-app/tsconfig.json b/examples/template-default/packages/app/tsconfig.json similarity index 91% rename from examples/sir/packages/sir-app/tsconfig.json rename to examples/template-default/packages/app/tsconfig.json index bb6f7219..c8c5cae6 100644 --- a/examples/sir/packages/sir-app/tsconfig.json +++ b/examples/template-default/packages/app/tsconfig.json @@ -8,8 +8,8 @@ // Configure path aliases "paths": { // The following lines enable path aliases within the app - "@core": ["../sir-core/src"], - "@core-strings": ["../sir-core/strings"], + "@core": ["../core/src"], + "@core-strings": ["../core/strings"], "@prep/*": ["../../sde-prep/*"] }, // XXX: The following two lines work around a TS/VSCode issue where this config diff --git a/examples/sir/packages/sir-app/vite.config.js b/examples/template-default/packages/app/vite.config.js similarity index 89% rename from examples/sir/packages/sir-app/vite.config.js rename to examples/template-default/packages/app/vite.config.js index c636b0d7..d318371a 100644 --- a/examples/sir/packages/sir-app/vite.config.js +++ b/examples/template-default/packages/app/vite.config.js @@ -32,8 +32,8 @@ export default defineConfig(env => { resolve: { alias: { - '@core': resolve(appDir, '..', 'sir-core', 'src'), - '@core-strings': resolve(appDir, '..', 'sir-core', 'strings'), + '@core': resolve(appDir, '..', 'core', 'src'), + '@core-strings': resolve(appDir, '..', 'core', 'strings'), '@prep': resolve(projDir, 'sde-prep') } }, @@ -57,8 +57,8 @@ export default defineConfig(env => { }, server: { - // Run the dev server at `localhost:8091` by default - port: 8091, + // Run the dev server at `localhost:8080` by default + port: 8080, // Open the app in the browser by default open: '/index.html' diff --git a/examples/sir/packages/sir-core/.gitignore b/examples/template-default/packages/core/.gitignore similarity index 100% rename from examples/sir/packages/sir-core/.gitignore rename to examples/template-default/packages/core/.gitignore diff --git a/examples/sir/packages/sir-core/package.json b/examples/template-default/packages/core/package.json similarity index 95% rename from examples/sir/packages/sir-core/package.json rename to examples/template-default/packages/core/package.json index 38129448..d4f63ac9 100644 --- a/examples/sir/packages/sir-core/package.json +++ b/examples/template-default/packages/core/package.json @@ -1,5 +1,5 @@ { - "name": "sir-core", + "name": "core", "version": "1.0.0", "private": true, "files": [ diff --git a/examples/sir/packages/sir-core/src/config/config.ts b/examples/template-default/packages/core/src/config/config.ts similarity index 100% rename from examples/sir/packages/sir-core/src/config/config.ts rename to examples/template-default/packages/core/src/config/config.ts diff --git a/examples/sir/packages/sir-core/src/index.ts b/examples/template-default/packages/core/src/index.ts similarity index 100% rename from examples/sir/packages/sir-core/src/index.ts rename to examples/template-default/packages/core/src/index.ts diff --git a/examples/sir/packages/sir-core/src/model/inputs.ts b/examples/template-default/packages/core/src/model/inputs.ts similarity index 100% rename from examples/sir/packages/sir-core/src/model/inputs.ts rename to examples/template-default/packages/core/src/model/inputs.ts diff --git a/examples/sir/packages/sir-core/src/model/model.ts b/examples/template-default/packages/core/src/model/model.ts similarity index 100% rename from examples/sir/packages/sir-core/src/model/model.ts rename to examples/template-default/packages/core/src/model/model.ts diff --git a/examples/sir/packages/sir-core/tsconfig.json b/examples/template-default/packages/core/tsconfig.json similarity index 100% rename from examples/sir/packages/sir-core/tsconfig.json rename to examples/template-default/packages/core/tsconfig.json diff --git a/examples/sir/packages/sir-core/vite.config.js b/examples/template-default/packages/core/vite.config.js similarity index 100% rename from examples/sir/packages/sir-core/vite.config.js rename to examples/template-default/packages/core/vite.config.js diff --git a/examples/sir/sde.config.js b/examples/template-default/sde.config.js similarity index 88% rename from examples/sir/sde.config.js rename to examples/template-default/sde.config.js index 2445e086..f267132a 100644 --- a/examples/sir/sde.config.js +++ b/examples/template-default/sde.config.js @@ -7,17 +7,16 @@ import { vitePlugin } from '@sdeverywhere/plugin-vite' import { wasmPlugin } from '@sdeverywhere/plugin-wasm' import { workerPlugin } from '@sdeverywhere/plugin-worker' -const baseName = 'sir' const __dirname = dirname(fileURLToPath(import.meta.url)) const configDir = joinPath(__dirname, 'config') const packagePath = (...parts) => joinPath(__dirname, 'packages', ...parts) -const appPath = (...parts) => packagePath(`${baseName}-app`, ...parts) -const corePath = (...parts) => packagePath(`${baseName}-core`, ...parts) +const appPath = (...parts) => packagePath('app', ...parts) +const corePath = (...parts) => packagePath('core', ...parts) export async function config() { return { // Specify the Vensim model to read - modelFiles: ['model/sir.mdl'], + modelFiles: ['model/MODEL_NAME.mdl'], // The following files will be hashed to determine whether the model needs // to be rebuilt when watch mode is active @@ -48,7 +47,7 @@ export async function config() { // Build or serve the model explorer app vitePlugin({ - name: `${baseName}-app`, + name: 'app', apply: { development: 'serve' }, diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts index f9b4cdc5..ccac17ef 100644 --- a/packages/create/src/index.ts +++ b/packages/create/src/index.ts @@ -112,8 +112,8 @@ async function chooseProjectDir(): Promise { // so this should be re-evaluated later. async function runDegit(templateTarget: string, hash: string, dstDir: string, spinner: Ora): Promise { const emitter = degit(`${templateTarget}${hash}`, { - cache: false, - force: true + cache: false + // force: true }) try { @@ -311,6 +311,7 @@ export async function main(): Promise { // TODO: See if project contains an mdl file, if not, use hello-world // TODO: Set up sde.config.js file to use selected mdl file + // TODO: Fill in default values for config files ora({ text: green('Setup complete!') }).succeed() diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 395ccd76..796a85f4 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -1,6 +1,5 @@ packages: - packages/* - examples/* - # TODO: This next line can be removed once we remove sir/packages - - examples/sir/packages/* + - '!examples/template-*' - tests From bb6c5e6c0299857ff09a8754652dc80e72efb960 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Tue, 20 Sep 2022 17:36:25 -0700 Subject: [PATCH 03/23] fix: copy template files without overwriting existing files --- packages/create/package.json | 2 + packages/create/src/index.ts | 44 ++++++++++++++------ pnpm-lock.yaml | 81 ++++++++++++++---------------------- 3 files changed, 65 insertions(+), 62 deletions(-) diff --git a/packages/create/package.json b/packages/create/package.json index 65991276..fb7edb1c 100644 --- a/packages/create/package.json +++ b/packages/create/package.json @@ -27,6 +27,7 @@ "dependencies": { "degit": "^2.8.4", "execa": "^6.1.0", + "fs-extra": "^10.1.0", "kleur": "^4.1.5", "ora": "^6.1.2", "prompts": "^2.4.2", @@ -35,6 +36,7 @@ }, "devDependencies": { "@types/degit": "^2.8.3", + "@types/fs-extra": "^9.0.13", "@types/node": "^16.11.7", "@types/prompts": "^2.0.14", "@types/which-pm-runs": "^1.0.0", diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts index ccac17ef..aab0b954 100644 --- a/packages/create/src/index.ts +++ b/packages/create/src/index.ts @@ -1,7 +1,9 @@ // Copyright (c) 2022 Climate Interactive / New Venture Fund -import { existsSync } from 'fs' -import { relative, resolve as resolvePath } from 'path' +import { existsSync, mkdtempSync, readdirSync } from 'fs' +import { copy } from 'fs-extra' +import { tmpdir } from 'os' +import { relative, join as joinPath, resolve as resolvePath } from 'path' import degit from 'degit' import { execa, execaCommand } from 'execa' @@ -111,23 +113,39 @@ async function chooseProjectDir(): Promise { // It contains workarounds for degit issues that may or may not be relevant for SDE, // so this should be re-evaluated later. async function runDegit(templateTarget: string, hash: string, dstDir: string, spinner: Ora): Promise { + // Enable verbose degit logging if the --verbose flag is used + const verbose = args.verbose + + // Set up degit (we will be writing to a temporary directory, so force is safe) const emitter = degit(`${templateTarget}${hash}`, { - cache: false - // force: true + cache: false, + force: true, + verbose }) try { - // emitter.on('info', info => { - // logger.debug(info.message) - // }) - await emitter.clone(dstDir) + if (verbose) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + emitter.on('info', (info: any) => { + console.log(info.message) + }) + } + + // Make degit write to a temporary directory + const tmpDir = mkdtempSync(joinPath(tmpdir(), 'sde-create-')) + await emitter.clone(tmpDir) // degit does not return an error when an invalid template is provided, as such we // need to handle this manually - // if (isEmpty(cwd)) { - // fs.rmdirSync(cwd) - // throw new Error(`Error: The provided template (${cyan(options.template)}) does not exist`) - // } + if (!existsSync(tmpDir) || readdirSync(tmpDir).length === 0) { + throw new Error('The requested template failed to download') + } + + // Copy files to destination without overwriting + await copy(tmpDir, dstDir, { + overwrite: false, + errorOnExist: false + }) } catch (e) { spinner.fail() @@ -213,7 +231,7 @@ async function chooseTemplate(projDir: string): Promise { // Handle response const templateSpinner = ora('Copying project files...').start() - const templateTarget = `climateinteractive/sdeverywhere/examples/${options.template}` + const templateTarget = `climateinteractive/SDEverywhere/examples/${options.template}` // TODO: Fix this before branch is merged const hash = '#chris/228-create-package' diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 23260001..93cb92b2 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -91,47 +91,6 @@ importers: '@sdeverywhere/check-core': link:../../packages/check-core assert-never: 1.2.1 - examples/sir: - specifiers: - '@sdeverywhere/cli': ^0.7.0 - '@sdeverywhere/plugin-check': ^0.1.0 - '@sdeverywhere/plugin-config': ^0.1.0 - '@sdeverywhere/plugin-vite': ^0.1.1 - '@sdeverywhere/plugin-wasm': ^0.1.0 - '@sdeverywhere/plugin-worker': ^0.1.0 - dependencies: - '@sdeverywhere/cli': link:../../packages/cli - '@sdeverywhere/plugin-check': link:../../packages/plugin-check - '@sdeverywhere/plugin-config': link:../../packages/plugin-config - '@sdeverywhere/plugin-vite': link:../../packages/plugin-vite - '@sdeverywhere/plugin-wasm': link:../../packages/plugin-wasm - '@sdeverywhere/plugin-worker': link:../../packages/plugin-worker - - examples/sir/packages/sir-app: - specifiers: - '@types/chart.js': ^2.9.34 - bootstrap-slider: 10.6.2 - chart.js: ^2.9.4 - jquery: ^3.5.1 - sir-core: ^1.0.0 - vite: ^2.9.12 - dependencies: - bootstrap-slider: 10.6.2 - chart.js: 2.9.4 - jquery: 3.6.1 - sir-core: link:../sir-core - devDependencies: - '@types/chart.js': 2.9.37 - vite: 2.9.12 - - examples/sir/packages/sir-core: - specifiers: - '@sdeverywhere/runtime': ^0.1.0 - '@sdeverywhere/runtime-async': ^0.1.0 - dependencies: - '@sdeverywhere/runtime': link:../../../../packages/runtime - '@sdeverywhere/runtime-async': link:../../../../packages/runtime-async - packages/build: specifiers: '@types/cross-spawn': ^6.0.2 @@ -249,12 +208,14 @@ importers: packages/create: specifiers: '@types/degit': ^2.8.3 + '@types/fs-extra': ^9.0.13 '@types/node': ^16.11.7 '@types/prompts': ^2.0.14 '@types/which-pm-runs': ^1.0.0 '@types/yargs-parser': ^21.0.0 degit: ^2.8.4 execa: ^6.1.0 + fs-extra: ^10.1.0 kleur: ^4.1.5 ora: ^6.1.2 prompts: ^2.4.2 @@ -263,6 +224,7 @@ importers: dependencies: degit: 2.8.4 execa: 6.1.0 + fs-extra: 10.1.0 kleur: 4.1.5 ora: 6.1.2 prompts: 2.4.2 @@ -270,6 +232,7 @@ importers: yargs-parser: 21.1.1 devDependencies: '@types/degit': 2.8.3 + '@types/fs-extra': 9.0.13 '@types/node': 16.11.40 '@types/prompts': 2.0.14 '@types/which-pm-runs': 1.0.0 @@ -639,6 +602,12 @@ packages: resolution: {integrity: sha512-QJ1znjr9CDax2L17rgBnDOfNHsC1XtVAMswu+lRWvWb+kANhHA0slUNSSBsG8FVNvM4I4yXlN9doJRot3A2hkQ==} dev: true + /@types/fs-extra/9.0.13: + resolution: {integrity: sha512-nEnwB++1u5lVDM2UI4c1+5R+FYaKfaAzS4OococimjVm3nQw3TuzH5UNsocrcTBbhnerblyHj4A49qXbIiZdpA==} + dependencies: + '@types/node': 17.0.42 + dev: true + /@types/json-schema/7.0.11: resolution: {integrity: sha512-wOuvG1SN4Us4rez+tylwwwCV1psiNVOkJeM3AUWUNWg/jDQY2+HE/444y5gc+jBmRqASOm2Oeh5c1axHobwRKQ==} dev: true @@ -978,10 +947,6 @@ packages: readable-stream: 3.6.0 dev: false - /bootstrap-slider/10.6.2: - resolution: {integrity: sha512-8JTPZB9QVOdrGzYF3YgC3YW6ssfPeBvBwZnXffiZ7YH/zz1D0EKlZvmQsm/w3N0XjVNYQEoQ0ax+jHrErV4K1Q==} - dev: false - /brace-expansion/1.1.11: resolution: {integrity: sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==} dependencies: @@ -1892,6 +1857,15 @@ packages: engines: {node: '>=0.8'} dev: false + /fs-extra/10.1.0: + resolution: {integrity: sha512-oRXApq54ETRj4eMiFzGnHWGy+zo5raudjuxN0b8H7s/RU2oW0Wvsx9O0ACRN/kRq9E8Vu/ReskGB5o3ji+FzHQ==} + engines: {node: '>=12'} + dependencies: + graceful-fs: 4.2.10 + jsonfile: 6.1.0 + universalify: 2.0.0 + dev: false + /fs.realpath/1.0.0: resolution: {integrity: sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw==} @@ -2320,10 +2294,6 @@ packages: engines: {node: '>=10'} dev: true - /jquery/3.6.1: - resolution: {integrity: sha512-opJeO4nCucVnsjiXOE+/PcCgYw9Gwpvs/a6B1LL/lQhwWwpbVEVYDZ1FokFr8PRc7ghYlrFPuyHuiiDNTQxmcw==} - dev: false - /js-stringify/1.0.2: resolution: {integrity: sha512-rtS5ATOo2Q5k1G+DADISilDA6lv79zIiwFd6CcjuIxGKLFm5C+RLImRscVap9k55i+MOZwgliw+NejvkLuGD5g==} dev: true @@ -2363,6 +2333,14 @@ packages: resolution: {integrity: sha512-fQzRfAbIBnR0IQvftw9FJveWiHp72Fg20giDrHz6TdfB12UH/uue0D3hm57UB5KgAVuniLMCaS8P1IMj9NR7cA==} dev: true + /jsonfile/6.1.0: + resolution: {integrity: sha512-5dgndWOriYSm5cnYaJNhalLNDKOqFwyDB/rr1E9ZsGciGvKPs8R2xYGCacuf3z6K1YKDz182fd+fY3cn3pMqXQ==} + dependencies: + universalify: 2.0.0 + optionalDependencies: + graceful-fs: 4.2.10 + dev: false + /jstransformer/1.0.0: resolution: {integrity: sha1-7Yvwkh4vPx7U1cGkT2hwntJHIsM=} dependencies: @@ -3808,6 +3786,11 @@ packages: which-boxed-primitive: 1.0.2 dev: true + /universalify/2.0.0: + resolution: {integrity: sha512-hAZsKq7Yy11Zu1DE0OzWjw7nnLZmJZYTDZZyEFHZdUhV8FkH5MCfoU1XMaxXovpyW5nq5scPqq0ZDP9Zyl04oQ==} + engines: {node: '>= 10.0.0'} + dev: false + /uri-js/4.4.1: resolution: {integrity: sha512-7rKUyy33Q1yc98pQ1DAmLtwX109F7TIfWlW1Ydo8Wl1ii1SeHieeh0HHfPeL2fMXK6z0s8ecKs9frCuLJvndBg==} dependencies: From e0f6a1b8fb986888e21b71127cb53a88ec341ee7 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Tue, 20 Sep 2022 21:43:36 -0700 Subject: [PATCH 04/23] fix: allow user to choose mdl if there are multiple --- packages/create/src/index.ts | 90 +++++++++++++++++++++++++++++++----- 1 file changed, 79 insertions(+), 11 deletions(-) diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts index aab0b954..3a94f2f2 100644 --- a/packages/create/src/index.ts +++ b/packages/create/src/index.ts @@ -1,6 +1,7 @@ // Copyright (c) 2022 Climate Interactive / New Venture Fund import { existsSync, mkdtempSync, readdirSync } from 'fs' +import { readdir } from 'fs/promises' import { copy } from 'fs-extra' import { tmpdir } from 'os' import { relative, join as joinPath, resolve as resolvePath } from 'path' @@ -10,6 +11,7 @@ import { execa, execaCommand } from 'execa' import { bgCyan, black, bold, cyan, dim, green, red, reset, yellow } from 'kleur/colors' import type { Ora } from 'ora' import ora from 'ora' +import type { Choice } from 'prompts' import prompts from 'prompts' import detectPackageManager from 'which-pm-runs' import yargs from 'yargs-parser' @@ -46,19 +48,17 @@ async function chooseProjectDir(): Promise { } const showValidDirMsg = (dir: string) => { - const ack = ora({ + ora({ color: 'green', - text: `Using "${bold(dir)}" as project directory.` - }) - ack.succeed() + text: green(`Using "${bold(dir)}" as the project directory.`) + }).succeed() } const showInvalidDirMsg = (dir: string) => { - const reject = ora({ + ora({ color: 'red', text: `"${bold(dir)}" contains existing 'package.json' and/or 'packages' directory, stopping.` - }) - reject.fail() + }).fail() } // See if project directory is provided on command line @@ -244,6 +244,69 @@ async function chooseTemplate(projDir: string): Promise { templateSpinner.succeed() } +async function chooseMdlFile(projDir: string): Promise { + // Find all `.mdl` files in the project directory + // From https://stackoverflow.com/a/45130990 + async function getFiles(dir: string): Promise { + const dirents = await readdir(dir, { withFileTypes: true }) + const files = await Promise.all( + dirents.map(dirent => { + const res = resolvePath(dir, dirent.name) + return dirent.isDirectory() ? getFiles(res) : res + }) + ) + return files.flat() + } + const allFiles = await getFiles(projDir) + const mdlFiles = allFiles.filter(f => f.endsWith('.mdl')).map(f => relative(projDir, f)) + const mdlChoices = mdlFiles.map(f => { + return { + title: f, + value: f + } as Choice + }) + + let mdlFile: string + if (mdlFiles.length === 0) { + // No mdl files found; print error message and exit + // TODO: Offer to add a basic mdl to get the user started + ora({ + color: 'red', + text: `No mdl files were found in "${projDir}". Add your mdl file to that directory and try again.` + }).fail() + process.exit(1) + } else if (mdlFiles.length === 1) { + // Only one mdl file + ora().succeed(`Found "${mdlFiles[0]}", will configure the project to use that mdl file.`) + mdlFile = mdlFiles[0] + } else { + // Multiple mdl files found; allow the user to choose one + // TODO: Eventually we should allow the user to choose to flatten if there are multiple submodels + const options = await prompts( + [ + { + type: 'select', + name: 'mdlFile', + message: 'It looks like there are multiple mdl files. Which one would you like to use?', + choices: mdlChoices + } + ], + { onCancel: () => ora().info(dim('Operation cancelled.')) } + ) + if (!options.mdlFile) { + process.exit(1) + } + mdlFile = options.mdlFile + } + + ora({ + color: 'green', + text: green(`Using "${bold(mdlFile)}" as the model for the project.`) + }).succeed() + + return mdlFile +} + async function chooseInstall(projDir: string): Promise { // Prompt the user const installResponse = await prompts( @@ -321,16 +384,21 @@ export async function main(): Promise { // Prompt the user to select a template await chooseTemplate(projDir) + // Prompt the user to select an mdl file + const mdlPath = await chooseMdlFile(projDir) + // TODO + console.log(mdlPath) + + // TODO: Prompt the user to choose input/output vars (if default template chosen) + + // TODO: Prompt the user to install Emscripten SDK + // Prompt the user to install dependencies await chooseInstall(projDir) // Prompt the user to initialize a git repo await chooseGitInit(projDir) - // TODO: See if project contains an mdl file, if not, use hello-world - // TODO: Set up sde.config.js file to use selected mdl file - // TODO: Fill in default values for config files - ora({ text: green('Setup complete!') }).succeed() console.log(`\n${bgCyan(black(' Next steps '))}\n`) From 012ca85a5887f84a7ff63e930d7269ce9024a4f0 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Tue, 20 Sep 2022 22:50:03 -0700 Subject: [PATCH 05/23] feat: add support for installing Emscripten SDK --- packages/create/src/index.ts | 105 ++++++++++++++++++++++++--- packages/create/src/install-emsdk.ts | 54 ++++++++++++++ 2 files changed, 148 insertions(+), 11 deletions(-) create mode 100644 packages/create/src/install-emsdk.ts diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts index 3a94f2f2..a778b339 100644 --- a/packages/create/src/index.ts +++ b/packages/create/src/index.ts @@ -1,6 +1,6 @@ // Copyright (c) 2022 Climate Interactive / New Venture Fund -import { existsSync, mkdtempSync, readdirSync } from 'fs' +import { existsSync, mkdtempSync, readdirSync, rmSync } from 'fs' import { readdir } from 'fs/promises' import { copy } from 'fs-extra' import { tmpdir } from 'os' @@ -16,6 +16,8 @@ import prompts from 'prompts' import detectPackageManager from 'which-pm-runs' import yargs from 'yargs-parser' +import { installEmscripten } from './install-emsdk' + export const TEMPLATES = [ { title: 'Default project', @@ -93,6 +95,9 @@ async function chooseProjectDir(): Promise { }, { onCancel: () => ora().info(dim('Operation cancelled.')) } ) + if (!projDir) { + process.exit(1) + } projDir = dirResponse.directory if (projDir === '') { projDir = process.cwd() @@ -146,6 +151,9 @@ async function runDegit(templateTarget: string, hash: string, dstDir: string, sp overwrite: false, errorOnExist: false }) + + // Remove the temporary directory + rmSync(tmpDir, { recursive: true, force: true }) } catch (e) { spinner.fail() @@ -307,7 +315,78 @@ async function chooseMdlFile(projDir: string): Promise { return mdlFile } -async function chooseInstall(projDir: string): Promise { +async function chooseInstallEmsdk(projDir: string): Promise { + // TODO: Use findUp and skip this step if emsdk directory already exists + + // Prompt the user + const underParentDir = resolvePath(projDir, '..', 'emsdk') + const underProjDir = joinPath(projDir, 'emsdk') + const installResponse = await prompts( + { + type: 'select', + name: 'install', + message: `Would you like to install the Emscripten SDK?`, + choices: [ + // ${reset(dim('(recommended)')) + { + title: `Install under parent directory (${bold(underParentDir)})`, + description: 'This is recommended so that it can be shared by multiple projects', + value: 'parent' + }, + { + title: `Install under project directory (${bold(underProjDir)})"`, + description: 'This is useful for keeping everything under a single project directory', + value: 'project' + }, + { + title: `Don't install`, + description: `It's OK, you can install it later`, + value: 'skip' + } + ] + }, + { + onCancel: () => { + ora().info( + dim( + 'Operation cancelled. Your project folder has been created, but the Emscripten SDK and other dependencies have not been installed.' + ) + ) + process.exit(1) + } + } + ) + + // Handle response + if (args.dryRun) { + ora().info(dim(`--dry-run enabled, skipping.`)) + return + } else if (installResponse.install === 'skip') { + ora().info( + dim('No problem! Be sure to install the Emscripten SDK and configure it in `sde.config.js` after setup.') + ) + return + } + + const installDir = installResponse.install === 'parent' ? underParentDir : underProjDir + try { + // TODO: Use spinner here + await installEmscripten(installDir) + } catch (e) { + ora({ + color: 'red', + text: red(`Failed to install Emscripten SDK: ${e.message}`) + }).fail() + process.exit(1) + } + + ora({ + color: 'green', + text: green(`Installed the Emscripten SDK in "${bold(installDir)}"`) + }).succeed() +} + +async function chooseInstallDeps(projDir: string): Promise { // Prompt the user const installResponse = await prompts( { @@ -367,7 +446,7 @@ async function chooseGitInit(projDir: string): Promise { ora().info(dim(`--dry-run enabled, skipping.`)) } else if (gitResponse.git) { await execaCommand('git init', { cwd: projDir }) - ora().succeed('Git repository created!') + ora().succeed(green('Git repository created!')) } else { ora().info(dim(`No problem! You can come back and run ${cyan(`git init`)} later.`)) } @@ -381,24 +460,28 @@ export async function main(): Promise { // Prompt the user to select a project directory const projDir = await chooseProjectDir() - // Prompt the user to select a template - await chooseTemplate(projDir) + if (args.TODO) { + // Prompt the user to select a template + await chooseTemplate(projDir) - // Prompt the user to select an mdl file - const mdlPath = await chooseMdlFile(projDir) - // TODO - console.log(mdlPath) + // Prompt the user to select an mdl file + const mdlPath = await chooseMdlFile(projDir) + // TODO + console.log(mdlPath) + } // TODO: Prompt the user to choose input/output vars (if default template chosen) - // TODO: Prompt the user to install Emscripten SDK + // Prompt the user to install Emscripten SDK + await chooseInstallEmsdk(projDir) // Prompt the user to install dependencies - await chooseInstall(projDir) + await chooseInstallDeps(projDir) // Prompt the user to initialize a git repo await chooseGitInit(projDir) + console.log() ora({ text: green('Setup complete!') }).succeed() console.log(`\n${bgCyan(black(' Next steps '))}\n`) diff --git a/packages/create/src/install-emsdk.ts b/packages/create/src/install-emsdk.ts new file mode 100644 index 00000000..6a35e1ef --- /dev/null +++ b/packages/create/src/install-emsdk.ts @@ -0,0 +1,54 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { existsSync, rmSync } from 'fs' +import { join as joinPath } from 'path' + +import { execa } from 'execa' + +// TODO: Make this configurable; we're using this older version for now because it is +// relatively stable and has been used to build En-ROADS and C-ROADS for a long time +const version = '2.0.34' + +/** + * Install the Emscripten SDK to the `emsdk` directory under the + * given directory (if `emsdk` is not already present), then + * activates the requested version (specified with `version`). + * + * The implementation is similar to the existing `setup-emsdk` action + * (https://github.com/mymindstorm/setup-emsdk) except that one has issues + * with the cache directory on Windows, so having our own script gives us + * more control over installation and caching behavior. + */ +export async function installEmscripten(emsdkDir: string /*, options?: { verbose?: boolean }*/): Promise { + // Only download if the emsdk directory wasn't restored from a cache + if (!existsSync(emsdkDir)) { + console.log(`Downloading Emscripten SDK to ${emsdkDir}`) + // console.log() + await execa('git', ['clone', 'https://github.com/emscripten-core/emsdk.git', emsdkDir]) + } else { + console.log(`Found existing Emscripten SDK directory: ${emsdkDir}`) + } + // console.log() + + if (process.env.CI) { + // On GitHub Actions, remove the `.git` directory to keep the cache smaller. + // When we update to a new version, the cache key will miss and the emsdk + // repo will be redownloaded. + console.log('CI detected, removing .git directory...') + const gitDir = joinPath(emsdkDir, '.git') + rmSync(gitDir, { recursive: true, force: true }) + } else { + // For local development, pull to get the latest + console.log('Local development detected, performing git pull...') + await execa('git', ['pull'], { cwd: emsdkDir }) + // console.log() + } + + const emsdkCmd = async (...args: string[]) => { + return execa('python3', ['emsdk.py', ...args], { cwd: emsdkDir }) + } + + console.log(`Activating Emscripten SDK ${version}...`) + await emsdkCmd('install', version) + await emsdkCmd('activate', version) +} From 34c36ce4a4dcc1eec579892cf62f7a3f4cf3e0e2 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Tue, 20 Sep 2022 23:12:40 -0700 Subject: [PATCH 06/23] refactor: split index file into separate modules, one per step --- packages/create/src/index.ts | 480 ++------------------------ packages/create/src/install-emsdk.ts | 54 --- packages/create/src/step-deps.ts | 47 +++ packages/create/src/step-directory.ts | 84 +++++ packages/create/src/step-emsdk.ts | 129 +++++++ packages/create/src/step-git.ts | 33 ++ packages/create/src/step-mdl.ts | 72 ++++ packages/create/src/step-template.ts | 171 +++++++++ 8 files changed, 560 insertions(+), 510 deletions(-) delete mode 100644 packages/create/src/install-emsdk.ts create mode 100644 packages/create/src/step-deps.ts create mode 100644 packages/create/src/step-directory.ts create mode 100644 packages/create/src/step-emsdk.ts create mode 100644 packages/create/src/step-git.ts create mode 100644 packages/create/src/step-mdl.ts create mode 100644 packages/create/src/step-template.ts diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts index a778b339..7d33cf94 100644 --- a/packages/create/src/index.ts +++ b/packages/create/src/index.ts @@ -1,485 +1,53 @@ // Copyright (c) 2022 Climate Interactive / New Venture Fund -import { existsSync, mkdtempSync, readdirSync, rmSync } from 'fs' -import { readdir } from 'fs/promises' -import { copy } from 'fs-extra' -import { tmpdir } from 'os' -import { relative, join as joinPath, resolve as resolvePath } from 'path' +import { relative } from 'path' -import degit from 'degit' -import { execa, execaCommand } from 'execa' -import { bgCyan, black, bold, cyan, dim, green, red, reset, yellow } from 'kleur/colors' -import type { Ora } from 'ora' +import { bgCyan, black, bold, cyan, green } from 'kleur/colors' import ora from 'ora' -import type { Choice } from 'prompts' import prompts from 'prompts' import detectPackageManager from 'which-pm-runs' import yargs from 'yargs-parser' -import { installEmscripten } from './install-emsdk' +import { chooseInstallDeps } from './step-deps' +import { chooseProjectDir } from './step-directory' +import { chooseInstallEmsdk } from './step-emsdk' +import { chooseGitInit } from './step-git' +import { chooseMdlFile } from './step-mdl' +import { chooseTemplate } from './step-template' -export const TEMPLATES = [ - { - title: 'Default project', - description: 'Includes recommended structure with config files, app, core library, model-check, etc', - value: 'template-default' - }, - { - title: 'Minimal project', - description: 'Includes simple config for model-check', - value: 'template-minimal' - } -] - -// Detect the package manager -const pkgManager = detectPackageManager()?.name || 'npm' - -// Parse command line arguments -const args = yargs(process.argv) -prompts.override(args) - -async function chooseProjectDir(): Promise { - /** - * Return true if the given directory does not exist, or it does not contain important files - * like `package.json`. - */ - function isValidDir(dir: string): boolean { - const packageJson = resolvePath(dir, 'package.json') - const packagesDir = resolvePath(dir, 'packages') - return !existsSync(dir) || (!existsSync(packageJson) && !existsSync(packagesDir)) - } - - const showValidDirMsg = (dir: string) => { - ora({ - color: 'green', - text: green(`Using "${bold(dir)}" as the project directory.`) - }).succeed() - } - - const showInvalidDirMsg = (dir: string) => { - ora({ - color: 'red', - text: `"${bold(dir)}" contains existing 'package.json' and/or 'packages' directory, stopping.` - }).fail() - } - - // See if project directory is provided on command line - let projDir = args['_'][2] as string - if (projDir) { - // Directory was provided, see if it is valid - if (isValidDir(projDir)) { - // The provided directory is valid, so proceed - showValidDirMsg(projDir) - } else { - // The provided directory is not valid, so show error message and exit - showInvalidDirMsg(projDir) - process.exit(1) - } - } else { - // Directory was not provided, so prompt the user - const dirResponse = await prompts( - { - type: 'text', - name: 'directory', - message: 'Where would you like to create your new project?', - initial: '' - // validate(value) { - // if (value === '') { - // value = process.cwd() - // } - // if (!isValidDir(value)) { - // return notValidMsg(value) - // } - // return true - // } - }, - { onCancel: () => ora().info(dim('Operation cancelled.')) } - ) - if (!projDir) { - process.exit(1) - } - projDir = dirResponse.directory - if (projDir === '') { - projDir = process.cwd() - } - if (isValidDir(projDir)) { - showValidDirMsg(projDir) - } else { - showInvalidDirMsg(projDir) - process.exit(1) - } - } - - return projDir -} - -// XXX: This is mostly copied from Astro's create package: -// https://github.com/withastro/astro/blob/main/packages/create-astro/src/index.ts -// It contains workarounds for degit issues that may or may not be relevant for SDE, -// so this should be re-evaluated later. -async function runDegit(templateTarget: string, hash: string, dstDir: string, spinner: Ora): Promise { - // Enable verbose degit logging if the --verbose flag is used - const verbose = args.verbose - - // Set up degit (we will be writing to a temporary directory, so force is safe) - const emitter = degit(`${templateTarget}${hash}`, { - cache: false, - force: true, - verbose - }) - - try { - if (verbose) { - // eslint-disable-next-line @typescript-eslint/no-explicit-any - emitter.on('info', (info: any) => { - console.log(info.message) - }) - } - - // Make degit write to a temporary directory - const tmpDir = mkdtempSync(joinPath(tmpdir(), 'sde-create-')) - await emitter.clone(tmpDir) - - // degit does not return an error when an invalid template is provided, as such we - // need to handle this manually - if (!existsSync(tmpDir) || readdirSync(tmpDir).length === 0) { - throw new Error('The requested template failed to download') - } - - // Copy files to destination without overwriting - await copy(tmpDir, dstDir, { - overwrite: false, - errorOnExist: false - }) - - // Remove the temporary directory - rmSync(tmpDir, { recursive: true, force: true }) - } catch (e) { - spinner.fail() - - // degit is compiled, so the stacktrace is pretty noisy. Only report the stacktrace when using verbose mode. - // logger.debug(err) - console.error(red(e.message)) - - // TODO: Handle common degit issues like below; for now, just log the error and exit - console.error(yellow('There was a problem copying the template.')) - console.error( - yellow( - 'Please file a new issue with the command output here: https://github.com/climateinteractive/sdeverywhere/issues' - ) - ) - process.exit(1) - - // // Warning for issue #655 and other corrupted cache issue - // if (e.message === 'zlib: unexpected end of file' || e.message === 'TAR_BAD_ARCHIVE: Unrecognized archive format') { - // console.log( - // yellow( - // // 'Local degit cache seems to be corrupted.' - // // 'For more information check out this issue: https://github.com/withastro/astro/issues/655.' - // ) - // ) - // const cacheIssueResponse = await prompts({ - // type: 'confirm', - // name: 'cache', - // message: 'Would you like us to clear the cache and try again?', - // initial: true - // }) - // if (cacheIssueResponse.cache) { - // const homeDirectory = os.homedir() - // const cacheDir = joinPath(homeDirectory, '.degit', 'github', '@sdeverywhere') - // rmSync(cacheDir, { recursive: true, force: true, maxRetries: 3 }) - // spinner = ora('Copying project files...').start() - // try { - // await emitter.clone(dstDir) - // } catch (e) { - // // logger.debug(e) - // console.error(red(e.message)) - // } - // } else { - // console.log( - // "Okay, no worries! To fix this manually, remove the folder '~/.degit/github/withastro' and rerun the command." - // ) - // } - // } - - // // Helpful message when encountering the "could not find commit hash for ..." error - // if (e.code === 'MISSING_REF') { - // console.log( - // yellow( - // "This seems to be an issue with degit. Please check if you have 'git' installed on your system, and if you don't, go here to install: https://git-scm.com" - // ) - // ) - // console.log( - // yellow( - // "If you do have 'git' installed, please file a new issue with the command output here: https://github.com/climateinteractive/sdeverywhere/issues" - // ) - // ) - // } - - // process.exit(1) - } -} - -async function chooseTemplate(projDir: string): Promise { - // Prompt the user - const options = await prompts( - [ - { - type: 'select', - name: 'template', - message: 'Which template would you like to use?', - choices: TEMPLATES - } - ], - { onCancel: () => ora().info(dim('Operation cancelled.')) } - ) - if (!options.template) { - process.exit(1) - } - - // Handle response - const templateSpinner = ora('Copying project files...').start() - const templateTarget = `climateinteractive/SDEverywhere/examples/${options.template}` - // TODO: Fix this before branch is merged - const hash = '#chris/228-create-package' - - // Copy the template files to the project directory - if (!args.dryRun) { - await runDegit(templateTarget, hash, projDir, templateSpinner) - } - - templateSpinner.text = green('Template copied!') - templateSpinner.succeed() -} - -async function chooseMdlFile(projDir: string): Promise { - // Find all `.mdl` files in the project directory - // From https://stackoverflow.com/a/45130990 - async function getFiles(dir: string): Promise { - const dirents = await readdir(dir, { withFileTypes: true }) - const files = await Promise.all( - dirents.map(dirent => { - const res = resolvePath(dir, dirent.name) - return dirent.isDirectory() ? getFiles(res) : res - }) - ) - return files.flat() - } - const allFiles = await getFiles(projDir) - const mdlFiles = allFiles.filter(f => f.endsWith('.mdl')).map(f => relative(projDir, f)) - const mdlChoices = mdlFiles.map(f => { - return { - title: f, - value: f - } as Choice - }) - - let mdlFile: string - if (mdlFiles.length === 0) { - // No mdl files found; print error message and exit - // TODO: Offer to add a basic mdl to get the user started - ora({ - color: 'red', - text: `No mdl files were found in "${projDir}". Add your mdl file to that directory and try again.` - }).fail() - process.exit(1) - } else if (mdlFiles.length === 1) { - // Only one mdl file - ora().succeed(`Found "${mdlFiles[0]}", will configure the project to use that mdl file.`) - mdlFile = mdlFiles[0] - } else { - // Multiple mdl files found; allow the user to choose one - // TODO: Eventually we should allow the user to choose to flatten if there are multiple submodels - const options = await prompts( - [ - { - type: 'select', - name: 'mdlFile', - message: 'It looks like there are multiple mdl files. Which one would you like to use?', - choices: mdlChoices - } - ], - { onCancel: () => ora().info(dim('Operation cancelled.')) } - ) - if (!options.mdlFile) { - process.exit(1) - } - mdlFile = options.mdlFile - } - - ora({ - color: 'green', - text: green(`Using "${bold(mdlFile)}" as the model for the project.`) - }).succeed() - - return mdlFile -} - -async function chooseInstallEmsdk(projDir: string): Promise { - // TODO: Use findUp and skip this step if emsdk directory already exists - - // Prompt the user - const underParentDir = resolvePath(projDir, '..', 'emsdk') - const underProjDir = joinPath(projDir, 'emsdk') - const installResponse = await prompts( - { - type: 'select', - name: 'install', - message: `Would you like to install the Emscripten SDK?`, - choices: [ - // ${reset(dim('(recommended)')) - { - title: `Install under parent directory (${bold(underParentDir)})`, - description: 'This is recommended so that it can be shared by multiple projects', - value: 'parent' - }, - { - title: `Install under project directory (${bold(underProjDir)})"`, - description: 'This is useful for keeping everything under a single project directory', - value: 'project' - }, - { - title: `Don't install`, - description: `It's OK, you can install it later`, - value: 'skip' - } - ] - }, - { - onCancel: () => { - ora().info( - dim( - 'Operation cancelled. Your project folder has been created, but the Emscripten SDK and other dependencies have not been installed.' - ) - ) - process.exit(1) - } - } - ) - - // Handle response - if (args.dryRun) { - ora().info(dim(`--dry-run enabled, skipping.`)) - return - } else if (installResponse.install === 'skip') { - ora().info( - dim('No problem! Be sure to install the Emscripten SDK and configure it in `sde.config.js` after setup.') - ) - return - } - - const installDir = installResponse.install === 'parent' ? underParentDir : underProjDir - try { - // TODO: Use spinner here - await installEmscripten(installDir) - } catch (e) { - ora({ - color: 'red', - text: red(`Failed to install Emscripten SDK: ${e.message}`) - }).fail() - process.exit(1) - } - - ora({ - color: 'green', - text: green(`Installed the Emscripten SDK in "${bold(installDir)}"`) - }).succeed() -} - -async function chooseInstallDeps(projDir: string): Promise { - // Prompt the user - const installResponse = await prompts( - { - type: 'confirm', - name: 'install', - message: `Would you like to install ${pkgManager} dependencies? ${reset(dim('(recommended)'))}`, - initial: true - }, - { - onCancel: () => { - ora().info( - dim('Operation cancelled. Your project folder has been created, but no dependencies have been installed.') - ) - process.exit(1) - } - } - ) - - // Handle response - if (args.dryRun) { - ora().info(dim(`--dry-run enabled, skipping.`)) - } else if (installResponse.install) { - const installExec = execa(pkgManager, ['install'], { cwd: projDir }) - const installingPackagesMsg = 'Installing packages...' - const installSpinner = ora(installingPackagesMsg).start() - await new Promise((resolve, reject) => { - installExec.stdout?.on('data', function (data) { - installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` - }) - installExec.on('error', error => reject(error)) - installExec.on('close', () => resolve()) - }) - installSpinner.text = green('Packages installed!') - installSpinner.succeed() - } else { - ora().info(dim(`No problem! Remember to install dependencies after setup.`)) - } -} +export async function main(): Promise { + // Detect the package manager + const pkgManager = detectPackageManager()?.name || 'npm' -async function chooseGitInit(projDir: string): Promise { - // Prompt the user - const gitResponse = await prompts( - { - type: 'confirm', - name: 'git', - message: `Would you like to initialize a new git repository? ${reset(dim('(optional)'))}`, - initial: true - }, - { - onCancel: () => { - ora().info(dim('Operation cancelled. Your project folder has already been created.')) - process.exit(1) - } - } - ) - if (args.dryRun) { - ora().info(dim(`--dry-run enabled, skipping.`)) - } else if (gitResponse.git) { - await execaCommand('git init', { cwd: projDir }) - ora().succeed(green('Git repository created!')) - } else { - ora().info(dim(`No problem! You can come back and run ${cyan(`git init`)} later.`)) - } -} + // Parse command line arguments + const args = yargs(process.argv) + prompts.override(args) -export async function main(): Promise { // Display welcome message console.log(`\n${bold('Welcome to SDEverywhere!')}`) console.log(`Let's create a new SDEverywhere project for your model.\n`) // Prompt the user to select a project directory - const projDir = await chooseProjectDir() + const projDir = await chooseProjectDir(args) - if (args.TODO) { - // Prompt the user to select a template - await chooseTemplate(projDir) + // Prompt the user to select a template + await chooseTemplate(projDir, args) - // Prompt the user to select an mdl file - const mdlPath = await chooseMdlFile(projDir) - // TODO - console.log(mdlPath) - } + // Prompt the user to select an mdl file + const mdlPath = await chooseMdlFile(projDir) + // TODO + console.log(mdlPath) // TODO: Prompt the user to choose input/output vars (if default template chosen) // Prompt the user to install Emscripten SDK - await chooseInstallEmsdk(projDir) + await chooseInstallEmsdk(projDir, args) // Prompt the user to install dependencies - await chooseInstallDeps(projDir) + await chooseInstallDeps(projDir, args, pkgManager) // Prompt the user to initialize a git repo - await chooseGitInit(projDir) + await chooseGitInit(projDir, args) console.log() ora({ text: green('Setup complete!') }).succeed() diff --git a/packages/create/src/install-emsdk.ts b/packages/create/src/install-emsdk.ts deleted file mode 100644 index 6a35e1ef..00000000 --- a/packages/create/src/install-emsdk.ts +++ /dev/null @@ -1,54 +0,0 @@ -// Copyright (c) 2022 Climate Interactive / New Venture Fund - -import { existsSync, rmSync } from 'fs' -import { join as joinPath } from 'path' - -import { execa } from 'execa' - -// TODO: Make this configurable; we're using this older version for now because it is -// relatively stable and has been used to build En-ROADS and C-ROADS for a long time -const version = '2.0.34' - -/** - * Install the Emscripten SDK to the `emsdk` directory under the - * given directory (if `emsdk` is not already present), then - * activates the requested version (specified with `version`). - * - * The implementation is similar to the existing `setup-emsdk` action - * (https://github.com/mymindstorm/setup-emsdk) except that one has issues - * with the cache directory on Windows, so having our own script gives us - * more control over installation and caching behavior. - */ -export async function installEmscripten(emsdkDir: string /*, options?: { verbose?: boolean }*/): Promise { - // Only download if the emsdk directory wasn't restored from a cache - if (!existsSync(emsdkDir)) { - console.log(`Downloading Emscripten SDK to ${emsdkDir}`) - // console.log() - await execa('git', ['clone', 'https://github.com/emscripten-core/emsdk.git', emsdkDir]) - } else { - console.log(`Found existing Emscripten SDK directory: ${emsdkDir}`) - } - // console.log() - - if (process.env.CI) { - // On GitHub Actions, remove the `.git` directory to keep the cache smaller. - // When we update to a new version, the cache key will miss and the emsdk - // repo will be redownloaded. - console.log('CI detected, removing .git directory...') - const gitDir = joinPath(emsdkDir, '.git') - rmSync(gitDir, { recursive: true, force: true }) - } else { - // For local development, pull to get the latest - console.log('Local development detected, performing git pull...') - await execa('git', ['pull'], { cwd: emsdkDir }) - // console.log() - } - - const emsdkCmd = async (...args: string[]) => { - return execa('python3', ['emsdk.py', ...args], { cwd: emsdkDir }) - } - - console.log(`Activating Emscripten SDK ${version}...`) - await emsdkCmd('install', version) - await emsdkCmd('activate', version) -} diff --git a/packages/create/src/step-deps.ts b/packages/create/src/step-deps.ts new file mode 100644 index 00000000..d66bc64d --- /dev/null +++ b/packages/create/src/step-deps.ts @@ -0,0 +1,47 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { execa } from 'execa' +import { bold, dim, green, reset } from 'kleur/colors' +import ora from 'ora' +import prompts from 'prompts' +import type { Arguments } from 'yargs-parser' + +export async function chooseInstallDeps(projDir: string, args: Arguments, pkgManager: string): Promise { + // Prompt the user + const installResponse = await prompts( + { + type: 'confirm', + name: 'install', + message: `Would you like to install ${pkgManager} dependencies? ${reset(dim('(recommended)'))}`, + initial: true + }, + { + onCancel: () => { + ora().info( + dim('Operation cancelled. Your project folder has been created, but no dependencies have been installed.') + ) + process.exit(1) + } + } + ) + + // Handle response + if (args.dryRun) { + ora().info(dim(`--dry-run enabled, skipping.`)) + } else if (installResponse.install) { + const installExec = execa(pkgManager, ['install'], { cwd: projDir }) + const installingPackagesMsg = 'Installing packages...' + const installSpinner = ora(installingPackagesMsg).start() + await new Promise((resolve, reject) => { + installExec.stdout?.on('data', function (data) { + installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` + }) + installExec.on('error', error => reject(error)) + installExec.on('close', () => resolve()) + }) + installSpinner.text = green('Packages installed!') + installSpinner.succeed() + } else { + ora().info(dim(`No problem! Remember to install dependencies after setup.`)) + } +} diff --git a/packages/create/src/step-directory.ts b/packages/create/src/step-directory.ts new file mode 100644 index 00000000..be635a9c --- /dev/null +++ b/packages/create/src/step-directory.ts @@ -0,0 +1,84 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { existsSync } from 'fs' +import { resolve as resolvePath } from 'path' + +import { bold, dim, green } from 'kleur/colors' +import ora from 'ora' +import prompts from 'prompts' +import type { Arguments } from 'yargs-parser' + +export async function chooseProjectDir(args: Arguments): Promise { + /** + * Return true if the given directory does not exist, or it does not contain important files + * like `package.json`. + */ + function isValidDir(dir: string): boolean { + const packageJson = resolvePath(dir, 'package.json') + const packagesDir = resolvePath(dir, 'packages') + return !existsSync(dir) || (!existsSync(packageJson) && !existsSync(packagesDir)) + } + + const showValidDirMsg = (dir: string) => { + ora({ + color: 'green', + text: green(`Using "${bold(dir)}" as the project directory.`) + }).succeed() + } + + const showInvalidDirMsg = (dir: string) => { + ora({ + color: 'red', + text: `"${bold(dir)}" contains existing 'package.json' and/or 'packages' directory, stopping.` + }).fail() + } + + // See if project directory is provided on command line + let projDir = args['_'][2] as string + if (projDir) { + // Directory was provided, see if it is valid + if (isValidDir(projDir)) { + // The provided directory is valid, so proceed + showValidDirMsg(projDir) + } else { + // The provided directory is not valid, so show error message and exit + showInvalidDirMsg(projDir) + process.exit(1) + } + } else { + // Directory was not provided, so prompt the user + const dirResponse = await prompts( + { + type: 'text', + name: 'directory', + message: 'Where would you like to create your new project?', + initial: '' + // validate(value) { + // if (value === '') { + // value = process.cwd() + // } + // if (!isValidDir(value)) { + // return notValidMsg(value) + // } + // return true + // } + }, + { onCancel: () => ora().info(dim('Operation cancelled.')) } + ) + if (!projDir) { + process.exit(1) + } + projDir = dirResponse.directory + if (projDir === '') { + projDir = process.cwd() + } + if (isValidDir(projDir)) { + showValidDirMsg(projDir) + } else { + showInvalidDirMsg(projDir) + process.exit(1) + } + } + + return projDir +} diff --git a/packages/create/src/step-emsdk.ts b/packages/create/src/step-emsdk.ts new file mode 100644 index 00000000..0303fb26 --- /dev/null +++ b/packages/create/src/step-emsdk.ts @@ -0,0 +1,129 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { existsSync, rmSync } from 'fs' +import { join as joinPath, resolve as resolvePath } from 'path' + +import { execa } from 'execa' +import { bold, dim, green, red } from 'kleur/colors' +import ora from 'ora' +import prompts from 'prompts' +import type { Arguments } from 'yargs-parser' + +// TODO: Make this configurable; we're using this older version for now because it is +// relatively stable and has been used to build En-ROADS and C-ROADS for a long time +const version = '2.0.34' + +export async function chooseInstallEmsdk(projDir: string, args: Arguments): Promise { + // TODO: Use findUp and skip this step if emsdk directory already exists + + // Prompt the user + const underParentDir = resolvePath(projDir, '..', 'emsdk') + const underProjDir = joinPath(projDir, 'emsdk') + const installResponse = await prompts( + { + type: 'select', + name: 'install', + message: `Would you like to install the Emscripten SDK?`, + choices: [ + // ${reset(dim('(recommended)')) + { + title: `Install under parent directory (${bold(underParentDir)})`, + description: 'This is recommended so that it can be shared by multiple projects', + value: 'parent' + }, + { + title: `Install under project directory (${bold(underProjDir)})"`, + description: 'This is useful for keeping everything under a single project directory', + value: 'project' + }, + { + title: `Don't install`, + description: `It's OK, you can install it later`, + value: 'skip' + } + ] + }, + { + onCancel: () => { + ora().info( + dim( + 'Operation cancelled. Your project folder has been created, but the Emscripten SDK and other dependencies have not been installed.' + ) + ) + process.exit(1) + } + } + ) + + // Handle response + if (args.dryRun) { + ora().info(dim(`--dry-run enabled, skipping.`)) + return + } else if (installResponse.install === 'skip') { + ora().info( + dim('No problem! Be sure to install the Emscripten SDK and configure it in `sde.config.js` after setup.') + ) + return + } + + const installDir = installResponse.install === 'parent' ? underParentDir : underProjDir + try { + // TODO: Use spinner here + await installEmscripten(installDir) + } catch (e) { + ora({ + color: 'red', + text: red(`Failed to install Emscripten SDK: ${e.message}`) + }).fail() + process.exit(1) + } + + ora({ + color: 'green', + text: green(`Installed the Emscripten SDK in "${bold(installDir)}"`) + }).succeed() +} + +/** + * Install the Emscripten SDK to the `emsdk` directory under the + * given directory (if `emsdk` is not already present), then + * activates the requested version (specified with `version`). + * + * The implementation is similar to the existing `setup-emsdk` action + * (https://github.com/mymindstorm/setup-emsdk) except that one has issues + * with the cache directory on Windows, so having our own script gives us + * more control over installation and caching behavior. + */ +async function installEmscripten(emsdkDir: string /*, options?: { verbose?: boolean }*/): Promise { + // Only download if the emsdk directory wasn't restored from a cache + if (!existsSync(emsdkDir)) { + console.log(`Downloading Emscripten SDK to ${emsdkDir}`) + // console.log() + await execa('git', ['clone', 'https://github.com/emscripten-core/emsdk.git', emsdkDir]) + } else { + console.log(`Found existing Emscripten SDK directory: ${emsdkDir}`) + } + // console.log() + + if (process.env.CI) { + // On GitHub Actions, remove the `.git` directory to keep the cache smaller. + // When we update to a new version, the cache key will miss and the emsdk + // repo will be redownloaded. + console.log('CI detected, removing .git directory...') + const gitDir = joinPath(emsdkDir, '.git') + rmSync(gitDir, { recursive: true, force: true }) + } else { + // For local development, pull to get the latest + console.log('Local development detected, performing git pull...') + await execa('git', ['pull'], { cwd: emsdkDir }) + // console.log() + } + + const emsdkCmd = async (...args: string[]) => { + return execa('python3', ['emsdk.py', ...args], { cwd: emsdkDir }) + } + + console.log(`Activating Emscripten SDK ${version}...`) + await emsdkCmd('install', version) + await emsdkCmd('activate', version) +} diff --git a/packages/create/src/step-git.ts b/packages/create/src/step-git.ts new file mode 100644 index 00000000..c352d88c --- /dev/null +++ b/packages/create/src/step-git.ts @@ -0,0 +1,33 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { execaCommand } from 'execa' +import { cyan, dim, green, reset } from 'kleur/colors' +import ora from 'ora' +import prompts from 'prompts' +import type { Arguments } from 'yargs-parser' + +export async function chooseGitInit(projDir: string, args: Arguments): Promise { + // Prompt the user + const gitResponse = await prompts( + { + type: 'confirm', + name: 'git', + message: `Would you like to initialize a new git repository? ${reset(dim('(optional)'))}`, + initial: true + }, + { + onCancel: () => { + ora().info(dim('Operation cancelled. Your project folder has already been created.')) + process.exit(1) + } + } + ) + if (args.dryRun) { + ora().info(dim(`--dry-run enabled, skipping.`)) + } else if (gitResponse.git) { + await execaCommand('git init', { cwd: projDir }) + ora().succeed(green('Git repository created!')) + } else { + ora().info(dim(`No problem! You can come back and run ${cyan(`git init`)} later.`)) + } +} diff --git a/packages/create/src/step-mdl.ts b/packages/create/src/step-mdl.ts new file mode 100644 index 00000000..f0c9bb5f --- /dev/null +++ b/packages/create/src/step-mdl.ts @@ -0,0 +1,72 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { readdir } from 'fs/promises' +import { relative, resolve as resolvePath } from 'path' + +import { bold, dim, green } from 'kleur/colors' +import ora from 'ora' +import type { Choice } from 'prompts' +import prompts from 'prompts' + +export async function chooseMdlFile(projDir: string): Promise { + // Find all `.mdl` files in the project directory + // From https://stackoverflow.com/a/45130990 + async function getFiles(dir: string): Promise { + const dirents = await readdir(dir, { withFileTypes: true }) + const files = await Promise.all( + dirents.map(dirent => { + const res = resolvePath(dir, dirent.name) + return dirent.isDirectory() ? getFiles(res) : res + }) + ) + return files.flat() + } + const allFiles = await getFiles(projDir) + const mdlFiles = allFiles.filter(f => f.endsWith('.mdl')).map(f => relative(projDir, f)) + const mdlChoices = mdlFiles.map(f => { + return { + title: f, + value: f + } as Choice + }) + + let mdlFile: string + if (mdlFiles.length === 0) { + // No mdl files found; print error message and exit + // TODO: Offer to add a basic mdl to get the user started + ora({ + color: 'red', + text: `No mdl files were found in "${projDir}". Add your mdl file to that directory and try again.` + }).fail() + process.exit(1) + } else if (mdlFiles.length === 1) { + // Only one mdl file + ora().succeed(`Found "${mdlFiles[0]}", will configure the project to use that mdl file.`) + mdlFile = mdlFiles[0] + } else { + // Multiple mdl files found; allow the user to choose one + // TODO: Eventually we should allow the user to choose to flatten if there are multiple submodels + const options = await prompts( + [ + { + type: 'select', + name: 'mdlFile', + message: 'It looks like there are multiple mdl files. Which one would you like to use?', + choices: mdlChoices + } + ], + { onCancel: () => ora().info(dim('Operation cancelled.')) } + ) + if (!options.mdlFile) { + process.exit(1) + } + mdlFile = options.mdlFile + } + + ora({ + color: 'green', + text: green(`Using "${bold(mdlFile)}" as the model for the project.`) + }).succeed() + + return mdlFile +} diff --git a/packages/create/src/step-template.ts b/packages/create/src/step-template.ts new file mode 100644 index 00000000..2048ee88 --- /dev/null +++ b/packages/create/src/step-template.ts @@ -0,0 +1,171 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { existsSync, mkdtempSync, readdirSync, rmSync } from 'fs' +import { copy } from 'fs-extra' +import { tmpdir } from 'os' +import { join as joinPath } from 'path' + +import degit from 'degit' +import { dim, green, red, yellow } from 'kleur/colors' +import type { Ora } from 'ora' +import ora from 'ora' +import prompts from 'prompts' +import type { Arguments } from 'yargs-parser' + +const TEMPLATES = [ + { + title: 'Default project', + description: 'Includes recommended structure with config files, app, core library, model-check, etc', + value: 'template-default' + }, + { + title: 'Minimal project', + description: 'Includes simple config for model-check', + value: 'template-minimal' + } +] + +export async function chooseTemplate(projDir: string, args: Arguments): Promise { + // Prompt the user + const options = await prompts( + [ + { + type: 'select', + name: 'template', + message: 'Which template would you like to use?', + choices: TEMPLATES + } + ], + { onCancel: () => ora().info(dim('Operation cancelled.')) } + ) + if (!options.template) { + process.exit(1) + } + + // Handle response + const templateSpinner = ora('Copying project files...').start() + const templateTarget = `climateinteractive/SDEverywhere/examples/${options.template}` + // TODO: Fix this before branch is merged + const hash = '#chris/228-create-package' + + // Copy the template files to the project directory + if (!args.dryRun) { + await runDegit(templateTarget, hash, projDir, args, templateSpinner) + } + + templateSpinner.text = green('Template copied!') + templateSpinner.succeed() +} + +// XXX: This is mostly copied from Astro's create package: +// https://github.com/withastro/astro/blob/main/packages/create-astro/src/index.ts +// It contains workarounds for degit issues that may or may not be relevant for SDE, +// so this should be re-evaluated later. +async function runDegit( + templateTarget: string, + hash: string, + dstDir: string, + args: Arguments, + spinner: Ora +): Promise { + // Enable verbose degit logging if the --verbose flag is used + const verbose = args.verbose + + // Set up degit (we will be writing to a temporary directory, so force is safe) + const emitter = degit(`${templateTarget}${hash}`, { + cache: false, + force: true, + verbose + }) + + try { + if (verbose) { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + emitter.on('info', (info: any) => { + console.log(info.message) + }) + } + + // Make degit write to a temporary directory + const tmpDir = mkdtempSync(joinPath(tmpdir(), 'sde-create-')) + await emitter.clone(tmpDir) + + // degit does not return an error when an invalid template is provided, as such we + // need to handle this manually + if (!existsSync(tmpDir) || readdirSync(tmpDir).length === 0) { + throw new Error('The requested template failed to download') + } + + // Copy files to destination without overwriting + await copy(tmpDir, dstDir, { + overwrite: false, + errorOnExist: false + }) + + // Remove the temporary directory + rmSync(tmpDir, { recursive: true, force: true }) + } catch (e) { + spinner.fail() + + // degit is compiled, so the stacktrace is pretty noisy. Only report the stacktrace when using verbose mode. + // logger.debug(err) + console.error(red(e.message)) + + // TODO: Handle common degit issues like below; for now, just log the error and exit + console.error(yellow('There was a problem copying the template.')) + console.error( + yellow( + 'Please file a new issue with the command output here: https://github.com/climateinteractive/sdeverywhere/issues' + ) + ) + process.exit(1) + + // // Warning for issue #655 and other corrupted cache issue + // if (e.message === 'zlib: unexpected end of file' || e.message === 'TAR_BAD_ARCHIVE: Unrecognized archive format') { + // console.log( + // yellow( + // // 'Local degit cache seems to be corrupted.' + // // 'For more information check out this issue: https://github.com/withastro/astro/issues/655.' + // ) + // ) + // const cacheIssueResponse = await prompts({ + // type: 'confirm', + // name: 'cache', + // message: 'Would you like us to clear the cache and try again?', + // initial: true + // }) + // if (cacheIssueResponse.cache) { + // const homeDirectory = os.homedir() + // const cacheDir = joinPath(homeDirectory, '.degit', 'github', '@sdeverywhere') + // rmSync(cacheDir, { recursive: true, force: true, maxRetries: 3 }) + // spinner = ora('Copying project files...').start() + // try { + // await emitter.clone(dstDir) + // } catch (e) { + // // logger.debug(e) + // console.error(red(e.message)) + // } + // } else { + // console.log( + // "Okay, no worries! To fix this manually, remove the folder '~/.degit/github/withastro' and rerun the command." + // ) + // } + // } + + // // Helpful message when encountering the "could not find commit hash for ..." error + // if (e.code === 'MISSING_REF') { + // console.log( + // yellow( + // "This seems to be an issue with degit. Please check if you have 'git' installed on your system, and if you don't, go here to install: https://git-scm.com" + // ) + // ) + // console.log( + // yellow( + // "If you do have 'git' installed, please file a new issue with the command output here: https://github.com/climateinteractive/sdeverywhere/issues" + // ) + // ) + // } + + // process.exit(1) + } +} From cd7d9a3a43e84b98422f486ac3bc50796c578f55 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 11:09:55 -0700 Subject: [PATCH 07/23] test: rename test to match source name --- .../tests/{create-dirs.spec.ts => step-directory.spec.ts} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename packages/create/tests/{create-dirs.spec.ts => step-directory.spec.ts} (98%) diff --git a/packages/create/tests/create-dirs.spec.ts b/packages/create/tests/step-directory.spec.ts similarity index 98% rename from packages/create/tests/create-dirs.spec.ts rename to packages/create/tests/step-directory.spec.ts index a55a9b54..95df04b0 100644 --- a/packages/create/tests/create-dirs.spec.ts +++ b/packages/create/tests/step-directory.spec.ts @@ -28,7 +28,7 @@ function runCreate(args: string[] = []) { } } -describe('create directories', () => { +describe('step - create project directory', () => { // it('should stop if directory provided on command line contains important files', () => { // // TODO // }) From 4e7572642c8c0d7a2313126a5be61cdb0dc6c160 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 11:11:59 -0700 Subject: [PATCH 08/23] fix: use exit code 0 since tool is interactive and exit code is less important --- packages/create/src/step-deps.ts | 2 +- packages/create/src/step-directory.ts | 13 +++++++++---- packages/create/src/step-emsdk.ts | 4 ++-- packages/create/src/step-git.ts | 2 +- packages/create/src/step-mdl.ts | 12 +++++++----- packages/create/src/step-template.ts | 12 +++++++----- 6 files changed, 27 insertions(+), 18 deletions(-) diff --git a/packages/create/src/step-deps.ts b/packages/create/src/step-deps.ts index d66bc64d..855fd947 100644 --- a/packages/create/src/step-deps.ts +++ b/packages/create/src/step-deps.ts @@ -20,7 +20,7 @@ export async function chooseInstallDeps(projDir: string, args: Arguments, pkgMan ora().info( dim('Operation cancelled. Your project folder has been created, but no dependencies have been installed.') ) - process.exit(1) + process.exit(0) } } ) diff --git a/packages/create/src/step-directory.ts b/packages/create/src/step-directory.ts index be635a9c..2f19acab 100644 --- a/packages/create/src/step-directory.ts +++ b/packages/create/src/step-directory.ts @@ -43,7 +43,7 @@ export async function chooseProjectDir(args: Arguments): Promise { } else { // The provided directory is not valid, so show error message and exit showInvalidDirMsg(projDir) - process.exit(1) + process.exit(0) } } else { // Directory was not provided, so prompt the user @@ -63,10 +63,15 @@ export async function chooseProjectDir(args: Arguments): Promise { // return true // } }, - { onCancel: () => ora().info(dim('Operation cancelled.')) } + { + onCancel: () => { + ora().info(dim('Operation cancelled.')) + process.exit(0) + } + } ) if (!projDir) { - process.exit(1) + process.exit(0) } projDir = dirResponse.directory if (projDir === '') { @@ -76,7 +81,7 @@ export async function chooseProjectDir(args: Arguments): Promise { showValidDirMsg(projDir) } else { showInvalidDirMsg(projDir) - process.exit(1) + process.exit(0) } } diff --git a/packages/create/src/step-emsdk.ts b/packages/create/src/step-emsdk.ts index 0303fb26..4c9bd8ee 100644 --- a/packages/create/src/step-emsdk.ts +++ b/packages/create/src/step-emsdk.ts @@ -50,7 +50,7 @@ export async function chooseInstallEmsdk(projDir: string, args: Arguments): Prom 'Operation cancelled. Your project folder has been created, but the Emscripten SDK and other dependencies have not been installed.' ) ) - process.exit(1) + process.exit(0) } } ) @@ -75,7 +75,7 @@ export async function chooseInstallEmsdk(projDir: string, args: Arguments): Prom color: 'red', text: red(`Failed to install Emscripten SDK: ${e.message}`) }).fail() - process.exit(1) + process.exit(0) } ora({ diff --git a/packages/create/src/step-git.ts b/packages/create/src/step-git.ts index c352d88c..8255d7a5 100644 --- a/packages/create/src/step-git.ts +++ b/packages/create/src/step-git.ts @@ -18,7 +18,7 @@ export async function chooseGitInit(projDir: string, args: Arguments): Promise { ora().info(dim('Operation cancelled. Your project folder has already been created.')) - process.exit(1) + process.exit(0) } } ) diff --git a/packages/create/src/step-mdl.ts b/packages/create/src/step-mdl.ts index f0c9bb5f..194dfe54 100644 --- a/packages/create/src/step-mdl.ts +++ b/packages/create/src/step-mdl.ts @@ -38,7 +38,7 @@ export async function chooseMdlFile(projDir: string): Promise { color: 'red', text: `No mdl files were found in "${projDir}". Add your mdl file to that directory and try again.` }).fail() - process.exit(1) + process.exit(0) } else if (mdlFiles.length === 1) { // Only one mdl file ora().succeed(`Found "${mdlFiles[0]}", will configure the project to use that mdl file.`) @@ -55,11 +55,13 @@ export async function chooseMdlFile(projDir: string): Promise { choices: mdlChoices } ], - { onCancel: () => ora().info(dim('Operation cancelled.')) } + { + onCancel: () => { + ora().info(dim('Operation cancelled.')) + process.exit(0) + } + } ) - if (!options.mdlFile) { - process.exit(1) - } mdlFile = options.mdlFile } diff --git a/packages/create/src/step-template.ts b/packages/create/src/step-template.ts index 2048ee88..c401b126 100644 --- a/packages/create/src/step-template.ts +++ b/packages/create/src/step-template.ts @@ -36,11 +36,13 @@ export async function chooseTemplate(projDir: string, args: Arguments): Promise< choices: TEMPLATES } ], - { onCancel: () => ora().info(dim('Operation cancelled.')) } + { + onCancel: () => { + ora().info(dim('Operation cancelled.')) + process.exit(0) + } + } ) - if (!options.template) { - process.exit(1) - } // Handle response const templateSpinner = ora('Copying project files...').start() @@ -118,7 +120,7 @@ async function runDegit( 'Please file a new issue with the command output here: https://github.com/climateinteractive/sdeverywhere/issues' ) ) - process.exit(1) + process.exit(0) // // Warning for issue #655 and other corrupted cache issue // if (e.message === 'zlib: unexpected end of file' || e.message === 'TAR_BAD_ARCHIVE: Unrecognized archive format') { From 321a0c28d88bde1766a1578ee620d19ff3660838 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 11:25:28 -0700 Subject: [PATCH 09/23] fix: handle errors in install deps step --- packages/create/src/step-deps.ts | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/packages/create/src/step-deps.ts b/packages/create/src/step-deps.ts index 855fd947..c4256d57 100644 --- a/packages/create/src/step-deps.ts +++ b/packages/create/src/step-deps.ts @@ -1,7 +1,7 @@ // Copyright (c) 2022 Climate Interactive / New Venture Fund import { execa } from 'execa' -import { bold, dim, green, reset } from 'kleur/colors' +import { bold, cyan, dim, green, reset, yellow } from 'kleur/colors' import ora from 'ora' import prompts from 'prompts' import type { Arguments } from 'yargs-parser' @@ -32,15 +32,28 @@ export async function chooseInstallDeps(projDir: string, args: Arguments, pkgMan const installExec = execa(pkgManager, ['install'], { cwd: projDir }) const installingPackagesMsg = 'Installing packages...' const installSpinner = ora(installingPackagesMsg).start() - await new Promise((resolve, reject) => { - installExec.stdout?.on('data', function (data) { - installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` + try { + await new Promise((resolve, reject) => { + installExec.stdout?.on('data', function (data) { + installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` + }) + installExec.stderr?.on('data', function (data) { + installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` + }) + installExec.on('error', error => reject(error)) + installExec.on('exit', code => reject(`Install failed (code=${code})`)) + installExec.on('close', () => resolve()) }) - installExec.on('error', error => reject(error)) - installExec.on('close', () => resolve()) - }) - installSpinner.text = green('Packages installed!') - installSpinner.succeed() + installSpinner.text = green('Packages installed!') + installSpinner.succeed() + } catch (e) { + installSpinner.text = yellow( + `There was an error installing packages. Try running ${cyan( + `${pkgManager} install` + )} in your project directory later.` + ) + installSpinner.warn() + } } else { ora().info(dim(`No problem! Remember to install dependencies after setup.`)) } From 43b424797bc6861956d799d45287af8b20e6b390 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 11:38:12 -0700 Subject: [PATCH 10/23] refactor: make --dry-run handling consistent --- packages/create/src/step-deps.ts | 57 +++++++++++++++------------- packages/create/src/step-git.ts | 13 +++++-- packages/create/src/step-template.ts | 15 ++++---- 3 files changed, 47 insertions(+), 38 deletions(-) diff --git a/packages/create/src/step-deps.ts b/packages/create/src/step-deps.ts index c4256d57..18fffed4 100644 --- a/packages/create/src/step-deps.ts +++ b/packages/create/src/step-deps.ts @@ -28,33 +28,36 @@ export async function chooseInstallDeps(projDir: string, args: Arguments, pkgMan // Handle response if (args.dryRun) { ora().info(dim(`--dry-run enabled, skipping.`)) - } else if (installResponse.install) { - const installExec = execa(pkgManager, ['install'], { cwd: projDir }) - const installingPackagesMsg = 'Installing packages...' - const installSpinner = ora(installingPackagesMsg).start() - try { - await new Promise((resolve, reject) => { - installExec.stdout?.on('data', function (data) { - installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` - }) - installExec.stderr?.on('data', function (data) { - installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` - }) - installExec.on('error', error => reject(error)) - installExec.on('exit', code => reject(`Install failed (code=${code})`)) - installExec.on('close', () => resolve()) - }) - installSpinner.text = green('Packages installed!') - installSpinner.succeed() - } catch (e) { - installSpinner.text = yellow( - `There was an error installing packages. Try running ${cyan( - `${pkgManager} install` - )} in your project directory later.` - ) - installSpinner.warn() - } - } else { + return + } else if (!installResponse.install) { ora().info(dim(`No problem! Remember to install dependencies after setup.`)) + return + } + + // Install dependencies + const installExec = execa(pkgManager, ['install'], { cwd: projDir }) + const installingPackagesMsg = 'Installing packages...' + const installSpinner = ora(installingPackagesMsg).start() + try { + await new Promise((resolve, reject) => { + installExec.stdout?.on('data', function (data) { + installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` + }) + installExec.stderr?.on('data', function (data) { + installSpinner.text = `${installingPackagesMsg}\n${bold(`[${pkgManager}]`)} ${data}` + }) + installExec.on('error', error => reject(error)) + installExec.on('exit', code => reject(`Install failed (code=${code})`)) + installExec.on('close', () => resolve()) + }) + installSpinner.text = green('Packages installed!') + installSpinner.succeed() + } catch (e) { + installSpinner.text = yellow( + `There was an error installing packages. Try running ${cyan( + `${pkgManager} install` + )} in your project directory later.` + ) + installSpinner.warn() } } diff --git a/packages/create/src/step-git.ts b/packages/create/src/step-git.ts index 8255d7a5..ecfc2d20 100644 --- a/packages/create/src/step-git.ts +++ b/packages/create/src/step-git.ts @@ -22,12 +22,17 @@ export async function chooseGitInit(projDir: string, args: Arguments): Promise Date: Wed, 21 Sep 2022 12:10:27 -0700 Subject: [PATCH 11/23] fix: remove accidental exit --- packages/create/src/step-directory.ts | 3 --- 1 file changed, 3 deletions(-) diff --git a/packages/create/src/step-directory.ts b/packages/create/src/step-directory.ts index 2f19acab..82405bb2 100644 --- a/packages/create/src/step-directory.ts +++ b/packages/create/src/step-directory.ts @@ -70,9 +70,6 @@ export async function chooseProjectDir(args: Arguments): Promise { } } ) - if (!projDir) { - process.exit(0) - } projDir = dirResponse.directory if (projDir === '') { projDir = process.cwd() From ab9cd71783e19744ff0db80638899738b6552fad Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 15:37:45 -0700 Subject: [PATCH 12/23] fix: add workspaces to template-default package.json --- examples/template-default/package.json | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/examples/template-default/package.json b/examples/template-default/package.json index f980e43e..af295b7b 100644 --- a/examples/template-default/package.json +++ b/examples/template-default/package.json @@ -8,6 +8,10 @@ "dev": "sde dev", "start": "sde dev" }, + "workspaces": [ + "packages/core", + "packages/app" + ], "dependencies": { "@sdeverywhere/cli": "^0.7.0", "@sdeverywhere/plugin-check": "^0.1.0", From 331bf9f639b561033507612e2f1a81932cc6c81a Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 15:38:05 -0700 Subject: [PATCH 13/23] fix: change colors in template-default --- examples/template-default/config/colors.csv | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/examples/template-default/config/colors.csv b/examples/template-default/config/colors.csv index 84c72c7c..d8665198 100644 --- a/examples/template-default/config/colors.csv +++ b/examples/template-default/config/colors.csv @@ -1,3 +1,6 @@ id,hex code,name,comment -black,#000000,, blue,#0072b2,, +red,#d33700,, +green,#53bb37,, +gray,#a7a9ac,, +black,#000000,, From 7a1cd26abc54e16167128db84879a3035fb6ed61 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 15:38:26 -0700 Subject: [PATCH 14/23] fix: change model name placeholders in template-default --- examples/template-default/sde.config.js | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/examples/template-default/sde.config.js b/examples/template-default/sde.config.js index f267132a..b2c374bf 100644 --- a/examples/template-default/sde.config.js +++ b/examples/template-default/sde.config.js @@ -16,16 +16,16 @@ const corePath = (...parts) => packagePath('core', ...parts) export async function config() { return { // Specify the Vensim model to read - modelFiles: ['model/MODEL_NAME.mdl'], + modelFiles: ['MODEL_NAME.mdl'], // The following files will be hashed to determine whether the model needs // to be rebuilt when watch mode is active - modelInputPaths: ['model/*.mdl'], + modelInputPaths: ['MODEL_NAME.mdl'], // The following files will cause the model to be rebuilt when watch mode is // is active. Note that these are globs so we use forward slashes regardless // of platform. - watchPaths: ['config/**', 'model/*.mdl'], + watchPaths: ['config/**', 'MODEL_NAME.mdl'], // Read csv files from `config` directory modelSpec: configProcessor({ From 5b61fab5c5d8f578dcc3b4bcaf95c4a427f171d4 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 15:39:25 -0700 Subject: [PATCH 15/23] feat: add support for generating graphs.csv and inputs.csv --- packages/create/package.json | 2 + packages/create/src/index.ts | 21 +- packages/create/src/step-config.ts | 381 +++++++++++++++++++++++++++ packages/create/src/step-emsdk.ts | 6 +- packages/create/src/step-mdl.ts | 5 +- packages/create/src/step-template.ts | 4 +- packages/create/src/types.d.ts | 4 + pnpm-lock.yaml | 4 + 8 files changed, 415 insertions(+), 12 deletions(-) create mode 100644 packages/create/src/step-config.ts create mode 100644 packages/create/src/types.d.ts diff --git a/packages/create/package.json b/packages/create/package.json index fb7edb1c..1184c5b5 100644 --- a/packages/create/package.json +++ b/packages/create/package.json @@ -25,6 +25,7 @@ "ci:build": "run-s clean lint prettier:check type-check build test:ci" }, "dependencies": { + "@sdeverywhere/compile": "^0.7.0", "degit": "^2.8.4", "execa": "^6.1.0", "fs-extra": "^10.1.0", @@ -32,6 +33,7 @@ "ora": "^6.1.2", "prompts": "^2.4.2", "which-pm-runs": "^1.1.0", + "yaml": "^2.1.1", "yargs-parser": "^21.1.1" }, "devDependencies": { diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts index 7d33cf94..657dff18 100644 --- a/packages/create/src/index.ts +++ b/packages/create/src/index.ts @@ -8,6 +8,7 @@ import prompts from 'prompts' import detectPackageManager from 'which-pm-runs' import yargs from 'yargs-parser' +import { chooseGenConfig, updateSdeConfig } from './step-config' import { chooseInstallDeps } from './step-deps' import { chooseProjectDir } from './step-directory' import { chooseInstallEmsdk } from './step-emsdk' @@ -29,27 +30,37 @@ export async function main(): Promise { // Prompt the user to select a project directory const projDir = await chooseProjectDir(args) + console.log() // Prompt the user to select a template - await chooseTemplate(projDir, args) + const templateName = await chooseTemplate(projDir, args) + console.log() // Prompt the user to select an mdl file const mdlPath = await chooseMdlFile(projDir) - // TODO - console.log(mdlPath) - // TODO: Prompt the user to choose input/output vars (if default template chosen) + // Update the `sde.config.js` file to use the chosen mdl file + await updateSdeConfig(projDir, mdlPath) + console.log() + + // If the user chose the default template, offer to set up CSV files + if (templateName === 'template-default' && !args.dryRun) { + await chooseGenConfig(projDir, mdlPath) + console.log() + } // Prompt the user to install Emscripten SDK await chooseInstallEmsdk(projDir, args) + console.log() // Prompt the user to install dependencies await chooseInstallDeps(projDir, args, pkgManager) + console.log() // Prompt the user to initialize a git repo await chooseGitInit(projDir, args) - console.log() + ora({ text: green('Setup complete!') }).succeed() console.log(`\n${bgCyan(black(' Next steps '))}\n`) diff --git a/packages/create/src/step-config.ts b/packages/create/src/step-config.ts new file mode 100644 index 00000000..0d2a52de --- /dev/null +++ b/packages/create/src/step-config.ts @@ -0,0 +1,381 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +import { mkdir, readFile, writeFile } from 'fs/promises' +import { dirname, join as joinPath, parse as parsePath, resolve as resolvePath } from 'path' + +import { bold, cyan, dim, green, reset, yellow } from 'kleur/colors' +import ora from 'ora' +import type { Choice } from 'prompts' +import prompts from 'prompts' +import yaml from 'yaml' + +import { parseAndGenerate, preprocessModel } from '@sdeverywhere/compile' + +interface MdlConstVariable { + kind: 'const' + name: string + value: number +} + +interface MdlAuxVariable { + kind: 'aux' + name: string +} + +interface MdlLevelVariable { + kind: 'level' + name: string +} + +type MdlVariable = MdlConstVariable | MdlAuxVariable | MdlLevelVariable + +export async function updateSdeConfig(projDir: string, mdlPath: string): Promise { + // Read the `sde.config.js` file from the template + const configPath = joinPath(projDir, 'sde.config.js') + let configText = await readFile(configPath, 'utf8') + + // Replace instances of `MODEL_NAME.mdl` with the path to the chosen mdl file + configText = configText.replaceAll('MODEL_NAME.mdl', mdlPath) + + // Write the updated file + await writeFile(configPath, configText) +} + +export async function chooseGenConfig(projDir: string, mdlPath: string): Promise { + // TODO: For now we eagerly read the mdl file; maybe change this to only load it if + // the user chooses to generate graph and/or slider config + let mdlVars: MdlVariable[] + try { + // Get the list of variables available in the model + mdlVars = await readModelVars(projDir, mdlPath) + } catch (e) { + console.log(e) + ora( + yellow('The mdl file failed to load. We will continue setting things up, and you can diagnose the issue later.') + ).warn() + return + } + + // See if the user wants to generate graph config + await chooseGenGraphConfig(projDir, mdlVars) + console.log() + + // See if the user wants to generate slider config + await chooseGenSliderConfig(projDir, mdlVars) +} + +async function chooseGenGraphConfig(projDir: string, mdlVars: MdlVariable[]): Promise { + // Prompt the user + const genResponse = await prompts( + { + type: 'confirm', + name: 'genGraph', + message: `Would you like to configure a graph to get you started? ${reset(dim('(recommended)'))}`, + initial: true + }, + { + onCancel: () => { + ora().info(dim('Operation cancelled.')) + process.exit(0) + } + } + ) + + // Handle response + if (!genResponse.genGraph) { + ora().info(dim(`No problem! You can edit the "${cyan('config/graphs.csv')}" file later.`)) + return + } + + // Offer multi-select with available output variables + const outputVarNames: string[] = [] + for (const mdlVar of mdlVars) { + if (mdlVar.kind === 'aux' || mdlVar.kind === 'level') { + outputVarNames.push(mdlVar.name) + } + } + outputVarNames.sort((a, b) => { + return a.toLowerCase().localeCompare(b.toLowerCase()) + }) + const choices = outputVarNames.map(f => { + return { + title: f, + value: f + } as Choice + }) + const varsResponse = await prompts( + { + type: 'autocompleteMultiselect', + name: 'vars', + message: 'Choose up to three output variables to display in the graph', + choices, + max: 3 + }, + { + onCancel: () => { + ora().info(dim('Operation cancelled.')) + process.exit(0) + } + } + ) + + if (varsResponse.vars.length === 0) { + ora().info(dim(`No variables selected. You can edit the "${cyan('config/graphs.csv')}" file later.`)) + return + } + + // Add line to `graphs.csv` + const graphsCsvFile = joinPath(projDir, 'config', 'graphs.csv') + const csvLine = graphsCsvLine(varsResponse.vars) + let graphsCsvContent = await readFile(graphsCsvFile, 'utf8') + graphsCsvContent += `${csvLine}\n` + await writeFile(graphsCsvFile, graphsCsvContent) + + ora( + green(`Added graph to "${bold('config/graphs.csv')}". ${dim('You can configure graphs in that file later.')}`) + ).succeed() +} + +// TODO: Ideally the `plugin-config` package would expose an API for generating CSV, but +// until then, we will generate a comma-separated line of values in hardcoded fashion +function graphsCsvLine(outputVarNames: string[]): string { + const colors = ['blue', 'red', 'green'] + + // Fill the line with blanks initially, then set the small subset of fields + const a = Array(131).fill('') + // id + a[0] = '1' + // parent menu + a[2] = 'Graphs' + // graph title + a[3] = 'Graph Title' + // kind + a[7] = 'line' + // Plots start at 26 + let index = 26 + let colorIndex = 0 + for (const outputVarName of outputVarNames) { + // Surround the name in quotes if it contains a comma + const escapedName = escapeCsvField(outputVarName) + // plot N variable + a[index + 0] = escapedName + // plot N style + a[index + 2] = 'line' + // plot N label + a[index + 3] = escapedName + // plot N color + a[index + 4] = colors[colorIndex] + index += 7 + colorIndex++ + } + + return a.join(',') +} + +async function chooseGenSliderConfig(projDir: string, mdlVars: MdlVariable[]): Promise { + // Prompt the user + const genResponse = await prompts( + { + type: 'confirm', + name: 'genSliders', + message: `Would you like to configure a few sliders to get you started? ${reset(dim('(recommended)'))}`, + initial: true + }, + { + onCancel: () => { + ora().info(dim('Operation cancelled.')) + process.exit(0) + } + } + ) + + // Handle response + if (!genResponse.genSliders) { + ora().info(dim(`No problem! You can edit the "${cyan('config/inputs.csv')}" file later.`)) + return + } + + // Offer multi-select with available input variables + const inputVarNames: string[] = [] + for (const mdlVar of mdlVars) { + if (mdlVar.kind === 'const') { + inputVarNames.push(mdlVar.name) + } + } + inputVarNames.sort((a, b) => { + return a.toLowerCase().localeCompare(b.toLowerCase()) + }) + const choices = inputVarNames.map(f => { + return { + title: f, + value: f + } as Choice + }) + const varsResponse = await prompts( + { + type: 'autocompleteMultiselect', + name: 'vars', + message: 'Choose up to three input variables to control with sliders', + choices, + max: 3 + }, + { + onCancel: () => { + ora().info(dim('Operation cancelled.')) + process.exit(0) + } + } + ) + + if (varsResponse.vars.length === 0) { + ora().info(dim(`No variables selected. You can edit the "${cyan('config/inputs.csv')}" file later.`)) + return + } + + // Add lines to `inputs.csv` + const inputsCsvFile = joinPath(projDir, 'config', 'inputs.csv') + let inputsCsvContent = await readFile(inputsCsvFile, 'utf8') + let idNumber = 1 + for (const inputVarName of varsResponse.vars) { + const inputVar = mdlVars.find(v => v.name === inputVarName) + if (inputVar && inputVar.kind === 'const') { + const defaultValue = inputVar.value + const csvLine = inputsCsvLine(inputVarName, defaultValue, idNumber.toString()) + inputsCsvContent += `${csvLine}\n` + idNumber++ + } + } + await writeFile(inputsCsvFile, inputsCsvContent) + + const slidersText = varsResponse.vars.length > 1 ? 'sliders' : 'sliders' + ora( + green( + `Added ${slidersText} to "${bold('config/inputs.csv')}". ${dim('You can configure sliders in that file later.')}` + ) + ).succeed() +} + +// TODO: Ideally the `plugin-config` package would expose an API for generating CSV, but +// until then, we will generate a comma-separated line of values in hardcoded fashion +function inputsCsvLine(inputVarName: string, defaultValue: number, id: string): string { + // Surround the name in quotes if it contains a comma + const escapedName = escapeCsvField(inputVarName) + + // TODO: Be smarter about automatically choosing min/max/step values + const minValue = defaultValue - 1 + const maxValue = defaultValue + 1 + const step = 0.1 + + // Fill the line with blanks initially, then set the small subset of fields + const a = Array(28).fill('') + // id + a[0] = id + // input type + a[1] = 'slider' + // viewid + a[2] = 'view1' + // varname + a[3] = escapedName + // label + a[4] = escapedName + // group name + a[6] = 'Sliders' + // slider min + a[7] = minValue + // slider max + a[8] = maxValue + // slider default + a[9] = defaultValue + // slider step + a[10] = step + // units + a[11] = '(units)' + + return a.join(',') +} + +function escapeCsvField(s: string): string { + // Surround the value in quotes if it contains a comma + // TODO: Escape double quotes as well + return s.includes(',') ? `"${s}"` : s +} + +async function readModelVars(projDir: string, mdlPath: string): Promise { + // TODO: This function contains a subset of the logic from `sde-generate.js` in + // the `cli` package; should revisit + // let { modelDirname, modelName, modelPathname } = modelPathProps(model) + + // Ensure the `build` directory exists (under the `sde-prep` directory) + const buildDir = resolvePath(projDir, 'sde-prep', 'build') + await mkdir(buildDir, { recursive: true }) + + // Use an empty model spec; this will make SDE look at all variables in the mdl + const spec = {} + + // Try parsing the mdl file to generate the list of variables + // TODO: This depends on some `compile` package APIs that are not yet considered stable. + // Ideally we'd use an API that does not write files but instead returns an in-memory + // object in a specified format. + + // Read and preprocess the model + const mdlFile = resolvePath(projDir, mdlPath) + const preprocessed = preprocessModel(mdlFile, spec, 'genc', /*writeFiles=*/ false) + + // Parse the model and generate the variable list + const mdlDir = dirname(mdlFile) + const mdlName = parsePath(mdlFile).name + await parseAndGenerate(preprocessed, spec, 'printVarList', mdlDir, mdlName, buildDir) + + // Read `build/{mdl}_vars.yaml` + // TODO: For now the printVarList code only outputs txt and yaml files; we'll use the + // yaml file for now, but that means we have a dependency on the `yaml` package. Once + // we change the `compile` package to output JSON, we'll need to change this code. + const varsYamlFile = joinPath(buildDir, `${mdlName}_vars.yaml`) + const varsYamlContent = await readFile(varsYamlFile, 'utf8') + const varObjs = yaml.parse(varsYamlContent) + + // Create a simplified array of variables + const mdlVars: MdlVariable[] = [] + for (const varObj of varObjs) { + // Skip special control variables + const varName = varObj.modelLHS.toLowerCase() + switch (varName) { + case 'final time': + case 'initial time': + case 'saveper': + case 'time': + case 'time step': + continue + default: + break + } + + // Only include certain variables for now + // TODO: Include "data" vars + switch (varObj.varType) { + case 'const': + mdlVars.push({ + kind: 'const', + name: varObj.modelLHS, + value: Number.parseFloat(varObj.modelFormula) + }) + break + case 'aux': + mdlVars.push({ + kind: 'aux', + name: varObj.modelLHS + }) + break + case 'level': + mdlVars.push({ + kind: 'level', + name: varObj.modelLHS + }) + break + default: + break + } + } + + return mdlVars +} diff --git a/packages/create/src/step-emsdk.ts b/packages/create/src/step-emsdk.ts index 4c9bd8ee..44d7d141 100644 --- a/packages/create/src/step-emsdk.ts +++ b/packages/create/src/step-emsdk.ts @@ -4,7 +4,7 @@ import { existsSync, rmSync } from 'fs' import { join as joinPath, resolve as resolvePath } from 'path' import { execa } from 'execa' -import { bold, dim, green, red } from 'kleur/colors' +import { bold, cyan, dim, green, red } from 'kleur/colors' import ora from 'ora' import prompts from 'prompts' import type { Arguments } from 'yargs-parser' @@ -61,7 +61,9 @@ export async function chooseInstallEmsdk(projDir: string, args: Arguments): Prom return } else if (installResponse.install === 'skip') { ora().info( - dim('No problem! Be sure to install the Emscripten SDK and configure it in `sde.config.js` after setup.') + dim( + `No problem! Be sure to install the Emscripten SDK and configure it in "${cyan('sde.config.js')}" after setup.` + ) ) return } diff --git a/packages/create/src/step-mdl.ts b/packages/create/src/step-mdl.ts index 194dfe54..3ac2df0b 100644 --- a/packages/create/src/step-mdl.ts +++ b/packages/create/src/step-mdl.ts @@ -65,10 +65,7 @@ export async function chooseMdlFile(projDir: string): Promise { mdlFile = options.mdlFile } - ora({ - color: 'green', - text: green(`Using "${bold(mdlFile)}" as the model for the project.`) - }).succeed() + ora(green(`Using "${bold(mdlFile)}" as the model for the project.`)).succeed() return mdlFile } diff --git a/packages/create/src/step-template.ts b/packages/create/src/step-template.ts index c51fa510..adaef3d7 100644 --- a/packages/create/src/step-template.ts +++ b/packages/create/src/step-template.ts @@ -25,7 +25,7 @@ const TEMPLATES = [ } ] -export async function chooseTemplate(projDir: string, args: Arguments): Promise { +export async function chooseTemplate(projDir: string, args: Arguments): Promise { // Prompt the user const options = await prompts( [ @@ -58,6 +58,8 @@ export async function chooseTemplate(projDir: string, args: Arguments): Promise< await runDegit(templateTarget, hash, projDir, args, templateSpinner) templateSpinner.text = green('Template copied!') templateSpinner.succeed() + + return options.template } // XXX: This is mostly copied from Astro's create package: diff --git a/packages/create/src/types.d.ts b/packages/create/src/types.d.ts new file mode 100644 index 00000000..65aa5bd1 --- /dev/null +++ b/packages/create/src/types.d.ts @@ -0,0 +1,4 @@ +// Copyright (c) 2022 Climate Interactive / New Venture Fund + +// XXX: Workaround for lack of type declarations for the `compile` package +declare module '@sdeverywhere/compile' diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 93cb92b2..ed5a16ac 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -207,6 +207,7 @@ importers: packages/create: specifiers: + '@sdeverywhere/compile': ^0.7.0 '@types/degit': ^2.8.3 '@types/fs-extra': ^9.0.13 '@types/node': ^16.11.7 @@ -220,8 +221,10 @@ importers: ora: ^6.1.2 prompts: ^2.4.2 which-pm-runs: ^1.1.0 + yaml: ^2.1.1 yargs-parser: ^21.1.1 dependencies: + '@sdeverywhere/compile': link:../compile degit: 2.8.4 execa: 6.1.0 fs-extra: 10.1.0 @@ -229,6 +232,7 @@ importers: ora: 6.1.2 prompts: 2.4.2 which-pm-runs: 1.1.0 + yaml: 2.1.1 yargs-parser: 21.1.1 devDependencies: '@types/degit': 2.8.3 From 2cdea137fec168202c3a1883066415416122adeb Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 16:53:19 -0700 Subject: [PATCH 16/23] fix: update model.csv using initial/final time values from mdl --- packages/create/src/step-config.ts | 72 +++++++++++++++++++++++------- 1 file changed, 57 insertions(+), 15 deletions(-) diff --git a/packages/create/src/step-config.ts b/packages/create/src/step-config.ts index 0d2a52de..d16a799d 100644 --- a/packages/create/src/step-config.ts +++ b/packages/create/src/step-config.ts @@ -56,12 +56,67 @@ export async function chooseGenConfig(projDir: string, mdlPath: string): Promise return } + // Extract `INITIAL TIME` and `FINAL TIME` values and remove special control variables from the list + let initialTime: number + let finalTime: number + const validVars: MdlVariable[] = [] + for (const v of mdlVars) { + const varName = v.name.toLowerCase() + let skip = false + switch (varName) { + case 'final time': + skip = true + if (v.kind === 'const') { + finalTime = v.value + } + break + case 'initial time': + skip = true + if (v.kind === 'const') { + initialTime = v.value + } + break + case 'saveper': + case 'time': + case 'time step': + skip = true + break + default: + break + } + if (!skip) { + validVars.push(v) + } + } + + // Set the values in `model.csv` + if (initialTime === undefined) { + initialTime = 0 + } + if (finalTime === undefined) { + finalTime = 100 + } + + // TODO: Auto-detect dat files that are referenced by the mdl and include them here + const datFiles: string[] = [] + const datPart = datFiles.join(';') + + // Preserve the `model.csv` header but drop other existing content (if any) + const modelCsvFile = joinPath(projDir, 'config', 'model.csv') + const origModelCsvContent = await readFile(modelCsvFile, 'utf8') + const modelCsvHeader = origModelCsvContent.split('\n')[0] + + // Add line and write out updated `model.csv` + const modelCsvLine = `${initialTime},${finalTime},${initialTime},${finalTime},${datPart}` + const newModelCsvContent = `${modelCsvHeader}\n${modelCsvLine}\n` + await writeFile(modelCsvFile, newModelCsvContent) + // See if the user wants to generate graph config - await chooseGenGraphConfig(projDir, mdlVars) + await chooseGenGraphConfig(projDir, validVars) console.log() // See if the user wants to generate slider config - await chooseGenSliderConfig(projDir, mdlVars) + await chooseGenSliderConfig(projDir, validVars) } async function chooseGenGraphConfig(projDir: string, mdlVars: MdlVariable[]): Promise { @@ -337,19 +392,6 @@ async function readModelVars(projDir: string, mdlPath: string): Promise Date: Wed, 21 Sep 2022 18:21:08 -0700 Subject: [PATCH 17/23] fix: write pnpm-workspace.yaml if pnpm is detected --- packages/create/src/index.ts | 4 ++-- packages/create/src/step-template.ts | 12 +++++++++++- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts index 657dff18..253c8352 100644 --- a/packages/create/src/index.ts +++ b/packages/create/src/index.ts @@ -33,7 +33,7 @@ export async function main(): Promise { console.log() // Prompt the user to select a template - const templateName = await chooseTemplate(projDir, args) + const templateName = await chooseTemplate(projDir, args, pkgManager) console.log() // Prompt the user to select an mdl file @@ -61,7 +61,7 @@ export async function main(): Promise { await chooseGitInit(projDir, args) console.log() - ora({ text: green('Setup complete!') }).succeed() + ora(green('Setup complete!')).succeed() console.log(`\n${bgCyan(black(' Next steps '))}\n`) diff --git a/packages/create/src/step-template.ts b/packages/create/src/step-template.ts index adaef3d7..d68b628d 100644 --- a/packages/create/src/step-template.ts +++ b/packages/create/src/step-template.ts @@ -1,6 +1,7 @@ // Copyright (c) 2022 Climate Interactive / New Venture Fund import { existsSync, mkdtempSync, readdirSync, rmSync } from 'fs' +import { writeFile } from 'fs/promises' import { copy } from 'fs-extra' import { tmpdir } from 'os' import { join as joinPath } from 'path' @@ -25,7 +26,7 @@ const TEMPLATES = [ } ] -export async function chooseTemplate(projDir: string, args: Arguments): Promise { +export async function chooseTemplate(projDir: string, args: Arguments, pkgManager: string): Promise { // Prompt the user const options = await prompts( [ @@ -59,6 +60,15 @@ export async function chooseTemplate(projDir: string, args: Arguments): Promise< templateSpinner.text = green('Template copied!') templateSpinner.succeed() + if (options.template === 'template-default' && pkgManager === 'pnpm') { + // pnpm doesn't use the "workspaces" config from `package.json` (like npm and yarn use), + // so in the case of pnpm, write a `pnpm-workspace.yaml` file so that the default + // template monorepo layout is set up correctly + const workspaceFile = joinPath(projDir, 'pnpm-workspace.yaml') + const workspaceContent = `packages:\n - packages/*\n` + await writeFile(workspaceFile, workspaceContent) + } + return options.template } From 68010a4dfded4184655f6aabc7da723b8459f7c1 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 18:26:13 -0700 Subject: [PATCH 18/23] fix: allow for overriding commit/branch on command line --- packages/create/src/step-template.ts | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/packages/create/src/step-template.ts b/packages/create/src/step-template.ts index d68b628d..18cfabe1 100644 --- a/packages/create/src/step-template.ts +++ b/packages/create/src/step-template.ts @@ -51,10 +51,13 @@ export async function chooseTemplate(projDir: string, args: Arguments, pkgManage return } + // Allow branch name or commit hash to be overridden on command line + const defaultRev = 'main' + const commit = args.commit || defaultRev + // Copy the template files to the project directory const templateTarget = `climateinteractive/SDEverywhere/examples/${options.template}` - // TODO: Fix this before branch is merged - const hash = '#chris/228-create-package' + const hash = `#${commit}` const templateSpinner = ora('Copying project files...').start() await runDegit(templateTarget, hash, projDir, args, templateSpinner) templateSpinner.text = green('Template copied!') From 008ed5cc7c937b823828a95794a429816d14bb6a Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 18:34:38 -0700 Subject: [PATCH 19/23] fix: add a sample mdl file if none found --- packages/create/src/step-mdl.ts | 44 +++++++++++++++++++++++++-------- 1 file changed, 34 insertions(+), 10 deletions(-) diff --git a/packages/create/src/step-mdl.ts b/packages/create/src/step-mdl.ts index 3ac2df0b..f6322e37 100644 --- a/packages/create/src/step-mdl.ts +++ b/packages/create/src/step-mdl.ts @@ -1,13 +1,33 @@ // Copyright (c) 2022 Climate Interactive / New Venture Fund -import { readdir } from 'fs/promises' -import { relative, resolve as resolvePath } from 'path' +import { readdir, writeFile } from 'fs/promises' +import { join as joinPath, relative, resolve as resolvePath } from 'path' -import { bold, dim, green } from 'kleur/colors' +import { bold, cyan, dim, green, yellow } from 'kleur/colors' import ora from 'ora' import type { Choice } from 'prompts' import prompts from 'prompts' +const sampleMdlContent = `\ +{UTF-8} + +X = TIME + ~~| + +Y = 0 + ~ [-10,10,0.1] + ~ + | + +Z = X + Y + ~~| + +INITIAL TIME = 2000 ~~| +FINAL TIME = 2100 ~~| +TIME STEP = 1 ~~| +SAVEPER = TIME STEP ~~| +` + export async function chooseMdlFile(projDir: string): Promise { // Find all `.mdl` files in the project directory // From https://stackoverflow.com/a/45130990 @@ -32,13 +52,17 @@ export async function chooseMdlFile(projDir: string): Promise { let mdlFile: string if (mdlFiles.length === 0) { - // No mdl files found; print error message and exit - // TODO: Offer to add a basic mdl to get the user started - ora({ - color: 'red', - text: `No mdl files were found in "${projDir}". Add your mdl file to that directory and try again.` - }).fail() - process.exit(0) + // No mdl files found; write a sample mdl to get the user started + const sampleMdlFile = joinPath(projDir, 'sample.mdl') + await writeFile(sampleMdlFile, sampleMdlContent) + ora( + yellow( + `No mdl files were found in "${projDir}". A "${cyan( + 'sample.mdl' + )}" file has been added to the project to get you started.` + ) + ).warn() + mdlFile = 'sample.mdl' } else if (mdlFiles.length === 1) { // Only one mdl file ora().succeed(`Found "${mdlFiles[0]}", will configure the project to use that mdl file.`) From dae3e175963bbaf0f35b63718d902f7c1162f834 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 19:37:15 -0700 Subject: [PATCH 20/23] fix: simplify ora calls --- packages/create/src/step-directory.ts | 12 +++--------- packages/create/src/step-emsdk.ts | 10 ++-------- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/packages/create/src/step-directory.ts b/packages/create/src/step-directory.ts index 82405bb2..0901bc4f 100644 --- a/packages/create/src/step-directory.ts +++ b/packages/create/src/step-directory.ts @@ -3,7 +3,7 @@ import { existsSync } from 'fs' import { resolve as resolvePath } from 'path' -import { bold, dim, green } from 'kleur/colors' +import { bold, dim, green, red } from 'kleur/colors' import ora from 'ora' import prompts from 'prompts' import type { Arguments } from 'yargs-parser' @@ -20,17 +20,11 @@ export async function chooseProjectDir(args: Arguments): Promise { } const showValidDirMsg = (dir: string) => { - ora({ - color: 'green', - text: green(`Using "${bold(dir)}" as the project directory.`) - }).succeed() + ora(green(`Using "${bold(dir)}" as the project directory.`)).succeed() } const showInvalidDirMsg = (dir: string) => { - ora({ - color: 'red', - text: `"${bold(dir)}" contains existing 'package.json' and/or 'packages' directory, stopping.` - }).fail() + ora(red(`"${bold(dir)}" contains existing 'package.json' and/or 'packages' directory, stopping.`)).fail() } // See if project directory is provided on command line diff --git a/packages/create/src/step-emsdk.ts b/packages/create/src/step-emsdk.ts index 44d7d141..5ca66b52 100644 --- a/packages/create/src/step-emsdk.ts +++ b/packages/create/src/step-emsdk.ts @@ -73,17 +73,11 @@ export async function chooseInstallEmsdk(projDir: string, args: Arguments): Prom // TODO: Use spinner here await installEmscripten(installDir) } catch (e) { - ora({ - color: 'red', - text: red(`Failed to install Emscripten SDK: ${e.message}`) - }).fail() + ora(red(`Failed to install Emscripten SDK: ${e.message}`)).fail() process.exit(0) } - ora({ - color: 'green', - text: green(`Installed the Emscripten SDK in "${bold(installDir)}"`) - }).succeed() + ora(green(`Installed the Emscripten SDK in "${bold(installDir)}"`)).succeed() } /** From a0de6075434b3d2e381d1f9a71086bb9456218e3 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 22:01:42 -0700 Subject: [PATCH 21/23] feat: add minimal template --- examples/template-default/README.md | 15 +++++++++ examples/template-minimal/.gitignore | 1 + examples/template-minimal/README.md | 34 ++++++++++++++++++++ examples/template-minimal/package.json | 16 ++++++++++ examples/template-minimal/sde.config.js | 42 +++++++++++++++++++++++++ 5 files changed, 108 insertions(+) create mode 100644 examples/template-minimal/.gitignore create mode 100644 examples/template-minimal/README.md create mode 100644 examples/template-minimal/package.json create mode 100644 examples/template-minimal/sde.config.js diff --git a/examples/template-default/README.md b/examples/template-default/README.md index 8f3e4da5..94f06e54 100644 --- a/examples/template-default/README.md +++ b/examples/template-default/README.md @@ -3,6 +3,21 @@ This is a template that is used by the `@sdeverywhere/create` package to generate a new project that uses SDEverywhere. +The project includes: + +- a build process that converts a Vensim model to a WebAssembly module that + can run the model in any web browser or in a Node.js application +- a `config` directory that contains CSV files for configuring the generated + model and application +- a "core" package that provides a clean JavaScript / TypeScript API around the + WebAssembly model +- an "app" package containing a simple JavaScript / jQuery-based web application + that can be used to exercise the model +- a local development mode (`npm run dev`) that allows for rapid prototyping + of the model and app +- a "model-check" setup that allows for running checks and comparison tests using + the generated model + ## Quick Start ```sh diff --git a/examples/template-minimal/.gitignore b/examples/template-minimal/.gitignore new file mode 100644 index 00000000..3c0f0280 --- /dev/null +++ b/examples/template-minimal/.gitignore @@ -0,0 +1 @@ +sde-prep diff --git a/examples/template-minimal/README.md b/examples/template-minimal/README.md new file mode 100644 index 00000000..2b331bd0 --- /dev/null +++ b/examples/template-minimal/README.md @@ -0,0 +1,34 @@ +# template-minimal + +This is a template that is used by the `@sdeverywhere/create` package to generate a +new project that uses SDEverywhere. + +The generated project is more minimal than the "default" template, but may be +useful if you don't want to generate a library or app around your model and +if you are more interested in just running the "model-check" tool. + +Note that unlike the "default" template, this template does not use config files +(e.g., CSV files read from the `config` directory), so you can edit the +`sde.config.js` file to configure the input and output variables. + +(If you decide later to use config files, refer to `template-default` to see how +to add the `@sdeverywhere/plugin-config` package to your project.) + +## Quick Start + +```sh +# Create a new project (you can also use yarn or pnpm here, if preferred). +# Be sure to choose the "Minimal" template. +npm create @sdeverywhere + +# Enter development mode for your model. This will start a live +# development environment that will build a WebAssembly version of the +# model and run checks on it any time you make changes to: +# - the Vensim model file (.mdl) +# - the checks file (.check.yaml) +npm run dev +``` + +## License + +SDEverywhere is distributed under the MIT license. See `LICENSE` for more details. diff --git a/examples/template-minimal/package.json b/examples/template-minimal/package.json new file mode 100644 index 00000000..01d9bac2 --- /dev/null +++ b/examples/template-minimal/package.json @@ -0,0 +1,16 @@ +{ + "name": "project", + "version": "1.0.0", + "private": true, + "type": "module", + "scripts": { + "build": "sde bundle", + "dev": "sde dev" + }, + "dependencies": { + "@sdeverywhere/cli": "^0.7.0", + "@sdeverywhere/plugin-check": "^0.1.0", + "@sdeverywhere/plugin-wasm": "^0.1.0", + "@sdeverywhere/plugin-worker": "^0.1.0" + } +} diff --git a/examples/template-minimal/sde.config.js b/examples/template-minimal/sde.config.js new file mode 100644 index 00000000..44495bb2 --- /dev/null +++ b/examples/template-minimal/sde.config.js @@ -0,0 +1,42 @@ +import { checkPlugin } from '@sdeverywhere/plugin-check' +import { wasmPlugin } from '@sdeverywhere/plugin-wasm' +import { workerPlugin } from '@sdeverywhere/plugin-worker' + +export async function config() { + return { + modelFiles: ['MODEL_NAME.mdl'], + + modelSpec: async () => { + return { + // TODO: Change these values as desired (usually they will be the same as + // the `INITIAL TIME` and `FINAL TIME` values from the mdl, but you can + // use different values here) + startTime: 2000, + endTime: 2100, + inputs: [ + // TODO: List your input variables here + { varName: 'Y', defaultValue: 0, minValue: -10, maxValue: 10 } + ], + outputs: [ + // TODO: List your output variables here + { varName: 'Z' } + ], + datFiles: [ + // TODO: If your mdl refers to vdfx files, list dat files here (first + // convert vdfx to dat, since SDEverywhere only supports dat format) + ] + } + }, + + plugins: [ + // Generate a `wasm-model.js` file containing the Wasm model + wasmPlugin(), + + // Generate a `worker.js` file that runs the Wasm model in a worker + workerPlugin(), + + // Run model check + checkPlugin() + ] + } +} From 1112275be2df3591b6ee8a39b0f3c958c5b21369 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 22:14:09 -0700 Subject: [PATCH 22/23] fix: generate sample .check.yaml file and include check-core as dependency in templates to workaround schema resolution issue --- examples/template-default/package.json | 1 + examples/template-minimal/package.json | 1 + packages/create/src/index.ts | 6 ++-- packages/create/src/step-config.ts | 47 +++++++++++++++++++++++++- 4 files changed, 52 insertions(+), 3 deletions(-) diff --git a/examples/template-default/package.json b/examples/template-default/package.json index af295b7b..bbd938ab 100644 --- a/examples/template-default/package.json +++ b/examples/template-default/package.json @@ -13,6 +13,7 @@ "packages/app" ], "dependencies": { + "@sdeverywhere/check-core": "^0.1.0", "@sdeverywhere/cli": "^0.7.0", "@sdeverywhere/plugin-check": "^0.1.0", "@sdeverywhere/plugin-config": "^0.1.0", diff --git a/examples/template-minimal/package.json b/examples/template-minimal/package.json index 01d9bac2..9fd65000 100644 --- a/examples/template-minimal/package.json +++ b/examples/template-minimal/package.json @@ -8,6 +8,7 @@ "dev": "sde dev" }, "dependencies": { + "@sdeverywhere/check-core": "^0.1.0", "@sdeverywhere/cli": "^0.7.0", "@sdeverywhere/plugin-check": "^0.1.0", "@sdeverywhere/plugin-wasm": "^0.1.0", diff --git a/packages/create/src/index.ts b/packages/create/src/index.ts index 253c8352..6f8a4e62 100644 --- a/packages/create/src/index.ts +++ b/packages/create/src/index.ts @@ -8,7 +8,7 @@ import prompts from 'prompts' import detectPackageManager from 'which-pm-runs' import yargs from 'yargs-parser' -import { chooseGenConfig, updateSdeConfig } from './step-config' +import { chooseGenConfig, generateCheckYaml, updateSdeConfig } from './step-config' import { chooseInstallDeps } from './step-deps' import { chooseProjectDir } from './step-directory' import { chooseInstallEmsdk } from './step-emsdk' @@ -39,8 +39,10 @@ export async function main(): Promise { // Prompt the user to select an mdl file const mdlPath = await chooseMdlFile(projDir) - // Update the `sde.config.js` file to use the chosen mdl file + // Update the `sde.config.js` file to use the chosen mdl file and + // generate a sample `.check.yaml` file await updateSdeConfig(projDir, mdlPath) + await generateCheckYaml(projDir, mdlPath) console.log() // If the user chose the default template, offer to set up CSV files diff --git a/packages/create/src/step-config.ts b/packages/create/src/step-config.ts index d16a799d..e1675c76 100644 --- a/packages/create/src/step-config.ts +++ b/packages/create/src/step-config.ts @@ -1,7 +1,8 @@ // Copyright (c) 2022 Climate Interactive / New Venture Fund +import { existsSync } from 'fs' import { mkdir, readFile, writeFile } from 'fs/promises' -import { dirname, join as joinPath, parse as parsePath, resolve as resolvePath } from 'path' +import { dirname, join as joinPath, parse as parsePath, relative, resolve as resolvePath } from 'path' import { bold, cyan, dim, green, reset, yellow } from 'kleur/colors' import ora from 'ora' @@ -29,6 +30,23 @@ interface MdlLevelVariable { type MdlVariable = MdlConstVariable | MdlAuxVariable | MdlLevelVariable +const sampleCheckContent = `\ +# yaml-language-server: $schema=SCHEMA_PATH + +# NOTE: This is just a simple check to get you started. Replace "Some output" with +# the name of some variable you'd like to test. Additional tests can be developed +# in the "playground" (beta) inside the model-check report. +- describe: Some output + tests: + - it: should be > 0 for all input scenarios + scenarios: + - preset: matrix + datasets: + - name: Some output + predicates: + - gt: 0 +` + export async function updateSdeConfig(projDir: string, mdlPath: string): Promise { // Read the `sde.config.js` file from the template const configPath = joinPath(projDir, 'sde.config.js') @@ -41,6 +59,33 @@ export async function updateSdeConfig(projDir: string, mdlPath: string): Promise await writeFile(configPath, configText) } +export async function generateCheckYaml(projDir: string, mdlPath: string): Promise { + // Generate a sample `{mdl}.check.yaml` file if one doesn't already exist + // TODO: Make this optional (ask user first)? + const checkYamlFile = mdlPath.replace('.mdl', '.check.yaml') + const checkYamlPath = joinPath(projDir, checkYamlFile) + if (!existsSync(checkYamlPath)) { + // Get relative path from yaml file parent dir to project dir + let relProjPath = relative(dirname(checkYamlPath), projDir) + if (relProjPath.length === 0) { + relProjPath = './' + } + + // TODO: This path is normally different depending on whether using npm/yarn or + // pnpm. For npm/yarn, `check-core` is hoisted under top-level `node_modules`, + // but for pnpm, it is nested under `node_modules/.pnpm`. As an ugly workaround + // the templates declare `check-core` as a direct dependency even though it is + // not really needed (a transitive dependency via `plugin-check` would normally + // suffice). This allows us to use the same path here that works for all + // three package managers. + const nodeModulesPart = joinPath(relProjPath, 'node_modules') + const checkCorePart = '@sdeverywhere/check-core/schema/check.schema.json' + const schemaPath = `${nodeModulesPart}/${checkCorePart}` + const checkContent = sampleCheckContent.replace('SCHEMA_PATH', schemaPath) + await writeFile(checkYamlPath, checkContent) + } +} + export async function chooseGenConfig(projDir: string, mdlPath: string): Promise { // TODO: For now we eagerly read the mdl file; maybe change this to only load it if // the user chooses to generate graph and/or slider config From 4a94ff6e9bb9b4fcb9e18f17c79761250b2f54c6 Mon Sep 17 00:00:00 2001 From: Chris Campbell Date: Wed, 21 Sep 2022 22:35:20 -0700 Subject: [PATCH 23/23] fix: remove incorrectly copied CHANGELOG --- packages/create/CHANGELOG.md | 158 ----------------------------------- 1 file changed, 158 deletions(-) diff --git a/packages/create/CHANGELOG.md b/packages/create/CHANGELOG.md index e2f856e0..825c32f0 100644 --- a/packages/create/CHANGELOG.md +++ b/packages/create/CHANGELOG.md @@ -1,159 +1 @@ # Changelog - -## [0.7.1](https://github.com/climateinteractive/SDEverywhere/compare/cli-v0.7.0...cli-v0.7.1) (2022-08-06) - - -### Bug Fixes - -* correct handling of paths on Windows and those containing spaces ([#223](https://github.com/climateinteractive/SDEverywhere/issues/223)) ([c783e81](https://github.com/climateinteractive/SDEverywhere/commit/c783e811a43331e4c563438b8fa441792bdcfe28)), closes [#222](https://github.com/climateinteractive/SDEverywhere/issues/222) - - -### Dependencies - -* The following workspace dependencies were updated - * dependencies - * @sdeverywhere/build bumped from ^0.1.0 to ^0.1.1 - -## [0.7.0](https://github.com/climateinteractive/SDEverywhere/compare/cli-v0.6.0...cli-v0.7.0) (2022-06-28) - - -### ⚠ BREAKING CHANGES - -* The `sdeverywhere` package is deprecated and effectively replaced by `@sdeverywhere/cli`. Additionally, the `sde generate --genhtml` command and supporting code has been removed and will be replaced with a different solution in the near future. - -### Features - -* add build and plugin-{check,vite,wasm,worker} packages ([#206](https://github.com/climateinteractive/SDEverywhere/issues/206)) ([dd34cbf](https://github.com/climateinteractive/SDEverywhere/commit/dd34cbfcc0b8b3fb1655c8aa64fb919f9757b8be)), closes [#203](https://github.com/climateinteractive/SDEverywhere/issues/203) -* refactor into monorepo with compile and cli packages ([#192](https://github.com/climateinteractive/SDEverywhere/issues/192)) ([8946f18](https://github.com/climateinteractive/SDEverywhere/commit/8946f1854a116f7e9935d5e93d4485865d06d114)), closes [#191](https://github.com/climateinteractive/SDEverywhere/issues/191) - -## [0.6.0](https://github.com/climateinteractive/SDEverywhere/compare/0.5.3...sdeverywhere-v0.6.0) (2022-06-04) - - -### Features - -* add `runModelWithBuffers` entry point that takes pre-allocated input/output buffers ([#50](https://github.com/climateinteractive/SDEverywhere/issues/50)) ([083109a](https://github.com/climateinteractive/SDEverywhere/commit/083109a1f03d9648c6a30efbdfbd3d1543d81cfd)), closes [#49](https://github.com/climateinteractive/SDEverywhere/issues/49) -* add basic support for GET DATA BETWEEN TIMES function ([#42](https://github.com/climateinteractive/SDEverywhere/issues/42)) ([8a294aa](https://github.com/climateinteractive/SDEverywhere/commit/8a294aa5f65c1ffc76a397c77257c02687988770)), closes [#33](https://github.com/climateinteractive/SDEverywhere/issues/33) -* add sde command for flattening parent+submodels ([#59](https://github.com/climateinteractive/SDEverywhere/issues/59)) ([50e11ec](https://github.com/climateinteractive/SDEverywhere/commit/50e11ec69ff0c09d84b72a9b16a8ae3d7d3c1433)), closes [#58](https://github.com/climateinteractive/SDEverywhere/issues/58) -* add support for external data variables that use subscript or dimension ([#41](https://github.com/climateinteractive/SDEverywhere/issues/41)) ([035ab5c](https://github.com/climateinteractive/SDEverywhere/commit/035ab5cb88a7e1ff79423cccacde3b827ce8d0dd)), closes [#32](https://github.com/climateinteractive/SDEverywhere/issues/32) -* add support for external data variables with > 2 dimensions ([#47](https://github.com/climateinteractive/SDEverywhere/issues/47)) ([6683e63](https://github.com/climateinteractive/SDEverywhere/commit/6683e63a90cc45fe5109a39c5918ae6d05c878cd)), closes [#45](https://github.com/climateinteractive/SDEverywhere/issues/45) -* allow for specifying I/O variables using Vensim names in spec file ([#61](https://github.com/climateinteractive/SDEverywhere/issues/61)) ([a6bc98d](https://github.com/climateinteractive/SDEverywhere/commit/a6bc98d48f267cbc05244a0749e1108be680f3f7)), closes [#60](https://github.com/climateinteractive/SDEverywhere/issues/60) -* implement ALLOCATE AVAILABLE function ([#106](https://github.com/climateinteractive/SDEverywhere/issues/106)) ([ebd0311](https://github.com/climateinteractive/SDEverywhere/commit/ebd031160d80e52faa7b98f6e1f1d73fcbca1e15)) -* implement DELAY FIXED function ([#108](https://github.com/climateinteractive/SDEverywhere/issues/108)) ([bd3b3e8](https://github.com/climateinteractive/SDEverywhere/commit/bd3b3e8163bc95dc6957442277311e31b5c4bef9)), closes [#29](https://github.com/climateinteractive/SDEverywhere/issues/29) -* implement GET DIRECT CONSTANTS for CSV ([#86](https://github.com/climateinteractive/SDEverywhere/issues/86)) ([beedd4f](https://github.com/climateinteractive/SDEverywhere/commit/beedd4f1efec31490076badc910fd20003829044)) -* implement GET DIRECT DATA for CSV ([#82](https://github.com/climateinteractive/SDEverywhere/issues/82)) ([b40e738](https://github.com/climateinteractive/SDEverywhere/commit/b40e738389f315984206bac9f9d28f6555549180)), closes [#81](https://github.com/climateinteractive/SDEverywhere/issues/81) -* implement GET DIRECT LOOKUPS including a test model ([#99](https://github.com/climateinteractive/SDEverywhere/issues/99)) ([47f6fe5](https://github.com/climateinteractive/SDEverywhere/commit/47f6fe5d4188b3d14703ff7730d5c874c296df60)), closes [#98](https://github.com/climateinteractive/SDEverywhere/issues/98) -* implement GET DIRECT SUBSCRIPT for CSV ([#79](https://github.com/climateinteractive/SDEverywhere/issues/79)) ([51691e7](https://github.com/climateinteractive/SDEverywhere/commit/51691e7a209cb01911756ff54037dd756a5d439e)) -* implement NPV function ([#95](https://github.com/climateinteractive/SDEverywhere/issues/95)) ([9fa17eb](https://github.com/climateinteractive/SDEverywhere/commit/9fa17eb0f2b2a7ef63608771cb8234eaa3c2b35e)), closes [#94](https://github.com/climateinteractive/SDEverywhere/issues/94) -* implement QUANTUM function ([#89](https://github.com/climateinteractive/SDEverywhere/issues/89)) ([42225f9](https://github.com/climateinteractive/SDEverywhere/commit/42225f9508337cd6dbb0e31c2e5f73269f4b526a)), closes [#87](https://github.com/climateinteractive/SDEverywhere/issues/87) -* implement the <-> subscript alias operator ([#80](https://github.com/climateinteractive/SDEverywhere/issues/80)) ([a43917f](https://github.com/climateinteractive/SDEverywhere/commit/a43917f63e0830530c2faf2b76bc65dc90da7533)), closes [#78](https://github.com/climateinteractive/SDEverywhere/issues/78) -* increase the number of dimension and array loop index vars ([#138](https://github.com/climateinteractive/SDEverywhere/issues/138)) ([4c66470](https://github.com/climateinteractive/SDEverywhere/commit/4c6647069582632e89845b2f08b73348c5cc63e6)), closes [#137](https://github.com/climateinteractive/SDEverywhere/issues/137) -* remove inline comments in the preprocessor ([#74](https://github.com/climateinteractive/SDEverywhere/issues/74)) ([d23b1c3](https://github.com/climateinteractive/SDEverywhere/commit/d23b1c355c8b936d704f7ed4affa012b0eb27356)) -* sort equations alphabetically when preprocessing mdl file ([#56](https://github.com/climateinteractive/SDEverywhere/issues/56)) ([bb968f7](https://github.com/climateinteractive/SDEverywhere/commit/bb968f79348401ceb9c75930fb66ff32ac7c453f)), closes [#55](https://github.com/climateinteractive/SDEverywhere/issues/55) - - -### Bug Fixes - -* abort code generation on finding a lookup of size zero ([#162](https://github.com/climateinteractive/SDEverywhere/issues/162)) ([44c1202](https://github.com/climateinteractive/SDEverywhere/commit/44c1202aaa113505132ab86a49d8ad7f3bee8673)), closes [#161](https://github.com/climateinteractive/SDEverywhere/issues/161) -* add async and await to some chained cli functions ([#37](https://github.com/climateinteractive/SDEverywhere/issues/37)) ([afdbb77](https://github.com/climateinteractive/SDEverywhere/commit/afdbb773db6fdfb8f7ba8d35782254307389f895)), closes [#35](https://github.com/climateinteractive/SDEverywhere/issues/35) -* allow > 2 dimensions when generating Vensim array names ([#155](https://github.com/climateinteractive/SDEverywhere/issues/155)) ([6575ea9](https://github.com/climateinteractive/SDEverywhere/commit/6575ea9c3d8b72e4d172c4add00bf7f1d2cf33bf)), closes [#154](https://github.com/climateinteractive/SDEverywhere/issues/154) -* allow extra index subscripts in 2D const lists ([#110](https://github.com/climateinteractive/SDEverywhere/issues/110)) ([f5494af](https://github.com/climateinteractive/SDEverywhere/commit/f5494afd1837aebbea5d1cdb329447aa5904d264)), closes [#109](https://github.com/climateinteractive/SDEverywhere/issues/109) -* allow GET DIRECT CONSTANTS to use 2 subscripts in the same family ([#144](https://github.com/climateinteractive/SDEverywhere/issues/144)) ([e53d876](https://github.com/climateinteractive/SDEverywhere/commit/e53d876571a6b89705c007f5d2ebf0de366b09e3)), closes [#143](https://github.com/climateinteractive/SDEverywhere/issues/143) -* correct declarations when subscripted variable is initialized from data and constants ([#116](https://github.com/climateinteractive/SDEverywhere/issues/116)) ([7c51641](https://github.com/climateinteractive/SDEverywhere/commit/7c51641223b869d5cff7e5b28692344134d5e4ee)), closes [#115](https://github.com/climateinteractive/SDEverywhere/issues/115) -* correct GET DIRECT DATA with mapped dim by directly comparing indices ([#146](https://github.com/climateinteractive/SDEverywhere/issues/146)) ([fa2097b](https://github.com/climateinteractive/SDEverywhere/commit/fa2097b4b7a0e1ae6580334f93936f70fde19a36)), closes [#145](https://github.com/climateinteractive/SDEverywhere/issues/145) -* correct initialization of 2D arrays to allow dimensions with matching subscript names ([#101](https://github.com/climateinteractive/SDEverywhere/issues/101)) ([2ed7e42](https://github.com/climateinteractive/SDEverywhere/commit/2ed7e427d6af956a9c6b9c03ef38845700cb3ff3)), closes [#84](https://github.com/climateinteractive/SDEverywhere/issues/84) -* emit any expression for the offset arg of VECTOR ELM MAP ([#129](https://github.com/climateinteractive/SDEverywhere/issues/129)) ([bd0a724](https://github.com/climateinteractive/SDEverywhere/commit/bd0a724462aaab09e205294b98cb450180f619c9)), closes [#128](https://github.com/climateinteractive/SDEverywhere/issues/128) -* exclude data vars from initLevels ([#127](https://github.com/climateinteractive/SDEverywhere/issues/127)) ([93a49cf](https://github.com/climateinteractive/SDEverywhere/commit/93a49cf1e7562987a9836dab580a6fad4428d25d)), closes [#126](https://github.com/climateinteractive/SDEverywhere/issues/126) -* expand references to vars with any number of separated dimensions ([#112](https://github.com/climateinteractive/SDEverywhere/issues/112)) ([0d0d40e](https://github.com/climateinteractive/SDEverywhere/commit/0d0d40e1b5b39560ae67f440172e5ded90a6cd3a)), closes [#111](https://github.com/climateinteractive/SDEverywhere/issues/111) -* expand variables allowing for any number of indices in EXCEPT clause ([#118](https://github.com/climateinteractive/SDEverywhere/issues/118)) ([d92343e](https://github.com/climateinteractive/SDEverywhere/commit/d92343e0806f228537f905956d053777accf9e97)), closes [#117](https://github.com/climateinteractive/SDEverywhere/issues/117) -* fix LOOKUP FORWARD to correctly handle fractional inputs ([#38](https://github.com/climateinteractive/SDEverywhere/issues/38)) ([c1f9580](https://github.com/climateinteractive/SDEverywhere/commit/c1f95804533b70036b14fa1caee64d402aeacfeb)), closes [#36](https://github.com/climateinteractive/SDEverywhere/issues/36) -* generate correct references for the ALLOCATE AVAILABLE priority profile ([#136](https://github.com/climateinteractive/SDEverywhere/issues/136)) ([b1d8ae2](https://github.com/climateinteractive/SDEverywhere/commit/b1d8ae2c19b4717df49a4a3e001625ce9568a3ac)), closes [#135](https://github.com/climateinteractive/SDEverywhere/issues/135) -* get direct data offset from the separated dimension ([#114](https://github.com/climateinteractive/SDEverywhere/issues/114)) ([ebbaa01](https://github.com/climateinteractive/SDEverywhere/commit/ebbaa0161dd0d5272f43f5323c0658413d21da47)), closes [#113](https://github.com/climateinteractive/SDEverywhere/issues/113) -* handle subdimensions correctly for GET DIRECT CONSTANTS and fix EXCEPT handling ([#125](https://github.com/climateinteractive/SDEverywhere/issues/125)) ([2fdfb34](https://github.com/climateinteractive/SDEverywhere/commit/2fdfb343dad047b1583a321ed174fa8813107a6c)), closes [#124](https://github.com/climateinteractive/SDEverywhere/issues/124) [#134](https://github.com/climateinteractive/SDEverywhere/issues/134) -* handle subscripts correctly when nested in expr within array function call ([#48](https://github.com/climateinteractive/SDEverywhere/issues/48)) ([b2458ab](https://github.com/climateinteractive/SDEverywhere/commit/b2458ab7e764b4ec33bebf1ee9ac3a80de9b474b)), closes [#46](https://github.com/climateinteractive/SDEverywhere/issues/46) -* make `sde log` wait for DAT file to be fully written and improve error handling ([#123](https://github.com/climateinteractive/SDEverywhere/issues/123)) ([34f25f8](https://github.com/climateinteractive/SDEverywhere/commit/34f25f8b97a8f8d2d0c949b6b81ca655a5e048d1)), closes [#122](https://github.com/climateinteractive/SDEverywhere/issues/122) -* make browserify an optional dependency ([#53](https://github.com/climateinteractive/SDEverywhere/issues/53)) ([e9bbbc6](https://github.com/climateinteractive/SDEverywhere/commit/e9bbbc66dcb59d629b6053a51faeee83a347147d)), closes [#52](https://github.com/climateinteractive/SDEverywhere/issues/52) -* make flatten command work when equations don't include continuation backslash ([#173](https://github.com/climateinteractive/SDEverywhere/issues/173)) ([ac7c027](https://github.com/climateinteractive/SDEverywhere/commit/ac7c0277c3676f09222f4c36fc2abb23894fad26)) -* prevent memory leaks in fixed delay initialization ([#160](https://github.com/climateinteractive/SDEverywhere/issues/160)) ([e158b2f](https://github.com/climateinteractive/SDEverywhere/commit/e158b2f3abe0a1419fb3211fc5952e60b1daaac0)), closes [#159](https://github.com/climateinteractive/SDEverywhere/issues/159) -* record variants of a subscripted variable in removeUnusedVariables ([#65](https://github.com/climateinteractive/SDEverywhere/issues/65)) ([f6d7035](https://github.com/climateinteractive/SDEverywhere/commit/f6d70356f814d3936a5e8c13296088ea241a268c)), closes [#64](https://github.com/climateinteractive/SDEverywhere/issues/64) -* refine the ALLOCATE AVAILABLE search algorithm ([#141](https://github.com/climateinteractive/SDEverywhere/issues/141)) ([bca43a0](https://github.com/climateinteractive/SDEverywhere/commit/bca43a049f666289a2c536cc4174718d43efa4d3)), closes [#139](https://github.com/climateinteractive/SDEverywhere/issues/139) -* remove fcmp library and rewrite expressions without using its macros ([#131](https://github.com/climateinteractive/SDEverywhere/issues/131)) ([df76872](https://github.com/climateinteractive/SDEverywhere/commit/df768724c36ded22677ecdf54c0a3019f9cbab8f)), closes [#107](https://github.com/climateinteractive/SDEverywhere/issues/107) -* remove unnecessary memcpy loop in lookup data initialization ([#166](https://github.com/climateinteractive/SDEverywhere/issues/166)) ([b94d9c4](https://github.com/climateinteractive/SDEverywhere/commit/b94d9c4c2c14a08440fc2f769a6e08e33a165f2a)), closes [#165](https://github.com/climateinteractive/SDEverywhere/issues/165) -* set model directory in the sde causes command ([#142](https://github.com/climateinteractive/SDEverywhere/issues/142)) ([44d326e](https://github.com/climateinteractive/SDEverywhere/commit/44d326e3d684f25ad76d9bca3ca7bd2bb7379768)), closes [#140](https://github.com/climateinteractive/SDEverywhere/issues/140) -* take DELAY FIXED value from input when delay time = 0 ([#148](https://github.com/climateinteractive/SDEverywhere/issues/148)) ([328d050](https://github.com/climateinteractive/SDEverywhere/commit/328d05097aa741c09f8be057eed18c05350ce486)), closes [#147](https://github.com/climateinteractive/SDEverywhere/issues/147) -* terminate generated equations with ~~| ([#120](https://github.com/climateinteractive/SDEverywhere/issues/120)) ([44d1c2a](https://github.com/climateinteractive/SDEverywhere/commit/44d1c2a5fe36e11c57ac9828959f77303819c90e)), closes [#119](https://github.com/climateinteractive/SDEverywhere/issues/119) -* use a global replace to join multiple line Vensim equations in comments ([#175](https://github.com/climateinteractive/SDEverywhere/issues/175)) ([678f2bb](https://github.com/climateinteractive/SDEverywhere/commit/678f2bb4aa8f0a1bd5fdbe8e05c355bc4db5d517)) -* use case-insensitive sort and remove trailing whitespace in preprocessor ([#57](https://github.com/climateinteractive/SDEverywhere/issues/57)) ([d58390a](https://github.com/climateinteractive/SDEverywhere/commit/d58390ae9754562b38e54aeb7917605aab54b894)), closes [#55](https://github.com/climateinteractive/SDEverywhere/issues/55) -* use chunkedFunction to break up initLookups into smaller functions ([#133](https://github.com/climateinteractive/SDEverywhere/issues/133)) ([bad4580](https://github.com/climateinteractive/SDEverywhere/commit/bad4580d6d98c8ea66082bbe984542ce84ba9164)), closes [#132](https://github.com/climateinteractive/SDEverywhere/issues/132) -* use correct subdimension index for delay aux vars ([#92](https://github.com/climateinteractive/SDEverywhere/issues/92)) ([7158b0f](https://github.com/climateinteractive/SDEverywhere/commit/7158b0fb45af1cfb9396d22010a41a3df7c917f4)), closes [#91](https://github.com/climateinteractive/SDEverywhere/issues/91) -* use subdim indices for GET DIRECT DATA ([#164](https://github.com/climateinteractive/SDEverywhere/issues/164)) ([d7b46c6](https://github.com/climateinteractive/SDEverywhere/commit/d7b46c6b0720b716d466b40ef975103338e5398c)), closes [#163](https://github.com/climateinteractive/SDEverywhere/issues/163) -* wrap conditional branch expression in parentheses when optimizing IF THEN ELSE ([#153](https://github.com/climateinteractive/SDEverywhere/issues/153)) ([bd42d54](https://github.com/climateinteractive/SDEverywhere/commit/bd42d540e6ef573ce7f713e429eaad61e87398ce)), closes [#152](https://github.com/climateinteractive/SDEverywhere/issues/152) - - -### Performance Improvements - -* cache last input and last accessed index for faster lookups ([#43](https://github.com/climateinteractive/SDEverywhere/issues/43)) ([0933a89](https://github.com/climateinteractive/SDEverywhere/commit/0933a89819cf91000354f45459556fcb212312f3)), closes [#34](https://github.com/climateinteractive/SDEverywhere/issues/34) -* cache parsed CSV file data and replace Array with Set to improve code gen performance ([#168](https://github.com/climateinteractive/SDEverywhere/issues/168)) ([58a45ba](https://github.com/climateinteractive/SDEverywhere/commit/58a45bafda4ccec754b5e60f4fa0045d60b5dcab)), closes [#167](https://github.com/climateinteractive/SDEverywhere/issues/167) -* improve code gen performance by avoiding linear searches ([#63](https://github.com/climateinteractive/SDEverywhere/issues/63)) ([d4bf555](https://github.com/climateinteractive/SDEverywhere/commit/d4bf555bd8c568af8c837eccd135e79a490d5c0f)), closes [#62](https://github.com/climateinteractive/SDEverywhere/issues/62) -* optimize IF THEN ELSE for cases where condition expression resolves to a constant ([#103](https://github.com/climateinteractive/SDEverywhere/issues/103)) ([f9ef675](https://github.com/climateinteractive/SDEverywhere/commit/f9ef67539938ed949a17022df05707a2c06c558a)), closes [#102](https://github.com/climateinteractive/SDEverywhere/issues/102) -* remove variables that are not referenced by input or output variables ([#44](https://github.com/climateinteractive/SDEverywhere/issues/44)) ([6c80c59](https://github.com/climateinteractive/SDEverywhere/commit/6c80c5919d94b9d66c2df3d27f181989ac864000)), closes [#1](https://github.com/climateinteractive/SDEverywhere/issues/1) - -## 0.5.3 (2020-07-29) - -- improved performance of LOOKUP -- optimized dimension name references to avoid array accesses -- changed lookup initialization to use static arrays for improved Wasm performance -- replaced wrapper functions with C macros to reduce function call overhead -- split large functions reduce stack frame size (improves Wasm memory use and performance) - -## 0.5.2 (2020-06-03) - -- includes fixes that more fully automate conversion of complicated MDL model files -- moved tools to Python 3 -- use the updated ANTLR-Version package -- improved support for two-dimensional arrays -- added handling of 2D constant arrays with subscripts in any order -- added support for dimension name references -- added support for ELMCOUNT -- added support for PULSE TRAIN -- updated documentation -- updated npm package dependencies - -## 0.5.1 (2019-09-27) - -- support multiple chartDatfiles delimited by semicolons in app.csv -- override generated app styles in an optional custom.css file in the config folder -- add optional varname prefix to readDat - -## 0.5.0 (2019-07-24) - -- web app generation uses simpler CSV configuration instead of YAML -- three-dimensional arrays -- :EXCEPT: subscripts -- two-dimensional const arrays -- GET DIRECT DATA for Excel at code generation time -- read output variables from DAT files to WITH LOOKUP variables -- generate variable documention in text and YAML formats -- allow all special characters in variable names -- improved coverage of subrange and mapping edge cases - -## 0.4.1 (2018-03-11) - -- enable a blank cell in the HTML input panel with an empty value in `sliders` -- add `sde causes` command to print model variable dependencies -- fixed HTML generation on Linux -- corrected instructions for building from the repo - -## 0.4.0 (2018-02-05) - -- updated web app generation to use an improved template -- added new app.yaml web app specification file -- generate complete web app with the `sde generate --genhtml` command -- removed the Vensim grammar to an independent package -- removed the lotka sample model -- added the SIR sample model -- optimized performance by making high-precision floating point comparisons optional -- added support for generating code to run the model interactively -- removed unnecessary glib2 dependency -- added a warning message when an input or output variable does not exist in the model -- fill in all ref ids for a generated variable that is expanded over a non-apply-to-all array -- implement Vensim data variables in DAT files with lookups