feat: enable js proto generation #751

ilgooz · 2021-02-12T13:25:00Z

Running starport build or starport serve will start generating proto js types for the app under vue/generated.

Also moved protoc wrapper to its own pkg.

out dir can be configured via `build.proto.js.out`.

fadeev · 2021-02-12T18:35:44Z

For some reason, when I try building locally, it returns:

starport/pkg/protobufjs/js_darwin.go:1:1: expected 'package', found version
make: *** [build] Error 1

(I have pulled with LFS, however, after successfully pulling once, it now tells me Skipping object checkout, Git LFS is not installed., although it is installed)

Gitpod returns an error:

On other branches, Gitpod works.

ilgooz · 2021-02-12T18:55:32Z

Have you run git lfs install?

Gitpod error seems to be happening in their end. Tested with another PR, it's giving the same error.

ilgooz · 2021-02-12T18:56:46Z

For now we can use the branch link: https://gitpod.io/#https://github.com/tendermint/starport/tree/feat/enable-js-proto-gen

fadeev · 2021-02-12T19:07:31Z

Silly mistake 🙂 locally now works! Looking great! 👍

Gitpod still doesn't work, though. I wonder why that is?

@clockworkgr check out this branch!

clockworkgr · 2021-02-12T19:41:54Z

will do

fadeev · 2021-02-13T06:35:07Z

Gitpod doesn't throw an error when you just pushed, but if you start it after a couple of minutes, it doesn't work. I think there might be a problem with prebuilt images and LFS. Because when you just start it, it doesn't pull a prebuilt image, it does everything on the spot.

clockworkgr · 2021-02-13T06:38:46Z

relevant: https://community.gitpod.io/t/adding-git-lfs-support/582 ?

ilgooz · 2021-02-13T07:35:36Z

Branch link doesn't work for me anymore as well. Tho, it seems this is not relavent to this PR. I think we can manually test PRs for now, and I'll make a dedicated PR onto feat/codegen-fx for the fix ~as I find one.

ilgooz · 2021-02-13T15:12:15Z

should be good now. 🌮

lumtis

Everything looks good 👍
We just should handle every error case when reading files

Tested and seems to work fine :)

starport/pkg/protobufjs/protobufjs.go

starport/services/chain/build.go

fadeev

We may need to tweak config slightly, given what we do is build clients, perhaps, having build.client.js is more apt. Also why top-level frontend?

ilgooz · 2021-02-17T06:24:26Z

We may need to tweak config slightly, given what we do is build clients, perhaps, having build.client.js is more apt. Also why top-level frontend?

This generated proto file will never be used directly by the users and it will only contain proto types. Besides, we'll be genating it for each module in the next iteration. So a module will end up having a types.js, rest.js and index.js where index.js is a generated Module class/object from the template of ours and wraps the API of these other two files and provides a high level interface.

We added frontend section to be able to define and customize frontend app's location and ability to choose where to locate generated JS files inside the app. frontend name itself can be changed to whatever makes more sense. Any ideas?

* change cosmosprotoc name to cosmosproto

lumtis · 2021-02-19T13:53:27Z

starport/pkg/giturl/giturl.go

+	"strings"
+)
+
+// UserAndRepo returns only the user and repo portion of the git url u.


Suggested change

// UserAndRepo returns only the user and repo portion of the git url u.

// UserAndRepo returns only the user and repo portion of the git url.

u points to the variable u.

starport/services/scaffolder/init.go

* empty commit * feat: sdk msg discovery from app source (#734) * feat: sdk msg discovery from app source * `pkg/cosmosanalysis/msg.Discover()` discovers types that implements sdk.Msg. * added `pkg/protoanalysis` for proto file analysis. * fix linter * docs * docs Co-authored-by: Lucas Bertrand <[email protected]> Co-authored-by: Lucas Bertrand <[email protected]> * feat: add pkg/protobufjs (#744) * feat: add pkg/protobufjs protobufjs is made available for use as a Go pkg with the high level Generate() API. protobufjs binaries statically added to Starport's source code and they are instantly available. only the relavent protobufjs binary included to the final binary of Starport dependending on the OS. for Linux, it adds 17MB to the total binary size of Starport. * ci: enable lfs * fix lint * cleanup & docs * cleanup & an attempt to add ts defs for generated js * gitpod: install git-lfs * ci(pi): disable them temp because high LFS usage * rm pi files * gitpod: fix lfs * gitpod: fix lfs * gitpod: fix lfs * gitpod: fix lfs * docs: manual installation * gitpod: fix lfs * fix gitpod (#754) * fix(gitpod): drop lfs (#755) * fix(gitpod): drop lfs compress protobufjs binary manually instead of relying on go-bindata's. generated files now under 100MB. * tidy * cleanup * feat(pkg/cosmosanalysis): add more detailed msg info (#752) * feat(pkg/cosmosanalysis): add more detailed msg info most importantly, msg urls added to the analysis output. * docs Co-authored-by: Lucas Bertrand <[email protected]> Co-authored-by: Lucas Bertrand <[email protected]> * feat(nodetime): add sta pkg and refactor (#757) * feat(nodetime): add sta pkg and refactor a new pkg/ called nodetime added to host and use a single NodeJS runtime that has multiple CLIs bundled inside. * `swagger-typescript-api` CLI added alongside `protobufjs` CLI. * switched Starport's Go compiler to 1.16 beta to benefit from new the `embed` feature. it'll be switched to stable 1.16 release once it's available (likely this Feb). * fix integration tests * fix gitpod * feat: upgrade to stable Go 1.16 (#765) * fix: integration tests (#766) * feat: enable js proto generation (#751) * refactor(protoc): wrap protoc cmd as a pkg * enable js code generation for build & server out dir can be configured via `build.proto.js.out`. * generate js per module * change cosmosprotoc name to cosmosproto * docs * fix linter * chore(integration): disable parallel run to not bottleneck the CI * feat(codegen): generate rest client for js (#771) * feat(codegen): generate rest client for js * fix Go codegen * feat(codegen/ts-proto): generate app types with ts-proto (#795) * feat(codegen/ts-proto): generate app types with ts-proto previously, we were using `protobufjs` to generate app types. now, we have switched to `ts-proto` but also kept js support by placing `.js` and `.d.ts` files next to the generated `.ts` files. * refactored `nodetime` packaging. * added more detailed info to `pkg/cosmosanalysis/module.Discover()` output. * added more detailed info to `pkg/protoanalysis.DiscoverPackages()` output. * added plugin binary configurability option to `pkg/protoc` pkg. * improved `pkg/cosmosgen`'s public API. * added `pkg/nodetime/tsc`, `pkg/nodetime/ts-proto` and removed `pkg/nodetime/protobufjs`. * fix lint * fix err check * feat(codegen): generated js client (the wrapper) (#772) * feat(codegen): generate rest client for js * fix Go codegen * feat(codegen): generated js client (the wrapper) it wraps generated protobufjs types and rest client, uses cosmjs and exports high level `txClient()` and `queryClient()` funcs for: * creating messages, signing and broadcasting them, * querying. js client created individually for each module. within Vuex, it can be used like in below: ```js import { DirectSecp256k1HdWallet } from "@cosmjs/proto-signing"; import { txClient, queryClient } from "./store/tendermint/mars/mars/module"; async function main() { const wallet = await DirectSecp256k1HdWallet.fromMnemonic("alfa romeo..."); const t = await txClient(wallet); const q = await queryClient(); console.log(await t.signAndBroadcast([ t.msgCreateUser({ name: "mars", creator: wallet.address }) ])); console.log(await q.queryUserAll()); } main() ``` * fix(sta): sdk module name * docs * feat(codegen/ts-proto): generate app types with ts-proto previously, we were using `protobufjs` to generate app types. now, we have switched to `ts-proto` but also kept js support by placing `.js` and `.d.ts` files next to the generated `.ts` files. * refactored `nodetime` packaging. * added more detailed info to `pkg/cosmosanalysis/module.Discover()` output. * added more detailed info to `pkg/protoanalysis.DiscoverPackages()` output. * added plugin binary configurability option to `pkg/protoc` pkg. * improved `pkg/cosmosgen`'s public API. * added `pkg/nodetime/tsc`, `pkg/nodetime/ts-proto` and removed `pkg/nodetime/protobufjs`. * fix lint * fix err check * migrate wrapper to ts(-proto) and new version of cosmjs * several code generation improvements made. * add customizable fee * fix default amount * feat(codegen): add code generation for 3rd party modules (#797) * feat(codegen): generate rest client for js * fix Go codegen * feat(codegen): generated js client (the wrapper) it wraps generated protobufjs types and rest client, uses cosmjs and exports high level `txClient()` and `queryClient()` funcs for: * creating messages, signing and broadcasting them, * querying. js client created individually for each module. within Vuex, it can be used like in below: ```js import { DirectSecp256k1HdWallet } from "@cosmjs/proto-signing"; import { txClient, queryClient } from "./store/tendermint/mars/mars/module"; async function main() { const wallet = await DirectSecp256k1HdWallet.fromMnemonic("alfa romeo..."); const t = await txClient(wallet); const q = await queryClient(); console.log(await t.signAndBroadcast([ t.msgCreateUser({ name: "mars", creator: wallet.address }) ])); console.log(await q.queryUserAll()); } main() ``` * fix(sta): sdk module name * docs * feat(codegen/ts-proto): generate app types with ts-proto previously, we were using `protobufjs` to generate app types. now, we have switched to `ts-proto` but also kept js support by placing `.js` and `.d.ts` files next to the generated `.ts` files. * refactored `nodetime` packaging. * added more detailed info to `pkg/cosmosanalysis/module.Discover()` output. * added more detailed info to `pkg/protoanalysis.DiscoverPackages()` output. * added plugin binary configurability option to `pkg/protoc` pkg. * improved `pkg/cosmosgen`'s public API. * added `pkg/nodetime/tsc`, `pkg/nodetime/ts-proto` and removed `pkg/nodetime/protobufjs`. * fix lint * fix err check * migrate wrapper to ts(-proto) and new version of cosmjs * several code generation improvements made. * feat(codegen): add code generation for dependency modules optionally enable js related code code generation for the 3rd party modules -including the sdk- that used by an app. * several improvements mode on code generation logic. * fix lint & err checking * fix go codegen * scaffolding(stargate): add generated 3rd party js clients * enable code gen for 3rd party modules on demand * docs * fix: add chain to path (#802) Co-authored-by: Alex Megalokonomos <[email protected]> * chore: sync stargate scaffolding for new js gen path (#803) * feat: Updated vue template for codegen (#807) * feat: Updated vue template * package.json fixes * chrore(ui): update versions , add plain eslint config (#812) * feat: (codegen/vuex): add Vuex code generation & refactor (#824) * codegen(vuex): add Vuex code generation & refactor code generation related packages. * fix err handling & linter errors * fix err handling * fix msg discovery * cosmosanalysis: add metadata about grpc gateway * docs: fix typo * fix linter err * feat: provide more tpl data to loader & add user-side warnings (#857) * feat: provide more tpl data to loader & add user-side warnings about not modifying generated dirs/files. * cosmetic * fix: Template updates (#825) * fix: Template updates * feat: sendMsg* actions, Msg* message creation * feat: Add true Error()s * fix: Change `chain` to `generated` * fix: tsc ignore imported dependency * Updated generated JS/TS for SDK modules * feat: Add get all pages option to queries * fiix: Pass all option to subscription * feat: Add query returns from store * WIP v0.15 Layout * Regenerated JS/TS * fix: stale state, return from getters * Regenerated TS/JS * JS Template changes * Regenerated JS/TS * Additional template changes * fix: Template changes * Regenerated TS/JS * templates: sync pre generated js Co-authored-by: Lucas Bertrand <[email protected]> Co-authored-by: Alex Megalokonomos <[email protected]>

* refactor(protoc): wrap protoc cmd as a pkg * enable js code generation for build & server out dir can be configured via `build.proto.js.out`. * generate js per module * change cosmosprotoc name to cosmosproto * docs

* empty commit * feat: sdk msg discovery from app source (ignite#734) * feat: sdk msg discovery from app source * `pkg/cosmosanalysis/msg.Discover()` discovers types that implements sdk.Msg. * added `pkg/protoanalysis` for proto file analysis. * fix linter * docs * docs Co-authored-by: Lucas Bertrand <[email protected]> Co-authored-by: Lucas Bertrand <[email protected]> * feat: add pkg/protobufjs (ignite#744) * feat: add pkg/protobufjs protobufjs is made available for use as a Go pkg with the high level Generate() API. protobufjs binaries statically added to Starport's source code and they are instantly available. only the relavent protobufjs binary included to the final binary of Starport dependending on the OS. for Linux, it adds 17MB to the total binary size of Starport. * ci: enable lfs * fix lint * cleanup & docs * cleanup & an attempt to add ts defs for generated js * gitpod: install git-lfs * ci(pi): disable them temp because high LFS usage * rm pi files * gitpod: fix lfs * gitpod: fix lfs * gitpod: fix lfs * gitpod: fix lfs * docs: manual installation * gitpod: fix lfs * fix gitpod (ignite#754) * fix(gitpod): drop lfs (ignite#755) * fix(gitpod): drop lfs compress protobufjs binary manually instead of relying on go-bindata's. generated files now under 100MB. * tidy * cleanup * feat(pkg/cosmosanalysis): add more detailed msg info (ignite#752) * feat(pkg/cosmosanalysis): add more detailed msg info most importantly, msg urls added to the analysis output. * docs Co-authored-by: Lucas Bertrand <[email protected]> Co-authored-by: Lucas Bertrand <[email protected]> * feat(nodetime): add sta pkg and refactor (ignite#757) * feat(nodetime): add sta pkg and refactor a new pkg/ called nodetime added to host and use a single NodeJS runtime that has multiple CLIs bundled inside. * `swagger-typescript-api` CLI added alongside `protobufjs` CLI. * switched Starport's Go compiler to 1.16 beta to benefit from new the `embed` feature. it'll be switched to stable 1.16 release once it's available (likely this Feb). * fix integration tests * fix gitpod * feat: upgrade to stable Go 1.16 (ignite#765) * fix: integration tests (ignite#766) * feat: enable js proto generation (ignite#751) * refactor(protoc): wrap protoc cmd as a pkg * enable js code generation for build & server out dir can be configured via `build.proto.js.out`. * generate js per module * change cosmosprotoc name to cosmosproto * docs * fix linter * chore(integration): disable parallel run to not bottleneck the CI * feat(codegen): generate rest client for js (ignite#771) * feat(codegen): generate rest client for js * fix Go codegen * feat(codegen/ts-proto): generate app types with ts-proto (ignite#795) * feat(codegen/ts-proto): generate app types with ts-proto previously, we were using `protobufjs` to generate app types. now, we have switched to `ts-proto` but also kept js support by placing `.js` and `.d.ts` files next to the generated `.ts` files. * refactored `nodetime` packaging. * added more detailed info to `pkg/cosmosanalysis/module.Discover()` output. * added more detailed info to `pkg/protoanalysis.DiscoverPackages()` output. * added plugin binary configurability option to `pkg/protoc` pkg. * improved `pkg/cosmosgen`'s public API. * added `pkg/nodetime/tsc`, `pkg/nodetime/ts-proto` and removed `pkg/nodetime/protobufjs`. * fix lint * fix err check * feat(codegen): generated js client (the wrapper) (ignite#772) * feat(codegen): generate rest client for js * fix Go codegen * feat(codegen): generated js client (the wrapper) it wraps generated protobufjs types and rest client, uses cosmjs and exports high level `txClient()` and `queryClient()` funcs for: * creating messages, signing and broadcasting them, * querying. js client created individually for each module. within Vuex, it can be used like in below: ```js import { DirectSecp256k1HdWallet } from "@cosmjs/proto-signing"; import { txClient, queryClient } from "./store/tendermint/mars/mars/module"; async function main() { const wallet = await DirectSecp256k1HdWallet.fromMnemonic("alfa romeo..."); const t = await txClient(wallet); const q = await queryClient(); console.log(await t.signAndBroadcast([ t.msgCreateUser({ name: "mars", creator: wallet.address }) ])); console.log(await q.queryUserAll()); } main() ``` * fix(sta): sdk module name * docs * feat(codegen/ts-proto): generate app types with ts-proto previously, we were using `protobufjs` to generate app types. now, we have switched to `ts-proto` but also kept js support by placing `.js` and `.d.ts` files next to the generated `.ts` files. * refactored `nodetime` packaging. * added more detailed info to `pkg/cosmosanalysis/module.Discover()` output. * added more detailed info to `pkg/protoanalysis.DiscoverPackages()` output. * added plugin binary configurability option to `pkg/protoc` pkg. * improved `pkg/cosmosgen`'s public API. * added `pkg/nodetime/tsc`, `pkg/nodetime/ts-proto` and removed `pkg/nodetime/protobufjs`. * fix lint * fix err check * migrate wrapper to ts(-proto) and new version of cosmjs * several code generation improvements made. * add customizable fee * fix default amount * feat(codegen): add code generation for 3rd party modules (ignite#797) * feat(codegen): generate rest client for js * fix Go codegen * feat(codegen): generated js client (the wrapper) it wraps generated protobufjs types and rest client, uses cosmjs and exports high level `txClient()` and `queryClient()` funcs for: * creating messages, signing and broadcasting them, * querying. js client created individually for each module. within Vuex, it can be used like in below: ```js import { DirectSecp256k1HdWallet } from "@cosmjs/proto-signing"; import { txClient, queryClient } from "./store/tendermint/mars/mars/module"; async function main() { const wallet = await DirectSecp256k1HdWallet.fromMnemonic("alfa romeo..."); const t = await txClient(wallet); const q = await queryClient(); console.log(await t.signAndBroadcast([ t.msgCreateUser({ name: "mars", creator: wallet.address }) ])); console.log(await q.queryUserAll()); } main() ``` * fix(sta): sdk module name * docs * feat(codegen/ts-proto): generate app types with ts-proto previously, we were using `protobufjs` to generate app types. now, we have switched to `ts-proto` but also kept js support by placing `.js` and `.d.ts` files next to the generated `.ts` files. * refactored `nodetime` packaging. * added more detailed info to `pkg/cosmosanalysis/module.Discover()` output. * added more detailed info to `pkg/protoanalysis.DiscoverPackages()` output. * added plugin binary configurability option to `pkg/protoc` pkg. * improved `pkg/cosmosgen`'s public API. * added `pkg/nodetime/tsc`, `pkg/nodetime/ts-proto` and removed `pkg/nodetime/protobufjs`. * fix lint * fix err check * migrate wrapper to ts(-proto) and new version of cosmjs * several code generation improvements made. * feat(codegen): add code generation for dependency modules optionally enable js related code code generation for the 3rd party modules -including the sdk- that used by an app. * several improvements mode on code generation logic. * fix lint & err checking * fix go codegen * scaffolding(stargate): add generated 3rd party js clients * enable code gen for 3rd party modules on demand * docs * fix: add chain to path (ignite#802) Co-authored-by: Alex Megalokonomos <[email protected]> * chore: sync stargate scaffolding for new js gen path (ignite#803) * feat: Updated vue template for codegen (ignite#807) * feat: Updated vue template * package.json fixes * chrore(ui): update versions , add plain eslint config (ignite#812) * feat: (codegen/vuex): add Vuex code generation & refactor (ignite#824) * codegen(vuex): add Vuex code generation & refactor code generation related packages. * fix err handling & linter errors * fix err handling * fix msg discovery * cosmosanalysis: add metadata about grpc gateway * docs: fix typo * fix linter err * feat: provide more tpl data to loader & add user-side warnings (ignite#857) * feat: provide more tpl data to loader & add user-side warnings about not modifying generated dirs/files. * cosmetic * fix: Template updates (ignite#825) * fix: Template updates * feat: sendMsg* actions, Msg* message creation * feat: Add true Error()s * fix: Change `chain` to `generated` * fix: tsc ignore imported dependency * Updated generated JS/TS for SDK modules * feat: Add get all pages option to queries * fiix: Pass all option to subscription * feat: Add query returns from store * WIP v0.15 Layout * Regenerated JS/TS * fix: stale state, return from getters * Regenerated TS/JS * JS Template changes * Regenerated JS/TS * Additional template changes * fix: Template changes * Regenerated TS/JS * templates: sync pre generated js Co-authored-by: Lucas Bertrand <[email protected]> Co-authored-by: Alex Megalokonomos <[email protected]>

refactor(protoc): wrap protoc cmd as a pkg

93bf722

ilgooz changed the base branch from develop to feat/codegen-fx February 12, 2021 13:25

enable js code generation for build & server

3fb0b15

out dir can be configured via `build.proto.js.out`.

ilgooz force-pushed the feat/enable-js-proto-gen branch from 59dc2ae to 3fb0b15 Compare February 12, 2021 13:28

ilgooz marked this pull request as ready for review February 12, 2021 13:32

ilgooz requested review from fadeev and lumtis as code owners February 12, 2021 13:32

ilgooz added 2 commits February 13, 2021 00:23

Merge branch 'feat/codegen-fx' into feat/enable-js-proto-gen

16fbfbe

Merge branch 'feat/codegen-fx' into feat/enable-js-proto-gen

0a59fce

lumtis suggested changes Feb 16, 2021

View reviewed changes

starport/pkg/protobufjs/protobufjs.go Outdated Show resolved Hide resolved

starport/services/chain/build.go Show resolved Hide resolved

fadeev approved these changes Feb 16, 2021

View reviewed changes

ilgooz added 2 commits February 19, 2021 03:58

Merge branch 'feat/codegen-fx' into feat/enable-js-proto-gen

c504a86

generate js per module

0dbe9de

* change cosmosprotoc name to cosmosproto

ilgooz requested review from lumtis and fadeev February 19, 2021 13:46

lumtis approved these changes Feb 19, 2021

View reviewed changes

docs

c112e16

ilgooz merged commit 35b235a into feat/codegen-fx Feb 19, 2021

ilgooz deleted the feat/enable-js-proto-gen branch February 19, 2021 14:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: enable js proto generation #751

feat: enable js proto generation #751

ilgooz commented Feb 12, 2021 •

edited

Loading

fadeev commented Feb 12, 2021 •

edited

Loading

ilgooz commented Feb 12, 2021

ilgooz commented Feb 12, 2021

fadeev commented Feb 12, 2021 •

edited

Loading

clockworkgr commented Feb 12, 2021

fadeev commented Feb 13, 2021

clockworkgr commented Feb 13, 2021

ilgooz commented Feb 13, 2021 •

edited

Loading

ilgooz commented Feb 13, 2021

lumtis left a comment •

edited

Loading

fadeev left a comment

ilgooz commented Feb 17, 2021

lumtis Feb 19, 2021

ilgooz Feb 19, 2021

	// UserAndRepo returns only the user and repo portion of the git url u.
	// UserAndRepo returns only the user and repo portion of the git url.

feat: enable js proto generation #751

feat: enable js proto generation #751

Conversation

ilgooz commented Feb 12, 2021 • edited Loading

fadeev commented Feb 12, 2021 • edited Loading

ilgooz commented Feb 12, 2021

ilgooz commented Feb 12, 2021

fadeev commented Feb 12, 2021 • edited Loading

clockworkgr commented Feb 12, 2021

fadeev commented Feb 13, 2021

clockworkgr commented Feb 13, 2021

ilgooz commented Feb 13, 2021 • edited Loading

ilgooz commented Feb 13, 2021

lumtis left a comment • edited Loading

Choose a reason for hiding this comment

fadeev left a comment

Choose a reason for hiding this comment

ilgooz commented Feb 17, 2021

lumtis Feb 19, 2021

Choose a reason for hiding this comment

ilgooz Feb 19, 2021

Choose a reason for hiding this comment

ilgooz commented Feb 12, 2021 •

edited

Loading

fadeev commented Feb 12, 2021 •

edited

Loading

fadeev commented Feb 12, 2021 •

edited

Loading

ilgooz commented Feb 13, 2021 •

edited

Loading

lumtis left a comment •

edited

Loading