feat: refactor deployer to run locally without auth #810
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
iulianbarbu
reviewed
May 2, 2023
There was a problem hiding this comment.
I have few nits but other than this deploying the postgres-rocket-app fails with this error:
2023-05-02T15:58:26.640221000Z ERROR {error="Load error: Custom error: failed to provision shuttle_shared_db :: Postgres"} shuttle_deployer::deployment::run: service startup encountered an error
Does it work on your end?
LE: the postgres error is covered in the PR description. Following up on it in a separate PR is needed.
iulianbarbu
approved these changes
May 2, 2023
paulotten
pushed a commit
to mikegin/shuttle
that referenced
this pull request
May 4, 2023
* feat: refactor deployer to run locally without auth * docs: update contributing deployer guide * refactor: workspace dep * fix: bump regex to 1.8.1 to fix 1.8.0 bug * fix: provisioner port for local deployer * refactor: renaming * feat: refactor to request token from auth * refactor: remove claims feature from deployer common * refactor: use key/cookie from original request * refactor: implement scopebuilder * fix: clippy * refactor: address review * refactor: auth container command * refactor: cleanup builder in start fn * docs: local arg docs
iulianbarbu
added a commit
that referenced
this pull request
Jun 6, 2023
* cargo-shuttle: separated unix from windows local_run (#823) * docs: add note about init bug to readme (#824) Add a note about #821 to the readme. * ci: fix windows binary build (#825) * ci: test prod ci * ci: restore ci * chore: v0.15.0 (#820) * chore: v0.15.0 * chore: bump examples submodule * revert: protoc removal (#826) * revert: protoc removal * ci: test run release jobs * revert: dockerfile protoc install * revert: deployer prepare.sh protoc install * ci: restore ci * fix: wasm qa casing (#828) * fix: disable docker QA (#830) * fix: disable docker QA * ci: keep running QA * fix: Remove unused project list filtering (#832) * fix: Project filtering is disabled * remove filter flag --------- Signed-off-by: jonaro00 <[email protected]> * add star gif * docs: document how to generate protofiles (#836) * docs: document how to generate protofiles * docs: expand on crate description * feat: refactor deployer to run locally without auth (#810) * feat: refactor deployer to run locally without auth * docs: update contributing deployer guide * refactor: workspace dep * fix: bump regex to 1.8.1 to fix 1.8.0 bug * fix: provisioner port for local deployer * refactor: renaming * feat: refactor to request token from auth * refactor: remove claims feature from deployer common * refactor: use key/cookie from original request * refactor: implement scopebuilder * fix: clippy * refactor: address review * refactor: auth container command * refactor: cleanup builder in start fn * docs: local arg docs * added shuttle console sneak peek * typo-fix * fix: shuttle init --template, reorder subcommands, fix bugs (#792) * shuttle init --template, reorder subcommands, fix bugs * fix review feedback * ci: download sccache instead of compiling it (#859) * ci: download sccache instead of compiling it * ci: fix some typos * chore: more typos * build(docker): Change default provisioner port to 3000 (#852) On MacOs, port 5000 is used by AirPlay receiver which will make the 'make up' command described in CONTRIBUTING.md fail for Mac users * feat: ApiKey newtype to ensure key is always valid format (#835) * feat: ensure API key is valid * feat: use ApiKey in auth * refactor: clean up tests * refactor: don't allocate in parse unless it succeeds * fix: clippy * fix: missing anyhow * feat: impl debug/display for apikey * chore: add `.editorconfig` (#855) This adds a simple `.editorconfig` file to the root of the repository so that different editors/IDEs may pick up the settings. This makes it easier to have consistent formatting while editing, not just after running `cargo fmt`. See [here](https://editorconfig.org) for more details. Co-authored-by: AlphaKeks <[email protected]> * fix: `make test` (#858) * fix: `make test` * fix: update api-key * fix: some panic messages get lost (#854) * fix: some panic messages get lost * feat: tests for loader and bind panics --------- Co-authored-by: Thomas Grimm <[email protected]> * feat: remove /hello from tests/ci (#863) * feat: change "/hello" routes to "/" * feat: use updated examples * fix: clippy complaining about format!{"{url}") being useless post '/hello' removal * chore: run cargo fmt * chore: update examples --------- Co-authored-by: Paul Otten <[email protected]> Co-authored-by: Paul Otten <[email protected]> * feat: add on_new_span impl to runtime Logger (#864) * feat: add on_new_span impl to runtime Logger * PR suggestion Co-authored-by: Pieter <[email protected]> --------- Co-authored-by: Vlad Stepanov <[email protected]> Co-authored-by: Pieter <[email protected]> * misc: rename examples to shuttle-examples (#871) * misc: rename examples to shuttle-examples * misc: update examples repo * chore: v0.16.0 (#881) * chore: v0.16.0 * chore: bump examples * feat(Makefile): add option to disable --detach on make up (#878) * feat(Makefile): add option to disable --detach on make up * fix: minor formatting changes * fix: revert addition of apikey to auth (#886) * fix: revert addition of apikey to auth * fix: display impl is needed for key.to_string() * revert: revert #886 (#887) * Revert "fix: revert addition of apikey to auth (#886)" This reverts commit 7054e6a. * feat: add debug call for malformed api key * add option to use rustls instead of native-tls in `shuttle-shared-db` (#870) * add option to use rustls instead of native-tls in `shuttle-shared-db` * Update CircleCI config Allow specifying features by one for a specific crates. Test `shuttle-shared-db` features one by one. * more readable CI job name * Add comment to CircleCI config * Match doc links with Shuttle Service current doc url (#885) * match doc links with current url * missing test error comparisons * Update/syn 2.0 (#880) * update syn to 2.0 * feat: codegen upgraded to use syn v2 (close #875) --- Co-Authored-by: John Vandenberg <[email protected]> Co-Authored-by: Yatin Maan * fix: appropriate expect statement (#875) * chore: update aws crates (#897) * chore: update aws crates * fix: remove accidental comment * fix: set correct admin scopes in scopebuilder (#899) * chore: bump common to 0.16.2 (#900) * chore: bump common to 0.16.1 * chore: v0.16.2 * Update README.md updated capital * Update README.md updated capitalization of Shuttle * Reimplemented JwtAuthentication with struct-based Future. (#868) * Reimplemented JwtAuthentication with struct-based Future. * More effective encoding * Remove explicit lifetime * Code cleanup * Code cleanup --------- Co-authored-by: root <[email protected]> * docs: add installation instructions for Arch Linux (#902) * feat: show output of failed tests (#907) * feat: show output of failed tests * feat: remove cargo dependency * fix: cargo clippy warnings * fix: Cargo.lock version * ci: release automation on unstable (#816) * ci: push images to unstable on main with approval * ci: rename approve unstable push job * ci: test unstable ssh access * ci: test master access * ci: test ssh access with generated config * ci: try StrictHostKeyChecking no * ci: try circleci user * ci: try host admin.unstable * ci: try unstable deployment * ci: only deploy on push success * ci: restore other jobs * ci: remove sparse registry from deploy * ci: remove test * chore: upgrade salvo in shuttle-salvo (#901) * chore: upgrade salvo in shuttle-salvo * docs: hello_world at root * feat: allow resetting a user's API-key (#857) * feat: allow resetting a users API-key A user should be able to reset their api-key, this is important functionality to have in case users leak their key. This should generate a new api-key and persist it in the users table. Fixes #838 * clippy * chore: some cleanup, code style, etc. * account name from cookie or key * invalidate auth_layer cache * fixes * Add `logout --invalidate-api-key` * Rename to reset, add success msg and login url hint. * fmt --------- Co-authored-by: jonaro00 <[email protected]> * feat(cargo-shuttle): log reconnects and improved error messages (#853) * feat(cargo-shuttle): log reconnects and improved error messages * feat(cargo-shuttle): outputs about the reconnect * chore: cargo fmt * chore: cargo fmt * made last_state optional, added more verbose state outputs, removed Clone trait struct Client and DeployArgs. * refactor: refactoring cargo-shuttle library add optional to last_state, change state output messages to have more information, and removed unused clone trait from stucts * fix: typo in State: Unkown Co-authored-by: Iulian Barbu <[email protected]> * refactor: remove unused code remove the running state arm as this state should be unreachable and remove Eq and PartialEq since last_state is optional now --------- Co-authored-by: Forrest Walker <[email protected]> Co-authored-by: Iulian Barbu <[email protected]> * feat(runtime): Remove dependency on clap (#822) * docs: contributing updates (#918) * fix: deployment state shown as running on startup crash (#919) * fix: deployment state shown as running on startup crash * fix: remove second deployment fetch * chore: promote hyper-reverse-proxy to a workspace dependency (#921) this fixes `cargo vendor` * suggest next logical command (#915) * suggest next command * suggest next command * suggest next command * fix: suggest next command * fix: suggest new command * fix: suggest next command * fix: suggest next command * fix: suggest next command * ci: add windows qa (#812) * ci: add windows qa * ci: why??? * ci: windows qa * ci: stop on error * ci: test wasm * ci: test docker * ci: turn off docker for now * ci: add --template flag * ci: remove /hello from test endpoints * ci: figure out sleep time * ci: fix timeout * ci: disable wasm * ci: restore old * fix(gateway): handle certificate expiration as well (#932) * fix(gateway): handle certificate expiration as well * gateway: commented the renewal check * ci: production deployment automation (#920) * ci: production deployment automation initial commit * ci: incorrect requirement * ci: fix production ssh config * ci: test deduplicated deploy-images workflow * ci: test deduplicated deploy workflow for prod * ci: test deploying to unstable with envs in params * ci: try using $ for env vars in params * ci: fix password types * release(prod): test dry run * release(prod): enable unstable and prod image build/push... ...and deploy * release(prod): test cargo publish dry run * release(prod): cargo-shuttle publish crates conflicts with platform-test * release(prod): improved with speed ups --------- Co-authored-by: Iulian Barbu <[email protected]> * Chore/0.17.0 (#934) * chore: release 0.17.0 * chore: updated deps versions * examples: updated to latest shuttle-examples/main * release(prod): gate against local crates.io patch (#936) * release(prod): fix the missing line break escape (#937) * release(prod): add protoc dependency and fix the crates order (#938) * feat(shuttle-axum) Make AxumService generic to be able to use axum::State with it (#924) * docs: Update links and commands (#948) * update docs etc * remove comment * Update GitHub templates (#945) * chore: update Cargo.lock (#942) * Add helpful error if port cannot be used (#950) * Add helpful error if port cannot be used * Don't println on success * cargo format * fix: --name was ignored when not running from cargo folder (#929) * fix: --name was ignored when not running from cargo folder * fix: remove custom error message to fix tests * Use `unwrap_or` instead of explicit match statement Co-authored-by: jonaro00 <[email protected]> * Fix cargo fmt --------- Co-authored-by: jonaro00 <[email protected]> * chore: bump otel crates and remove protoc dep (#956) * fix: log files packed in archive (#931) * fix: log files packed in archive * fix: change log level to debug * refactor: sanitize all path on the user's proxy (#946) * feat(gateway, cargo-shuttle): implement pagination for project list (#862) * feat(gateway, cargo-shuttle): implement pagination for `project list` This change adds query parameters to the `/projects` endpoint that allow specifying an `offset` and a `limit` to allow for pagination. The `cargo-shuttle` CLI has also been updated to take those as optional parameters for the `project list` subcommand with default values of `0` for `offset` and `10` for `limit`. refactor(deployer, cargo-shuttle): implement pagination for `deployment list` This adds pagination functionality to `cargo shuttle deployment list` just like `cargo shuttle project list` and allows the same parameters. * feat(deployer, cargo-shuttle): implement pagination for `deployment list` This adds the same pagination functionality to `cargo shuttle deployment list` as `cargo shuttle project list`. * Updated from PR suggestions fix(cargo-shuttle): remove long name in page param fix(cargo-shuttle): reorder params in get_deployments path fix(cargo-shuttle): check it limit is 0 in deployments_list to prevent useless code fix(deployer, gateway): Move PaginationDetails struct to common fix(gateway): Chain query builder params * fix(cargo-shuttle, deployer): get rid of warnings Reorders arguments to be consistent between `get_projects_list` and `get_deployments`, and gets rid of an unused import. * feat(deployer): add missing pagination params to get_deployments utoipa * test: add test for out of bound pagination of deploy and projects list * feat(gateway): add order by clause for paginated endpoint * ref: revert having PaginationDetails in common * feat(cargo): improve error message when page == 1 vs page > 1 * feat(common): add message that more page might be available for projects * feat(cargo, deployment): update pagination message for deployment list * style(deployer): reformat perstistence/mod.rs file * feat(deployer): return the deployments starting from the latest updated * feat(cargo-shuttle): update next page message * feat(gateway): add created_at field on projects, use it to sort projects * style(cargo-shuttle): reformat files * test(deployer): fix test with new get_deployments ordering --------- Co-authored-by: AlphaKeks <[email protected]> Co-authored-by: Jocelyn Boullier <[email protected]> Co-authored-by: 73nko <[email protected]> * fix: crossterm/comfytable conflict (#959) * fix: crossterm/comfytable conflict * ci: remove unused protoc installs * feat: pre-installed build environment in deployer (#960) * feat: pre-installed build environment in deployer * Edit list * refactor: un-tangle crossterm/comfytable (#961) * refactor: un-tangle crossterm/comfytable * chore: bump crossterm * docs: comment typo * fix: Ignore span logs below WARN (#958) * fix: Ignore span logs below WARN * modify test * fmt * move to before JsonVisitor, fix test * Convert later * fmt * Flip the comparison like a burger * fix: remove cargo-sort from CONTRIBUTING.md (#966) * fix/release(prod): unstable AWS creds clashed with prod (#970) * chore: v0.18.0 (#972) * chore: v0.18.0 * chore: bump examples * fix: broken cargo.lock after merge --------- Signed-off-by: jonaro00 <[email protected]> Co-authored-by: Iulian Barbu <[email protected]> Co-authored-by: jonaro00 <[email protected]> Co-authored-by: Ivan <[email protected]> Co-authored-by: s e <[email protected]> Co-authored-by: Zisulin Morbrot <[email protected]> Co-authored-by: AlphaKeks <[email protected]> Co-authored-by: AlphaKeks <[email protected]> Co-authored-by: paulotten <[email protected]> Co-authored-by: piewhat <[email protected]> Co-authored-by: Thomas Grimm <[email protected]> Co-authored-by: mikegin <[email protected]> Co-authored-by: Paul Otten <[email protected]> Co-authored-by: Paul Otten <[email protected]> Co-authored-by: Valentin <[email protected]> Co-authored-by: Vlad Stepanov <[email protected]> Co-authored-by: Pieter <[email protected]> Co-authored-by: Xavi <[email protected]> Co-authored-by: Syed Fasiuddin <[email protected]> Co-authored-by: Artūras Šlajus <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: Orhun Parmaksız <[email protected]> Co-authored-by: piewhat <[email protected]> Co-authored-by: Heiko Seeberger <[email protected]> Co-authored-by: Forrest Walker <[email protected]> Co-authored-by: Kieren Davies <[email protected]> Co-authored-by: figsoda <[email protected]> Co-authored-by: Iulian Barbu <[email protected]> Co-authored-by: Boyd Kane <[email protected]> Co-authored-by: Raminder Singh <[email protected]> Co-authored-by: Jocelyn Boullier <[email protected]> Co-authored-by: 73nko <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
This PR makes it much easier to run deployer locally for development by requesting a token from auth in the deployer and setting it on every request. We will now only need to insert the admin user in the auth state, we won't need to set the jwt in the shuttle config.
How Has This Been Tested (if applicable)?
Tested running the deployer locally and sending commands, everything works except for deploying a service with a postgres database, but a mongodb service works fine. I tested the local deployer on main, and that has the same problem. We'll need to figure that out, but maybe in a separate PR.