Releases: shuttle-hq/shuttle
v0.19.0
shuttle: v0.19.0 update
We're excited to release shuttle v0.19.0! 🚀
Upgraded to Rust 1.70
Our deployers (the containers in which your service will be built and ran) have a pinned version of Rust. This has now been upgraded to 1.70, thanks @iamwacko from the Shuttle batch!
Init from any template
Our cargo shuttle init
command has been refactored to now clone templates from github repositories. To get a hello world example from our repository, you can simply run:
cargo shuttle init -t axum
.
You can also initialize a project from any template by passing in the url of any repository and the sub-folder you want to clone. Below is an example of how you would init an axum project from our websocket example.
cargo shuttle init --from https://github.com/shuttle-hq/shuttle-examples.git --subfolder axum/websocket
Thanks to @paulotten and @jonaro00 from the Shuttle batch for this great new functionality!
The sixth and seventh weeks of Shuttle Batch 2023
The Shuttle batch is nearing the end, but their contributions are still forming the majority of this release. We also have several large batch contributions implementing new features like DynamoDB, Turso and Sqlite in the pipeline.
- @iamwacko updated our pinned version of Rust to 1.70 and fixed some new clippy lints in #967
- @iamwacko removed the cargo dependency of the builder #922
- @jonaro00 fixed a bug where the
cargo shuttle logs --latest
was returning the oldest deployment’s logs due to the order being flipped in the recent pagination PR in #982 - @hseeberger and @jonaro00 added some relevant git data to the metadata of deployments, in #943. This will be very useful in our upcoming console!
- @iamwacko refactored the Shuttle codebase to use
rustls
rather thannative-tls
withopenssl
in #879 - @paulotten refactored our
init
code to rather clone examples from our repository using thecargo-generate
crate under the hood in #888 - @jonaro00 implemented IntoIter for our secretstore in #1006
Other contributions
- @laundmo documented the format of
Secrets.toml
in our docs in shuttle-hq/shuttle-docs#114 - @beyarkay expanded the
shuttle_aws_rds
documentation in our docs in shuttle-hq/shuttle-docs#115, added a cookbook to our CLI docs in shuttle-hq/shuttle-docs#126 - @beyarkay added a link to our docs in the CLI help output in #985
- @lecoqjacob improved the error message in the event that a user runs
cargo run
rather thancargo shuttle run
in #993
Upgrading
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall), cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the restart
project command. This will not delete any databases, and you will keep your project name:
cargo shuttle project restart
Finally, redeploy your shuttle service:
cargo shuttle deploy
Commits in this release
- feat(service): get rid of
cargo
dependency by @iamwacko in #922 - ci: bump pinned Rust to 1.70 by @iamwacko in #967
- fix: logs --latest flipped order by @jonaro00 in #982
- feat(deployer): add more deployment metadata by @hseeberger in #943
- feat: switch from native-tls to rustls by @iamwacko in #879
- feat: use
cargo generate
instead of hardcoding examples source code by @paulotten in #888 - fix: re-add
--version
flag by @paulotten in #998 - fix: remove auth login endpoint by @oddgrd in #1007
- Add commands cookbook to cargo-shuttle --help by @beyarkay in #985
- docs: add sections on PR and reviews by @oddgrd in #1016
- feat(secrets): Implement into_iter for SecretStore by @jonaro00 in #1006
- fix: mysql provisioning bug by @oddgrd in #1001
- provide better error message by @lecoqjacob in #993
- feat: shuttle init --from by @jonaro00 in #984
- chore: v0.19.0 by @oddgrd in #1019
New Contributors
- @lecoqjacob made their first contribution in #993
Full Changelog: v0.18.0...v0.19.0
v0.18.0
shuttle: v0.18.0 update
We're excited to release shuttle v0.18.0! 🚀
Removal of protoc
requirement
With the last of our upstream crates upgraded to no longer need protoc
available at compile-time, installingprotoc
will no longer be required to install or use Shuttle! 🥳
Pagination of list commands
Two of our list commands can now be paginated, namely project list
and deployment list
. These commands will now by default fetch the 10 latest deployments, a --page 2
flag can then be passed to fetch the next 10 deployments, and so on. The limit of how many to fetch per page can also be configured with the --limit
flag.
Thanks to @AlphaKeks and @Kazy from the Shuttle batch for this feature! PR: #862
Sanitizing paths in our proxy layer
@chesedo developed and published a new crate, [tower-sanitize-path](https://crates.io/crates/tower-sanitize-path), for sanitizing paths with tower, and implemented it in our proxy layer to protect users against path traversals in #946
The fourth and fifth weeks of Shuttle Batch 2023
The Shuttle batch continues with a lot of great contributions! We’ve also seen tremendous progress on larger tickets, so keep your eyes peeled for some larger new features in the next releases! Here are the batch contributions for the last two weeks:
- @morlinbrot added support for axum state #924, as well as an example for using it in shuttle-hq/shuttle-examples#52 (review)
- @jonaro00 updated some docs in the main repo in https://github.com/shuttle-hq/shuttle/pull/948/files.
- @jonaro00 updated and improved our github issue and PR templates in #945
- @AlphaKeks and @Kazy implemented pagination for the project/deployment list endpoints, also flags for this to our CLI in #862
- @jonaro00 added some common C build tools to our deployers, allowing Shuttle projects to use crates that depend on them in #960
- @jonaro00 fixed an issue where we were recording too much log output by default, since this was causing issues for certain projects, particularly serenity bots, in #958
- @paulotten updated our
CONTRIBUTING.md
document to no longer call for usingcargo-sort
, since it doesn’t work with workspace inheritance in #966
Other contributions
- @orhun updated our lockfile in #942
- @beyarkay improved the panic message in the event where the shuttle-runtime port is already taken in #950
- @imor fixed an issue where cargo-shuttle commands that were passed
--name
could not be ran outside a cargo project in #929 - @imor added a
tracing::debug!
call tocargo-shuttle
to output which files are archived for deployment in #931
Upgrading
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall), cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the restart
project command. This will not delete any databases, and you will keep your project name:
cargo shuttle project restart
Finally, redeploy your shuttle service:
cargo shuttle deploy
Commits for this release
- feat(shuttle-axum) Make AxumService generic to be able to use state by @morlinbrot in #924
- docs: Update links and commands by @jonaro00 in #948
- misc: Update GitHub templates by @jonaro00 in #945
- Update Cargo.lock by @orhun in #942
- Add helpful error if port cannot be used by @beyarkay in #950
- fix: --name was ignored when not running from cargo folder by @imor in #929
- chore: bump otel crates and remove protoc dep by @oddgrd in #956
- fix: log files packed in archive by @imor in #931
- refactor: sanitize all path on the user's proxy by @chesedo in #946
- feat(gateway, cargo-shuttle): implement pagination for project list by @AlphaKeks in #862
- fix: crossterm/comfytable conflict by @oddgrd in #959
- feat: pre-installed build environment in deployer by @jonaro00 in #960
- refactor: un-tangle crossterm/comfytable by @oddgrd in #961
- fix: Ignore span logs below WARN by @jonaro00 in #958
- fix: remove cargo-sort from CONTRIBUTING.md by @paulotten in #966
- fix/release(prod): unstable AWS creds clashed with prod by @iulianbarbu in #970
- chore: v0.18.0 by @oddgrd in #972
New Contributors
Full Changelog: v0.17.0...v0.18.0
v0.17.0
shuttle: v0.17.0 update
We're excited to release shuttle v0.17.0! 🚀
Second and third week of Shuttle Batch 2023
We are extremely excited to see how the batch participants progress on the small and medium-sized projects. We had all round contributions and enjoyed seeing the participants taking the lead and clarifying their projects with the maintainer team, seeking the biggest impact. We are very happy to see the help they offer to each other, the encouraging words, and the team spirit that makes them push through. Thanks to everyone and we’re looking forward to seeing more of this!
As for the contributions, here is the list:
- @iamwacko added an optional flag to allow not running our local development environment in detached mode in #878
- @XaviFP fixed a broken link in our codegen errors in #885
- @utterstep added a feature-flag to
shuttle-shared-db
to allow users to userustls
rather thannative-tls
if they want to in #870 - @SyedFasiuddin upgraded our codegen to syn 2.0 in #880
- @paulotten upgraded our aws sdk crates in #897
- @piewhat removed the
cargo
dependency from deployer, and also refactored the test run on deploys to show the output of failed tests in #907 - @jonaro00 and @hseeberge1r implemented a new feature that allows users to reset their api-keys, which is useful if you leaked your key, in #857
- @SyedFasiuddin contributed to adding next logical suggestions for various cargo-shuttle commands: #915.
Other contributions
- @arturaz improved the performance of our JwtAuthentication layer by manually implementing Future for it in #868
- @kierendavies removed the
clap
dependency fromshuttle-runtime
, replacing it with a declarative macro, ensuring we don’t leak theclap
dependency to users projects in #822 - @figsoda added
hyper-reverse-proxy
as a workspace dependency ****used by both the deployer and gateway in #921 - @merlinbrot added an awesome
request-scheduler
example showcasing how to do cron-like schedules for accessing a URL in shuttle-hq/shuttle-examples#47 - @joshua-mo-143 added a SaaS full-stack application powered by Next.js and Shuttle: shuttle-hq/shuttle-examples#50.
cargo-shuttle is now packaged for Arch Linux
@orhun has packaged cargo-shuttle
for Arch Linux, and he has graciously volunteered to keep the package updated. He added installation instructions to our readme in #902 Thanks!
You can now install cargo-shuttle on Arch Linux with pacman:
pacman -S cargo-shuttle
Shuttle Salvo integration version bump
shuttle-salvo has been upgraded to v0.41, this means that to use version 0.17.0 of shuttle-salvo
you’ll also need to update your project to use the new version.
Upgrading
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall), cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the restart
project command. This will not delete any databases, and you will keep your project name:
cargo shuttle project restart
Finally, redeploy your shuttle service:
cargo shuttle deploy
v0.16.0
shuttle: v0.16.0 update
We're excited to release shuttle v0.16.0! 🚀
First week of Shuttle Batch 2023
This marks the first release since the start of this year’s batch, bringing a lot of new contributors to Shuttle. We’ve already had several pull requests, solving a lot of long-standing issues.
- @piewhat and @pedromfedricci fixed a bug where panic messages from users services that were owned strings were not captured in the runtime logs in #854
- @vroussea and @utterstep fixed a bug were some tracing data was lost due to a missing
on_new_span
impl on ourLogger
in #864 - @paulotten and @mikegin refactored our examples to serve “Hello, world!” at
/
and updated our relevant tests in #863 - @paulotten fixed our end to end tests in #858
- @AlphaKeks added an
.editorconfig
in #855 - @morlinbrot changed the default port for the
provisioner
to make contributing easier for contributors with AirPlay receivers in #852 - @iamwacko improved the efficiency of our CI in #859
- @jonaro00 cleaned up some dead code in #832
Updated init command
@jonar00, also from the shuttle batch, made a series of improvement to our init
command, making it a lot easier to implement for new service implementations. He also made some changes to the API, to initialize with a certain framework you now first need to pass the new --template
(or -t
) flag.
# before
cargo shuttle init --axum
# now
cargo shuttle init --template axum
# or
cargo shuttle init -t axum
In the future we intend to refactor init
to initialize from any template by passing in a URL, for example a link to one of our examples.
Upgrading
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall), cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the restart
project command. This will not delete any databases, and you will keep your project name:
cargo shuttle project restart
Finally, redeploy your shuttle service:
cargo shuttle deploy
All commits for this release
- fix: wasm qa casing by @oddgrd in #828
- fix: disable docker QA by @oddgrd in #830
- fix: Remove unused project list filtering by @jonaro00 in #832
- docs: document how to generate protofiles by @oddgrd in #836
- feat: refactor deployer to run locally without auth by @oddgrd in #810
- fix: shuttle init --template, reorder subcommands, fix bugs by @jonaro00 in #792
- ci: download sccache instead of compiling it by @iamwacko in #859
- build(docker): Change default provisioner port to 3000 by @morlinbrot in #852
- feat: ApiKey newtype to ensure key is always valid format by @oddgrd in #835
- chore: add
.editorconfig
by @AlphaKeks in #855 - fix:
make test
by @paulotten in #858 - fix: some panic messages get lost by @piewhat in #854
- feat: remove /hello from tests/ci by @mikegin in #863
- feat: add on_new_span impl to runtime Logger by @vroussea in #864
- misc: rename examples to shuttle-examples by @chesedo in #871
- chore: v0.16.0 by @oddgrd in #881
New Contributors
- @iamwacko made their first contribution in #859
- @morlinbrot made their first contribution in #852
- @AlphaKeks made their first contribution in #855
- @paulotten made their first contribution in #858
- @piewhat made their first contribution in #854
- @mikegin made their first contribution in #863
- @vroussea made their first contribution in #864
Full Changelog: v0.15.0...v0.16.0
v0.15.0
shuttle: v0.15.0 update
We're excited to release shuttle v0.15.0! 🚀
Improving our pre-release QA
To prevent regressions we need to do thorough QA of the changes we make for new releases. To prevent this from slowing down releases and development, for this release we have started implementing automated QA in our CI. For this first iteration we will do an end-to-end test of installing cargo-shuttle
, doing a local run and a deploy on macOS and Linux (a workflow for windows should be ready for the next release).
Updated CLI command and first bounty 💎
For this release we had our first approved bounty contribution, congratulations to @jonaro00!
To claim the bounty they submitted an implementation of a new flag for the logs command in the CLI:
cargo shuttle logs —latest
This command is a convenient way to get the logs of the last deployment, so you will not need to do a deployment list
, get the ID and then a cargo shuttle logs <id>
to get the latest one. All the existing functionality of the command remains the same.
Error message improvements
For this release we have addressed some poor or missing error messages:
- Updated the error messages on commands that 404 when a deployment and/or service is missing with more information and a tip to resolve it.
- Validate the input in the interactive login from
cargo shuttle login
, to ensure incorrectly pasted keys will prompt the user to re-submit.
Bug fixes
- Fixed a bug where cancelling a local run or a local run crashing would not free the port bound by the service, leading to crashes on subsequent runs #805
- Fixed a bug where we were inadvertently logging secrets in the deployer #814
Contributions
- @jonaro00 added the
logs --latest
flag, and also ensured the list of deployments is correctly sorted in #799 - @sentinel1909 expanded our docs FAQ in shuttle-hq/shuttle-docs#70
- @jhawkesworth added an example of how to use rocket dynamic templates with shuttle in shuttle-hq/shuttle-examples#38
Upgrading
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall), cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the restart
project command. This will not delete any databases, and you will keep your project name:
cargo shuttle project restart
Finally, redeploy your shuttle service:
cargo shuttle deploy
Commits for this release
- feat: add algora shields to readme by @oddgrd in #793
- ci: linux QA by @chesedo in #800
- feat: Add
cargo shuttle logs --latest
by @jonaro00 in #799 - feat: improve deployer 404 messages by @oddgrd in #796
- deployer & gateway: added OpenAPI docs by @iulianbarbu in #794
- docs: clarify shuttle_static_folder will not be adjacent to executable by @jhawkesworth in #803
- fix: minor development bug fixes by @oddgrd in #806
- ci: add mac qa by @chesedo in #801
- chore: bump otel crates to remove protoc requirement by @oddgrd in #802
- feat: ensure interactive login API key is valid format by @oddgrd in #797
- fix: stop logging the full loadresponse by @oddgrd in #814
- fix(common): feature flagged utoipa dependency by @iulianbarbu in #817
- fix: apply admin layer to admin routes only by @oddgrd in #818
- cargo-shuttle: fix address in use error when service panicked in a previous run by @iulianbarbu in #805
- cargo-shuttle: separate local runs per target family by @iulianbarbu in #823
- docs: add note about init bug to readme by @oddgrd in #824
- ci: fix windows binary build by @oddgrd in #825
- chore: v0.15.0 by @oddgrd in #820
- revert: protoc removal by @oddgrd in #826
Full Changelog: v0.14.0...v0.15.0
v0.14.0
shuttle: v0.14.0 update
We're excited to release shuttle v0.14.0! 🚀
Workspace support
We now have full support for cargo workspaces! We have had limited support in the past, where only a workspace where the shuttle project was the root crate was deployable. Following this release, you will be able to have a regular workspace. We will search the workspace for shuttle projects, and in this first iteration we will start all of them for local runs, but only the first one we find for deployments.
We intend to allow starting several services in deployment too. The long term plan is to have one central service that routes to the other services in the project, sharing resources amongst them. Any feedback on what people would like to see in this area is most welcome.
Note: When deploying a workspace where the shuttle service uses shuttle-static-folder
, the static folder must be placed in the workspace root.
Changes to the project
commands
We have a new project
command: cargo shuttle project restart
. This command will replace the old flow of project rm
→ project new
to restart a project.
In addition to that, the following cargo shuttle project
commands have been renamed:
new
has been renamed tostart
rm
has been renamed tostop
Their functionality remains the same, they will start and stop your project’s container.
Thanks to @jonaro00 for suggesting, implementing and documenting these changes!
Removing the protoc
requirement for cargo-shuttle
We have removed the need for protoc
to install cargo-shuttle
, but protoc
will still be required to build, run and deploy shuttle services. We intend to remove the protoc
requirement entirely in the next release.
Bug fixes
- Fixed a bug where a deployment might return an error due to not waiting on the deployment state to update, thanks @jonaro00! #744
- Fixed a bug on windows where the path to the static folder was incorrect for local runs: #762
- Fixed a bug where setting the
local_uri
for databases would cause deployments to fail: #786
Contributions
- In a series of PRs to shuttle and upstream crates, @kierendavies has resolved the issues with using
podman
for local development of shuttle, as well as updating the documentation for how to do so. https://github.com/shuttle-hq/shuttle/pull/722 https://github.com/shuttle-hq/shuttle/pull/723 https://github.com/shuttle-hq/shuttle/pull/724 - @jonaro00 renamed our project commands and implemented a
restart
command. He also updated our docs and examples to reflect the change. #771 shuttle-hq/shuttle-docs#62 shuttle-hq/shuttle-examples#36 - @jhawkesworth updated our git submodule to allow cloning via both git and https in #776
- @sentinel1909 documented the ways to configure a shuttle project name in shuttle-hq/shuttle-docs#57
- @kierendavies enabled exhaustiveness check of cargo-shuttle command matching in #768
Upgrading
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall), cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the restart
project command. This will not delete any databases, and you will keep your project name:
cargo shuttle project restart
Finally, redeploy your shuttle service:
cargo shuttle deploy
Commits for this release
- gateway: expect on refreshing projects by @iulianbarbu in #757
- bug: blocked channel on gateway worker by @chesedo in #758
- gateway/certs: serve certificate as default by @iulianbarbu in #760
- Fix gateway clippy by @iulianbarbu in #761
- fix: windows path canonicalization bug in static folder by @oddgrd in #762
- chore: bump static folder to 0.13.1 by @oddgrd in #764
- chore: Update bollard to v0.14.0 by @kierendavies in #722
- Reference container images by full name by @kierendavies in #723
- Update Makefile for better Podman support by @kierendavies in #724
- feat: commit generated proto files by @oddgrd in #753
- feat: compile an entire workspace by @chesedo in #767
- tests: make provisioner test deterministic by @chesedo in #770
- refactor: getting project name by @chesedo in #774
- feat: use sparse registry in deployers by @oddgrd in #773
- feat: start all services in a workspace for local runs by @oddgrd in #772
- fix: Timing of deployment status and local run printouts by @jonaro00 in #744
- refactor: Enable exhaustiveness check of command matching by @kierendavies in #768
- refactor: get the static folder name from the by @chesedo in #780
- feat: Rename
project new/rm
tostart/stop
, addrestart
+ other args fixes by @jonaro00 in #771 - feat examples: switch to relative url for examples submodule to allow… by @jhawkesworth in #776
- fix: is_dirty path bug on windows by @oddgrd in #783
- Docs/add oss tenets by @oddgrd in #782
- Remove cargo from cargo shuttle by @gautamprikshit1 in #765
- docs: update deployer local run guide by @oddgrd in #784
- runtime: infer environment based on storage type by @iulianbarbu in #786
- fix: secrets not archived in workspace crates by @oddgrd in #785
- fix: revert use of portpicker for local run by @oddgrd in #787
- chore: v0.14.0 by @oddgrd in #788
- fix: cargo shuttle integration tests, project cmd renaming by @oddgrd in #789
Full Changelog: v0.13.0...v0.14.0
v0.13.0
shuttle: v0.13.0 update
We're excited to release shuttle v0.13.0! 🚀
Changes to the cargo-shuttle CLI
We added a new command to list all the provisioned resources for your project, like databases and secrets: cargo shuttle resource list
. With the addition of the new resource command, the cargo shuttle status
command will no longer list resources.
Note: Due to the changes to the deployer API required for the resource list
command, if you do a project rm
and project new
after this release you will also need to upgrade your CLI.
Upgrading our pinned Rust version
In this release we bump our pinned version of Rust to 1.68, previously 1.65, allowing users to use the new language features introduced in these updates. To be able to take advantage of this, you will need to update your project container by running cargo shuttle project rm
followed by cargo shuttle project new
.
Contributions
- @jonaro00 bumped our dependencies and moved all the remaining duplicated dependencies to the workspace manifest. Thank you! #736
- @jonaro00 clarified our protoc install guide and made some general improvements to the docs in shuttle-hq/shuttle-docs#56
- @jonaro00 created a feature request template for our repository in #737
- @d2weber made their first contribution to shuttle by updating our readme in #698
- @meskill made their first contribution to shuttle by fixing some broken buttons on our website: shuttle-hq/www#39
- @sentinel1909 updated one of our blog posts with the 0.12 changes in shuttle-hq/www#44
Upgrading
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall), cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the rm
and new
project commands. This will not delete any databases, and you will keep your project name:
cargo shuttle project rm
cargo shuttle project new
Finally, redeploy your shuttle service:
cargo shuttle deploy
Commits for this release
- feat: admin command to destroy projects by @chesedo in #729
- ci: protoc install by @chesedo in #731
- Update README.md by @d2weber in #698
- Add feature suggestion issue template by @jonaro00 in #737
- chore: bump rust and cargo to 1.68 by @oddgrd in #738
- Renew LetsEncrypt certificates by @iulianbarbu in #641
- feat: resources endpoint by @chesedo in #740
- chore: update workspace dependencies by @jonaro00 in #736
- Cargo.lock: remove patch unused occurrences by @iulianbarbu in #742
- feat: record resources in codegen by @chesedo in #746
- Feat: update resources with resourcebuilder changes by @oddgrd in #747
- docs: update contributing project structure by @oddgrd in #745
- refactor: switch to resource endpoint by @chesedo in #748
- fix: unknown resource type by @oddgrd in #749
- feat: polish CLI after 0.13 updates by @oddgrd in #750
- chore: v0.13.0 by @oddgrd in #755
Full Changelog: v0.12.0...v0.13.0
v0.12.0
shuttle: v0.12.0 update
We're excited to release shuttle v0.12.0! 🚀
Removing our unsafe FFI
As of this release Shuttle will no longer rely on compiling users projects as dynamic libraries, we have refactored the codegen to embed the required code to configure and add resources to a users project binary. This means we can get rid of the unsafe FFI needed to load the dynamic libraries, which was a big source of memory issues (especially for local runs). As a result of the refactor, we’ll be able to:
- Get rid of the
Sync
requirement for services. - Get rid of the extra
tokio::Runtime
that is currently required for shuttle resources to be started on the correct side of the FFI (e.g.shuttle_shared_db
,shuttle_secrets
). - Correctly stop deployments.
- Remove the version pin on
tokio
(as of writing it is pinned to 1.22).
Shuttle-next framework
Shuttle-next is a brand new WASM web-framework based on axum and hyper. There are many benefits to using WASM for a backend web-framework, but one of the key features is the isolation it offers. Since WASM modules have no notion of a file system, a shuttle-next project can only interact with the host through resources explicitly declared by us. In this first iteration, the only resource available is an HTTP stream to and from a project.
Shuttle-next is very much a work in progress, and we're releasing this iteration without some of the
features that make shuttle a pleasure to use, for example the ability to effortlessly deploy and connect to a database. It's also not currently possible to use any middleware. We're planning to implement this functionality, and more, in the near future. We're also hoping that the feedback we get from this release will help us see which features we should prioritize, and which areas of the framework's design need the most attention going forward.
Below is a small example of shuttle-next
, see the example in our docs for more information.
Note: shuttle-next does not currently work on Windows, it only works on unix systems (i.e mac, Linux and Windows Subsystem for Linux). We aim to resolve this in future releases.
// lib.rs
shuttle_next::app! {
#[shuttle_next::endpoint(method = get, route = "/hello")]
async fn hello() -> &'static str {
"Hello, World!"
}
}
Extracting the service integrations
The service implementations for frameworks like Axum, Rocket, Serenity etc., have now been extracted into separate crates. You will now need to depend on that crate, e.g shuttle_axum = "0.12.0"
, rather than shuttle-service = { version = "0.12.0", features = ["web-axum"] }
.
Bug fixes
- Fixed a bug where windows users had to do a
cargo build
before they couldcargo shuttle run
: #721 - Fixed a bug where the file contents of static assets in the
shuttle_static_files
directory were deleted in local runs: #717 - Fixed a bug where passing a path to
cargo shuttle init
would return an error: #706
Protoc requirement
Due to new dependencies we can’t go without users will now need to install protoc
to install and use cargo-shuttle
and our other crates. We tried to find a way around this requirement, but it came at the cost of significantly increased build times. We know this extra step required before being able to use shuttle is far from ideal, and we’ll try to find a good way around it in the future.
Mac and Linux
To install protoc
from the distributed binary, you can use this script:
# replace <arch> with your architecture, for example for linux x86_64 replace <arch> with: linux-x86_64
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v21.9/protoc-21.9-<arch>.zip &&\
sudo unzip -o protoc-21.9-<arch>.zip -d /usr/local bin/protoc &&\
sudo unzip -o protoc-21.9-<arch>.zip -d /usr/local 'include/*' &&\
rm -f protoc-21.9-<arch>.zip
If you'd like to do this yourself without the above script, see instructions here: https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os
On mac you can also install protoc with brew
: brew install protobuf
.
Note: installing protoc
on Linux with apt-get install protobuf-compiler
will not be enough, since it will install an old version of protoc
(v3.12) but it needs to be at least v3.19. Installing on mac using brew works.
Windows
On Windows the simplest way to install protoc
is to first install chocolatey, and then simply run: choco install protoc
Upgrading
Due to the nature of these changes, all projects will be stopped and forced to upgrade. Please see instructions here for how to upgrade, as well as our updated examples and updated docs
First, since your project will now be compiled as a binary and no longer a library, you need to convert it to a binary. That means renaming the lib.rs
file where your shuttle_service::main
function currently is to main.rs
, and you’ll also need to remove the [lib]
field from your Cargo.toml
.
We have also made some changes to the imports, you’ll now need to import shuttle-runtime
and tokio
, as well as the service integration for your framework. See the example below.
# Before for axum apps
axum = "0.6"
shuttle-service = { version = "0.11.0", features = ["web-axum"] }
sync_wrapper = "0.1.1"
# After for axum apps
axum = "0.6"
shuttle-axum = "0.12.0"
shuttle-runtime = "0.12.0"
tokio = "1.26.0"
# Before for serenity bots
serenity = { version = "0.11.5", default-features = false, features = ["client", "gateway", "rustls_backend", "model"] }
shuttle-secrets = "0.11.0"
shuttle-service = { version = "0.11.0", features = ["bot-serenity"] }
# After for serenity bots
serenity = { version = "0.11.5", default-features = false, features = ["client", "gateway", "rustls_backend", "model"] }
shuttle-runtime = "0.12.0"
shuttle-serenity = "0.12.0"
shuttle-secrets = "0.12.0"
tokio = "1.26.0"
Since the service integrations now live in a different crate from where the Service
trait is defined, they need to be wrapped in a new-type for us to be able to implement Service
for them. This means you need to wrap the return type from your shuttle_runtime::main
function in this type, but he have implemented From
from for all the integrations, so you can simply return Ok(router.into())
. See an example of upgrading an axum app below, and see our updated examples for other integrations:
// Before in src/lib.rs
use axum::{routing::get, Router};
use sync_wrapper::SyncWrapper;
async fn hello_world() -> &'static str {
"Hello, world!"
}
#[shuttle_service::main]
async fn axum() -> shuttle_service::ShuttleAxum {
let router = Router::new().route("/hello", get(hello_world));
let sync_wrapper = SyncWrapper::new(router);
Ok(sync_wrapper)
}
// After in src/main.rs
use axum::{routing::get, Router};
async fn hello_world() -> &'static str {
"Hello, world!"
}
#[shuttle_runtime::main]
async fn axum() -> shuttle_axum::ShuttleAxum {
let router = Router::new().route("/hello", get(hello_world));
Ok(router.into())
}
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using cargo-binstall, cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the rm
and new
project commands. This will not delete any databases, and you will keep your project name:
cargo shuttle project rm
cargo shuttle project new
Finally, redeploy your shuttle service:
cargo shuttle deploy
Contributions
- @kierendavies made their first contribution to shuttle by fixing a nefarious cyclical dependency bug in #700
- @blixus made some great improvements to our CLI docs in shuttle-hq/shuttle-docs#48
- @HexPanda made their first contribution to shuttle by fixing the indentation of the poise init code in #687
- @stavares843 made their first contribution to shuttle by fixing some typos in our repo in #682
Commits for this release
- feat: shuttle-serenity initial commit poc by @brokad in #429
- shuttle next wrapper POC by @oddgrd in #431
- refactor: create runtimes workspace by @chesedo in #432
- feat: add --provisioner-address arg to both runtimes by @chesedo in #433
- feat: create a control plane interface (part 1) by @chesedo in #436
- refactor: combine runtimes into one by @chesedo in #438
- feat(deployer): add support for building wasm projects by @akrantz01 in #437
- feat: move factory to runtime by @oddgrd in #444
- WIP feat: start runtime from deployer by @oddgrd in #450
- feat: get logs from runtime by @chesedo in #459
- [WI...
v0.11.2
shuttle: v0.11.2 update
We're excited to release shuttle v0.11.2! 🚀
Sleeping Idle Projects
Shuttle imposes no restrictions on how many projects a user can create. While we are in alpha this is a win-win, users creating many and varied projects is great for testing both our code and how we handle load. However, this has lead to a sizable amount of idle projects. To remedy this, we will start putting these idle projects to sleep, by default after 30 minutes of no activity.
By activity we mean both:
- CPU load of the project over a 30 minute window. The load is increased when a project receives traffic (for example, traffic to an API endpoint for an axum server, or calling the commands of a discord bot in a discord server). An HTTP server (eg. axum or rocket), will need about 2 requests per minute over the idle-time period to be considered active, a discord bot will need about 6 calls per minute.
- User interaction, for example running
cargo shuttle status
or any other CLI commands except forcargo shuttle project <cmd>
commands.
If the project is sleeping, traffic to the project or user interaction will wake it, and in our tests this usually takes around one second. Important: if a discord bot is sleeping due to inactivity, calling the bot commands will not wake it, only developer interaction through the CLI will (but traffic will keep it from going to sleep). See the below instructions for how to increase the idle limit.
Since we don’t really want to restrict active users yet, this limit can be freely modified. You can change it by passing in a flag when creating a project: cargo shuttle project new --idle-minutes 40
. If you pass in --idle-minutes 0
the idle timeout will be disabled and your project will never sleep. This is especially useful for discord bots, since they will not be awoken by traffic (people using the bot commands in a discord channel), only by developer interaction through the shuttle CLI.
Due to the nature of these changes, to be able to run cargo shuttle project new
, you will need to update your cargo-shuttle
CLI (so if you don’t need to run project new
you don’t have to update now).
To upgrade your shuttle CLI, run: cargo install cargo-shuttle
, or if you’re using cargo-binstall, cargo binstall cargo-shuttle
.
If you’d like to upgrade your project container, run the rm
and new
project commands. This will not delete any databases, and you will keep your project name:
cargo shuttle project rm
cargo shuttle project new
Finally, redeploy your shuttle service:
cargo shuttle deploy
Note: The versions of dependencies like shuttle-service
, shuttle-secrets
and the other resource libraries were not incremented for this release, they should stay at 0.11.0
.
New Contributors
- @jonaro00 made their first contribution to shuttle by doing some much needed housekeeping of our various markdown files and docs, as well as cleaning up some redundant dependencies in #666 , #664 and shuttle-hq/shuttle-docs#45
Commits for this release
- feat: add bug report and PR templates by @oddgrd in #661
- Remove time v0.1 dependency from main crates by @jonaro00 in #664
- Fix and update all markdown files + some more by @jonaro00 in #666
- Fix: ignore misc lockfiles by @oddgrd in #670
- chore: opt out of panamax by @brokad in #576
- feat: stop idle deployers by @chesedo in #627
- chore: v0.11.2 by @oddgrd in #672
Full Changelog: v0.11.1...v0.11.2
v0.11.0
shuttle: v0.11.0 update
We're excited to release shuttle v0.11.0! 🚀
Security Refactor
The core of this release is a major security overhaul, introducing the new auth
crate to the repository. You should not notice any difference in functionality, but this will ensure complete isolation between users projects, and guarantee that a resource can only be accessed by it’s owner.
This refactor also lays the foundation for the recently announced dashboard to be able to securely interface with the Shuttle API. This means that soon the CLI will not be the only way to manage your projects, though it will take some time for the dashboard to reach feature parity with the CLI.
Due to the nature of these changes all projects will have to manually update, as we will not be able to keep running older versions that don’t support the new authentication system. To upgrade your project, you need to take the following steps:
- To upgrade your shuttle CLI, run:
cargo install cargo-shuttle
, or if you’re using [cargo-binstall](https://github.com/cargo-bins/cargo-binstall),cargo binstall cargo-shuttle
. - Update your shuttle dependency(s) to
0.11.0
:
shuttle-service = { version = "0.11.0", features = ["web-axum"] }
shuttle-static-folder = "0.11.0"
# ...
- Optionally, test locally using:
cargo shuttle run
- Upgrade your remote project using the
rm
andnew
project commands. This will not delete any databases, and you will keep your project name:
cargo shuttle project rm
cargo shuttle project new
- Deploy your project again:
cargo shuttle deploy
New Contributors
- @MrCoolTheCucumber made their first contribution in #622, laying down a solid foundation for our upcoming resource deletion feature.
- @angelorendina made their first contribution in #637, fixing a bug with our upcoming custom domain feature.
- @sentinel1909 made their first contribution in #636, filling a hole in our contribution docs. They have also made several contributions to our docs repo, thanks!
All contributions for this release
- bug: revive by @chesedo in #631
- feat: create an auth project + clap by @chesedo in #630
- [auth] feat: axum with routes by @chesedo in #632
- [auth] feat: add an auth module to shuttle common by @chesedo in #635
- Feat: optimize sqlite db in deployer and gateway by @oddgrd in #623
- feat: add a users layer by @oddgrd in #633
- feat: session manager by @chesedo in #638
- feat: convert api key to jwt by @oddgrd in #640
- [auth] feat: public key endpoint by @chesedo in #639
- Fix for install docker buildx issue #29 by @sentinel1909 in #636
- feat: implement rpc method in provisioner that allows for the deletion of resources by @MrCoolTheCucumber in #622
- feat: Allow filtering projects by project status by @joshua-mo-143 in #613
- (#634) restore custom domain when recreating project by @angelorendina in #637
- docs: clean up the contributing doc by @oddgrd in #644
- [auth] refactor: update gateway deployer and provisioner by @chesedo in #642
- refactor: get project name from label by @oddgrd in #646
- Fix: remove target from bin build name by @oddgrd in #650
- feat: OpenTelemetry collector by @chesedo in #649
- feat: allow admin scoped user to recreate any project by @oddgrd in #651
- feat: auth cache by @oddgrd in #643
- Feat: unbox the ClaimService and Scoped futures by @oddgrd in #653
- Feat: cache public key by @oddgrd in #655
- refactor: update opentelemetry in all our crates by @chesedo in #652
- chore: update examples submodule by @oddgrd in #656
- chore: v0.11.0 by @oddgrd in #654
Full Changelog: v0.10.0...v0.11.0