Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate servers from tapir-loom #3304

Merged
merged 17 commits into from
Nov 8, 2023
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 30 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,31 +18,41 @@ jobs:
matrix:
scala-version: [ "2.12", "2.13", "3" ]
target-platform: [ "JVM", "JS", "Native" ]
java: [ "11", "21" ]
exclude:
- scala-version: "2.12"
target-platform: "Native"
- scala-version: "2.12"
java: "21"
- scala-version: "2.13"
target-platform: "Native"
- java: "21"
target-platform: "Native"
- java: "21"
target-platform: "JS"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so ... we are trying to only include specific axes? maybe we can either (1) use include instead of exlucde, if possible; (2) if not, at least add a comment, what are the desired combinations that should be included

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the hint, somehow I wasn't aware that there's include. Turns out it can be combined with exclude, because exclusions are resolved first, so our rules will become:

        exclude:
          - java: "21"
          - scala-version: "2.12"
            target-platform: "Native"
          - scala-version: "2.13"
            target-platform: "Native"
        include:
          - java: "21"
            scala-version: "2.13"
            target-platform: "JVM"
          - java: "21"
            scala-version: "3"
            target-platform: "JVM"

Looks clearer to me.

steps:
- name: Checkout
uses: actions/checkout@v3
- name: Set up JDK 11
- name: Set up JDK
uses: actions/setup-java@v3
with:
distribution: 'temurin'
cache: 'sbt'
java-version: 11
java-version: ${{ matrix.java }}
- name: Install sam cli
if: matrix.java == '11'
run: |
wget -q https://github.com/aws/aws-sam-cli/releases/latest/download/aws-sam-cli-linux-x86_64.zip
unzip -q aws-sam-cli-linux-x86_64.zip -d sam-installation
sudo ./sam-installation/install --update
sam --version
- name: Install NPM
if: matrix.java == '11'
run: |
sudo apt install npm
npm --version
- name: Install AWS CDK
if: matrix.java == '11'
run: |
npm install -g aws-cdk
cdk --version
Expand All @@ -52,10 +62,13 @@ jobs:
sudo apt-get update
sudo apt-get install libidn2-dev libcurl3-dev
echo "STTP_NATIVE=1" >> $GITHUB_ENV
- name: Enable Loom-specific modules
if: matrix.java == '21'
run: echo "JDK_LOOM=1" >> $GITHUB_ENV
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This env variable makes the project build only jdk21 servers. Checking java version in sbt instead of reading it might cause confusion in local development. For example, I use 21 as my default working JDK, but I still want to build the "JDK11 set". More developers may have such a case. Maybe we should rename the variable to something meaning more "build loom servers only"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense :) add this info to readme in Contributing?

- name: Compile
run: sbt $SBT_JAVA_OPTS -v "compileScoped ${{ matrix.scala-version }} ${{ matrix.target-platform }}"
- name: Compile documentation
if: matrix.target-platform == 'JVM'
if: matrix.target-platform == 'JVM' && matrix.java == '11'
run: sbt $SBT_JAVA_OPTS -v compileDocumentation
- name: Test
if: matrix.target-platform != 'JS'
Expand Down Expand Up @@ -109,21 +122,29 @@ jobs:
needs: [ci]
if: github.event_name != 'pull_request' && (startsWith(github.ref, 'refs/tags/v'))
runs-on: ubuntu-22.04
env:
STTP_NATIVE: 1
strategy:
matrix:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hopefully this will work and the two publishes will publish separate jars - no other way to find out than try ;)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, unfortunately there's no nice way to do a dry-run. We could probably make publishArtifacts dependent on some variables, but I'm not sure if it's worth adding complexity.

java: [ "11", "21" ]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Set up JDK 11
- name: Set up JDK
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: 11
java-version: ${{ matrix.java }}
cache: 'sbt'
- name: Install libidn2-dev libcurl3-dev
if: matrix.java == '11'
run: |
sudo apt-get update
sudo apt-get install libidn2-dev libcurl3-dev
- name: Enable Native-specific modules
if: matrix.java == '11'
run: echo "STTP_NATIVE=1" >> $GITHUB_ENV
- name: Enable Loom-specific modules
if: matrix.java == '21'
run: echo "JDK_LOOM=1" >> $GITHUB_ENV
- name: Compile
run: sbt $SBT_JAVA_OPTS compile
- name: Publish artifacts
Expand All @@ -134,12 +155,14 @@ jobs:
SONATYPE_USERNAME: ${{ secrets.SONATYPE_USERNAME }}
SONATYPE_PASSWORD: ${{ secrets.SONATYPE_PASSWORD }}
- name: Extract version from commit message
if: matrix.java == '11'
run: |
version=${GITHUB_REF/refs\/tags\/v/}
echo "VERSION=$version" >> $GITHUB_ENV
env:
COMMIT_MSG: ${{ github.event.head_commit.message }}
- name: Publish release notes
if: matrix.java == '11'
uses: release-drafter/release-drafter@v5
with:
config-name: release-drafter.yml
Expand Down
35 changes: 34 additions & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -221,8 +221,10 @@ lazy val rawAllAggregates = core.projectRefs ++
vertxServerZio1.projectRefs ++
jdkhttpServer.projectRefs ++
nettyServer.projectRefs ++
nettyServerLoom.projectRefs ++
nettyServerCats.projectRefs ++
nettyServerZio.projectRefs ++
nimaServer.projectRefs ++
zio1HttpServer.projectRefs ++
zioHttpServer.projectRefs ++
awsLambdaCore.projectRefs ++
Expand Down Expand Up @@ -251,13 +253,21 @@ lazy val rawAllAggregates = core.projectRefs ++
awsCdk.projectRefs

lazy val allAggregates: Seq[ProjectReference] = {
if (sys.env.isDefinedAt("STTP_NATIVE")) {
val filteredByNative = if (sys.env.isDefinedAt("STTP_NATIVE")) {
println("[info] STTP_NATIVE defined, including native in the aggregate projects")
rawAllAggregates
} else {
println("[info] STTP_NATIVE *not* defined, *not* including native in the aggregate projects")
rawAllAggregates.filterNot(_.toString.contains("Native"))
}
if (sys.env.isDefinedAt("JDK_LOOM")) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm maybe this should be called ONLY_LOOM or sth like that? otherwise it sounds additive

println("[info] JDK_LOOM defined, including only loom-based projects")
filteredByNative.filter(p => (p.toString.contains("Loom") || p.toString.contains("Nima")))
} else {
println("[info] JDK_LOOM *not* defined, *not* including loom-based-projects")
filteredByNative.filterNot(p => (p.toString.contains("Loom") || p.toString.contains("Nima")))
}

}

// separating testing into different Scala versions so that it's not all done at once, as it causes memory problems on CI
Expand Down Expand Up @@ -1443,6 +1453,17 @@ lazy val nettyServer: ProjectMatrix = (projectMatrix in file("server/netty-serve
.jvmPlatform(scalaVersions = scala2And3Versions)
.dependsOn(serverCore, serverTests % Test)

lazy val nettyServerLoom: ProjectMatrix =
ProjectMatrix("nettyServerLoom", file(s"server/netty-server/loom"))
.settings(commonJvmSettings)
.settings(
name := "tapir-netty-server-loom",
// needed because of https://github.com/coursier/coursier/issues/2016
useCoursier := false
)
.jvmPlatform(scalaVersions = scala2_13And3Versions)
.dependsOn(nettyServer, serverTests % Test)

lazy val nettyServerCats: ProjectMatrix = nettyServerProject("cats", catsEffect)
.settings(
libraryDependencies ++= Seq(
Expand Down Expand Up @@ -1471,6 +1492,18 @@ def nettyServerProject(proj: String, dependency: ProjectMatrix): ProjectMatrix =
.jvmPlatform(scalaVersions = scala2And3Versions)
.dependsOn(nettyServer, dependency, serverTests % Test)

lazy val nimaServer: ProjectMatrix = (projectMatrix in file("server/nima-server"))
.settings(commonJvmSettings)
.settings(
name := "tapir-nima-server",
libraryDependencies ++= Seq(
"io.helidon.webserver" % "helidon-webserver" % Versions.helidon,
"io.helidon.logging" % "helidon-logging-slf4j" % Versions.helidon
) ++ loggerDependencies
)
.jvmPlatform(scalaVersions = scala2_13And3Versions)
.dependsOn(serverCore, serverTests % Test)

lazy val vertxServer: ProjectMatrix = (projectMatrix in file("server/vertx-server"))
.settings(commonJvmSettings)
.settings(
Expand Down
2 changes: 2 additions & 0 deletions doc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ input and output parameters. An endpoint specification can be interpreted as:
* [Akka HTTP](server/akkahttp.md) `Route`s/`Directive`s
* [Http4s](server/http4s.md) `HttpRoutes[F]` (using cats-effect or [ZIO](server/zio-http4s.md))
* [Netty](server/netty.md) (using `Future`s, cats-effect or ZIO)
* [Helidon Níma](server/nima.md) (using JVM 21 Virtual Threads and direct style)
* [Finatra](server/finatra.md) `http.Controller`
* [Pekko HTTP](server/pekkohttp.md) `Route`s/`Directive`s
* [Play](server/play.md) `Route`
Expand Down Expand Up @@ -239,6 +240,7 @@ We offer commercial support for sttp and related technologies, as well as develo
server/http4s
server/zio-http4s
server/netty
server/nima
server/finatra
server/pekkohttp
server/play
Expand Down
6 changes: 1 addition & 5 deletions doc/other_interpreters.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@
At its core, tapir creates a data structure describing the HTTP endpoints. This data structure can be freely
interpreted also by code not included in the library. Below is a list of projects, which provide tapir interpreters.

## tapir-loom

The [tapir-loom](https://github.com/softwaremill/tapir-loom) project provides server interpreters which allow writing the server logic in the "direct" style (synchronous, using the `Id` effect). Depends on Java 19, which includes a preview of Project Loom (Virtual Threads for the JVM).

## GraphQL

[Caliban](https://github.com/ghostdogpr/caliban) allows you to easily turn your Tapir endpoints into a GraphQL API. More details in the [documentation](https://ghostdogpr.github.io/caliban/docs/interop.html#tapir).
Expand All @@ -25,4 +21,4 @@ layer that allows you to create traced http endpoints from tapir Endpoint defini

## tapir-http-session

[tapir-http-session](https://github.com/SOFTNETWORK-APP/tapir-http-session) provides integration with functionality of [akka-http-session](https://github.com/softwaremill/akka-http-session), which includes client-side session management in web and mobile applications.
[tapir-http-session](https://github.com/SOFTNETWORK-APP/tapir-http-session) provides integration with functionality of [akka-http-session](https://github.com/softwaremill/akka-http-session), which includes client-side session management in web and mobile applications.
22 changes: 21 additions & 1 deletion doc/server/netty.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@
To expose an endpoint using a [Netty](https://netty.io)-based server, first add the following dependency:

```scala
// if you are using Future or just exploring
// if you are using Future or just exploring:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server" % "@VERSION@"

// if you want to use Java 21 Loom virtual threads in direct style:
"com.softwaremill.sttp.tapir" %% "tapir-netty-loom" % "@VERSION@"

// if you are using cats-effect:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-cats" % "@VERSION@"

Expand All @@ -16,6 +19,7 @@ To expose an endpoint using a [Netty](https://netty.io)-based server, first add
Then, use:

- `NettyFutureServer().addEndpoints` to expose `Future`-based server endpoints.
- `NettyIdServer().addEndpoints` to expose `Loom`-based server endpoints.
- `NettyCatsServer().addEndpoints` to expose `F`-based server endpoints, where `F` is any cats-effect supported effect. [Streaming](../endpoint/streaming.md) request and response bodies is supported with fs2.
- `NettyZioServer().addEndpoints` to expose `ZIO`-based server endpoints, where `R` represents ZIO requirements supported effect. Streaming is supported with ZIO Streams.

Expand All @@ -40,6 +44,22 @@ val binding: Future[NettyFutureServerBinding] =
NettyFutureServer().addEndpoint(helloWorld).start()
```

The `tapir-netty-loom` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style.

```scala
import sttp.tapir._
import sttp.tapir.server.netty.loom.{Id, NettyIdServer, NettyIdServerBinding}

val helloWorld = endpoint
.get
.in("hello").in(query[String]("name"))
.out(stringBody)
.serverLogicSuccess[Id](name => s"Hello, $name!")

val binding: NettyIdServerBinding =
NettyIdServer().addEndpoint(helloWorld).start()
```

## Configuration

The interpreter can be configured by providing an `NettyFutureServerOptions` value, see [server options](options.md) for
Expand Down
44 changes: 44 additions & 0 deletions doc/server/nima.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Running as a Helidon Níma server

```eval_rst
.. note::
Helidon Níma requires JDK supporting Project Loom threading (JDK21 or newer).
```

To expose an endpoint as a [Helidon Níma](https://helidon.io/nima) server, first add the following
dependency:

```scala
"com.softwaremill.sttp.tapir" %% "tapir-nima-server" % "@VERSION@"
```

Loom-managed concurrency uses direct style instead of effect wrappers like `Future[T]` or `IO[T]`. Because of this,
Tapir endpoints defined for Nima server use `Id[T]`, which provides compatibility, while effectively means just `T`.

Such endpoints are then processed through `NimaServerInterpreter` in order to obtain an `io.helidon.webserver.http.Handler`:

```scala
import io.helidon.webserver.WebServer
import sttp.tapir._
import sttp.tapir.server.nima.{Id, NimaServerInterpreter}

val helloEndpoint = endpoint.get
.in("hello")
.out(stringBody)
.serverLogicSuccess[Id] { _ =>
Thread.sleep(1000)
"hello, world!"
}

val handler = NimaServerInterpreter().toHandler(List(helloEndpoint))

WebServer
.builder()
.routing { builder =>
builder.any(handler)
()
}
.port(8080)
.build()
.start()
```
3 changes: 2 additions & 1 deletion doc/stability.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ The modules are categorised using the following levels:
| armeria | stabilising |
| finatra | stabilising |
| http4s | stabilising |
| netty | experimental |
| netty | stabilising |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

| nima | experimental |
| pekko-http| stabilising |
| play | stabilising |
| vertx | stabilising |
Expand Down
1 change: 1 addition & 0 deletions project/Versions.scala
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ object Versions {
val circe = "0.14.6"
val circeGenericExtras = "0.14.3"
val circeYaml = "1.15.0"
val helidon = "4.0.0"
val sttp = "3.9.0"
val sttpModel = "1.7.6"
val sttpShared = "1.3.16"
Expand Down
Loading
Loading