Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename netty-server-loom to netty-server-sync #3704

Merged
merged 3 commits into from
Apr 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 16 additions & 13 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ lazy val rawAllAggregates = core.projectRefs ++
vertxServerZio.projectRefs ++
jdkhttpServer.projectRefs ++
nettyServer.projectRefs ++
nettyServerLoom.projectRefs ++
nettyServerSync.projectRefs ++
nettyServerCats.projectRefs ++
nettyServerZio.projectRefs ++
nimaServer.projectRefs ++
Expand Down Expand Up @@ -249,8 +249,14 @@ lazy val rawAllAggregates = core.projectRefs ++
derevo.projectRefs ++
awsCdk.projectRefs

def buildWithLoom(project: String): Boolean =
project.contains("Loom") || project.contains("nima") || project.contains("perfTests") || project.contains("examples3")
lazy val loomProjects: Seq[String] = Seq(nettyServerSync, nimaServer, examples).flatMap(_.projectRefs).flatMap(projectId)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perfTests no longer have to be built with loom, because I moved sync server runner out of it (because of Scala 3 because of Ox :)


def projectId(projectRef: ProjectReference): Option[String] =
projectRef match {
case ProjectRef(_, id) => Some(id)
case LocalProject(id) => Some(id)
case _ => None
}

lazy val allAggregates: Seq[ProjectReference] = {
val filteredByNative = if (sys.env.isDefinedAt("STTP_NATIVE")) {
Expand All @@ -262,15 +268,14 @@ lazy val allAggregates: Seq[ProjectReference] = {
}
if (sys.env.isDefinedAt("ONLY_LOOM")) {
println("[info] ONLY_LOOM defined, including only loom-based projects")
filteredByNative.filter(p => buildWithLoom(p.toString))
filteredByNative.filter(p => projectId(p).forall(loomProjects.contains))
} else if (sys.env.isDefinedAt("ALSO_LOOM")) {
println("[info] ALSO_LOOM defined, including also loom-based projects")
filteredByNative
} else {
println("[info] ONLY_LOOM *not* defined, *not* including loom-based-projects")
filteredByNative.filterNot(p => buildWithLoom(p.toString))
filteredByNative.filterNot(p => projectId(p).forall(loomProjects.contains))
}

}

// separating testing into different Scala versions so that it's not all done at once, as it causes memory problems on CI
Expand Down Expand Up @@ -1450,18 +1455,16 @@ lazy val nettyServer: ProjectMatrix = (projectMatrix in file("server/netty-serve
.jvmPlatform(scalaVersions = scala2And3Versions)
.dependsOn(serverCore, serverTests % Test)

lazy val nettyServerLoom: ProjectMatrix =
ProjectMatrix("nettyServerLoom", file("server/netty-server/loom"))
lazy val nettyServerSync: ProjectMatrix =
ProjectMatrix("nettyServerSync", file("server/netty-server/sync"))
.settings(commonJvmSettings)
.settings(
name := "tapir-netty-server-loom",
name := "tapir-netty-server-sync",
// needed because of https://github.com/coursier/coursier/issues/2016
useCoursier := false,
Test / run / fork := true,
libraryDependencies ++= Seq(
"com.softwaremill.ox" %% "core" % Versions.ox,
"org.reactivestreams" % "reactive-streams-tck" % Versions.reactiveStreams % Test,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 2 deps (reactive-streams-tck and weaver) were added during development of netty-loom websockets but are not needed after all.

"com.disneystreaming" %% "weaver-cats" % "0.8.4" % Test
"com.softwaremill.ox" %% "core" % Versions.ox
)
)
.jvmPlatform(scalaVersions = List(scala3))
Expand Down Expand Up @@ -2149,7 +2152,7 @@ lazy val examples: ProjectMatrix = (projectMatrix in file("examples"))
sttpClient,
swaggerUiBundle,
http4sServerZio,
nettyServerLoom,
nettyServerSync,
nettyServerZio,
zioHttpServer,
zioJson,
Expand Down
4 changes: 2 additions & 2 deletions doc/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ Let's use the name `enumeration` in Tapir codebase to represent these "true" enu

## JDK version

To ensure that Tapir can be used in a wide range of projects, the CI job uses JDK11 for most of the modules. There are exceptions (like `netty-server-loom` and `nima-server`) which require JDK version >= 21. This requirement is adressed by the build matrix in `.github/workflows/ci.yml`, which runs separate builds on a newer Java version, and sets a `ONLY_LOOM` env variable, used by build.sbt to recognise that it should limit the scope of an aggregated task to these projects only.
To ensure that Tapir can be used in a wide range of projects, the CI job uses JDK11 for most of the modules. There are exceptions (like `netty-server-sync` and `nima-server`) which require JDK version >= 21. This requirement is adressed by the build matrix in `.github/workflows/ci.yml`, which runs separate builds on a newer Java version, and sets a `ONLY_LOOM` env variable, used by build.sbt to recognise that it should limit the scope of an aggregated task to these projects only.
For local development, feel free to use any JDK >= 11. You can be on JDK 21, then with missing `ONLY_LOOM` variable you can still run sbt tasks on projects excluded from aggegate build, for example:
```scala
nimaServer/Test/test
nettyServerLoom/compile
nettyServerSync3/compile
// etc.
```

Expand Down
4 changes: 4 additions & 0 deletions doc/migrating.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Migrating

## From 10.0.4 to 10.0.5

- `tapir-server-netty-loom` has been renamed to `tapir-netty-server-sync`, and is availavble only for Scala 3. Use imports from `sttp.tapir.server.netty.sync`, and start your server using `NettySyncServer()`. See [examples/HelloWorldNettySyncServer.scala](https://github.com/softwaremill/tapir/blob/master/examples/src/main/scala/sttp/tapir/examples/HelloWorldNettySyncServer.scala) for a full example.

## From 1.9.3 to 1.9.4

- `NettyConfig.defaultNoStreaming` has been removed, use `NettyConfig.default`.
Expand Down
6 changes: 3 additions & 3 deletions doc/server/netty.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ To expose an endpoint using a [Netty](https://netty.io)-based server, first add
"com.softwaremill.sttp.tapir" %% "tapir-netty-server" % "@VERSION@"

// if you want to use Java 21 Loom virtual threads in direct style:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-loom" % "@VERSION@"
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-sync" % "@VERSION@"

// if you are using cats-effect:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-cats" % "@VERSION@"
Expand Down Expand Up @@ -44,7 +44,7 @@ val binding: Future[NettyFutureServerBinding] =
NettyFutureServer().addEndpoint(helloWorld).start()
```

The `tapir-netty-server-loom` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style. It is
The `tapir-netty-server-sync` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style. It is
available only for Scala 3.
See [examples/HelloWorldNettySyncServer.scala](https://github.com/softwaremill/tapir/blob/master/examples/src/main/scala/sttp/tapir/examples/HelloWorldNettySyncServer.scala) for a full example.
To learn more about handling concurrency with Ox, see the [documentation](https://ox.softwaremill.com/).
Expand Down Expand Up @@ -141,7 +141,7 @@ object WebSocketsNettyCatsServer extends ResourceApp.Forever {
}
```

### tapir-netty-server-loom
### tapir-netty-server-sync

In the Loom-based backend, Tapir uses [Ox](https://ox.softwaremill.com) to manage concurrency, and your transformation pipeline should be represented as `Ox ?=> Source[A] => Source[B]`. Any forks started within this function will be run under a safely isolated internal scope.
See [examples/websocket/WebSocketNettySyncServer.scala](https://github.com/softwaremill/tapir/blob/master/examples/src/main/scala/sttp/tapir/examples/websocket/WebSocketNettySyncServer.scala) for a full example.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package sttp.tapir.examples

import ox.*
import sttp.tapir.*
import sttp.tapir.server.netty.loom.{Id, NettySyncServer}
import sttp.tapir.server.netty.sync.{Id, NettySyncServer}

object HelloWorldNettySyncServer:
val helloWorld = endpoint.get
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ import ox.*
import ox.channels.*
import sttp.capabilities.WebSockets
import sttp.tapir.*
import sttp.tapir.server.netty.loom.Id
import sttp.tapir.server.netty.loom.OxStreams
import sttp.tapir.server.netty.loom.OxStreams.Pipe // alias for Ox ?=> Source[A] => Source[B]
import sttp.tapir.server.netty.loom.NettySyncServer
import sttp.tapir.server.netty.sync.Id
import sttp.tapir.server.netty.sync.OxStreams
import sttp.tapir.server.netty.sync.OxStreams.Pipe // alias for Ox ?=> Source[A] => Source[B]
import sttp.tapir.server.netty.sync.NettySyncServer
import sttp.ws.WebSocketFrame

import scala.concurrent.duration.*
Expand Down
2 changes: 1 addition & 1 deletion generated-doc/out/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Let's use the name `enumeration` in Tapir codebase to represent these "true" enu

## JDK version

To ensure that Tapir can be used in a wide range of projects, the CI job uses JDK11 for most of the modules. There are exceptions (like `netty-server-loom` and `nima-server`) which require JDK version >= 21. This requirement is adressed by the build matrix in `.github/workflows/ci.yml`, which runs separate builds on a newer Java version, and sets a `ONLY_LOOM` env variable, used by build.sbt to recognise that it should limit the scope of an aggregated task to these projects only.
To ensure that Tapir can be used in a wide range of projects, the CI job uses JDK11 for most of the modules. There are exceptions (like `netty-server-sync` and `nima-server`) which require JDK version >= 21. This requirement is adressed by the build matrix in `.github/workflows/ci.yml`, which runs separate builds on a newer Java version, and sets a `ONLY_LOOM` env variable, used by build.sbt to recognise that it should limit the scope of an aggregated task to these projects only.
For local development, feel free to use any JDK >= 11. You can be on JDK 21, then with missing `ONLY_LOOM` variable you can still run sbt tasks on projects excluded from aggegate build, for example:
```scala
nimaServer/Test/test
Expand Down
4 changes: 2 additions & 2 deletions generated-doc/out/server/netty.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ To expose an endpoint using a [Netty](https://netty.io)-based server, first add
"com.softwaremill.sttp.tapir" %% "tapir-netty-server" % "1.10.4"

// if you want to use Java 21 Loom virtual threads in direct style:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-loom" % "1.10.4"
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-sync" % "1.10.4"

// if you are using cats-effect:
"com.softwaremill.sttp.tapir" %% "tapir-netty-server-cats" % "1.10.4"
Expand Down Expand Up @@ -44,7 +44,7 @@ val binding: Future[NettyFutureServerBinding] =
NettyFutureServer().addEndpoint(helloWorld).start()
```

The `tapir-netty-server-loom` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style.
The `tapir-netty-server-sync` server uses `Id[T]` as its wrapper effect for compatibility, while `Id[A]` means in fact just `A`, representing direct style.

```scala
import sttp.tapir._
Expand Down
4 changes: 2 additions & 2 deletions perf-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,9 @@ For WebSockets we want to measure latency distribution, not throughput, so use g
```
perfTests/runMain sttp.tapir.perf.apis.ServerRunner http4s.Tapir
```
If you're testing `NettySyncServer` (tapir-server-netty-loom), its server runner is located elsewhere:
If you're testing `NettySyncServer` (tapir-server-netty-sync), its server runner is located elsewhere:
```
nettyServerLoom3/Test/runMain sttp.tapir.netty.loom.perf.NettySyncServerRunner
nettyServerSync3/Test/runMain sttp.tapir.netty.sync.perf.NettySyncServerRunner
```
This is caused by `perf-tests` using Scala 2.13 forced by Gatling, while `NettySyncServer` is written excluisively for Scala 3.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import ox.Ox
import ox.channels.Source
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import ox.*
import internal.ox.OxDispatcher
Expand All @@ -25,7 +25,7 @@ import scala.util.control.NonFatal
* [[NettySyncServerEndpointListOverriddenOptions]] is an intermediary helper type representing added endpoints, which have custom server
* options.
*/
private[loom] case class NettySyncServerEndpointListOverridenOptions(
private[sync] case class NettySyncServerEndpointListOverridenOptions(
ses: List[ServerEndpoint[OxStreams & WebSockets, Id]],
overridenOptions: NettySyncServerOptions
)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import internal.{NettySyncRequestBody, NettySyncToResponseBody}
import internal.ox.OxDispatcher
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import org.slf4j.LoggerFactory
import sttp.tapir.model.ServerRequest
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom.internal
package sttp.tapir.server.netty.sync.internal

import io.netty.handler.codec.http.HttpContent
import org.playframework.netty.http.StreamedHttpRequest
Expand All @@ -9,9 +9,9 @@ import sttp.tapir.TapirFile
import sttp.tapir.model.ServerRequest
import sttp.tapir.server.netty.internal.NettyRequestBody
import sttp.tapir.server.netty.internal.reactivestreams.{FileWriterSubscriber, SimpleSubscriber}
import sttp.tapir.server.netty.loom.*
import sttp.tapir.server.netty.sync.*

private[loom] class NettySyncRequestBody(val createFile: ServerRequest => TapirFile) extends NettyRequestBody[Id, OxStreams]:
private[sync] class NettySyncRequestBody(val createFile: ServerRequest => TapirFile) extends NettyRequestBody[Id, OxStreams]:

override given monad: MonadError[Id] = idMonad
override val streams: capabilities.Streams[OxStreams] = OxStreams
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom.internal
package sttp.tapir.server.netty.sync.internal

import _root_.ox.*
import io.netty.channel.ChannelHandlerContext
Expand All @@ -9,13 +9,13 @@ import sttp.tapir.server.interpreter.ToResponseBody
import sttp.tapir.server.netty.NettyResponse
import sttp.tapir.server.netty.NettyResponseContent.ReactiveWebSocketProcessorNettyResponseContent
import sttp.tapir.server.netty.internal.{NettyToResponseBody, RunAsync}
import sttp.tapir.server.netty.loom._
import sttp.tapir.server.netty.loom.internal.ox.OxDispatcher
import sttp.tapir.server.netty.sync._
import sttp.tapir.server.netty.sync.internal.ox.OxDispatcher
import sttp.tapir.*

import java.nio.charset.Charset

private[loom] class NettySyncToResponseBody(runAsync: RunAsync[Id], oxDispatcher: OxDispatcher)(using me: MonadError[Id])
private[sync] class NettySyncToResponseBody(runAsync: RunAsync[Id], oxDispatcher: OxDispatcher)(using me: MonadError[Id])
extends ToResponseBody[NettyResponse, OxStreams]:

val delegate = new NettyToResponseBody(runAsync)(me)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom.internal.ox
package sttp.tapir.server.netty.sync.internal.ox

import ox.*
import ox.channels.Actor
Expand All @@ -18,7 +18,7 @@ import ox.channels.Actor
* @param ox
* concurrency scope where a fork will be run, using a nested scope to isolate failures.
*/
private[loom] class OxDispatcher()(using ox: Ox):
private[sync] class OxDispatcher()(using ox: Ox):
private class Runner:
def runAsync(thunk: Ox ?=> Unit, onError: Throwable => Unit): Unit =
fork {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
package sttp.tapir.server.netty.loom.internal.reactivestreams
package sttp.tapir.server.netty.sync.internal.reactivestreams

import org.reactivestreams.{Subscriber, Subscription}
import ox.*
import ox.channels.*

/** Can be used together with an [[OxProcessor]] to read from a Source when there's demand. */
private[loom] class ChannelSubscription[A](
private[sync] class ChannelSubscription[A](
subscriber: Subscriber[? >: A],
source: Source[A]
) extends Subscription:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
package sttp.tapir.server.netty.loom.internal.reactivestreams
package sttp.tapir.server.netty.sync.internal.reactivestreams

import org.reactivestreams.Subscriber
import ox.*
import ox.channels.*
import org.reactivestreams.Subscription
import org.reactivestreams.Processor
import sttp.tapir.server.netty.loom.internal.ox.OxDispatcher
import sttp.tapir.server.netty.loom.OxStreams
import sttp.tapir.server.netty.sync.internal.ox.OxDispatcher
import sttp.tapir.server.netty.sync.OxStreams

/** A reactive Processor, which is both a Publisher and a Subscriber
*
Expand All @@ -18,7 +18,7 @@ import sttp.tapir.server.netty.loom.OxStreams
* an optional function allowing wrapping external subscribers, can be used to intercept onNext, onComplete and onError with custom
* handling. Can be just identity.
*/
private[loom] class OxProcessor[A, B](
private[sync] class OxProcessor[A, B](
oxDispatcher: OxDispatcher,
pipeline: OxStreams.Pipe[A, B],
wrapSubscriber: Subscriber[? >: B] => Subscriber[? >: B]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom.internal.ws
package sttp.tapir.server.netty.sync.internal.ws

import io.netty.channel.ChannelHandlerContext
import io.netty.handler.codec.http.websocketx.{CloseWebSocketFrame, WebSocketCloseStatus, WebSocketFrame => NettyWebSocketFrame}
Expand All @@ -8,14 +8,14 @@ import ox.*
import ox.channels.{ChannelClosedException, Source}
import sttp.tapir.model.WebSocketFrameDecodeFailure
import sttp.tapir.server.netty.internal.ws.WebSocketFrameConverters._
import sttp.tapir.server.netty.loom.OxStreams
import sttp.tapir.server.netty.loom.internal.ox.OxDispatcher
import sttp.tapir.server.netty.loom.internal.reactivestreams.OxProcessor
import sttp.tapir.server.netty.sync.OxStreams
import sttp.tapir.server.netty.sync.internal.ox.OxDispatcher
import sttp.tapir.server.netty.sync.internal.reactivestreams.OxProcessor
import sttp.tapir.{DecodeResult, WebSocketBodyOutput}
import sttp.ws.WebSocketFrame
import java.io.IOException

private[loom] object OxSourceWebSocketProcessor:
private[sync] object OxSourceWebSocketProcessor:

def apply[REQ, RESP](
oxDispatcher: OxDispatcher,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ package sttp.tapir.server.netty

import sttp.monad.MonadError

package object loom:
package object sync:
type Id[X] = X
type IdRoute = Route[Id]

private[loom] implicit val idMonad: MonadError[Id] = new MonadError[Id] {
private[sync] implicit val idMonad: MonadError[Id] = new MonadError[Id] {
override def unit[T](t: T): Id[T] = t
override def map[T, T2](fa: Id[T])(f: T => T2): Id[T2] = f(fa)
override def flatMap[T, T2](fa: Id[T])(f: T => Id[T2]): Id[T2] = f(fa)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import cats.data.NonEmptyList
import cats.effect.unsafe.implicits.global
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package sttp.tapir.server.netty.loom
package sttp.tapir.server.netty.sync

import cats.data.NonEmptyList
import cats.effect.{IO, Resource}
Expand Down Expand Up @@ -72,6 +72,5 @@ class NettySyncTestServerInterpreter(eventLoopGroup: NioEventLoopGroup)
NettyConfig.default.eventLoopGroup(eventLoopGroup).randomPort.withDontShutdownEventLoopGroupOnClose.noGracefulShutdown
val customizedConfig = gracefulShutdownTimeout.map(config.withGracefulShutdownTimeout).getOrElse(config)
val options = NettySyncServerOptions.default
val interpreter = NettySyncServerInterpreter(options)
useInScope(NettySyncServer(options, customizedConfig).addEndpoints(endpoints.toList).start())(_.stop())
}
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
package sttp.tapir.server.netty.loom.perf
package sttp.tapir.server.netty.sync.perf

import ox.*
import ox.channels.*
import sttp.tapir.server.netty.loom.NettySyncServerOptions
import sttp.tapir.server.netty.loom.NettySyncServerBinding
import sttp.tapir.server.netty.loom.NettySyncServer
import sttp.tapir.server.netty.sync.NettySyncServerOptions
import sttp.tapir.server.netty.sync.NettySyncServerBinding
import sttp.tapir.server.netty.sync.NettySyncServer

import sttp.tapir.*
import sttp.tapir.server.netty.loom.Id
import sttp.tapir.server.netty.sync.Id
import sttp.tapir.server.ServerEndpoint
import sttp.tapir.server.model.EndpointExtensions.*
import sttp.tapir.server.netty.loom.OxStreams
import sttp.tapir.server.netty.sync.OxStreams
import sttp.tapir.Endpoint
import sttp.capabilities.WebSockets
import scala.concurrent.duration._
Expand Down
Loading