Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use official ZIO-lambda library in product-move-api #2115

Merged
merged 7 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -568,6 +568,7 @@ lazy val `product-move-api` = lambdaProject(
"com.softwaremill.sttp.tapir" %% "tapir-core" % tapirVersion,
"com.softwaremill.sttp.tapir" %% "tapir-json-zio" % tapirVersion,
"com.softwaremill.sttp.tapir" %% "tapir-aws-lambda" % tapirVersion,
"com.softwaremill.sttp.tapir" %% "tapir-aws-lambda-zio" % tapirVersion,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the new library added in softwaremill/tapir#2975

"com.softwaremill.sttp.tapir" %% "tapir-swagger-ui-bundle" % tapirVersion,
awsSecretsManager,
upickle,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ object AwsCredentialsLive {

private val ProfileName = "membership"

val layer: Layer[String, AwsCredentialsProvider] =
ZLayer.scoped(ZIO.fromAutoCloseable(ZIO.attempt(impl))).mapError(_.toString)
val layer: Layer[Throwable, AwsCredentialsProvider] =
ZLayer.scoped(ZIO.fromAutoCloseable(ZIO.attempt(impl)))
Comment on lines +16 to +17
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the official zio interpreter expects Throwable as the error, rather than Any, so I had to make everything comply. That is the majority of the changes.


private def impl: AwsCredentialsProviderChain =
AwsCredentialsProviderChain
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ object DynamoLive {
case Stage.PROD => PROD
}

val layer: ZLayer[Stage, ErrorResponse, Dynamo] =
val layer: RLayer[Stage, Dynamo] =
ZLayer.scoped {
ZIO.service[Stage].map { stage =>
val dynamoService = SupporterDataDynamoService(getStage(stage))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ object GuStageLive {
enum Stage:
case PROD, CODE

val layer: Layer[String, Stage] =
val layer: Layer[Throwable, Stage] =
ZLayer {
for {
stageString <- System.envOrElse("Stage", "CODE")
stage <- ZIO.attempt(Stage.valueOf(stageString))
} yield stage
}.mapError(_.toString)
}

}
Original file line number Diff line number Diff line change
@@ -1,78 +1,149 @@
package com.gu.productmove

import com.amazonaws.services.lambda.runtime.*
import com.amazonaws.services.lambda.runtime.events.{APIGatewayV2HTTPEvent, APIGatewayV2HTTPResponse}
import com.gu.productmove.GuStageLive.Stage
import com.gu.productmove.endpoint.available.AvailableProductMovesEndpoint
import com.gu.productmove.endpoint.cancel.SubscriptionCancelEndpoint
import com.gu.productmove.endpoint.move.{ProductMoveEndpoint, ProductMoveEndpointTypes}
import com.gu.productmove.endpoint.updateamount.UpdateSupporterPlusAmountEndpoint
import com.gu.productmove.framework.ZIOApiGatewayRequestHandler
import com.gu.productmove.framework.ZIOApiGatewayRequestHandler.TIO
import com.gu.productmove.zuora.rest.{ZuoraClient, ZuoraClientLive, ZuoraGet, ZuoraGetLive}
import com.gu.productmove.zuora.{GetSubscription, GetSubscriptionLive}
import software.amazon.awssdk.auth.credentials.*
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.s3.S3Client
import software.amazon.awssdk.services.s3.model.{GetObjectRequest, S3Exception}
import software.amazon.awssdk.utils.SdkAutoCloseable
import io.circe.generic.semiauto.*
import io.circe.syntax.*
import sttp.apispec.openapi.Info
import sttp.capabilities.WebSockets
import sttp.capabilities.zio.ZioStreams
import sttp.client3.*
import sttp.client3.httpclient.zio.HttpClientZioBackend
import sttp.client3.logging.{Logger, LoggingBackend}
import sttp.model.*
import sttp.tapir.*
import sttp.tapir.docs.openapi.OpenAPIDocsInterpreter
import sttp.tapir.json.zio.*
import sttp.tapir.server.ServerEndpoint
import sttp.tapir.serverless.aws.lambda.*
import sttp.tapir.serverless.aws.lambda.{AwsHttp, AwsRequest, AwsRequestContext}
import zio.*
import zio.ZIO.attemptBlocking
import zio.json.*

import scala.concurrent.Future
import scala.jdk.CollectionConverters.*
import java.io.{ByteArrayInputStream, ByteArrayOutputStream}
import java.nio.charset.StandardCharsets
import scala.util.Try

// this handler contains all the endpoints
object Handler extends ZIOApiGatewayRequestHandler {
object Handler
extends ZIOApiGatewayRequestHandler(
List(
AvailableProductMovesEndpoint.server,
ProductMoveEndpoint.server,
UpdateSupporterPlusAmountEndpoint.server,
SubscriptionCancelEndpoint.server,
),
)
Comment on lines +25 to +33
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed all the test stuff from the handler and made the endpoints a constructor parameter.

This was because with inheritance and vals I was getting issues with initialisation order, and server was coming out as null.
Rather than just going down the lazy val option (which worked) I decided to separate/simplify appropriately.


object HandlerManualTests {

private val devSubscriptionNumber = "A-S00737111"

@main
// run this to test locally via console with some hard coded data
def testProductMove(): Unit = super.runTest(
def testProductMove(): Unit = runTest(
"POST",
"/product-move/A-S123",
Some(ProductMoveEndpointTypes.ExpectedInput(49.99, false, None, None).toJson),
"/product-move/recurring-contribution-to-supporter-plus/" + devSubscriptionNumber,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was not matching the actual routes definiton.

Some(ProductMoveEndpointTypes.ExpectedInput(49.99, preview = false, None, None).toJson),
)

@main
// run this to test locally via console with some hard coded data
def testAvailableMoves(): Unit = super.runTest(
def testAvailableMoves(): Unit = runTest(
"GET",
"/available-product-moves/A-S123",
"/available-product-moves/" + devSubscriptionNumber,
None,
)

@main
// this will output the yaml to the console
def testDocs(): Unit = {
Handler.runTest(
runTest(
"GET",
"/docs/docs.yaml",
None,
)
}

// this represents all the routes for the server
override val server: List[ServerEndpoint[Any, TIO]] = List(
AvailableProductMovesEndpoint.server,
ProductMoveEndpoint.server,
UpdateSupporterPlusAmountEndpoint.server,
SubscriptionCancelEndpoint.server,
)
@main
// this will output the HTML to the console
def testDocsHtml(): Unit = {
runTest(
"GET",
"/docs/",
None,
)
}

@main
def testRealDocsRequest(): Unit = {
val redactedJson = """{"resource":"/docs","path":"/docs/","httpMethod":"GET","headers":{"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en-US;q=0.9,en;q=0.8","cache-control":"max-age=0","Host":"product-move-api-code.support.guardianapis.com","if-modified-since":"Fri, 01 Jan 2010 00:00:00 GMT","if-none-match":"\"blahblah\"","sec-ch-ua":"\"Google Chrome\";v=\"119\", \"Chromium\";v=\"119\", \"Not?A_Brand\";v=\"24\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"macOS\"","sec-fetch-dest":"document","sec-fetch-mode":"navigate","sec-fetch-site":"none","sec-fetch-user":"?1","upgrade-insecure-requests":"1","User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36","X-Amzn-Trace-Id":"blahblah","X-Forwarded-For":"1.2.3.4","X-Forwarded-Port":"443","X-Forwarded-Proto":"https"},"multiValueHeaders":{"accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"accept-encoding":["gzip, deflate, br"],"accept-language":["en-GB,en-US;q=0.9,en;q=0.8"],"cache-control":["max-age=0"],"Host":["product-move-api-code.support.guardianapis.com"],"if-modified-since":["Fri, 01 Jan 2010 00:00:00 GMT"],"if-none-match":["\"blahblah\""],"sec-ch-ua":["\"Google Chrome\";v=\"119\", \"Chromium\";v=\"119\", \"Not?A_Brand\";v=\"24\""],"sec-ch-ua-mobile":["?0"],"sec-ch-ua-platform":["\"macOS\""],"sec-fetch-dest":["document"],"sec-fetch-mode":["navigate"],"sec-fetch-site":["none"],"sec-fetch-user":["?1"],"upgrade-insecure-requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"],"X-Amzn-Trace-Id":["blahblah"],"X-Forwarded-For":["1.2.3.4"],"X-Forwarded-Port":["443"],"X-Forwarded-Proto":["https"]},"queryStringParameters":null,"multiValueQueryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"resourceId":"blahblah","resourcePath":"/docs","httpMethod":"GET","extendedRequestId":"blahblah","requestTime":"28/Nov/2023:11:48:26 +0000","path":"/docs/","accountId":"1234","protocol":"HTTP/1.1","stage":"CODE","domainPrefix":"product-move-api-code","requestTimeEpoch":1701172106179,"requestId":"blahblah","identity":{"cognitoIdentityPoolId":null,"accountId":null,"cognitoIdentityId":null,"caller":null,"sourceIp":"1.2.3.4","principalOrgId":null,"accessKey":null,"cognitoAuthenticationType":null,"cognitoAuthenticationProvider":null,"userArn":null,"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36","user":null},"domainName":"product-move-api-code.support.guardianapis.com","apiId":"blahblah"},"body":null,"isBase64Encoded":false}"""
runStringTest(redactedJson)
}

// for testing
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this functions moved out of the superclass ZIOApiGatewayRequestHandler

def runTest(method: String, path: String, testInput: Option[String]): Unit = {
val inputValue = makeTestRequest(method, path, testInput)
val inputJson = inputValue.asJson(deriveEncoder).spaces2
runStringTest(inputJson)
}

def runStringTest(inputJson: String): Unit = {
val input = new ByteArrayInputStream(inputJson.getBytes(StandardCharsets.UTF_8))
val context = new TestContext()
val output: ByteArrayOutputStream = new ByteArrayOutputStream()
Handler.handleRequest(input, output, context)
val response = new String(output.toByteArray, StandardCharsets.UTF_8)
println(s"response was ${response.length} characters long")
}

private def makeTestRequest(method: String, path: String, testInput: Option[String]) = {
AwsRequest(
rawPath = path,
rawQueryString = "",
headers = Map.empty,
requestContext = AwsRequestContext(
domainName = None,
http = AwsHttp(
method = method,
path = path,
protocol = "",
sourceIp = "",
userAgent = "",
),
),
body = testInput,
isBase64Encoded = false,
)
}

}

class TestContext() extends Context {
override def getAwsRequestId: String = ???

override def getLogGroupName: String = ???

override def getLogStreamName: String = ???

override def getFunctionName: String = ???

override def getFunctionVersion: String = ???

override def getInvokedFunctionArn: String = ???

override def getIdentity: CognitoIdentity = ???

override def getClientContext: ClientContext = ???

override def getRemainingTimeInMillis: Int = ???

override def getMemoryLimitInMB: Int = ???

override def getLogger: LambdaLogger = new LambdaLogger:
override def log(message: String): Unit = {
val now = java.time.Instant.now().toString
println(s"$now: $message")
}

override def log(message: Array[Byte]): Unit = println(s"LOG BYTES: ${message.toString}")
}

// called from genDocs command in build.sbt
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,11 @@ object SQS {
}

object SQSLive {
val layer: ZLayer[AwsCredentialsProvider with Stage, ErrorResponse, SQS] =
val layer: RLayer[AwsCredentialsProvider & Stage, SQS] =
ZLayer.scoped(for {
stage <- ZIO.service[Stage]
sqsClient <- initializeSQSClient().mapError(ex =>
InternalServerError(s"Failed to initialize SQS Client with error: $ex"),
new Throwable(s"Failed to initialize SQS Client with error", ex),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we were returning InternalServerError but I don't know if it was actually getting used directly, or just converted into an internal server error later anyway

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you check? We obvs want to maintain the existing behaviour

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's a really good point I will check that for sure

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, existing code we end up with Any as the failure type in the ZIO, and that ends up here in the code


So no matter whether we called it InternalServerError or BadRequest etc, it it would still be an internal server error.

If/where we map the error to a ZIO.success then we could use it as a response.
I think the new (official) way of using Throwable rather than my way of using Any as the error type is better as it's pretty clear that a Throwable will be a 500 error, and if we try to use a BadRequest or something, expecting it will really be a bad request, it won't compile unless we recover it into a success.

)
emailQueueName = EmailQueueName.value
emailQueueUrlResponse <- getQueue(emailQueueName, sqsClient)
Expand Down Expand Up @@ -154,14 +154,14 @@ object SQSLive {
private def getQueue(
queueName: String,
sqsAsyncClient: SqsAsyncClient,
): ZIO[Any, ErrorResponse, GetQueueUrlResponse] =
): Task[GetQueueUrlResponse] =
val queueUrl = GetQueueUrlRequest.builder.queueName(queueName).build()

ZIO
.fromCompletableFuture(
sqsAsyncClient.getQueueUrl(queueUrl),
)
.mapError { ex => InternalServerError(s"Failed to get sqs queue name: $queueName, error: ${ex.getMessage}") }
.mapError { ex => new Throwable(s"Failed to get sqs queue name: $queueName", ex) }

private def impl(creds: AwsCredentialsProvider): SqsAsyncClient =
SqsAsyncClient.builder
Expand Down
Loading