Skip to content

Latest commit

 

History

History
642 lines (526 loc) · 17.2 KB

spring-knative-cloudevents.adoc

File metadata and controls

642 lines (526 loc) · 17.2 KB

Processing CloudEvents with Spring and Knative

Introduction

This document describes how to get started writing SpringBoot apps that can process CloudEvents when deployed to Knative.

Prerequisites:

Tip
 * The Kubernetes cluster must be version v1.15 or later and must support the LoadBalancer service type to run the ingress service. We have tested these instructions with Google GKE, Docker Desktop for Mac, Docker Desktop for Windows, MicroK8s with MetalLB Loadbalancer on Ubuntu and Minikube with minikube tunnel. Try starting with a clean cluster to avoid any conflicts. Make sure to create a cluster with 4 or more cpus.
Note
 The commands in this guide are targeted for use in a Bash shell on Linux or macOS and PowerShell on Windows 10.

Initialize a project

Initialize a Spring Boot function application from start.spring.io:

curl https://start.spring.io/starter.tgz \
 -d dependencies=webflux,actuator,cloud-function \
 -d language=java \
 -d javaVersion=11 \
 -d bootVersion=2.2.5.RELEASE \
 -d type=maven-project \
 -d groupId=com.example \
 -d artifactId=spring-events \
 -d name=spring-events \
 -d packageName=com.example.spring-events \
 -d baseDir=spring-events | tar -xzvf -
cd spring-events
Note
on Windows use:
curl.exe https://start.spring.io/starter.tgz `
 -d dependencies=webflux,actuator,cloud-function `
 -d language=java `
 -d javaVersion=11 `
 -d bootVersion=2.2.5.RELEASE `
 -d type=maven-project `
 -d groupId=com.example `
 -d artifactId=spring-events `
 -d name=spring-events `
 -d packageName=com.example.spring-events `
 -d baseDir=spring-events `
 -o spring-events.tgz
tar.exe -xzvf spring-events.tgz
rm spring-events.tgz
cd spring-events

Add the function code

Add CloudEvents API as a dependency in pom.xml:

        <dependency>
            <groupId>io.cloudevents</groupId>
            <artifactId>cloudevents-api</artifactId>
            <version>1.2.0</version>
        </dependency>

Also add the jsonschema2pojo plugin to the build section in pom.xml:

            <plugin>
                <groupId>org.jsonschema2pojo</groupId>
                <artifactId>jsonschema2pojo-maven-plugin</artifactId>
                <version>1.0.2</version>
                <configuration>
                    <sourceDirectory>${basedir}/src/main/resources/schema</sourceDirectory>
                    <targetPackage>com.example.types</targetPackage>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>generate</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

Next add the schema for the SpringEvent class.

We expect this type being part of a type registry in Knative Eventing. In this case we haven’t registered a source that will provide this type. Instead we’ll add this manually to our project.

Create the file:

mkdir -p src/main/resources/schema
touch src/main/resources/schema/spring-event.json
Note
on Windows use:
mkdir -p src/main/resources/schema
New-Item -type file src/main/resources/schema/spring-event.json

Then, add this content:

src/main/resources/schema/spring-event.json
{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "SpringEvent",
    "description" : "This is the schema for the SpringEvent type.",
    "type": "object",
    "properties": {
        "releaseDate": {
            "type": "string",
            "format": "date-time"
        },
        "releaseName": {
            "type": "string"
        },
        "version": {
            "type": "string"
        }
    },
    "additionalProperties": false
}

We now need to add a CloudEventMapper that can map incoming JSON data to the generated SpringEvent class and also add the header information to a CloudEvent class.",

We expect this type of functionality to be added to Spring Cloud Function in the near future, but for now we need to provide this mapping ourselves.

Just to keep this simple, we will copy a simple mapper implementation from the trisberg/cloud-event-mapper repository.

Copy the file:

mkdir -p src/main/java/com/springdeveloper/support/cloudevents
curl https://raw.githubusercontent.com/trisberg/cloud-event-mapper/master/src/main/java/com/springdeveloper/support/cloudevents/CloudEventMapper.java \
  -o src/main/java/com/springdeveloper/support/cloudevents/CloudEventMapper.java
Note
on Windows use:
mkdir -p src/main/java/com/springdeveloper/support/cloudevents
curl.exe https://raw.githubusercontent.com/trisberg/cloud-event-mapper/master/src/main/java/com/springdeveloper/support/cloudevents/CloudEventMapper.java `
  -o src/main/java/com/springdeveloper/support/cloudevents/CloudEventMapper.java

Finally, we need to add the function code to handle the CloudEvent in the SpringEventsApplication class:

src/main/java/com/example/springevents/SpringEventsApplication.java
package com.example.springevents;

import java.util.function.Function;

import com.example.types.SpringEvent;
import com.fasterxml.jackson.databind.JsonNode;
import com.springdeveloper.support.cloudevents.CloudEventMapper;
import io.cloudevents.CloudEvent;
import io.cloudevents.v03.AttributesImpl;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.messaging.Message;
import org.springframework.messaging.support.MessageBuilder;

@SpringBootApplication
public class SpringEventsApplication {

    private Logger log = LoggerFactory.getLogger(SpringEventsApplication.class);

    @Bean
    public Function<Message<JsonNode>, Message<String>> fun() {
        return (in) -> {
            CloudEvent<AttributesImpl, SpringEvent> cloudEvent = CloudEventMapper.convert(in, SpringEvent.class);
            String results = "Processed: " + cloudEvent.getData();
            log.info(results);
            return MessageBuilder.withPayload(results).build();
        };
    }

    public static void main(String[] args) {
        SpringApplication.run(SpringEventsApplication.class, args);
    }

}

Build and test locally

Build and run:

./mvnw spring-boot:run

In a separate terminal:

curl -w'\n' localhost:8080 \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.springevent" \
 -H "Ce-Source: spring.io/spring-event" \
 -H "Content-Type: application/json" \
 -H "Ce-Id: 0001" \
 -d '{"releaseDate":"2004-03-24", "releaseName":"Spring Framework", "version":"1.0"}'
Note
on Windows use:
curl.exe -w'\n' localhost:8080 `
 -H "Ce-Specversion: 1.0" `
 -H "Ce-Type: com.example.springevent" `
 -H "Ce-Source: spring.io/spring-event" `
 -H "Content-Type: application/json" `
 -H "Ce-Id: 0001" `
 -d '{\"releaseDate\":\"2004-03-24\", \"releaseName\":\"Spring Framework\", \"version\":\"1.0\"}'

Build and test on k8s

Install Knative Serving

First we need to install Knative Serving on a Kubernetes cluster.

kubectl apply -f https://github.com/knative/serving/releases/download/v0.13.0/serving-crds.yaml
kubectl apply -f https://github.com/knative/serving/releases/download/v0.13.0/serving-core.yaml

We also need an ingress service. Here we will use Contour.

kubectl apply -f https://github.com/knative/net-contour/releases/download/v0.13.0/contour.yaml
kubectl apply -f https://github.com/knative/net-contour/releases/download/v0.13.0/net-contour.yaml

Now we need to configure Knative Serving to use this ingress service.

kubectl patch configmap/config-network \
  --namespace knative-serving \
  --type merge \
  --patch '{"data":{"ingress.class":"contour.ingress.networking.knative.dev"}}'
Note
on Windows use:
kubectl patch configmap/config-network `
  --namespace knative-serving `
  --type merge `
  --patch '{\"data\":{\"ingress.class\":\"contour.ingress.networking.knative.dev\"}}'

Configure Skaffold

Tip
 Skaffold sends color codes to the terminal output which might make it hard to read when using Windows Powershell. You can add a --color=0 option to any Skaffold command to minimize the color codes and make the output more readable.

Create a Knative Service manifest:

cat <<EOF > knative-service.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: spring-events
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: spring-events
EOF
Note
on Windows use:
@"
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: spring-events
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: spring-events
"@ | Out-File knative-service.yaml -enc ascii

Initialize skaffold:

skaffold init --skip-build

Modify skaffold.yaml and add the build section

apiVersion: skaffold/v2alpha4
kind: Config
metadata:
  name: spring-events
build:
  local:
    push: true
  artifacts:
    - image: spring-events
      buildpack:
        builder: "cloudfoundry/cnb:bionic"
  tagPolicy:
    sha256: {}
deploy:
  kubectl:
    manifests:
    - knative-service.yaml

Set your own prefix for the repository name, here we use the current user logged in. This should match your Docker ID, if it doesn’t just replace it with your Docker ID instead.

skaffold config set default-repo $USER
Note
on Windows use:
skaffold config set default-repo $env:UserName

Deploy to Kubernetes

Build and deploy to Kubernetes cluster:

skaffold run

Look up Ingress external IP or hostname.

For most clusters like GKE, microk8s etc use:

INGRESS=$(kubectl get --namespace contour-external service/envoy \
 -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
Note
on Windows use:
$INGRESS=$(kubectl get --namespace contour-external service/envoy `
 -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
Note
on Mac with Docker Desktop use:
INGRESS=$(kubectl get --namespace contour-external service/envoy \
 -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
Note
on Windows with Docker Desktop use:
$INGRESS=$(kubectl get --namespace contour-external service/envoy `
 -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Send a message:

curl -w'\n' $INGRESS \
 -H "Host: spring-events.default.example.com" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.springevent" \
 -H "Ce-Source: spring.io/spring-event" \
 -H "Content-Type: application/json" \
 -H "Ce-Id: 0001" \
 -d '{"releaseDate":"2004-03-24", "releaseName":"Spring Framework", "version":"1.0"}'
Note
on Windows use:
curl.exe -w'\n' $INGRESS `
 -H "Host: spring-events.default.example.com" `
 -H "Ce-Specversion: 1.0" `
 -H "Ce-Type: com.example.springevent" `
 -H "Ce-Source: spring.io/spring-event" `
 -H "Content-Type: application/json" `
 -H "Ce-Id: 0001" `
 -d '{\"releaseDate\":\"2004-03-24\", \"releaseName\":\"Spring Framework\", \"version\":\"1.0\"}'

Check the logs:

kubectl logs -c user-container -l serving.knative.dev/configuration=spring-events

Use Knative eventing to dispatch the CloudEvents

Install Knative Eventing

kubectl apply -f https://github.com/knative/eventing/releases/download/v0.13.0/eventing-crds.yaml
kubectl apply -f https://github.com/knative/eventing/releases/download/v0.13.0/eventing-core.yaml
kubectl apply -f https://github.com/knative/eventing/releases/download/v0.13.0/in-memory-channel.yaml
kubectl apply -f https://github.com/knative/eventing/releases/download/v0.13.0/channel-broker.yaml

Enable the default broker on the default namespace

Add a label to the namespace to have the eventing default broker start up:

kubectl label namespace default knative-eventing-injection=enabled

Verify that the broker is running:

kubectl -n default get broker.eventing.knative.dev default

Create a trigger

We need a trigger to respond to the SpringEvents CloudEvents:

cat <<EOF > knative-trigger.yaml
apiVersion: eventing.knative.dev/v1beta1
kind: Trigger
metadata:
  name: spring-events
spec:
  filter:
    attributes:
      type: com.example.springevent
  subscriber:
    ref:
     apiVersion: serving.knative.dev/v1
     kind: Service
     name: spring-events
EOF
Note
on Windows use:
@"
apiVersion: eventing.knative.dev/v1beta1
kind: Trigger
metadata:
  name: spring-events
spec:
  filter:
    attributes:
      type: com.example.springevent
  subscriber:
    ref:
     apiVersion: serving.knative.dev/v1
     kind: Service
     name: spring-events
"@ | Out-File knative-trigger.yaml -enc ascii

Now, apply this trigger manifest:

kubectl -n default apply -f knative-trigger.yaml

Post some test data

Now we can try posting events to the broker:

First create a shell inside the cluster where you can execute curl commands:

kubectl run curl --generator=run-pod/v1 --image=radial/busyboxplus:curl -i --tty --rm
curl -v "http://default-broker.default.svc.cluster.local" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.springevent" \
 -H "Ce-Source: spring.io/spring-event" \
 -H "Content-Type: application/json" \
 -H "Ce-Id: 0001" \
 -d '{"releaseDate":"2004-03-24", "releaseName":"Spring Framework", "version":"1.0"}'
curl -v "http://default-broker.default.svc.cluster.local" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.springevent" \
 -H "Ce-Source: spring.io/spring-event" \
 -H "Content-Type: application/json" \
 -H "Ce-Id: 0007" \
 -d '{"releaseDate":"2017-09-28", "releaseName":"Spring Framework", "version":"5.0"}'
curl -v "http://default-broker.default.svc.cluster.local" \
 -H "Ce-Specversion: 1.0" \
 -H "Ce-Type: com.example.springevent" \
 -H "Ce-Source: spring.io/spring-event" \
 -H "Content-Type: application/json" \
 -H "Ce-Id: 0008" \
 -d '{"releaseDate":"2018-03-01", "releaseName":"Spring Boot", "version":"2.0"}'

Check the logs in a separate terminal window:

kubectl logs -c user-container -l serving.knative.dev/configuration=spring-events

Close the curl shell inside the cluster by entering exit command.

Clean up:

Delete the trigger and the service:

kubectl delete trigger.eventing.knative.dev/spring-events
skaffold delete

Delete the default broker:

kubectl label namespace default knative-eventing-injection-
kubectl delete broker.eventing.knative.dev/default

Delete any left over Knative resources:

kubectl delete knative --all --all-namespaces

Uninstall Knative Eventing:

kubectl delete -f https://github.com/knative/eventing/releases/download/v0.13.0/channel-broker.yaml
kubectl delete -f https://github.com/knative/eventing/releases/download/v0.13.0/in-memory-channel.yaml
kubectl delete -f https://github.com/knative/eventing/releases/download/v0.13.0/eventing-core.yaml

Uninstall Knative Serving:

kubectl delete -f https://github.com/knative/net-contour/releases/download/v0.13.0/contour.yaml
kubectl delete -f https://github.com/knative/net-contour/releases/download/v0.13.0/net-contour.yaml
kubectl delete -f https://github.com/knative/serving/releases/download/v0.13.0/serving-core.yaml