Skip to content

Latest commit

 

History

History
202 lines (152 loc) · 8.46 KB

dev_guide.md

File metadata and controls

202 lines (152 loc) · 8.46 KB
Raw

Pravega - Creating Your First Application

Learn how to create a Hello World Pravega app. This guide covers:

  • Starting Pravega standalone
  • Creating a Pravega Stream.
  • Creating an Event Writer and write events into a Pravega Stream.
  • Create an Event Reader and read events from the Pravega Stream.

1. Prerequisites

To complete this guide, you need:

  • less than 15 minutes

  • an IDE

  • JDK 11+ installed with JAVA_HOME configured appropriately

  • Gradle 6.5.1+ Installation : https://gradle.org/install/

    !! Verify Gradle is using the Java you expect. You can verify which JDK Gradle uses by running gradle --version.!!

2. Goal

In this guide, we will create a straightforward application that creates a Stream on Pravega and writes an event into the Stream and reads back from it. We recommend that you follow the instructions from Bootstrapping project onwards to create the application step by step. However, you can go straight to the completed example at Pravega-samples-repo.

Command to clone the pravega-samples repo 

git clone https://github.com/pravega/pravega-samples.git

The solution is located in the pravega-client-examples directory.

3. Starting Pravega Standalone.

In standalone mode, the Pravega server is accessible from clients through the localhost interface only. Controller REST APIs, however, are accessible from remote hosts/machines. You can launch a standalone mode server using either of the following options:

  1. From source code
Commands to start standalone from source code

Checkout the source code:

$ git clone https://github.com/pravega/pravega.git
$ cd pravega

Build the Pravega standalone mode distribution:

./gradlew startStandalone

  1. From installation package
Commands to start standalone from installation package

Download the Pravega release from the GitHub Releases.

$ tar xfvz pravega-<version>.tgz

Download and extract either tarball or zip files. Follow the instructions provided for the tar files (same can be applied for zip file) to launch all the components of Pravega on your local machine.

Run Pravega Standalone:

$ pravega-<version>/bin/pravega-standalone

4. Bootstrapping the Project.

The easiest way to bootstrap a sample application against Pravega is to run the following command in a folder of your choice.

 gradle init --type java-application

Add the below snippet to dependencies section of build.gradle in the app directory

// https://mvnrepository.com/artifact/io.pravega/pravega-client
implementation group: 'io.pravega', name: 'pravega-client', version: '0.8.1'

Invoke gradle run to run the project.

Expected output 

osboxes@osboxes:/tmp/demo$ gradle run

> Task :app:run
Hello World!

BUILD SUCCESSFUL in 890ms
2 actionable tasks: 2 executed

4.1 Create a Pravega Stream

Let’s first get to know Pravega’s client APIs by creating a stream with a fixed scaling policy of 1 segment. We’ll need a StreamConfiguration to define this:

StreamConfiguration streamConfig = StreamConfiguration.builder()
        .scalingPolicy(ScalingPolicy.fixed(1))
        .build();

With this streamConfig, we can create streams that feel a bit more like traditional append-only files. Streams exist within scopes, which provide a namespace for related streams. We use a StreamManager to tell a Pravega cluster controller to create our scope and our streams. Since we are using a standalone Pravega let's use the below controller address.

URI controllerURI = URI.create("tcp://localhost:9090");

The code to create a Pravega stream is as follows

try (StreamManager streamManager = StreamManager.create(controllerURI)) {
        streamManager.createScope("examples");
        streamManager.createStream("examples", "helloStream", streamConfig);
}

Executing the above lines should ensure we have created a Pravega scope called examples and a Pravega Stream called helloStream.

4.2 Create a Pravega Event Writer and write events into the stream.

Let's create a Pravega Event Writer using the EventStreamClientFactory

  try (EventStreamClientFactory clientFactory = EventStreamClientFactory.withScope("examples",
                ClientConfig.builder().controllerURI(controllerURI).build());
             EventStreamWriter<String> writer = clientFactory.createEventWriter("helloStream",
                     new UTF8StringSerializer(), EventWriterConfig.builder().build())) {

            writer.writeEvent("helloRoutingKey", "hello world!"); // write an event.
  }

The above snippet creates an Event Writer and writes an event into the Pravega stream. Note that writeEvent() returns a CompletableFuture, which can be captured for use or will be resolved when calling flush() or close(), and, if destined for the same segment, the futures write in the order writeEvent() is called.

When instantiating the EventStreamWriter above, we passed in a UTF8StringSerializer instance. Pravega uses a Serializer interface in its writers and readers to simplify the act of writing and reading an object’s bytes to and from streams. The JavaSerializer can handle any Serializable object.

4.3 Create a Pravega Event Reader and read the event back from the stream.

Readers are associated with reader groups, which track the readers’ progress and allow more than one reader to coordinate over which segments they’ll read. A ReaderGroupManager is used to create a new reader group on the Pravega Stream.

The below snippet create a ReaderGroup

     try (ReaderGroupManager readerGroupManager = ReaderGroupManager.withScope("examples", controllerURI)) {
            ReaderGroupConfig readerGroupConfig = ReaderGroupConfig.builder()
                    .stream(Stream.of("examples", "helloStream"))
                    .disableAutomaticCheckpoints()
                    .build();
            readerGroupManager.createReaderGroup("readerGroup", readerGroupConfig);
     }

We can attach an Pravega Event Reader to this reader group and read the data from the Pravega Stream helloStream. The below snippet creates an EventReader called reader and reads the value from the Pravega Stream.

  try (EventStreamClientFactory clientFactory = EventStreamClientFactory.withScope("examples",
                ClientConfig.builder().controllerURI(controllerURI).build());
             EventStreamReader<String> reader = clientFactory.createReader("reader",
                     "readerGroup",
                     new UTF8StringSerializer(),
                     ReaderConfig.builder().build())) {

            String event = reader.readNextEvent(5000).getEvent();
            System.out.println(event);
  }

5. What’s next?

This guide covered the creation of a application that writes and reads from Pravega. However, there is much more. We recommend continuing the journey by going through Pravega-client-101 and other samples present in the Pravega Samples repo.