Working With Scylla DynamoDB compatible API from an AWS Lambda Function

TL;DR

Works out of the box, by changing the AWS endpoint to Scylla IP

AWS.config.update({
  endpoint: "http://17.17.17.171:8000"
});

What are Lambda functions

AWS Lambda is an event-driven, serverless computing platform. It is a computing service that runs code in response to events, for example, REST API calls, and performs computation and calls to other services.

Note the “serverless” above. As such it is impossible to build a stateful service on Lambda alone. To build a stateful service, one needs a stateful backend to store the service state, and have the Lambda functions call it to store and restore this state. One of the popular backends for such a service is AWS DynamoDB. Scylla new DynamoDB compatible API, allows you to use the same functions with Scylla.

DynamoDB example

We took a trivial JS example[1] from an excellent AWS online tutorial: "Build a Serverless Web Application" [2]. The example implements a Unicorn Rides service (Grab for Unicorns ;) ) where the user asks for a ride, providing his location using a REST POST command, and the service allocates a horse from the fleet by random and writes the ride to the DB.

The tutorial walks you through

1. Create a Rides table on DynamoDB or its compatible ScyllaDB
2. Create AWS IAM role with access permission to this table
3. Create a requestUnicorn.js Lambda function which, as a response to API (REST) call, inserts a row to the Rides table using the AMI role.
4. Create a simple test for the function, and run it.

Next, we will run the same example with Scylla DynamoDB compatible API

Setup Scylla with DynamoDB compatible API

Follow the instructions on docs/alternator/getting-started.md (TBD: need to update) to start Scylla with docs/alternator/getting-started.md. Install AWS CLI, and validate it works with Scylla [TBD link] Create the Rides table:

aws --endpoint-url 'http://scylla-ip:8000' dynamodb create-table --table-name Rides --attribute-definitions AttributeName=RideId,AttributeType=S --key-schema AttributeName=RideId,KeyType=HASH

If you are running this AWS CLI on the same server as Scylla, you can use localhost for scylla-ip above.

Note that the Scylla instance needs an IP available from the external world, or in the same VPC, which the lambda function can access.

Now, update the example code to work with Scylla DynamoDB compatible API: At the top of the JS function, you will find the following line:

const AWS = require('aws-sdk');

This command creates an AWS object[3], which later will be used to create a DynamoDB client object. Before using the AWS object, we will update its configuration to use Scylla as an endpoint. Add the following just after the AWS creation

AWS.config.update({
  endpoint: "http://scylla-ip:8000"
});

Where scylla-ip is the IP of your instance with Alternator enabled, and 8000 is the port number chosen for the DynamoDB-compatible API, configured via the alternator-port parameter in scylla.yaml

And…. Done. Your example now works with Scylla. If you created the TestRequestEvent earlier, you can simply click the Test button to run the test. (instructions can be found in Step 4 of the AWS tutorial [2]) If all goes well, you should see successful results, for example

{
  "statusCode": 201,
  "body": "{\"RideId\":\"1VH81JkZaLc9l5Q5SNnIRg\",\"Unicorn\":{\"Name\":\"Rocinante\",\"Color\":\"Yellow\",\"Gender\":\"Female\"},\"UnicornName\":\"Rocinante\",\"Eta\":\"30 seconds\",\"Rider\":\"the_username\"}",
  "headers": {
    "Access-Control-Allow-Origin": "*"
  }
}

To validate this is indeed the case you can use the AWS CLI to scan the table:

aws  --endpoint-url 'http://scylla-ip:8000' dynamodb scan --table-name Rides

Security

Exposing the Scylla server to the public Internet is a bad practice. It is recommended to run Scylla in a VPC, and attach the Lambda function to this VPC[4]

When using DynamoDB, you can create an IAM role that limits the Lambda function to one DynamoDB table and one operation (Rides::putItem). This granularity is not available with Scylla DynamoDB compatible API yet (although Scylla does support Role Base Access Control [5]).

[1] https://github.com/aws-samples/aws-serverless-workshops/blob/master/WebApplication/3_ServerlessBackend/requestUnicorn.js

[2] https://aws.amazon.com/getting-started/projects/build-serverless-web-app-lambda-apigateway-s3-dynamodb-cognito/module-3/

[3] https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS.html

[4] https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html

[5] https://docs.scylladb.com/operating-scylla/security/rbac_usecase/