Skip to content

Commit

Permalink
Add Neptune Workbench CFN template (#442)
Browse files Browse the repository at this point in the history
Co-authored-by: Michael Chin <[email protected]>
  • Loading branch information
michaelnchin and michaelnchin authored Mar 14, 2023
1 parent 4794003 commit bc89959
Show file tree
Hide file tree
Showing 6 changed files with 200 additions and 5 deletions.
9 changes: 4 additions & 5 deletions additional-databases/sagemaker/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
## Launching graph-notebook using Amazon SageMaker
You can easily launch instances of graph-notebook on Amazon SageMaker by using a lifecycle configuration. To learn more about lifecycle configurations and how to create one, see [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html).
## Using graph-notebook on Amazon Sagemaker

Use the sample lifecycle configuration in this folder, [`install-graph-notebook-lc.sh`](install-graph-notebook-lc.sh) ([`install-graph-notebook-lc-cn.sh`](install-graph-notebook-lc-cn.sh) if using `cn-north-1` or `cn-northwest-1` region) or create your own shell script.
Amazon Sagemaker Notebooks provide an easy and effective solution for hosting, configuring, and running `graph-notebook` against a graph database. These notebooks also serve as the base platform for [Neptune Workbench](https://docs.aws.amazon.com/neptune/latest/userguide/graph-notebooks.html).

After you create a lifecycle configuration on SageMaker, you can create new notebook instances by specifying a saved lifecycle configuration:
If you would like to manually deploy a Neptune Workbench instance via AWS CloudFormation, please see the instructions in the [`neptune-notebook-cloudformation`](https://github.com/aws/graph-notebook/blob/main/additional-databases/sagemaker/neptune-notebook-cloudformation) folder.

![create-a-notebook](/././images/Create-Notebook-Instance.png)
For non-Neptune use cases, you can follow the instructions in the [`sagemaker-notebook-lifecycle`](https://github.com/aws/graph-notebook/blob/main/additional-databases/sagemaker/sagemaker-notebook-lifecycle) folder.
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
## Launching graph-notebook as Amazon Neptune Workbench via AWS CloudFormation

The AWS CloudFormation template in this folder, [`neptune-workbench-stack.yaml`](neptune-workbench-stack.yaml), deploys Amazon Neptune workbench notebooks as resources, and includes the base 'Getting Started' notebooks. The workbench lets you work with your Amazon Neptune cluster using Jupyter notebooks hosted by Amazon SageMaker. You are billed for workbench resources through Amazon SageMaker, separately from your Neptune billing.

### Parameter details
#### Minimum permissions for the SageMakerNotebookRole
This is the ARN for the AWS IAM role that the notebook instance will assume. Make sure that this role has at least the following minimum permissions within its service role policy:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::aws-neptune-notebook",
"arn:aws:s3:::aws-neptune-notebook/*"
]
},
{
"Effect": "Allow",
"Action": "neptune-db:connect",
"Resource": [
"your-cluster-arn/*"
]
}
]
}
```

The role should also establish the following trust relationship:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "sagemaker.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
```

#### How to populate the 'Cluster' value within the AWS Console for Amazon Neptune Notebooks
Add the following tags manually to the notebook instance.

| Key | Value |
| ------------- |-------------|
| **aws-neptune-cluster-id** | Amazon Neptune database cluster ID (found under *DB cluster id* under *Configuration* of the selected cluster in the AWS console) |
| **aws-neptune-resource-id** | Amazon Neptune cluster resource ID (found under *Resource id* under *Configuration* of the selected cluster in the AWS console) |
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
AWSTemplateFormatVersion: '2010-09-09'

Description: A template to deploy Neptune Notebooks using CloudFormation resources.

Parameters:
NotebookInstanceType:
Description: The notebook instance type.
Type: String
Default: ml.t2.medium
AllowedValues:
- ml.t2.medium
- ml.t2.large
- ml.t2.xlarge
- ml.t2.2xlarge
- ml.t3.2xlarge
- ml.t3.large
- ml.t3.medium
- ml.t3.xlarge
- ml.m4.xlarge
- ml.m4.2xlarge
- ml.m4.4xlarge
- ml.m4.10xlarge
- ml.m4.16xlarge
- ml.m5.12xlarge
- ml.m5.24xlarge
- ml.m5.2xlarge
- ml.m5.4xlarge
- ml.m5.xlarge
- ml.p2.16xlarge
- ml.p2.8xlarge
- ml.p2.xlarge
- ml.p3.16xlarge
- ml.p3.2xlarge
- ml.p3.8xlarge
- ml.c4.2xlarge
- ml.c4.4xlarge
- ml.c4.8xlarge
- ml.c4.xlarge
- ml.c5.18xlarge
- ml.c5.2xlarge
- ml.c5.4xlarge
- ml.c5.9xlarge
- ml.c5.xlarge
- ml.c5d.18xlarge
- ml.c5d.2xlarge
- ml.c5d.4xlarge
- ml.c5d.9xlarge
- ml.c5d.xlarge
ConstraintDescription: Must be a valid SageMaker instance type.

NeptuneClusterEndpoint:
Description: The cluster endpoint of an existing Neptune cluster.
Type: String

NeptuneClusterPort:
Description: 'OPTIONAL: The Port of an existing Neptune cluster (default 8182).'
Type: String
Default: '8182'

NeptuneClusterSecurityGroups:
Description: The VPC security group IDs. The security groups must be for the same VPC as specified in the subnet.
Type: List<AWS::EC2::SecurityGroup::Id>

NeptuneClusterSubnetId:
Description: The ID of the subnet in a VPC to which you would like to have a connectivity from your ML compute instance.
Type: AWS::EC2::Subnet::Id

SageMakerNotebookRole:
Description: The ARN for the IAM role that the notebook instance will assume.
Type: String
AllowedPattern: ^arn:aws[a-z\-]*:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@\-_/]+$

SageMakerNotebookName:
Description: The name of the Neptune notebook.
Type: String

Resources:
NeptuneNotebookInstance:
Type: AWS::SageMaker::NotebookInstance
Properties:
NotebookInstanceName: !Join
- ''
- - 'aws-neptune-'
- !Ref SageMakerNotebookName
InstanceType:
Ref: NotebookInstanceType
SubnetId:
Ref: NeptuneClusterSubnetId
SecurityGroupIds:
Ref: NeptuneClusterSecurityGroups
RoleArn:
Ref: SageMakerNotebookRole
LifecycleConfigName:
Fn::GetAtt:
- NeptuneNotebookInstanceLifecycleConfig
- NotebookInstanceLifecycleConfigName

NeptuneNotebookInstanceLifecycleConfig:
Type: AWS::SageMaker::NotebookInstanceLifecycleConfig
Properties:
OnStart:
- Content:
Fn::Base64:
Fn::Join:
- ''
- - "#!/bin/bash\n"
- sudo -u ec2-user -i << 'EOF'
- "\n"
- echo 'export GRAPH_NOTEBOOK_AUTH_MODE=
- "DEFAULT' >> ~/.bashrc\n"
- echo 'export GRAPH_NOTEBOOK_HOST=
- !Ref NeptuneClusterEndpoint
- "' >> ~/.bashrc\n"
- echo 'export GRAPH_NOTEBOOK_PORT=
- !Ref NeptuneClusterPort
- "' >> ~/.bashrc\n"
- echo 'export NEPTUNE_LOAD_FROM_S3_ROLE_ARN=
- "' >> ~/.bashrc\n"
- echo 'export AWS_REGION=
- !Ref AWS::Region
- "' >> ~/.bashrc\n"
- aws s3 cp s3://aws-neptune-notebook/graph_notebook.tar.gz /tmp/graph_notebook.tar.gz
- "\n"
- rm -rf /tmp/graph_notebook
- "\n"
- tar -zxvf /tmp/graph_notebook.tar.gz -C /tmp
- "\n"
- /tmp/graph_notebook/install.sh
- "\n"
- EOF
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
## Launching graph-notebook on Amazon SageMaker using a lifecycle
You can easily configure graph-notebook to run on an Amazon SageMaker Notebook instance by using a lifecycle configuration. To learn more about lifecycle configurations and how to create one, see [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html).

Use the sample lifecycle configuration in this folder, [`install-graph-notebook-lc.sh`](install-graph-notebook-lc.sh) ([`install-graph-notebook-lc-cn.sh`](install-graph-notebook-lc-cn.sh) if using `cn-north-1` or `cn-northwest-1` region) or create your own shell script.

After you create a lifecycle configuration on SageMaker, you can create new notebook instances by specifying a saved lifecycle configuration:

![create-a-notebook](/images/Create-Notebook-Instance.png)

0 comments on commit bc89959

Please sign in to comment.