-
Notifications
You must be signed in to change notification settings - Fork 170
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add Neptune Workbench CFN template (#442)
Co-authored-by: Michael Chin <[email protected]>
- Loading branch information
1 parent
4794003
commit bc89959
Showing
6 changed files
with
200 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,8 +1,7 @@ | ||
## Launching graph-notebook using Amazon SageMaker | ||
You can easily launch instances of graph-notebook on Amazon SageMaker by using a lifecycle configuration. To learn more about lifecycle configurations and how to create one, see [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html). | ||
## Using graph-notebook on Amazon Sagemaker | ||
|
||
Use the sample lifecycle configuration in this folder, [`install-graph-notebook-lc.sh`](install-graph-notebook-lc.sh) ([`install-graph-notebook-lc-cn.sh`](install-graph-notebook-lc-cn.sh) if using `cn-north-1` or `cn-northwest-1` region) or create your own shell script. | ||
Amazon Sagemaker Notebooks provide an easy and effective solution for hosting, configuring, and running `graph-notebook` against a graph database. These notebooks also serve as the base platform for [Neptune Workbench](https://docs.aws.amazon.com/neptune/latest/userguide/graph-notebooks.html). | ||
|
||
After you create a lifecycle configuration on SageMaker, you can create new notebook instances by specifying a saved lifecycle configuration: | ||
If you would like to manually deploy a Neptune Workbench instance via AWS CloudFormation, please see the instructions in the [`neptune-notebook-cloudformation`](https://github.com/aws/graph-notebook/blob/main/additional-databases/sagemaker/neptune-notebook-cloudformation) folder. | ||
|
||
![create-a-notebook](/././images/Create-Notebook-Instance.png) | ||
For non-Neptune use cases, you can follow the instructions in the [`sagemaker-notebook-lifecycle`](https://github.com/aws/graph-notebook/blob/main/additional-databases/sagemaker/sagemaker-notebook-lifecycle) folder. |
58 changes: 58 additions & 0 deletions
58
additional-databases/sagemaker/neptune-notebook-cloudformation/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
## Launching graph-notebook as Amazon Neptune Workbench via AWS CloudFormation | ||
|
||
The AWS CloudFormation template in this folder, [`neptune-workbench-stack.yaml`](neptune-workbench-stack.yaml), deploys Amazon Neptune workbench notebooks as resources, and includes the base 'Getting Started' notebooks. The workbench lets you work with your Amazon Neptune cluster using Jupyter notebooks hosted by Amazon SageMaker. You are billed for workbench resources through Amazon SageMaker, separately from your Neptune billing. | ||
|
||
### Parameter details | ||
#### Minimum permissions for the SageMakerNotebookRole | ||
This is the ARN for the AWS IAM role that the notebook instance will assume. Make sure that this role has at least the following minimum permissions within its service role policy: | ||
|
||
```json | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"s3:GetObject", | ||
"s3:ListBucket" | ||
], | ||
"Resource": [ | ||
"arn:aws:s3:::aws-neptune-notebook", | ||
"arn:aws:s3:::aws-neptune-notebook/*" | ||
] | ||
}, | ||
{ | ||
"Effect": "Allow", | ||
"Action": "neptune-db:connect", | ||
"Resource": [ | ||
"your-cluster-arn/*" | ||
] | ||
} | ||
] | ||
} | ||
``` | ||
|
||
The role should also establish the following trust relationship: | ||
|
||
```json | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Principal": { | ||
"Service": "sagemaker.amazonaws.com" | ||
}, | ||
"Action": "sts:AssumeRole" | ||
} | ||
] | ||
} | ||
``` | ||
|
||
#### How to populate the 'Cluster' value within the AWS Console for Amazon Neptune Notebooks | ||
Add the following tags manually to the notebook instance. | ||
|
||
| Key | Value | | ||
| ------------- |-------------| | ||
| **aws-neptune-cluster-id** | Amazon Neptune database cluster ID (found under *DB cluster id* under *Configuration* of the selected cluster in the AWS console) | | ||
| **aws-neptune-resource-id** | Amazon Neptune cluster resource ID (found under *Resource id* under *Configuration* of the selected cluster in the AWS console) | |
130 changes: 130 additions & 0 deletions
130
additional-databases/sagemaker/neptune-notebook-cloudformation/neptune-workbench-stack.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
AWSTemplateFormatVersion: '2010-09-09' | ||
|
||
Description: A template to deploy Neptune Notebooks using CloudFormation resources. | ||
|
||
Parameters: | ||
NotebookInstanceType: | ||
Description: The notebook instance type. | ||
Type: String | ||
Default: ml.t2.medium | ||
AllowedValues: | ||
- ml.t2.medium | ||
- ml.t2.large | ||
- ml.t2.xlarge | ||
- ml.t2.2xlarge | ||
- ml.t3.2xlarge | ||
- ml.t3.large | ||
- ml.t3.medium | ||
- ml.t3.xlarge | ||
- ml.m4.xlarge | ||
- ml.m4.2xlarge | ||
- ml.m4.4xlarge | ||
- ml.m4.10xlarge | ||
- ml.m4.16xlarge | ||
- ml.m5.12xlarge | ||
- ml.m5.24xlarge | ||
- ml.m5.2xlarge | ||
- ml.m5.4xlarge | ||
- ml.m5.xlarge | ||
- ml.p2.16xlarge | ||
- ml.p2.8xlarge | ||
- ml.p2.xlarge | ||
- ml.p3.16xlarge | ||
- ml.p3.2xlarge | ||
- ml.p3.8xlarge | ||
- ml.c4.2xlarge | ||
- ml.c4.4xlarge | ||
- ml.c4.8xlarge | ||
- ml.c4.xlarge | ||
- ml.c5.18xlarge | ||
- ml.c5.2xlarge | ||
- ml.c5.4xlarge | ||
- ml.c5.9xlarge | ||
- ml.c5.xlarge | ||
- ml.c5d.18xlarge | ||
- ml.c5d.2xlarge | ||
- ml.c5d.4xlarge | ||
- ml.c5d.9xlarge | ||
- ml.c5d.xlarge | ||
ConstraintDescription: Must be a valid SageMaker instance type. | ||
|
||
NeptuneClusterEndpoint: | ||
Description: The cluster endpoint of an existing Neptune cluster. | ||
Type: String | ||
|
||
NeptuneClusterPort: | ||
Description: 'OPTIONAL: The Port of an existing Neptune cluster (default 8182).' | ||
Type: String | ||
Default: '8182' | ||
|
||
NeptuneClusterSecurityGroups: | ||
Description: The VPC security group IDs. The security groups must be for the same VPC as specified in the subnet. | ||
Type: List<AWS::EC2::SecurityGroup::Id> | ||
|
||
NeptuneClusterSubnetId: | ||
Description: The ID of the subnet in a VPC to which you would like to have a connectivity from your ML compute instance. | ||
Type: AWS::EC2::Subnet::Id | ||
|
||
SageMakerNotebookRole: | ||
Description: The ARN for the IAM role that the notebook instance will assume. | ||
Type: String | ||
AllowedPattern: ^arn:aws[a-z\-]*:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@\-_/]+$ | ||
|
||
SageMakerNotebookName: | ||
Description: The name of the Neptune notebook. | ||
Type: String | ||
|
||
Resources: | ||
NeptuneNotebookInstance: | ||
Type: AWS::SageMaker::NotebookInstance | ||
Properties: | ||
NotebookInstanceName: !Join | ||
- '' | ||
- - 'aws-neptune-' | ||
- !Ref SageMakerNotebookName | ||
InstanceType: | ||
Ref: NotebookInstanceType | ||
SubnetId: | ||
Ref: NeptuneClusterSubnetId | ||
SecurityGroupIds: | ||
Ref: NeptuneClusterSecurityGroups | ||
RoleArn: | ||
Ref: SageMakerNotebookRole | ||
LifecycleConfigName: | ||
Fn::GetAtt: | ||
- NeptuneNotebookInstanceLifecycleConfig | ||
- NotebookInstanceLifecycleConfigName | ||
|
||
NeptuneNotebookInstanceLifecycleConfig: | ||
Type: AWS::SageMaker::NotebookInstanceLifecycleConfig | ||
Properties: | ||
OnStart: | ||
- Content: | ||
Fn::Base64: | ||
Fn::Join: | ||
- '' | ||
- - "#!/bin/bash\n" | ||
- sudo -u ec2-user -i << 'EOF' | ||
- "\n" | ||
- echo 'export GRAPH_NOTEBOOK_AUTH_MODE= | ||
- "DEFAULT' >> ~/.bashrc\n" | ||
- echo 'export GRAPH_NOTEBOOK_HOST= | ||
- !Ref NeptuneClusterEndpoint | ||
- "' >> ~/.bashrc\n" | ||
- echo 'export GRAPH_NOTEBOOK_PORT= | ||
- !Ref NeptuneClusterPort | ||
- "' >> ~/.bashrc\n" | ||
- echo 'export NEPTUNE_LOAD_FROM_S3_ROLE_ARN= | ||
- "' >> ~/.bashrc\n" | ||
- echo 'export AWS_REGION= | ||
- !Ref AWS::Region | ||
- "' >> ~/.bashrc\n" | ||
- aws s3 cp s3://aws-neptune-notebook/graph_notebook.tar.gz /tmp/graph_notebook.tar.gz | ||
- "\n" | ||
- rm -rf /tmp/graph_notebook | ||
- "\n" | ||
- tar -zxvf /tmp/graph_notebook.tar.gz -C /tmp | ||
- "\n" | ||
- /tmp/graph_notebook/install.sh | ||
- "\n" | ||
- EOF |
8 changes: 8 additions & 0 deletions
8
additional-databases/sagemaker/sagemaker-notebook-lifecycle/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
## Launching graph-notebook on Amazon SageMaker using a lifecycle | ||
You can easily configure graph-notebook to run on an Amazon SageMaker Notebook instance by using a lifecycle configuration. To learn more about lifecycle configurations and how to create one, see [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html). | ||
|
||
Use the sample lifecycle configuration in this folder, [`install-graph-notebook-lc.sh`](install-graph-notebook-lc.sh) ([`install-graph-notebook-lc-cn.sh`](install-graph-notebook-lc-cn.sh) if using `cn-north-1` or `cn-northwest-1` region) or create your own shell script. | ||
|
||
After you create a lifecycle configuration on SageMaker, you can create new notebook instances by specifying a saved lifecycle configuration: | ||
|
||
![create-a-notebook](/images/Create-Notebook-Instance.png) |
File renamed without changes.
File renamed without changes.