Add sequential notebook guidelines #3434
Conversation
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
CONTRIBUTING.md
Outdated
| @@ -217,6 +217,41 @@ Please remember to: | |||
| * Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. | |||
|
|
|||
|
|
|||
| ## Writing Sequential Notebooks | |||
|
|
|||
| Most notebooks are singular - all information and code needed for that example is stored in one notebook. However, there are a few cases in which an example may be split into multiple notebooks. These are called sequential notebooks, as the sequence of the example is split among multiple notebooks. An example you can look at is [this series of sequential notebooks that demonstrate how to build a music recommender](https://github.com/aws/amazon-sagemaker-examples/tree/main/end_to_end/music_recommendation). | |||
There was a problem hiding this comment.
One clarifying point - a singular notebook can also include other python, docker, etc files. But only one ipynb file.
CONTRIBUTING.md
Outdated
| You may want to consider using sequential notebooks to write your example if the following conditions apply: | ||
|
|
||
| * Your example takes over two hours to execute | ||
| * Your notebook is extremely lengthy to the point that it hinders the learning experience |
There was a problem hiding this comment.
Try to clarify how to define "lengthy"... number of code cells? Number of sections? Otherwise this will be inconsistent across contributors' judgments.
There was a problem hiding this comment.
It is very subjective - I think I will just take it out, it is very hard to quantify
CONTRIBUTING.md
Outdated
| * *Each notebook in the series must independently run end-to-end so that it can be tested in the daily CI (i.e. the CI test amazon-sagemaker-example-pr must pass).* | ||
| * This may include generating intermediate artifacts which can be immediately loaded up for use in later notebooks, etc. Depending on the situation, intermediate artifacts can be stored in the following places: | ||
| * The repo in the same folder where your notebook is stored: This is possible for very small files (on the order of KB) | ||
| * The sagemaker-sample-files S3 bucket: This is for larger files (on or above the order of MB). To upload a file to this bucket, please submit a ticket to the SageMaker Notebook Team using [this ticket template](https://t.corp.amazon.com/create/templates/af6cee93-e0b6-49af-85fb-ee9dc52262d3). If the files are acceptable, one of the SageMaker Notebook Team members will then upload your files on your behalf. |
There was a problem hiding this comment.
Don't link an external ticket template in the open-source file. Maybe we can share this in an internal wiki or pinned Slack message instead.
| * The sagemaker-sample-files S3 bucket: This is for larger files (on or above the order of MB). To upload a file to this bucket, please submit a ticket to the SageMaker Notebook Team using [this ticket template](https://t.corp.amazon.com/create/templates/af6cee93-e0b6-49af-85fb-ee9dc52262d3). If the files are acceptable, one of the SageMaker Notebook Team members will then upload your files on your behalf. | ||
| * Each notebook must have a ‘Background Section’ clearly stating that the notebook is part of a notebook sequence. It must contain the following elements below. You can look at the 'Background' section in [Music Recommender Data Exploration](https://github.com/aws/amazon-sagemaker-examples/blob/main/end_to_end/music_recommendation/01_data_exploration.ipynb) for an example. | ||
| * The objective and/or short summary of the notebook series | ||
| * A statement that the notebook is part of a notebook series. |
There was a problem hiding this comment.
Style nit: be consistent ending lines with or without a period.
There was a problem hiding this comment.
Noted, have put all lines ending with a period
CONTRIBUTING.md
Outdated
| * This may include generating intermediate artifacts which can be immediately loaded up for use in later notebooks, etc. Depending on the situation, intermediate artifacts can be stored in the following places: | ||
| * The repo in the same folder where your notebook is stored: This is possible for very small files (on the order of KB) | ||
| * The sagemaker-sample-files S3 bucket: This is for larger files (on or above the order of MB). To upload a file to this bucket, please submit a ticket to the SageMaker Notebook Team using [this ticket template](https://t.corp.amazon.com/create/templates/af6cee93-e0b6-49af-85fb-ee9dc52262d3). If the files are acceptable, one of the SageMaker Notebook Team members will then upload your files on your behalf. | ||
| * Each notebook must have a ‘Background Section’ clearly stating that the notebook is part of a notebook sequence. It must contain the following elements below. You can look at the 'Background' section in [Music Recommender Data Exploration](https://github.com/aws/amazon-sagemaker-examples/blob/main/end_to_end/music_recommendation/01_data_exploration.ipynb) for an example. |
There was a problem hiding this comment.
Formatting nit: watch out for "fancy" quotes (ie ‘’ instead of regular ones '')
CONTRIBUTING.md
Outdated
| * A statement communicating that the customer can choose to run the notebook by itself or as part of the series | ||
| * List and link to the other notebooks in the series. | ||
| * Clearly display where the current notebook fits in relation to the other notebooks (i.e. it is the 3rd notebook in the series). | ||
| * If you have a README that contains more introductory information about the notebook series as a whole, link to it. For example, it is nice to have an architecture diagram showing how the services interact across different notebooks - the README would be a good place to put such information. |
There was a problem hiding this comment.
Do you want to link an example of a set of sequential notebooks that includes an architecture diagram?
CONTRIBUTING.md
Outdated
| * When you first use an intermediate artifact in a notebook, add a link to the notebook that is responsible for generating that artifact. That way, customers can easily look up how that artifact was created if they wanted to. | ||
| * Use links to shorten the length of your notebook and keep it simple and organized. Instead of writing a long passage about how a feature works (i.e Batch Transform), it is better to link to the documentation for it. | ||
| * Design your notebook series such that the customer can get benefit from both the individual notebooks and the whole series. For example, each notebook should have clear takeaway points for the customer (i.e. one notebook teaches data preparation and feature engineering, the next notebook teaches training, etc.) | ||
| * Put the sequence order in the notebook file name. For example, the first notebook should start with “1_”, the second notebook with “2_”, etc. |
There was a problem hiding this comment.
Formatting nit: more fancy quotes to remove here.
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Issue #, if available:
Description of changes:
The goal of this PR is to add contributing guidelines for anyone wanting to write sequential notebooks.
Testing done:
None needed.
Merge Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.tox -e black-format,black-nb-formatBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.