Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix batch_transform_pca_dbscan_movie_clusters_notebook.ipynb #3566

Merged
merged 8 commits into from
Aug 19, 2022
Merged

fix batch_transform_pca_dbscan_movie_clusters_notebook.ipynb #3566

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a7417ae
fix batch_transform_pca_dbscan_movie_clusters.ipynb
Aug 18, 2022
e7a5448
lower test sample
Aug 18, 2022
0a97459
cleanup
Aug 18, 2022
666e3f8
lower test percentage
Aug 18, 2022
0cade21
Merge branch 'main' into fix-byoc-fix-batch-transform-pca-dbscan-movi…
atqy Aug 18, 2022
d6fdf2e
lower test percentage
Aug 18, 2022
4b0e46e
Merge branch 'main' into fix-byoc-fix-batch-transform-pca-dbscan-movi…
atqy Aug 18, 2022
9b9fc94
lower test percentage
Aug 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 26 additions & 2 deletions sagemaker_batch_transform/introduction_to_batch_transform/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,33 @@ RUN apt-get -y update && apt-get install -y --no-install-recommends \
wget \
r-base \
r-base-dev \
ca-certificates
ca-certificates

RUN R -e "install.packages(c('dbscan', 'plumber'), repos='https://cloud.r-project.org')"
RUN R -e "install.packages(c('Rcpp', 'BH', 'R6', 'jsonlite', 'crayon'), repos='https://cloud.r-project.org')"

RUN wget http://cran.r-project.org/src/contrib/Archive/stringi/stringi_1.2.4.tar.gz
RUN R CMD INSTALL stringi_1.2.4.tar.gz

RUN wget http://cran.r-project.org/src/contrib/Archive/rlang/rlang_0.2.2.tar.gz
RUN R CMD INSTALL rlang_0.2.2.tar.gz

RUN wget http://cran.r-project.org/src/contrib/Archive/magrittr/magrittr_1.5.tar.gz
RUN R CMD INSTALL magrittr_1.5.tar.gz

RUN wget http://cran.r-project.org/src/contrib/Archive/later/later_0.7.5.tar.gz
RUN R CMD INSTALL later_0.7.5.tar.gz

RUN wget http://cran.r-project.org/src/contrib/Archive/promises/promises_1.0.1.tar.gz
RUN R CMD INSTALL promises_1.0.1.tar.gz

RUN wget http://cran.r-project.org/src/contrib/Archive/httpuv/httpuv_1.4.4.2.tar.gz
RUN R CMD INSTALL httpuv_1.4.4.2.tar.gz

RUN wget http://cran.r-project.org/src/contrib/Archive/dbscan/dbscan_1.1-2.tar.gz
RUN R CMD INSTALL dbscan_1.1-2.tar.gz

RUN wget http://cran.r-project.org/src/contrib/Archive/plumber/plumber_0.4.6.tar.gz
RUN R CMD INSTALL plumber_0.4.6.tar.gz

COPY dbscan.R /opt/ml/dbscan.R
COPY plumber.R /opt/ml/plumber.R
Expand Down
4 changes: 2 additions & 2 deletions ...transform/introduction_to_batch_transform/batch_transform_pca_dbscan_movie_clusters.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we'll setup to split our dataset into train and test. Dimensionality reduction and clustering don't always require a holdout set to test accuracy, but it will allow us to illustrate how batch prediction might be used when new data arrives. In this case, our test dataset will be a simple 10% sample of items."
"Now, we'll setup to split our dataset into train and test. Dimensionality reduction and clustering don't always require a holdout set to test accuracy, but it will allow us to illustrate how batch prediction might be used when new data arrives. In this case, our test dataset will be a simple 0.5% sample of items."
]
},
{
Expand All @@ -270,7 +270,7 @@
"metadata": {},
"outputs": [],
"source": [
"test_products = products.sample(frac=0.1)\n",
"test_products = products.sample(frac=0.005)\n",
"train_products = products[~(products.index.isin(test_products.index))]"
]
},
Expand Down
2 changes: 1 addition & 1 deletion sagemaker_batch_transform/introduction_to_batch_transform/dbscan.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ parse_file <- function(file) {
# Second helper function for apply
parse_json <- function(line) {
if (validate(line)) {
return(do.call(rbind, fromJSON(line)[['projections']][[1]]))}}
return(do.call(rbind, fromJSON(line)))}}


# Setup scoring function
Expand Down
2 changes: 1 addition & 1 deletion sagemaker_batch_transform/introduction_to_batch_transform/plumber.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,4 @@ parse_file <- function(file) {
# Second helper function for apply
parse_json <- function(line) {
if (validate(line)) {
return(do.call(rbind, fromJSON(line)[['projections']][[1]]))}}
return(do.call(rbind, fromJSON(line)))}}