Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Client notebook for operator deployments #4815

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

feat: Client notebook for operator deployments #4815

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a9b2829
client notebook for operator deployments
dmartinol Dec 5, 2024
d902503
unnedded file
dmartinol Dec 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions infra/feast-operator/demo/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Installing the sample notebook
Follow this procedure to install and configure a sample Jupyter Notebook in your Kubernetes environment that can
connect a deployed `FeatureStore` instance.

## Requirement
* `FeatureStore` instance deployed using the Feast Operator.
* Local environment logged to the target Kubernetes cluster with `admin` privileges.
* `kubectl` CLI
* A Notebook instance created in the Kubernetes cluster (either with KubeFlow, OpedDataHub or RedHat OpenShift AI)
* No specific image is needed for the purpose, assuming that it includes the Python interpreter.

## Mounting the Feast resources
**Important Notes**:
* This procedure causes a restart of the running notebook to apply the mounted resources. Save your changes, before
proceeding.
* This procedure updates the `Notebook` custom resource and must be executed just once. Next executions will break
the consistency of the manifest. In this case, edit the resource manifest and remove the duplicated settings.

Launch this command to mount the Feast client ConfigMap and the required TLS secrets to the only Notebook in the namespace:
```bash
./init_feast_notebook.sh
```

In case you have multiple Notebooks or Feast instances, provide their names using the optional parameters:
```bash
% ./init_feast_notebook.sh --help
Usage: init_feast_notebook.sh [--notebook <notebook_name>] [--feast <feast_name>] [--help]

Options:
--notebook Specify the notebook name. Defaults to the only Notebook instance in the current namespace.
--feast Specify the Feast instance name. Defaults to the only Feast instance in the current namespace.
--help Display this help message.
```

Output example:
```console
Notebook Name: feast-client
Feast Name: example
ConfigMap Mount Path: /feast/feature_repository
Connecting Notebook 'feast-client' with Feast 'example'...
Patching Notebook feast-client with ConfigMap feast-example-client
notebook.kubeflow.org/feast-client patched
Offline TLS Mount Path:
Online TLS Mount Path: /tls/online
Registry TLS Mount Path: /tls/registry
Patching Notebook feast-client with Secret feast-example-online-tls mounted at /tls/online
notebook.kubeflow.org/feast-client patched
Patching Notebook feast-client with Secret feast-example-registry-tls mounted at /tls/registry
notebook.kubeflow.org/feast-client patched
```

## Validating the Notebook

Wait until the notebook is ready (replace NOTEBOOK_NAME with your notebook name):
```bash
kubectl wait --for=condition=ready pod/NOTEBOOK_NAME-0 --timeout=2m
```

Then verify the content of the mounted folders (update `/feast` and `/tls` with your actual paths, if needed)
```bash
kubectl exec pod/NOTEBOOK_NAME-0 -c NOTEBOOK_NAME -- bash -c "find /feast; find /tls"
```

Output example:
```console
/feast
/feast/feature_repository
/feast/feature_repository/..2024_12_05_09_26_57.3976272882
/feast/feature_repository/..2024_12_05_09_26_57.3976272882/feature_store.yaml
/feast/feature_repository/..data
/feast/feature_repository/feature_store.yaml
/tls
/tls/online
/tls/online/tls.key
/tls/online/tls.crt
/tls/online/..data
/tls/online/..2024_12_05_09_26_57.1747038009
/tls/online/..2024_12_05_09_26_57.1747038009/tls.key
/tls/online/..2024_12_05_09_26_57.1747038009/tls.crt
/tls/registry
/tls/registry/tls.key
/tls/registry/tls.crt
/tls/registry/..data
/tls/registry/..2024_12_05_09_26_57.3730005170
/tls/registry/..2024_12_05_09_26_57.3730005170/tls.key
/tls/registry/..2024_12_05_09_26_57.3730005170/tls.crt
```

## Importing the quickstart Jupyter Notebook
The provided [quickstart](./quickstart.ipynb) notebook provides you a quick-start client application to connect the
deployed Feast instance:
* Install `feast` from either `git` (for latest developments) or `pypi` (for released versions).
* Connect to the deplpoyed Feast instance and validate the content:
* Using `feast` CLI.
* With `feast` python package.

Copy the notebook in your workbench, using either `git clone`, `kubectl cp` or `wget`/`curl` and enjoy the experience!
161 changes: 161 additions & 0 deletions infra/feast-operator/demo/init_feast_notebook.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
#!/bin/bash

NOTEBOOK_NAME=""
FEAST_NAME=""
CM_MOUNT_PATH="/feast/feature_repository"

show_help() {
echo "Usage: init_feast_notebook.sh [--notebook <notebook_name>] [--feast <feast_name>] [--help]"
echo
echo "Options:"
echo " --notebook Specify the notebook name. Defaults to the only Notebook instance in the current namespace."
echo " --feast Specify the Feast instance name. Defaults to the only Feast instance in the current namespace."
echo " --help Display this help message."
exit 0
}

get_single_resource_name() {
local resource_type=$1
local resource_count
resource_count=$(kubectl get "$resource_type" --no-headers | wc -l)

if [ "$resource_count" -eq 1 ]; then
kubectl get "$resource_type" --no-headers | awk '{print $1}'
else
echo ""
fi
}

patch_config_map() {
local notebook_name=$1
local config_map_name=$2
local mount_path=$3
echo "Patching Notebook $notebook_name with ConfigMap $config_map_name"
kubectl patch notebook "$notebook_name" -n feast --type='json' -p="[
{
\"op\": \"add\",
\"path\": \"/spec/template/spec/volumes/-\",
\"value\": {
\"name\": \"$config_map_name\",
\"configMap\": {
\"name\": \"$config_map_name\"
}
}
},
{
\"op\": \"add\",
\"path\": \"/spec/template/spec/containers/0/volumeMounts/-\",
\"value\": {
\"name\": \"$config_map_name\",
\"mountPath\": \"$mount_path\"
}
}
]"
}

get_tls_secret_name() {
local service_type=$1
local config_map_name=$2
kubectl get cm $config_map_name -oyaml | yq '.data."feature_store.yaml"' | yq ".${service_type}.cert" | xargs dirname
}

patch_tls_secret(){
local notebook_name=$1
local secret_name=$2
local mount_path=$3

echo "Patching Notebook $notebook_name with Secret $secret_name mounted at $mount_path"
kubectl patch notebook "$notebook_name" -n feast --type='json' -p="[
{
\"op\": \"add\",
\"path\": \"/spec/template/spec/volumes/-\",
\"value\": {
\"name\": \"$secret_name\",
\"secret\": {
\"defaultMode\": 420,
\"secretName\": \"$secret_name\"
}
}
},
{
\"op\": \"add\",
\"path\": \"/spec/template/spec/containers/0/volumeMounts/-\",
\"value\": {
\"name\": \"$secret_name\",
\"mountPath\": \"$mount_path\"
}
}
]"
}

while [[ $# -gt 0 ]]; do
key="$1"
case $key in
--notebook)
NOTEBOOK_NAME="$2"
shift
shift
;;
--feast)
FEAST_NAME="$2"
shift
shift
;;
--help)
show_help
;;
*)
echo "Unknown option $1"
exit 1
;;
esac
done

if [ -z "$NOTEBOOK_NAME" ]; then
NOTEBOOK_NAME=$(get_single_resource_name "notebook")
if [ -z "$NOTEBOOK_NAME" ]; then
echo "Error: Multiple or no Notebook instances found. Specify the --notebook parameter."
exit 1
fi
fi

if [ -z "$FEAST_NAME" ]; then
FEAST_NAME=$(get_single_resource_name "feast")
if [ -z "$FEAST_NAME" ]; then
echo "Error: Multiple or no Feast instances found. Specify the --feast parameter."
exit 1
fi
fi

echo "Notebook Name: $NOTEBOOK_NAME"
echo "Feast Name: $FEAST_NAME"
echo "ConfigMap Mount Path: $CM_MOUNT_PATH"

echo "Connecting Notebook '$NOTEBOOK_NAME' with Feast '$FEAST_NAME'..."

client_configmap_name="feast-${FEAST_NAME}-client"
patch_config_map $NOTEBOOK_NAME $client_configmap_name $CM_MOUNT_PATH

offline_tls_mount_path=$(get_tls_secret_name "offline_store" $client_configmap_name)
online_tls_mount_path=$(get_tls_secret_name "online_store" $client_configmap_name)
registry_tls_mount_path=$(get_tls_secret_name "registry" $client_configmap_name)
echo "Offline TLS Mount Path: $offline_tls_secret_name"
echo "Online TLS Mount Path: $online_tls_mount_path"
echo "Registry TLS Mount Path: $registry_tls_mount_path"

if [ "$offline_tls_mount_path" != "." ]; then
secret_name="feast-${FEAST_NAME}-offline-tls"
patch_tls_secret $NOTEBOOK_NAME $secret_name ${offline_tls_mount_path}
fi
if [ "$online_tls_mount_path" != "." ]; then
secret_name="feast-${FEAST_NAME}-online-tls"
patch_tls_secret $NOTEBOOK_NAME $secret_name ${online_tls_mount_path}
fi
if [ "$registry_tls_mount_path" != "." ]; then
secret_name="feast-${FEAST_NAME}-registry-tls"
patch_tls_secret $NOTEBOOK_NAME $secret_name ${registry_tls_mount_path}
fi




Loading
Loading