Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Operator support for spec.feastProject & status.applied fields #4656

Merged
merged 1 commit into from
Oct 29, 2024

Conversation

tchughesiv
Copy link
Contributor

@tchughesiv tchughesiv commented Oct 20, 2024

What this PR does / why we need it:

Initial Operator functionality. Adds ability to provision a feast registry w/ ephemeral sqlite file-based DB. Also adds key fields to the FeatureStore.status... including conditions, feastVersion, serviceUrls, clientConfigMap, phase, & applied spec. With this change the operator will create the following objects for each FeatureStore CR -

  • Registry Deployment
  • Registry Service
  • Client ConfigMap w/ feature_store.yaml configured for remote registry access

Which issue(s) this PR fixes:

relates to #4561

Misc

% oc get feast   
NAME      STATUS   AGE
example   Ready    15h
$ oc get feast example -oyaml

apiVersion: feast.dev/v1alpha1
kind: FeatureStore
metadata:
  creationTimestamp: "2024-10-20T23:41:29Z"
  generation: 1
  name: example
  namespace: feast-operator-system
  resourceVersion: "855212"
  uid: 48f0d556-3778-4288-88ac-9a1d846ae30c
spec:
  feastProject: test
status:
  applied:
    feastProject: test
  clientConfigMap: feast-example-client
  conditions:
  - lastTransitionTime: "2024-10-20T23:41:29Z"
    message: Registry installation complete
    reason: Ready
    status: "True"
    type: Registry
  - lastTransitionTime: "2024-10-20T23:41:29Z"
    message: Client installation complete
    reason: Ready
    status: "True"
    type: Client
  - lastTransitionTime: "2024-10-20T23:41:29Z"
    message: FeatureStore installation complete
    reason: Ready
    status: "True"
    type: FeatureStore
  feastVersion: 0.40.0
  phase: Ready
  serviceUrls:
    registry: feast-example-registry.feast-operator-system.svc.cluster.local:80
$ oc get all -l "feast.dev/name=example"

NAME                                          READY   STATUS    RESTARTS   AGE
pod/feast-example-registry-86b9c8f6c4-pkztz   1/1     Running   0          78s

NAME                             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/feast-example-registry   ClusterIP   172.30.224.170   <none>        80/TCP    78s

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/feast-example-registry   1/1     1            1           78s

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/feast-example-registry-86b9c8f6c4   1         1         1       78s
$ oc get configmap feast-example-client -oyaml

apiVersion: v1
data:
  feature_store.yaml: |
    project: test
    provider: local
    registry:
        path: feast-example-registry.feast-operator-system.svc.cluster.local:80
        registry_type: remote
    entity_key_serialization_version: 3
kind: ConfigMap
metadata:
  creationTimestamp: "2024-10-20T23:41:29Z"
  labels:
    feast.dev/name: example
    feast.dev/service-type: client
  name: feast-example-client
  namespace: feast-operator-system
  ownerReferences:
  - apiVersion: feast.dev/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: FeatureStore
    name: example
    uid: 48f0d556-3778-4288-88ac-9a1d846ae30c
  resourceVersion: "760628"
  uid: a9e962ef-0472-4c61-9650-8111620ad5eb

@tchughesiv tchughesiv force-pushed the operator-controller-new branch 2 times, most recently from d2139dc to ae33a32 Compare October 21, 2024 14:50
@tchughesiv tchughesiv marked this pull request as ready for review October 21, 2024 14:50
@tchughesiv tchughesiv force-pushed the operator-controller-new branch 2 times, most recently from 02892b3 to db56abc Compare October 21, 2024 15:02
Copy link
Contributor

@dmartinol dmartinol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@franciscojavierarceo
Copy link
Member

franciscojavierarceo commented Oct 22, 2024

@tchughesiv is this ready for review?

@tchughesiv
Copy link
Contributor Author

@franciscojavierarceo yes it is

@tchughesiv tchughesiv force-pushed the operator-controller-new branch from ba5f275 to 14bf924 Compare October 28, 2024 16:26
@tchughesiv tchughesiv requested a review from a team as a code owner October 28, 2024 16:26
@tchughesiv tchughesiv force-pushed the operator-controller-new branch from 14bf924 to 320b559 Compare October 28, 2024 16:28
Copy link
Contributor

@redhatHameed redhatHameed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

ClientFailedReason = "ClientDeploymentFailed"

// Feast condition messages:
ReadyMessage = "FeatureStore installation complete"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe change to Feast installation complete

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, if the kind is FeatureStore, it was not that wrong. Anyway, it's fine. Can I merge it @tchughesiv?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree actually... made more sense the other way. I've reverted that last commit... and yes, it can be merged!

Copy link
Contributor

@dmartinol dmartinol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@tchughesiv tchughesiv force-pushed the operator-controller-new branch from 96f7b55 to 320b559 Compare October 29, 2024 11:25
@dmartinol dmartinol merged commit 430ac53 into feast-dev:master Oct 29, 2024
39 of 40 checks passed
lokeshrangineni pushed a commit to lokeshrangineni/feast that referenced this pull request Oct 29, 2024
…lds (feast-dev#4656)

spec.feastProject & status.applied logic

Signed-off-by: Tommy Hughes <[email protected]>
@tchughesiv tchughesiv mentioned this pull request Nov 11, 2024
franciscojavierarceo pushed a commit that referenced this pull request Dec 5, 2024
# [0.42.0](v0.41.0...v0.42.0) (2024-12-05)

### Bug Fixes

* Add adapters for sqlite datetime conversion ([#4797](#4797)) ([e198b17](e198b17))
* Added grpcio extras to default feature-server image ([#4737](#4737)) ([e9cd373](e9cd373))
* Changing node version in release ([7089918](7089918))
* Feast create empty online table when FeatureView attribute online=False ([#4666](#4666)) ([237c453](237c453))
* Fix db store types in Operator CRD ([#4798](#4798)) ([f09339e](f09339e))
* Fix the config issue for postgres ([#4776](#4776)) ([a36f7e5](a36f7e5))
* Fixed example materialize-incremental and improved explanation ([#4734](#4734)) ([ca8a7ab](ca8a7ab))
* Fixed SparkSource docstrings so it wouldn't used inhereted class docstrings ([#4722](#4722)) ([32e6aa1](32e6aa1))
* Fixing PGVector integration tests ([#4778](#4778)) ([88a0320](88a0320))
* Incorrect type passed to assert_permissions in materialize endpoints ([#4727](#4727)) ([b72c2da](b72c2da))
* Issue of DataSource subclasses using parent abstract class docstrings ([#4730](#4730)) ([b24acd5](b24acd5))
* Operator envVar positioning & tls.SecretRef.Name ([#4806](#4806)) ([1115d96](1115d96))
* Populates project created_time correctly according to created ti… ([#4686](#4686)) ([a61b93c](a61b93c))
* Reduce feast-server container image size & fix dev image build ([#4781](#4781)) ([ccc9aea](ccc9aea))
* Removed version func from feature_store.py ([#4748](#4748)) ([f902bb9](f902bb9))
* Support registry instantiation for read-only users ([#4719](#4719)) ([ca3d3c8](ca3d3c8))
* Syntax Error in BigQuery While Retrieving Columns that Start wit… ([#4713](#4713)) ([60fbc62](60fbc62))
* Update release version in a pertinent Operator file ([#4708](#4708)) ([764a8a6](764a8a6))

### Features

* Add api contract to fastapi docs ([#4721](#4721)) ([1a165c7](1a165c7))
* Add Couchbase as an online store ([#4637](#4637)) ([824859b](824859b))
* Add Operator support for spec.feastProject & status.applied fields ([#4656](#4656)) ([430ac53](430ac53))
* Add services functionality to Operator ([#4723](#4723)) ([d1d80c0](d1d80c0))
* Add TLS support to the Operator ([#4796](#4796)) ([a617a6c](a617a6c))
* Added feast Go operator db stores support ([#4771](#4771)) ([3302363](3302363))
* Added support for setting env vars in feast services in feast controller  ([#4739](#4739)) ([84b24b5](84b24b5))
* Adding docs outlining native Python transformations on singletons ([#4741](#4741)) ([0150278](0150278))
* Adding first feast operator e2e test. ([#4791](#4791)) ([8339f8d](8339f8d))
* Adding github action to run the operator end-to-end tests. ([#4762](#4762)) ([d8ccb00](d8ccb00))
* Adding ssl support for registry server. ([#4718](#4718)) ([ccf7a55](ccf7a55))
* Adding SSL support for the React UI server and feast UI command. ([#4736](#4736)) ([4a89252](4a89252))
* Adding support for native Python transformations on a single dictionary  ([#4724](#4724)) ([9bbc1c6](9bbc1c6))
* Adding TLS support for offline server. ([#4744](#4744)) ([5d8d03f](5d8d03f))
* Building the feast image ([#4775](#4775)) ([6635dde](6635dde))
* File persistence definition and implementation ([#4742](#4742)) ([3bad4a1](3bad4a1))
* Object store persistence in operator ([#4758](#4758)) ([0ae86da](0ae86da))
* OIDC authorization in Feast Operator ([#4801](#4801)) ([eb111d6](eb111d6))
* Operator will create k8s serviceaccount for each feast service ([#4767](#4767)) ([cde5760](cde5760))
* Printing more verbose logs when we start the offline server  ([#4660](#4660)) ([9d8d3d8](9d8d3d8))
* PVC configuration and impl ([#4750](#4750)) ([785a190](785a190))
* Qdrant vectorstore support ([#4689](#4689)) ([86573d2](86573d2))
* RBAC Authorization in Feast Operator ([#4786](#4786)) ([0ef5acc](0ef5acc))
* Support for nested timestamp fields in Spark Offline store ([#4740](#4740)) ([d4d94f8](d4d94f8))
* Update the go feature server from Expedia code repo. ([#4665](#4665)) ([6406625](6406625))
* Updated feast Go operator db stores ([#4809](#4809)) ([2c5a6b5](2c5a6b5))
* Updated sample secret following review ([#4811](#4811)) ([dc9f825](dc9f825))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants