Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move sparkctl to cmd directory #2347

Merged
merged 3 commits into from
Dec 6, 2024
Merged

Move sparkctl to cmd directory #2347

merged 3 commits into from
Dec 6, 2024

Conversation

ChenYi015
Copy link
Contributor

@ChenYi015 ChenYi015 commented Dec 5, 2024 •
edited
Loading

Purpose of this PR

Proposed changes:

  • Move sparkctl to cmd/sparkctl

Change Category

  • Bugfix (non-breaking change which fixes an issue)
  • Feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that could affect existing functionality)
  • Documentation update

Rationale

Checklist

  • I have conducted a self-review of my own code.
  • I have updated documentation accordingly.
  • I have added tests that prove my changes are effective or that my feature works.
  • Existing unit tests pass locally with my changes.

Additional Notes

@ChenYi015
Copy link
Contributor Author

/assign @jacobsalway @ImpSy

jacobsalway
jacobsalway reviewed Dec 5, 2024
View reviewed changes
cmd/operator/main.go Outdated Show resolved Hide resolved
jacobsalway
jacobsalway approved these changes Dec 6, 2024
View reviewed changes
Copy link
Member

@jacobsalway jacobsalway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@google-oss-prow google-oss-prow bot added the lgtm label Dec 6, 2024
@jacobsalway
Copy link
Member

@ChenYi015 you'll have to self-approve for the merge automation to pass. Alternatively if you're open to it we could promote myself or @ImpSy to contributor?

@ChenYi015
Copy link
Contributor Author

@ChenYi015 you'll have to self-approve for the merge automation to pass. Alternatively if you're open to it we could promote myself or @ImpSy to contributor?

@jacobsalway Yes, I agree to promote you as an approver so that we can merge PRs more easily. We can discuss this during the next community meeting call with the other maintainers.

@ChenYi015
Copy link
Contributor Author

/approve

@google-oss-prow Google OSS Prow
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ChenYi015, jacobsalway

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 2375a30 into kubeflow:master Dec 6, 2024
11 checks passed
@ChenYi015 ChenYi015 deleted the refactor/sparkctl branch December 6, 2024 05:20
ChenYi015 added a commit to ChenYi015/spark-operator that referenced this pull request Dec 10, 2024
@ChenYi015
Move sparkctl to cmd directory (kubeflow#2347)
b4e653d 
* Move spark-operator

Signed-off-by: Yi Chen <[email protected]>

* Move sparkctl to cmd directory

Signed-off-by: Yi Chen <[email protected]>

* Remove unnecessary app package/directory

Signed-off-by: Yi Chen <[email protected]>

---------

Signed-off-by: Yi Chen <[email protected]>
(cherry picked from commit 2375a30)
@ChenYi015 ChenYi015 mentioned this pull request Dec 10, 2024
Merged
google-oss-prow bot pushed a commit that referenced this pull request Dec 11, 2024
@ChenYi015 @Aransh
@npgretz @Cian911 @jacobsalway @dependabot @c-h-afzal @Tom-Newton @Aakcht
Release v2.1.0 (#2354)
664b9d0 
* Allow setting automountServiceAccountToken (#2298)

* Allow setting automountServiceAccountToken on workloads and serviceAccounts

Signed-off-by: Aran Shavit <[email protected]>

* update helm docs

Signed-off-by: Aran Shavit <[email protected]>

---------

Signed-off-by: Aran Shavit <[email protected]>
(cherry picked from commit 515d805)

* Fix: executor container security context does not work (#2306)

Signed-off-by: Yi Chen <[email protected]>
(cherry picked from commit 171e429)

* Fix: should not add emptyDir sizeLimit conf if it is nil (#2305)

Signed-off-by: Yi Chen <[email protected]>
(cherry picked from commit 763682d)

* Allow the Controller and Webhook Containers to run with the securityContext: readOnlyRootfilesystem: true (#2282)

* create a tmp dir for the controller to write Spark artifacts to and set the controller to readOnlyRootFilesystem

Signed-off-by: Nick Gretzon <[email protected]>

* mount a dir for the webhook container to generate its certificates in and set readOnlyRootFilesystem: true for the webhook pod

Signed-off-by: Nick Gretzon <[email protected]>

* update the securityContext in the controller deployment test

Signed-off-by: Nick Gretzon <[email protected]>

* update securityContext of the webhook container in the deployment_test

Signed-off-by: Nick Gretzon <[email protected]>

* update README

Signed-off-by: Nick Gretzon <[email protected]>

* remove -- so comments are not rendered in the README.md

Signed-off-by: Nick Gretzon <[email protected]>

* recreate README.md after removal of comments for volumes and volumeMounts

Signed-off-by: Nick Gretzon <[email protected]>

* make indentation for volumes and volumeMounts consistent with rest of values.yaml

Signed-off-by: Nick Gretzon <[email protected]>

* Revert "make indentation for volumes and volumeMounts consistent with rest of values.yaml"

This reverts commit dba97fc.

Signed-off-by: Nick Gretzon <[email protected]>

* fix indentation in webhook and controller deployment templates for volumes and volumeMounts

Signed-off-by: Nick Gretzon <[email protected]>

* Update charts/spark-operator-chart/values.yaml

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>

* Update charts/spark-operator-chart/values.yaml

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>

* Update charts/spark-operator-chart/values.yaml

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>

* Update charts/spark-operator-chart/values.yaml

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>

* Update charts/spark-operator-chart/templates/controller/deployment.yaml

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>

* Update charts/spark-operator-chart/templates/controller/deployment.yaml

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>

* Update charts/spark-operator-chart/templates/webhook/deployment.yaml

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>

* Update charts/spark-operator-chart/templates/webhook/deployment.yaml

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>

* add additional securityContext to the controller deployment_test.yaml

Signed-off-by: Nick Gretzon <[email protected]>

---------

Signed-off-by: Nick Gretzon <[email protected]>
Signed-off-by: Nicholas Gretzon <[email protected]>
Co-authored-by: Yi Chen <[email protected]>
(cherry picked from commit 72107fd)

* Fix: should not add emptyDir sizeLimit conf on executor pods if it is nil (#2316)

Signed-off-by: Cian Gallagher <[email protected]>
(cherry picked from commit 2999546)

* Bump `volcano.sh/apis` to 1.10.0 (#2320)

Signed-off-by: Jacob Salway <[email protected]>
(cherry picked from commit 22e4fb8)

* Truncate UI service name if over 63 characters (#2311)

* Truncate UI service name if over 63 characters

Signed-off-by: Jacob Salway <[email protected]>

* Also truncate ingress name

Signed-off-by: Jacob Salway <[email protected]>

---------

Signed-off-by: Jacob Salway <[email protected]>
(cherry picked from commit 43c1888)

* Bump aquasecurity/trivy-action from 0.28.0 to 0.29.0 (#2332)

Bumps [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](aquasecurity/trivy-action@0.28.0...0.29.0)

---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 270b09e)

* Bump github.com/onsi/ginkgo/v2 from 2.20.2 to 2.22.0 (#2335)

Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.20.2 to 2.22.0.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](onsi/ginkgo@v2.20.2...v2.22.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 40423d5)

* The webhook-key-name command-line param isn't taking effect (#2344)

Signed-off-by: C. H. Afzal <[email protected]>
(cherry picked from commit a261523)

* Robustness to driver pod taking time to create (#2315)

* Retry after driver pod now found if recent submission

Signed-off-by: Thomas Newton <[email protected]>

* Add a test

Signed-off-by: Thomas Newton <[email protected]>

* Make grace period configurable

Signed-off-by: Thomas Newton <[email protected]>

* Update test

Signed-off-by: Thomas Newton <[email protected]>

* Add an extra test with the driver pod

Signed-off-by: Thomas Newton <[email protected]>

* Separate context to create and delete the driver pod

Signed-off-by: Thomas Newton <[email protected]>

* Tidy

Signed-off-by: Thomas Newton <[email protected]>

* Autoformat

Signed-off-by: Thomas Newton <[email protected]>

* Update error message

Signed-off-by: Thomas Newton <[email protected]>

* Add helm paramater

Signed-off-by: Thomas Newton <[email protected]>

* Update internal/controller/sparkapplication/controller.go

Co-authored-by: Yi Chen <[email protected]>
Signed-off-by: Thomas Newton <[email protected]>

* Newlines between helm tests

Signed-off-by: Thomas Newton <[email protected]>

---------

Signed-off-by: Thomas Newton <[email protected]>
Co-authored-by: Yi Chen <[email protected]>
(cherry picked from commit d815e78)

* Use NSS_WRAPPER_PASSWD instead of /etc/passwd as in spark-operator image entrypoint.sh (#2312)

Signed-off-by: Aakcht <[email protected]>
(cherry picked from commit 5dd91c4)

* Move sparkctl to cmd directory (#2347)

* Move spark-operator

Signed-off-by: Yi Chen <[email protected]>

* Move sparkctl to cmd directory

Signed-off-by: Yi Chen <[email protected]>

* Remove unnecessary app package/directory

Signed-off-by: Yi Chen <[email protected]>

---------

Signed-off-by: Yi Chen <[email protected]>
(cherry picked from commit 2375a30)

* Spark Operator Official Release v2.1.0

Signed-off-by: Yi Chen <[email protected]>

---------

Signed-off-by: Yi Chen <[email protected]>
Co-authored-by: Aran Shavit <[email protected]>
Co-authored-by: Nicholas Gretzon <[email protected]>
Co-authored-by: Cian (Keen) Gallagher <[email protected]>
Co-authored-by: Jacob Salway <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: C. H. Afzal <[email protected]>
Co-authored-by: Thomas Newton <[email protected]>
Co-authored-by: Aakcht <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jacobsalway jacobsalway jacobsalway approved these changes

@ImpSy ImpSy Awaiting requested review from ImpSy

Assignees

@ImpSy ImpSy

@jacobsalway jacobsalway

Labels
approved lgtm size/M
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ChenYi015 @jacobsalway @ImpSy