Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement secrets pkg: backend and filedriver #375

Merged
merged 1 commit into from
Jan 6, 2021

Conversation

ashley-cui
Copy link
Member

This is the implementation of the backend of secrets. pkg/secrets takes a secret name and data and does operations on that secret data, most namely store, delete, lookup, and list, using a secretsmanager.

The first driver implemented here is a filedriver - where the data is stored unencrypted on disk in a file.

The secrets package can be easily expanded to use more drivers as the package implements an interface to accept different drivers

Related: containers/podman#3264, containers/buildah#1684

Signed-off-by: Ashley Cui [email protected]

@rhatdan
Copy link
Member

rhatdan commented Dec 8, 2020

@vrothberg @umohnani8 @mheon PTAL

@rhatdan
Copy link
Member

rhatdan commented Dec 8, 2020

Very nice work.

Should the file driver have an encryption options, perhaps defaulted to None?
Then we could add encryption types in the future?

Do you have a PR in Podman that uses this, to show how Secrets would work in the wild?

Copy link
Member

@vrothberg vrothberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, @ashley-cui!

I am still torn regarding the driver but gravitate towards always specifying a driver in (SecretsManager).Store(name, data, driver). This way, it's easy to add a --driver flag to the CLI that allows to use custom secrets drivers.

I assume that docker has some pre-defined API/protocol to interact with those drivers. If that's the case, secrets drivers could be configured in the containers.conf as a map[string]string:

[secretdrivers]
"driver-a"="/usr/libexec/secretdriver-a`
"..."="..."

pkg/secrets/secrets.lock should be deleted.

pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secrets.go Show resolved Hide resolved
pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secretsdb.go Outdated Show resolved Hide resolved
@rhatdan
Copy link
Member

rhatdan commented Dec 15, 2020

/approve

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ashley-cui, rhatdan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secrets_test.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver_test.go Outdated Show resolved Hide resolved
pkg/secrets/secrets.go Outdated Show resolved Hide resolved
@ashley-cui
Copy link
Member Author

Updated. The bug that was haunting me ended up being a simple using the wrong variable bug 🤦

pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver.go Outdated Show resolved Hide resolved
pkg/secrets/filedriver/filedriver_test.go Outdated Show resolved Hide resolved
pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secretsdb.go Outdated Show resolved Hide resolved
pkg/secrets/secretsdb.go Outdated Show resolved Hide resolved
pkg/secrets/secretsdb.go Outdated Show resolved Hide resolved
pkg/secrets/secretsdb.go Outdated Show resolved Hide resolved
pkg/secrets/secretsdb.go Outdated Show resolved Hide resolved
@ashley-cui ashley-cui force-pushed the secrets branch 2 times, most recently from 9cf854b to b0297e8 Compare January 4, 2021 20:16
Copy link
Member

@vrothberg vrothberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final nits from me. Really nice work, @ashley-cui!

Let's get that in, and we can tackle Podman :) We're free to change things in the future, if we need to.

pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secrets.go Outdated Show resolved Hide resolved
pkg/secrets/secrets_test.go Outdated Show resolved Hide resolved
pkg/secrets/secretsdb.go Outdated Show resolved Hide resolved
@ashley-cui
Copy link
Member Author

@vrothberg Thanks for all the thorough reviews :)

This is the implementation of the backend of secrets. pkg/secrets takes a secret name and data and does these operations on that secret data:  store, delete, lookup, and list, using a secretsmanager.

The first driver implemented here is a filedriver - where the data is stored unencrypted on disk in a file.

The secrets package can be easily expanded to use more drivers as the package implements an interface to accept different drivers

Signed-off-by: Ashley Cui <[email protected]>
const secretIDLength = 25

// errInvalidPath indicates that the secrets path is invalid
var errInvalidPath = errors.New("invalid secrets path")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I ask why these aren't public? We usually do public errors in Podman-land so that errors.Cause() can be used to easily identify what went wrong.

Copy link
Member Author

@ashley-cui ashley-cui Jan 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#375 (comment)

I can definitely make them public again if that's the right move

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't. It would be very confusing to expose that to users as they cannot really treat the errors in a special way. If we need some in the future, we can selectively make them public.

@ashley-cui ashley-cui force-pushed the secrets branch 3 times, most recently from 13ac506 to d7d9efa Compare January 5, 2021 15:06
@ashley-cui
Copy link
Member Author

@containers/podman-maintainers PTAL

Copy link
Member

@vrothberg vrothberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rhatdan
Copy link
Member

rhatdan commented Jan 6, 2021

Nice job @ashley-cui
/lgtm

@openshift-merge-robot openshift-merge-robot merged commit 969de12 into containers:master Jan 6, 2021
@ashley-cui ashley-cui deleted the secrets branch June 29, 2021 21:15
M1cha pushed a commit to M1cha/common that referenced this pull request Dec 20, 2022
aardvark,commit: acquire fs lock when performing commit to avoid `race` across parallel invocations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants