-
Notifications
You must be signed in to change notification settings - Fork 880
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not crash if an empty YAML file is encrypted #1290
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wondering if it wouldn't be better to abort the operation if len(branches) <= 0
, instead of continuing to execute a variety of code paths.
In addition, wondering if this is the right place to address this issue. As it appears to be YAML specific; what do the other stores do when they run into an empty document? If they e.g. error out, could this behavior then be replicated within the YAML store?
YAML is different from other file formats such JSON, INI etc. in that the other file formats always contain exactly one document, so an empty JSON/INI/... results in one empty branch. But an empty YAML file contains no document (https://yaml.org/spec/1.2.2/#92-streams), so you get zero branches. This is different from a YAML file which contains a single empty document. The following are examples of a YAML file with a single empty document: --- or {} So we could either simply not support zero-document YAML files, or we convert them to single empty document YAML files (which is kind of wrong), or we allow to 'encrypt' them (which is what the PR right now does - the result cannot be decrypted though, though the decryption code could also be adjusted to convert an empty YAML document back to an empty YAML document). |
Think having the encrypt and decrypt command handle empty documents gracefully is the best option to go with, as I think the behavior most users will be after is running encryption (or decryption) on a range of files expecting success (or an error, but not because the document is empty). In addition, I think that changing it to |
Hmm, while looking at how to implement that, I found some more problems:
For the first three points, it might be easier to disallow empty inputs completely. What do you prefer (given the above points)? |
With "disallow", you are aiming at erroring out? |
Yes. Which would be similar to the current behavior, except that the message is a nice text and not a stacktrace :-) |
(It would still be possible to change this at some point later in time, as a new feature. But at least the crash would go away for now.) |
This sounds good to me, we can ask the community for more feedback and use-cases afterwards. |
I completely changed the implementation. There's now a new error which is used when a completely empty document is provided for encryption. |
This only affects empty YAML files, since only these can contain zero documents. Signed-off-by: Felix Fontein <[email protected]>
@hiddeco thanks for reviewing this one as well! |
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/getsops/sops/v3](https://togithub.com/getsops/sops) | require | patch | `v3.8.0` -> `v3.8.1` | --- ### Release Notes <details> <summary>getsops/sops (github.com/getsops/sops/v3)</summary> ### [`v3.8.1`](https://togithub.com/getsops/sops/releases/tag/v3.8.1) [Compare Source](https://togithub.com/getsops/sops/compare/v3.8.0...v3.8.1) #### Note from the Maintainers In this release of SOPS, we have focused on landing a variety of bug fixes to improve the overall user experience. For a comprehensive list of changes, please refer to [`CHANGELOG.rst`](https://togithub.com/getsops/sops/blob/v3.8.1/CHANGELOG.rst). ##### Important Information for SOPS SDK Users With the project transitioning from the Mozilla Foundation to the CNCF, the Go module path has been updated to reflect this change in ownership. If you use `go.mozilla.org/sops/v3` as a library, going forward, import the Go Module using `github.com/getsops/sops/v3`. Apart from this small adjustment, the SDK's API remains fully backward compatible. For a one-liner to quickly implement this change throughout your codebase, please refer to: [https://github.com/getsops/sops/issues/1246#issuecomment-1625526429](https://togithub.com/getsops/sops/issues/1246#issuecomment-1625526429) #### Installation To install `sops`, download one of the pre-built binaries provided for your platform from the artifacts attached to this release. For instance, if you are using Linux on an AMD64 architecture: ```shell ### Download the binary curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.linux.amd64 ### Move the binary in to your PATH mv sops-v3.8.1.linux.amd64 /usr/local/bin/sops ### Make the binary executable chmod +x /usr/local/bin/sops ``` ##### Verify checksums file signature The checksums file provided within the artifacts attached to this release is signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of this file, run the following commands: ```shell ### Download the checksums file, certificate and signature curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.checksums.txt curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.checksums.pem curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.checksums.sig ### Verify the checksums file cosign verify-blob sops-v3.8.1.checksums.txt \ --certificate sops-v3.8.1.checksums.pem \ --signature sops-v3.8.1.checksums.sig \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com ``` ##### Verify binary integrity To verify the integrity of the downloaded binary, you can utilize the checksums file after having validated its signature: ```shell ### Verify the binary using the checksums file sha256sum -c sops-v3.8.1.checksums.txt --ignore-missing ``` ##### Verify artifact provenance The [SLSA provenance](https://slsa.dev/provenance/v0.2) of the binaries, packages, and SBOMs can be found within the artifacts associated with this release. It is presented through an [in-toto](https://in-toto.io/) link metadata file named `sops-v3.8.1.intoto.jsonl`. To verify the provenance of an artifact, you can utilize the [`slsa-verifier`](https://togithub.com/slsa-framework/slsa-verifier#artifacts) tool: ```shell ### Download the metadata file curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.intoto.jsonl ### Verify the provenance of the artifact slsa-verifier verify-artifact <artifact> \ --provenance-path sops-v3.8.1.intoto.jsonl \ --source-uri github.com/getsops/sops \ --source-tag v3.8.1 ``` #### Container Images The `sops` binaries are also available as container images, based on Debian (slim) and Alpine Linux. The Debian-based container images include any dependencies which may be required to make use of certain key services, such as GnuPG, AWS KMS, Azure Key Vault, and Google Cloud KMS. The Alpine-based container images are smaller in size, but do not include these dependencies. These container images are available for the following architectures: `linux/amd64` and `linux/arm64`. ##### GitHub Container Registry - `ghcr.io/getsops/sops:v3.8.1` - `ghcr.io/getsops/sops:v3.8.1-alpine` ##### Quay.io - `quay.io/getsops/sops:v3.8.1` - `quay.io/getsops/sops:v3.8.1-alpine` ##### Verify container image signature The container images are signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of an image, run the following command: ```shell cosign verify ghcr.io/getsops/sops:v3.8.1 \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com \ -o text ``` ##### Verify container image provenance The container images include [SLSA provenance](https://slsa.dev/provenance/v0.2) attestations. For more information around the verification of this, please refer to the [`slsa-verifier` documentation](https://togithub.com/slsa-framework/slsa-verifier#containers). #### Software Bill of Materials The Software Bill of Materials (SBOM) for each binary is accessible within the artifacts enclosed with this release. It is presented as an [SPDX](https://spdx.dev/) JSON file, formatted as `<binary>.spdx.sbom.json`. #### What's Changed - build(deps): Bump the ci group with 3 updates by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/getsops/sops/pull/1295](https://togithub.com/getsops/sops/pull/1295) - pgp: improve handling of GnuPG home dir by [@​hiddeco](https://togithub.com/hiddeco) in [https://github.com/getsops/sops/pull/1298](https://togithub.com/getsops/sops/pull/1298) - Delete sops encrypted file we don't have keys for by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1288](https://togithub.com/getsops/sops/pull/1288) - Improve handling of errors when binary store handles bad data by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1289](https://togithub.com/getsops/sops/pull/1289) - On macOS, prefer XDG_CONFIG_HOME over os.UserConfigDir() by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1291](https://togithub.com/getsops/sops/pull/1291) - Do not crash if an empty YAML file is encrypted by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1290](https://togithub.com/getsops/sops/pull/1290) - Fix descriptions of unencrypted-regex and encrypted-regex flags, and ensure unencrypted_regex is considered in config validation by [@​mitar](https://togithub.com/mitar) in [https://github.com/getsops/sops/pull/1300](https://togithub.com/getsops/sops/pull/1300) - build(deps): Bump the go group with 4 updates by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/getsops/sops/pull/1306](https://togithub.com/getsops/sops/pull/1306) - build(deps): Bump the ci group with 1 update by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/getsops/sops/pull/1301](https://togithub.com/getsops/sops/pull/1301) - Handle return values of dec.Token() to improve error messages by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1307](https://togithub.com/getsops/sops/pull/1307) - pgp: make error the last return value by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1310](https://togithub.com/getsops/sops/pull/1310) - pgp: do not require abs path for SopsGpgExecEnv by [@​holiman](https://togithub.com/holiman) in [https://github.com/getsops/sops/pull/1309](https://togithub.com/getsops/sops/pull/1309) - decrypt: fix dropped error by [@​alrs](https://togithub.com/alrs) in [https://github.com/getsops/sops/pull/1304](https://togithub.com/getsops/sops/pull/1304) - Handle errors by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1311](https://togithub.com/getsops/sops/pull/1311) - Report key rotation errors by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1317](https://togithub.com/getsops/sops/pull/1317) - cmd/sops/main.go: make sure to wrap raw errors with toExitError() by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1318](https://togithub.com/getsops/sops/pull/1318) - build(deps): Bump the go group with 7 updates by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/getsops/sops/pull/1319](https://togithub.com/getsops/sops/pull/1319) - Enrich AWS authentication documentation by [@​nsantiago2719](https://togithub.com/nsantiago2719) in [https://github.com/getsops/sops/pull/1272](https://togithub.com/getsops/sops/pull/1272) - Better error reporting for missing gpg binary by [@​makkes](https://togithub.com/makkes) in [https://github.com/getsops/sops/pull/1286](https://togithub.com/getsops/sops/pull/1286) - Improve RST and MD files by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1320](https://togithub.com/getsops/sops/pull/1320) - Add linting for RST and MD files by [@​felixfontein](https://togithub.com/felixfontein) in [https://github.com/getsops/sops/pull/1287](https://togithub.com/getsops/sops/pull/1287) - Update dependencies by [@​hiddeco](https://togithub.com/hiddeco) in [https://github.com/getsops/sops/pull/1325](https://togithub.com/getsops/sops/pull/1325) - Prepare v3.8.1 by [@​hiddeco](https://togithub.com/hiddeco) in [https://github.com/getsops/sops/pull/1324](https://togithub.com/getsops/sops/pull/1324) #### New Contributors - [@​mitar](https://togithub.com/mitar) made their first contribution in [https://github.com/getsops/sops/pull/1300](https://togithub.com/getsops/sops/pull/1300) - [@​holiman](https://togithub.com/holiman) made their first contribution in [https://github.com/getsops/sops/pull/1309](https://togithub.com/getsops/sops/pull/1309) - [@​alrs](https://togithub.com/alrs) made their first contribution in [https://github.com/getsops/sops/pull/1304](https://togithub.com/getsops/sops/pull/1304) - [@​nsantiago2719](https://togithub.com/nsantiago2719) made their first contribution in [https://github.com/getsops/sops/pull/1272](https://togithub.com/getsops/sops/pull/1272) - [@​makkes](https://togithub.com/makkes) made their first contribution in [https://github.com/getsops/sops/pull/1286](https://togithub.com/getsops/sops/pull/1286) **Full Changelog**: getsops/sops@v3.8.0...v3.8.1 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/markussiebert/cdk-sops-secrets).
An empty YAML file results in no tree branch, which trips over encrypt since it assumes there's at least one.
I'm not sure this is the correct way to fix this, since the encrypted result will be empty as well (zero tree branches).
On the other hand, if you encrypt a YAML file with
{}
or---
in it (and nothing else), there will be at least one tree branch. Decrypting the resulting file yields{}
. Decrypting an empty file yields ansops metadata not found
error.