docs: add ratify v2 architecture design proposal

[binbin-li]

Description

What this PR does / why we need it:

This PR adds more details and addressed feedback from the previous discussion on the Ratify architecture design v2 proposal: #1942

We already have another PR discussing the v2 scope, this PR just focuses on the system architecture refactoring.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):

Fixes #1942

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Helm Chart Change (any edit/addition/update that is necessary for changes merged to the main branch)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

• Test A
• Test B

Checklist:

• Does the affected code have corresponding tests?
• Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
• Does this introduce breaking changes that would require an announcement or bumping the major version?
• Do all new files have appropriate license header?

Post Merge Requirements

• MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

see 4 files with indirect coverage changes

[susanshi]

maintainers, please review @FeynmanZhou, @akashsinghal , @susanshi

[FeynmanZhou]

/cc @DahuK for a review if you are interested

[DahuK]

/cc @DahuK for a review if you are interested

Thanks Feynman! As an adopter, we are very looking forward to the evolution of v2, especially Image Verification as an important feature of containerd 2.0. If ratify service can be adapted to the Image Transfer service, we will actively embrace such an evolution, and try to integrate this to provide a more flexible way of checking signatures and get rid of the dependency on k8s webhook. And I understand the importance of a loosely coupled and extensible architecture for the project, and will actively cooperate with the integration and transformation work under the v2 framework.

I can't give more suggestions on the implementation details, but we have received some feedback from the users. Here is my two cents: I know that image signing and verification itself is a complex scenario, which depends on the integration of multiple tools and services, and Ratify provides a variety of built-in CRD (store, keymanagementproviders, verifier), which makes it difficult for beginners to troubleshoot and find some common configuration errors, and when the artifacts has multiple signatures, it is also difficult to locate the specific verification report detail in addition to querying the ratify pod logs when encountering signature verification failures.

In the V2 version, is there any consideration for simplifying the cost of configuration? cause some users may only need to the basic verification of the image signature to complete the compliance requirements, it would be better if there were a simple minimal configuration set or best practice guidelines, is it possible for user to do all of the configurations in the v2 config.json with an easy way?

[binbin-li]

/cc @DahuK for a review if you are interested

Thanks Feynman! As an adopter, we are very looking forward to the evolution of v2, especially Image Verification as an important feature of containerd 2.0. If ratify service can be adapted to the Image Transfer service, we will actively embrace such an evolution, and try to integrate this to provide a more flexible way of checking signatures and get rid of the dependency on k8s webhook. And I understand the importance of a loosely coupled and extensible architecture for the project, and will actively cooperate with the integration and transformation work under the v2 framework.

I can't give more suggestions on the implementation details, but we have received some feedback from the users. Here is my two cents: I know that image signing and verification itself is a complex scenario, which depends on the integration of multiple tools and services, and Ratify provides a variety of built-in CRD (store, keymanagementproviders, verifier), which makes it difficult for beginners to troubleshoot and find some common configuration errors, and when the artifacts has multiple signatures, it is also difficult to locate the specific verification report detail in addition to querying the ratify pod logs when encountering signature verification failures.

In the V2 version, is there any consideration for simplifying the cost of configuration? cause some users may only need to the basic verification of the image signature to complete the compliance requirements, it would be better if there were a simple minimal configuration set or best practice guidelines, is it possible for user to do all of the configurations in the v2 config.json with an easy way?

thanks for the feedback and suggestion! Ratify v2 is designed to be more extensible to new scenarios including the containerD.
We have considered supporting this scenario before, and @akashsinghal already made a POC. We will allocate more resources to this feature after the V2 framework is complete. In the meantime, we also encourage everyone to contribute to these new features.

For the configuration refactoring, we have another PR referring to it. We also heard similar feedback on the setting up the configuration. Therefore, it's also in v2 scope to make it more straighforward to configure Ratify. And if you have any ideas on better user experience, feel free to leave comments.

As for troubleshooting, we also kept improving the user experience since v1, and will continue in v2. One possible improvement is to refactor the Policy Provider to make it return a more clear report when errors happen.

[susanshi]

PR looks great, having the diagram is very helpful Binbin, we should include them on the Ratify website! Left some questions.

[susanshi]

please merge once all conversation is resolved.