feat: refactor certStore and KMP to support multi-tenancy [multi-tenancy PR 10]

[binbin-li]

Description

What this PR does / why we need it:

This is the 10th PR for multi-tenancy support. Please review #1422 first. Check diff between PR 9 and PR 10 at: binbin-li#141

1. Updated the implementation of pkg/customresources/certificatestores.go to support multi-tenancy and backward compatible. Moved prependNamespace logic to GetCertsFromStore method. Removed extra namespace mapping from ScopedCertStores, now the layout of certStore is similar to KMP.
2. Updated pkg/keymanagementprovider/keymanagementprovider.go to support multi-tenancy.
3. Removed getCertStoreNamespace from verifier_controller as we will not prepend namespace to certStore configs of Notation Verifiers.
4. Removed prependNamespaceToKMPName from cosign/trustpolicy.go as we will not prepend namespace to store configs of Cosign Verifiers.
5. Now the matching between namespace and resource(KMP/certStore) only exists in KMP or certStore implementation. Verifier will keep the original config from users since we'll have clustered and namespaced verifiers, no more assumption need to be made on the empty namespace. So notation verifier and cosign trustPolicy should not prepend namespace to the resource path but each resource should judge if the given scope has access to it.

User Bahavior

Since we don't have a cluster-wide CertStore CRD, default namespace will be reserved for cluster-wide CertStore usage.

Access level of verifier	Resource type	Resource name provided by user	expected behavior
cluster	certStore	cert1	return certs from gatekeeper-system/cert1 if it exists
cluster	certStore	namespace1/cert1	return certs from namespace1/cert1 if it exists
cluster	certStore	default/cert1	return certs from default/cert1 if it exists (default stands for a cluster-wide namespace)
cluster	kmp	kmp1	return certs from clustered kmp1 if it exists
cluster	kmp	namespace1/kmp1	cannot access namespaced resource
namespace1	certStore	cert1	cannot access gatekeeper-system/cert1
namespace1	certStore	namespace1/cert1	return certs from namespace1/cert1 if it exists
namespace1	certStore	default/cert1	return certs from default/cert1 if it exists
namespace1	kmp	kmp1	return certs from clustered kmp1 if it exists
namespace1	kmp	namespace1/kmp1	return certs from namespace1/kmp1 if it exists
namespace2	kmp	namespace1/kmp1	cannot access different namespace

We can notice that behavior is different between certStore and kmp. Because KMP is following the multi-tenancy design but certStore keep the previous behavior to avoid breaking change.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):

Fixes #

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Helm Chart Change (any edit/addition/update that is necessary for changes merged to the main branch)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

• Test A
• Test B

Checklist:

• Does the affected code have corresponding tests?
• Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
• Does this introduce breaking changes that would require an announcement or bumping the major version?
• Do all new files have appropriate license header?

Post Merge Requirements

• MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

[codecov]

Codecov Report

Attention: Patch coverage is 78.46154% with 14 lines in your changes are missing coverage. Please review.

Project coverage is 66.31%. Comparing base (1bd347c) to head (d0042d4).
Report is 3 commits behind head on dev.

❗ Current head d0042d4 differs from pull request most recent head 2474502. Consider uploading reports for the commit 2474502 to get more accurate results

Files	Patch %	Lines
...omresources/certificatestores/certificatestores.go	88.00%	2 Missing and 1 partial ⚠️
...s/namespaceresource/certificatestore_controller.go	50.00%	2 Missing ⚠️
pkg/keymanagementprovider/keymanagementprovider.go	88.23%	1 Missing and 1 partial ⚠️
pkg/manager/manager.go	0.00%	2 Missing ⚠️
pkg/verifier/cosign/trustpolicy.go	71.42%	2 Missing ⚠️
pkg/verifier/notation/truststore.go	75.00%	1 Missing and 1 partial ⚠️
pkg/controllers/verifier_controller.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1423      +/-   ##
==========================================
- Coverage   66.76%   66.31%   -0.45%     
==========================================
  Files         116      117       +1     
  Lines        6030     6089      +59     
==========================================
+ Hits         4026     4038      +12     
- Misses       1620     1668      +48     
+ Partials      384      383       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[akashsinghal]

nit (not blocking): I think this is out of date? "default" is not reserved right? It's the EmptyNamespace instead?

[binbin-li]

thanks for catching! I did update in my last batch but forgot to push it. Actually as we disuccsed last week, there should be no cluster-wide concept while users keep using certStore since certStore will always prefixed with the namespace (default if users don't provide it). And a cluster-wide verifier could always access certStores in all namespaces as v1.1 supports.

[susanshi]

left some clarification questions, thanks!

[susanshi]

do we still need the namespace var, ? are we moving local from verifier to certStore?

[binbin-li]

We won't need the namespace var since now. In the future, verifier will just keep the original configurations from users, like the ref to certStore/KMP. For the logics matching reference to certStore/KMP, it's moved to certStore/KMP implementation so that we could have different behavior on them. And it would be easier to manage it than splitting namespace fetching, namespace prefix and matching into different places.

[susanshi]

clarification , doe ctxUtils.GetNamespace(ctx) get the namespace of the verification request or the verifier.

[binbin-li]

It's fetching the namespace of the request, i.e. the namespace where image is deployed. I should update the comment to be more clear. Thanks for calling it out!

[akashsinghal]

lgtm