-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: add using include/exclude kinds and namespaces #395
base: main
Are you sure you want to change the base?
Conversation
FYI this might be related aquasecurity/trivy#7107 |
thanks. I saw it |
so today in order to run any scan, the user needs cluster permissions? if so, indeed sounds like a bug, since we should support the use case of scanning only specific namespaced workloads.
needless to say, when the user includes something, we should scan only that, and when they exclude something we should scan everything but this (we can discuss what "everything" means, if we want to never scan some resources). Then there is the question of if Trivy should error and fail when attempting to scan unauthorised resources. I think that for included resources it should - similar to how Trivy would fail for scanning unreachable container image. But for non-excluded resources, maybe it shouldn't fail (since the user isn't aware of everything), and just report it. If supporting this behavior is out of scope for this PR, it's fine I'm just documenting my thoughts. |
yes, after v0.51.0 there is no way to set up a specific namespace, so all request run at the cluster scope.
right now, "everything" for resources means next namespaced items: Deployments, Pods, ReplicaSets, ReplicationControllers, StatefulSets, DaemonSets, CronJobs, Jobs, Services, ServiceAccounts, ConfigMaps, Roles, RoleBindings, NetworkPolicies, Ingresses, ResourceQuotas, LimitRanges. so a user can exclude any kind of this list. there is a problem for exclude namespaces. I found out only one way to do it: get all namespaces and exclude specified ones. but such request requires a cluster role again. |
do you mean there is no way to include a specific namespace? meaning to scan just a specific namespace?
do you mean there's a problem in our specific implementation, or a general problem supporting such behavior? |
yes, a user can't specify a specific namespace to scan now,
it's a good question, I'll take a look at another implementation |
my thought is next: UPD: so it seems It's a general problem such behavior. |
I understand.. in order to exclude namespaces you need to know all available namespaces. can you please point me to the relevant piece of code where this happens? |
sure, in the current code: trivy-kubernetes/pkg/trivyk8s/trivyk8s.go Lines 189 to 206 in b2f739e
|
Wouldn't all namespaces be cluster level resources anyway? Maybe I don't understand this logic https://github.com/afdesk/trivy-kubernetes/blob/3cee92f75e565d7ea68352319cd001a4bad720d1/pkg/trivyk8s/trivyk8s.go#L542-L546 |
yes, you're right. a customer needs a cluster role to get all namespaces. |
I think there's no way around it, so let's just make the experience clear:
|
To summarize,
I believe this PR addresses point 2, but we should address point 1 if we don't already. |
I've added a specific log message if there is no permission to get a namespaces list: For example, if a limited user tries to exclude $ ./trivy k8s --report summary --kubeconfig myconfig --exclude-namespaces kube-system
2024-12-03T16:09:23+06:00 FATAL Fatal error get k8s artifacts with node info error: 'exclude namespaces' option requires a cluster role with permissions to list namespaces If a customer uses a limited role without some permissions, there will be shown an error message. 2024-12-03T15:48:57+06:00 ERROR Unable to list resources error="failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=roles - roles.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"roles\" in API group \"rbac.authorization.k8s.io\" in the namespace \"dev\""
2024-12-03T15:48:57+06:00 ERROR Unable to list resources error="failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=rolebindings - rolebindings.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"rolebindings\" in API group \"rbac.authorization.k8s.io\" in the namespace \"dev\"" |
Just FYI. |
@simar7 PTAL, when you have time. thanks! |
} | ||
|
||
// getNamespaces collects scannable namespaces | ||
func (c *client) getNamespaces() ([]string, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function seems untested, can we add some tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done 75fd906
if errors.IsForbidden(err) { | ||
return result, fmt.Errorf("'exclude namespaces' option requires a cluster role with permissions to list namespaces") | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can occur if a user passes in --include-namespaces
and they don't have cluster level permissions as well right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can occur if a user passes in
--include-namespaces
and they don't have cluster level permissions as well right?
no, it can occur only if len(c.excludeNamespaces) != 0
also a cluster role isn't required for --include-namespaces
} | ||
return result, nil | ||
} | ||
|
||
// ListArtifacts returns kubernetes scannable artifacs. | ||
func (c *client) ListArtifacts(ctx context.Context) ([]*artifacts.Artifact, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add some tests for this too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about it.
ListArtifacts
consists from several functions, which already tested: c.initResourceList()
and c.getNamespaces()
.
c.ListSpecificArtifacts(ctx)
had no tests, and I suggest to do it in a separated PR. wdyt?
pkg/trivyk8s/trivyk8s.go
Outdated
|
||
// collect only included kinds | ||
if len(c.includeKinds) != 0 { | ||
// `includeKinds` are already low cased. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you mean by this comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the comment was improved eed731d
Problem
After v0.51.0 to successfully scan a Kubernetes cluster, Trivy must be executed under a role that has read permissions at the cluster scope.
If a user tries to run
trivy k8s
under a limited account there will be a lot of error message (ex here).I can reproduce it for
limiteduser
with the role:Reason
Trivy must be able to access information about all cluster resources, including pods, deployments etc.
trivy-kubernetes/pkg/trivyk8s/trivyk8s.go
Lines 174 to 187 in b070991
Flags
include/exclude
kinds and namespaces are used only for the result filter.trivy-kubernetes/pkg/trivyk8s/trivyk8s.go
Lines 189 to 206 in b070991
Solution
This PR suggests next solution:
--include-kinds
/--exclude-kinds
flags.--include-namespaces
/--exclude-namespaces
flags. If the namespaces are not provided.resources
under eachnamespace
.resources
andnamespaces
are not provided, Trivy will run as currently at the cluster scope".exclude-namespaces
a user has to have a cluster role to receive all namespaces and filter excluded ones.Result:
$ ./trivy k8s --report summary --disable-node-collector --kubeconfig ./k8s/configs/limiteduser-kubeconfig --include-namespaces default --include-kinds pod 1 / 1 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 0 p/s Summary Report for default Workload Assessment ┌───────────┬────────────────────────────────────┬───────────────────────────────┬────────────────────┬───────────────────┐ │ Namespace │ Resource │ Vulnerabilities │ Misconfigurations │ Secrets │ │ │ ├─────┬──────┬──────┬──────┬────┼───┬───┬───┬────┬───┼───┬───┬───┬───┬───┤ │ │ │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │ ├───────────┼────────────────────────────────────┼─────┼──────┼──────┼──────┼────┼───┼───┼───┼────┼───┼───┼───┼───┼───┼───┤ │ default │ Pod/my-web-deploy-6dbcdb8c54-kbptl │ 157 │ 1336 │ 3108 │ 1529 │ 85 │ │ 1 │ 4 │ 10 │ │ │ │ │ │ │ └───────────┴────────────────────────────────────┴─────┴──────┴──────┴──────┴────┴───┴───┴───┴────┴───┴───┴───┴───┴───┴───┘ Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN Infra Assessment ┌───────────┬──────────┬───────────────────┬───────────────────┬───────────────────┐ │ Namespace │ Resource │ Vulnerabilities │ Misconfigurations │ Secrets │ │ │ ├───┬───┬───┬───┬───┼───┬───┬───┬───┬───┼───┬───┬───┬───┬───┤ │ │ │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │ └───────────┴──────────┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┘ Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN