Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Work around a bug in cAdvisor on ppc64le by removing \0 from SystemUUID #95

Merged
merged 1 commit into from
Feb 24, 2021

Conversation

jlpadilla
Copy link
Contributor

@jlpadilla jlpadilla commented Feb 23, 2021

Original PR: #94

Description of changes

Remove trailing null character from SystemUUID.

See google/cadvisor#2811 for the original bug.

A trailing null character in a Go string (_systemUUID) is confusing Redis on the hub:

# oc logs -n open-cluster-management search-prod-d2cf2-search-aggregator-8695c876b4-wfmvp
...
I0222 08:39:39.698067       1 resyncCluster.go:29] Resync for cluster: ocp-dal12-53e5 edges to insert: 2633
E0222 08:39:39.841503       1 redisinterfacev2.go:46] Error fetching results from RedisGraph V2 : errMsg: Invalid input at end of input: expected ' line: 1, column: 353, offset: 352 errCtx: ...des', kind:'node', created:'2021-02-09T21:53:14Z', _systemUUID:'IBM,0678B8300 errCtxOffset: 80
E0222 08:39:39.843059       1 redisinterfacev2.go:46] Error fetching results from RedisGraph V2 : errMsg: Invalid input at end of input: expected ' line: 1, column: 556, offset: 555 errCtx: ..., kind:'node', _clusterNamespace:'ocp-dal12-53e5', _systemUUID:'IBM,0678B8300 errCtxOffset: 80
E0222 08:39:39.844247       1 redisinterfacev2.go:46] Error fetching results from RedisGraph V2 : errMsg: Invalid input at end of input: expected ' line: 1, column: 193, offset: 192 errCtx: ..., apiversion:'v1', created:'2021-02-09T21:53:14Z', _systemUUID:'IBM,0678B8300 errCtxOffset: 80
W0222 08:39:39.844272       1 insert.go:36] Rejecting Resource ocp-dal12-53e5/03043655-7d3e-4510-baec-f237efde1661: errMsg: Invalid input at end of input: expected ' line: 1, column: 193, offset: 192 errCtx: ..., apiversion:'v1', created:'2021-02-09T21:53:14Z', _systemUUID:'IBM,0678B8300 errCtxOffset: 80

On the spoke:

I0211 12:26:46.698846       1 sender.go:229] First time sending or last Sync cycle failed, sending complete payload
I0211 12:26:46.747321       1 sender.go:176] Sending Resources { request: 683963, add: 4250, update: 0, delete: 0 edge add: 2625 edge delete: 0 }
E0211 12:27:28.193966       1 sender.go:233] Sync sender error. Aggregator reported wrong number of total intra edges. Expected 2625, got 2323
E0211 12:27:28.193987       1 sender.go:285] SEND ERROR: Aggregator reported wrong number of total intra edges. Expected 2625, got 2323
W0211 12:27:28.193996       1 sender.go:299] Backing off send interval because of error response from aggregator. Sleeping for 10s

A pull request has been submitted to cAdvisor to remove this trailing null character from the string.
However, in the meantime, we would like to work around this issue in the search-collector so we can support older versions of kubelet that do not have the fix.
The proposed patch has been tested on ppc64le and solves the issue mentioned above.

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jlpadilla, xrajesh
To complete the pull request process, please assign after the PR has been reviewed.
You can assign the PR to them by writing /assign in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jlpadilla jlpadilla merged commit ab171ee into main Feb 24, 2021
@jlpadilla jlpadilla deleted the jorge/ppc64le-system-uid branch February 24, 2021 02:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants