Upgrade kubernetes-client-api to 6.8.1 #1342

Vlatombe · 2023-03-27T07:47:11Z

Downstream of jenkinsci/kubernetes-client-api-plugin#221

Breaking changes (from library):

Migration to SnakeYAML engine (regression in octal notation support)

Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
Ensure that the pull request title represents the desired changelog entry
Please describe what you did
Link to relevant issues in GitHub or Jira
Link to relevant pull requests, esp. upstream and downstream changes
Ensure you have provided tests - that demonstrates feature works or fixes the issue

For backward compatibility, converts decimal to octal notation

Remove usage of HttpClientUtils.getProxyUrl as it got removed in newer version of kubernetes-client

julieheard · 2023-08-17T15:41:56Z

I notice that this plugin has a dependency on kubernetes-credentials-plugin. Do we need to make sure jenkinsci/kubernetes-credentials-plugin#39 is merged and released first, so we can use the release version for that in this plugin?

Vlatombe · 2023-08-17T15:44:33Z

It's probably safer.

Vlatombe · 2023-08-25T07:27:24Z

pom.xml

-      <version>0.10.0</version>
+      <!-- TODO https://github.com/jenkinsci/kubernetes-credentials-plugin/pull/39 @ f62dc9ee13617d84ec899274ed897668a4703242 -->
+      <version>0.10.1-rc150.f62dc9ee1361</version>


jenkinsci/kubernetes-credentials-plugin#39

Vlatombe · 2023-08-25T07:27:36Z

pom.xml

+        <!-- TODO https://github.com/jenkinsci/snakeyaml-api-plugin/pull/94 @ 4960f40749788fcfe6ddc7def8e0bbc0ce937046-->
+        <version>2.1-110.v4960f4074978</version>


jenkinsci/snakeyaml-api-plugin#94

Vlatombe · 2023-08-25T07:28:16Z

pom.xml

-      <version>6.4.1-215.v2ed17097a_8e9</version>
+      <version>6.8.1-220.v1f66736f9a_42</version>


jenkinsci/kubernetes-client-api-plugin#221

Vlatombe · 2023-08-25T08:40:25Z

@jglick Any clue about this new warning raised in org.csanchez.jenkins.plugins.kubernetes.pipeline.KubernetesPipelineTest.runInPod ?

   7.725 [id=1175]	WARNING	o.j.p.w.flow.FlowExecutionList#unregister: Owner[run In Pod/1:run In Pod #1] was not in the list to begin with: []

Looks like it's called twice, and it seems like a normal condition? In that case it should be logged with level FINE ?!

Stacktraces

Breakpoint reached
at org.jenkinsci.plugins.workflow.flow.FlowExecutionList.unregister(FlowExecutionList.java:148)
at org.jenkinsci.plugins.workflow.flow.FlowExecutionList$1.computeNext(FlowExecutionList.java:93)
at org.jenkinsci.plugins.workflow.flow.FlowExecutionList$1.computeNext(FlowExecutionList.java:83)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:146)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:141)
at org.jenkinsci.plugins.workflow.flow.FlowExecutionList$StepExecutionIteratorImpl.apply(FlowExecutionList.java:229)
at org.jenkinsci.plugins.workflow.steps.StepExecution.applyAll(StepExecution.java:179)
at org.jenkinsci.plugins.workflow.steps.StepExecution.applyAll(StepExecution.java:187)
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$AgentReconnectionListener.check(DurableTaskStep.java:761)
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$AgentReconnectionListener.onOffline(DurableTaskStep.java:748)
at hudson.slaves.SlaveComputer.lambda$closeChannel$1(SlaveComputer.java:944)
at hudson.slaves.SlaveComputer$$Lambda$1255/0x0000009000dafa28.accept(Unknown Source:-1)
at jenkins.util.Listeners.lambda$notify$0(Listeners.java:59)
at jenkins.util.Listeners$$Lambda$825/0x0000009000a1eef0.run(Unknown Source:-1)
at jenkins.util.Listeners.notify(Listeners.java:67)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:944)
at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:820)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
at jenkins.util.ErrorLoggingExecutorService$$Lambda$997/0x0000009000c05238.run(Unknown Source:-1)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:833)
Unregistering Owner[run In Pod/1:run In Pod #1]
Breakpoint reached
at org.jenkinsci.plugins.workflow.flow.FlowExecutionList.unregister(FlowExecutionList.java:148)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.finish(WorkflowRun.java:673)
at org.jenkinsci.plugins.workflow.job.WorkflowRun$GraphL.onNewHead(WorkflowRun.java:1069)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1530)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:511)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:38)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
at jenkins.util.ErrorLoggingExecutorService$$Lambda$997/0x0000009000c05238.run(Unknown Source:-1)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:833)
Unregistering Owner[run In Pod/1:run In Pod #1]

vs. only one call in master

Stacktrace

Breakpoint reached
	at org.jenkinsci.plugins.workflow.flow.FlowExecutionList.unregister(FlowExecutionList.java:133)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.finish(WorkflowRun.java:673)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun$GraphL.onNewHead(WorkflowRun.java:1069)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1587)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:509)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:38)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
	at jenkins.util.ErrorLoggingExecutorService$$Lambda$945/0x000000b000c12550.run(Unknown Source:-1)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.lang.Thread.run(Thread.java:833)

Seems related to jenkinsci/workflow-durable-task-step-plugin#323, combined with https://github.com/jenkinsci/workflow-api-plugin/blob/2c63e4d244199a60ac7003c9b8813f36aad5f638/src/main/java/org/jenkinsci/plugins/workflow/flow/FlowExecutionList.java#L83-L104 which has a side-effect of calling FlowExecutionList#unregister in some cases.

Vlatombe · 2023-08-25T12:58:28Z

jenkinsci/workflow-api-plugin#304 seems to fix it

jglick · 2023-08-25T13:27:49Z

Judging by the stack traces, it looks like a race condition: depending on timing, the AgentReconnectionListener may or may not call StepExecution.applyAll after the FlowExecution.complete but before WorkflowRun.finish gets to that point. Indeed we do not need to unregister anything from StepExecution.applyAll.

pom.xml

amuniz · 2023-10-10T14:41:50Z

As part of the snakeyaml (transitive) update a breaking change was introduced. The new snakeyaml-engine is implementing YAML 1.2.2, in which the merge operator (<<) is not defined anymore (it was removed from the spec). So any yaml using it will fail to parse.

I think it's worth noting in release notes @Vlatombe?

Vlatombe · 2023-10-10T14:52:10Z

Do you have a concrete use case for it ? I've never seen this operator being used in the context of this plugin

Vlatombe · 2023-10-10T14:54:18Z

Probably a general mention that snakeyaml-engine implements a new version of the yaml spec, and so anything that was relying on specifics of yaml 1.1 that got changed or removed in yaml 1.2 no longer works.

Vlatombe · 2023-10-12T07:57:29Z

Updated https://github.com/jenkinsci/kubernetes-plugin/releases/tag/4029.v5712230ccb_f8

vicecea · 2024-01-18T21:34:54Z

Facing the same issues as https://issues.jenkins.io/browse/JENKINS-71956
Any chance we can get an additional option that is backwards compatible?

Would there be a way to fix this in snakeyaml-engine ?

Thanks for any response!

Vlatombe added 5 commits March 27, 2023 09:46

Upgrade kubernetes-client-api to 6.5.1

1083015

align kubernetes-server-mock with client

e6cb569

Add some logs

7d83ca2

New SnakeYAML engine does not support octal notation.

5b812a7

For backward compatibility, converts decimal to octal notation

Spotbugs is stupid here

19f6aa3

jtnord mentioned this pull request Jun 6, 2023

Bump revision from 6.4.1 to 6.7.0 jenkinsci/kubernetes-client-api-plugin#208

Closed

Vlatombe added 3 commits June 23, 2023 17:02

Proxy handling code was not mapping username.

45a5985

Remove usage of HttpClientUtils.getProxyUrl as it got removed in newer version of kubernetes-client

Update kubernetes-client to 6.7.2

e188048

Handle absence of project volume

c77f398

julieheard mentioned this pull request Aug 11, 2023

Bump revision from 6.4.1 to 6.8.0 jenkinsci/kubernetes-client-api-plugin#219

Closed

Vlatombe added 2 commits August 16, 2023 17:35

Merge branch 'master' into kubernetes-client-api-6.5.1

d98e231

Update to 6.8.1

06591e0

Vlatombe changed the title ~~Upgrade kubernetes-client-api to 6.5.1~~ Upgrade kubernetes-client-api to 6.8.1 Aug 16, 2023

rsandell approved these changes Aug 22, 2023

View reviewed changes

Vlatombe mentioned this pull request Aug 24, 2023

Bump revision from 6.4.1 to 6.8.1 jenkinsci/kubernetes-client-api-plugin#221

Merged

Vlatombe added 4 commits August 24, 2023 08:28

Merge branch 'master' into kubernetes-client-api-6.5.1

801be98

Spotbugs is stupid

32f2225

Use kubernetes-credentials incrementals build

c711b6e

Update dependencies

c7dc8ff

Vlatombe commented Aug 25, 2023

View reviewed changes

Vlatombe added 2 commits August 25, 2023 09:48

Fix InjectedTest

a3e44e4

Closing a watch seems async now.

8b373d8

Vlatombe mentioned this pull request Aug 25, 2023

Do not unregister completed executions when iterating over FlowExecutionList to avoid unnecessary log warnings jenkinsci/workflow-api-plugin#304

Merged

Vlatombe added 3 commits August 28, 2023 19:00

Bump workflow-api to pick up jenkinsci/workflow-api-plugin#304

d8b62e5

Remove snakeyaml-api dependency

9c8fbdf

Update kubernetes-credentials

dc2971c

Vlatombe commented Aug 29, 2023

View reviewed changes

pom.xml Outdated Show resolved Hide resolved

Apply suggestions from code review

cee0992

Vlatombe commented Aug 29, 2023

View reviewed changes

pom.xml Outdated Show resolved Hide resolved

Upgrade to kubernetes-credentials release

a3a6df8

Vlatombe marked this pull request as ready for review August 29, 2023 13:50

Vlatombe requested a review from a team as a code owner August 29, 2023 13:50

Vlatombe added the enhancement Improvements label Aug 29, 2023

julieheard approved these changes Aug 29, 2023

View reviewed changes

Vlatombe enabled auto-merge August 29, 2023 14:00

Vlatombe merged commit 5712230 into jenkinsci:master Aug 29, 2023

Vlatombe deleted the kubernetes-client-api-6.5.1 branch August 29, 2023 14:18

Vlatombe mentioned this pull request Jan 19, 2024

[JENKINS-71956] Octal notation fixes #1503

Merged

Dohbedoh mentioned this pull request Apr 24, 2024

[JENKINS-71956] Switching YAML serializer to Jackson to support Octal values #1537

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade kubernetes-client-api to 6.8.1 #1342

Upgrade kubernetes-client-api to 6.8.1 #1342

Vlatombe commented Mar 27, 2023 •

edited

Loading

julieheard commented Aug 17, 2023

Vlatombe commented Aug 17, 2023

Vlatombe Aug 25, 2023

Vlatombe Aug 25, 2023

Vlatombe Aug 25, 2023

Vlatombe commented Aug 25, 2023 •

edited

Loading

Vlatombe commented Aug 25, 2023

jglick commented Aug 25, 2023

amuniz commented Oct 10, 2023

Vlatombe commented Oct 10, 2023

Vlatombe commented Oct 10, 2023

Vlatombe commented Oct 12, 2023

vicecea commented Jan 18, 2024 •

edited

Loading

		<!-- TODO https://github.com/jenkinsci/snakeyaml-api-plugin/pull/94 @ 4960f40749788fcfe6ddc7def8e0bbc0ce937046-->
		<version>2.1-110.v4960f4074978</version>

		<version>6.4.1-215.v2ed17097a_8e9</version>
		<version>6.8.1-220.v1f66736f9a_42</version>

Upgrade kubernetes-client-api to 6.8.1 #1342

Upgrade kubernetes-client-api to 6.8.1 #1342

Conversation

Vlatombe commented Mar 27, 2023 • edited Loading

julieheard commented Aug 17, 2023

Vlatombe commented Aug 17, 2023

Vlatombe Aug 25, 2023

Choose a reason for hiding this comment

Vlatombe Aug 25, 2023

Choose a reason for hiding this comment

Vlatombe Aug 25, 2023

Choose a reason for hiding this comment

Vlatombe commented Aug 25, 2023 • edited Loading

Vlatombe commented Aug 25, 2023

jglick commented Aug 25, 2023

amuniz commented Oct 10, 2023

Vlatombe commented Oct 10, 2023

Vlatombe commented Oct 10, 2023

Vlatombe commented Oct 12, 2023

vicecea commented Jan 18, 2024 • edited Loading

Vlatombe commented Mar 27, 2023 •

edited

Loading

Vlatombe commented Aug 25, 2023 •

edited

Loading

vicecea commented Jan 18, 2024 •

edited

Loading