Flink with auto instrumentation throws NotSerializableException on GlobalOpenTelemetry$ObfuscatedOpenTelemetry #7597

MosheElisha · 2023-01-17T16:59:45Z

Describe the bug
Deploying a task on Apache Flink Task Manager with javaagent opentelemetry-javaagent.jar version 1.22.1 results in the exception below.
With opentelemetry-javaagent.jar version 1.13.1, the issue was not there.

What version are you using?
opentelemetry-javaagent.jar version 1.22.1

Environment
Apache Flink version: 1.14.6, Scala: 2.12
JVM: OpenJDK 64-Bit Server VM - Eclipse Adoptium - 11/11.0.16.1+1

Additional context

java.lang.RuntimeException: Error while translating UnboundedSource: org.apache.beam.sdk.io.kafka.KafkaUnboundedSource@1325d10
	at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:250) ~[ue-mobility.jar:?]
	at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:336) ~[ue-mobility.jar:?]
	at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform(FlinkStreamingPipelineTranslator.java:161) ~[ue-mobility.jar:?]
	at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:140) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:593) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:585) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:585) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:585) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500(TransformHierarchy.java:240) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:214) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:469) ~[ue-mobility.jar:?]
	at org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38) ~[ue-mobility.jar:?]
	at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:92) ~[ue-mobility.jar:?]
	at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:115) ~[ue-mobility.jar:?]
	at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:104) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:323) ~[ue-mobility.jar:?]
	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:309) ~[ue-mobility.jar:?]
	...
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
	at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:261) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$2(ApplicationDispatcherBootstrap.java:224) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
	at org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:171) ~[flink-rpc-akka_ddd88b46-989c-4ed2-a236-1a43ef9b0a47.jar:1.14.6]
	at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_ddd88b46-989c-4ed2-a236-1a43ef9b0a47.jar:1.14.6]
	at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$withContextClassLoader$0(ClassLoadingUtils.java:41) ~[flink-rpc-akka_ddd88b46-989c-4ed2-a236-1a43ef9b0a47.jar:1.14.6]
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) [flink-rpc-akka_ddd88b46-989c-4ed2-a236-1a43ef9b0a47.jar:1.14.6]
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) [flink-rpc-akka_ddd88b46-989c-4ed2-a236-1a43ef9b0a47.jar:1.14.6]
	at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?]
	at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) [?:?]
	at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?]
	at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?]
	at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?]
Caused by: org.apache.flink.api.common.InvalidProgramException: The implementation of the RichParallelSourceFunction is not serializable. The object probably contains or references non serializable fields.
	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:164) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:69) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:2138) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1818) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1801) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1770) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1756) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:229) ~[ue-mobility.jar:?]
	... 38 more
Caused by: java.io.NotSerializableException: io.opentelemetry.javaagent.shaded.io.opentelemetry.api.GlobalOpenTelemetry$ObfuscatedOpenTelemetry
	at java.io.ObjectOutputStream.writeObject0(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeObject(Unknown Source) ~[?:?]
	at java.util.HashMap.internalWriteEntries(Unknown Source) ~[?:?]
	at java.util.HashMap.writeObject(Unknown Source) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
	at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
	at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeSerialData(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeObject0(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeSerialData(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeObject0(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeSerialData(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeObject0(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeObject(Unknown Source) ~[?:?]
	at java.util.ArrayList.writeObject(Unknown Source) ~[?:?]
	at jdk.internal.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
	at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
	at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeSerialData(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeObject0(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeSerialData(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeObject0(Unknown Source) ~[?:?]
	at java.io.ObjectOutputStream.writeObject(Unknown Source) ~[?:?]
	at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:143) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:69) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:2138) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1818) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1801) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1770) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1756) ~[flink-dist_2.12-1.14.6.jar:1.14.6]
	at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:229) ~[ue-mobility.jar:?]
	... 38 more

The text was updated successfully, but these errors were encountered:

trask · 2023-01-18T05:59:19Z

hi @MosheElisha! can you put together a small repro that can be used to investigate? it's unclear to me why GlobalOpenTelemetry$ObfuscatedOpenTelemetry is being serialized and what we can do to avoid that

MosheElisha · 2023-01-18T21:00:04Z

Thanks, @trask . I will try to create a small repo but it isn't easy.
I was able to narrow down the issue a bit - opentelemetry-javaagent version 1.17.0 is working fine. The issue started with version 1.18.0.
Perhaps this will give you a hint.

trask · 2023-01-18T21:52:46Z

if you could build from git and narrow down the commit that would be very helpful

laurit · 2023-01-19T09:01:55Z

I would guess that it is probably #6533 Try with -Dotel.instrumentation.kafka.metric-reporter.enabled=false and let us know whether it helps.

MosheElisha · 2023-01-19T11:13:30Z

Thanks, @laurit ! You are correct. With this flag, I was able to use 1.18.0 and also 1.22.1.

I hope now the issue is clear to you.

BTW - I am currently using OTEL_METRICS_EXPORTER=none maybe I am mistaken but shouldn't that influence the default of otel.instrumentation.kafka.metric-reporter.enabled to be false ?

Resolves #7597 I wasn't able to reproduce this. Figuring out how to run beam, flink and kafka together feels like too much effort. Without reproducing it is too hard to tell why the configuration is serialized, but my hunch is that it is enough to ensure that the configuration can be serialized.

MosheElisha · 2023-02-09T07:16:05Z

Thanks, @laurit ! I can help with verifying the fix once a new release is available.

laurit · 2023-02-09T09:32:17Z

@MosheElisha could you try with nightly build https://oss.sonatype.org/content/repositories/snapshots/io/opentelemetry/javaagent/opentelemetry-javaagent/1.23.0-SNAPSHOT/opentelemetry-javaagent-1.23.0-20230209.072034-109.jar

MosheElisha · 2023-02-09T18:31:46Z

Hi @laurit ,

I used the javaagent from the nightly build and removed the otel.instrumentation.kafka.metric-reporter.enabled=false.
I get the exception below.
Maybe it happens because my code also have a dependency on opentelemetry version 1.22.0 for creating custom traces. I can't easily upgrade that to the snapshot version.

[otel.javaagent 2023-02-09 17:46:16:697 +0000] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 1.23.0-SNAPSHOT
...
2023-02-09 17:46:47,687 WARN  org.apache.flink.runtime.taskmanager.Task                    [] - Source: readFromKafka2/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource) -> Flat Map -> ConvertUnsubscribeToSubscriptionInfo/ParMultiDo(ConvertToNetworkPerfSubscriptionInfo) -> windowUnsubscriptions/Window.Assign.out -> ToBinaryKeyedWorkItem (2/2)#0 (863e0f78d99dd97cfae81eab59205803) switched from INITIALIZING to FAILED with failure cause: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: io.opentelemetry.javaagent.shaded.instrumentation.kafka.internal.OpenTelemetrySupplier
ClassLoader info: URL ClassLoader:
    file: '/tmp/blobStore-e6596ffd-ad09-4251-89d6-a7e9fefb7b9f/job_40ff36b0b22a4d0f10fee2c55fda2647/blob_p-af6209068c71cfc9c3f5604ac09b99414bd0d69d-2d60374c8d702db11d34346a32110424' (valid JAR)
Class was actually found in classloader - deserialization issue.
	at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:338)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:155)
	at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.<init>(RegularOperatorChain.java:63)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:663)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:651)
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassNotFoundException: io.opentelemetry.javaagent.shaded.instrumentation.kafka.internal.OpenTelemetrySupplier
	at java.base/java.net.URLClassLoader.findClass(Unknown Source)
	at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
	at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
	at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
	at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
	at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
	at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Unknown Source)
	at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
	at java.base/java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
	at java.base/java.io.ObjectInputStream.readClassDesc(Unknown Source)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
	at java.base/java.util.HashMap.readObject(Unknown Source)
	at java.base/jdk.internal.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at java.base/java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source)
	at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source)
	at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
	at java.base/java.util.ArrayList.readObject(Unknown Source)
	at java.base/jdk.internal.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at java.base/java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source)
	at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source)
	at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source)
	at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
	at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:617)
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:602)
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:589)
	at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:543)
	at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:324)
	... 9 more

Hopefully resolves #7597 Without reproducing the issue it is hard to tell whether this will help. Another issue that could arise is that we add our metrics class in `metric.reporters` property which will probably break if this configuration is used to build consumer or producer after deserializing as our classes don't seem to be available there. If this fails we'll need to ask the issue reporter for instructions how to reproduce and find a different strategy for fixing this.

laurit · 2023-02-13T16:50:54Z

@MosheElisha could you try https://oss.sonatype.org/content/repositories/snapshots/io/opentelemetry/javaagent/opentelemetry-javaagent/1.23.0-SNAPSHOT/opentelemetry-javaagent-1.23.0-20230213.154905-134.jar

MosheElisha · 2023-02-13T20:23:57Z

Thanks, @laurit ! I can confirm that Flink started properly with this new build.

MosheElisha added the bug Something isn't working label Jan 17, 2023

laurit mentioned this issue Feb 7, 2023

Ensure kafka configuration remains serializable #7754

Merged

laurit closed this as completed in #7754 Feb 9, 2023

trask reopened this Feb 9, 2023

laurit mentioned this issue Feb 10, 2023

Second attempt at fixing serializing kafka configuration #7789

Merged

mateuszrzeszutek closed this as completed in #7789 Feb 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flink with auto instrumentation throws NotSerializableException on GlobalOpenTelemetry$ObfuscatedOpenTelemetry #7597

Flink with auto instrumentation throws NotSerializableException on GlobalOpenTelemetry$ObfuscatedOpenTelemetry #7597

MosheElisha commented Jan 17, 2023

trask commented Jan 18, 2023

MosheElisha commented Jan 18, 2023

trask commented Jan 18, 2023

laurit commented Jan 19, 2023

MosheElisha commented Jan 19, 2023

MosheElisha commented Feb 9, 2023

laurit commented Feb 9, 2023

MosheElisha commented Feb 9, 2023

laurit commented Feb 13, 2023

MosheElisha commented Feb 13, 2023

Flink with auto instrumentation throws NotSerializableException on GlobalOpenTelemetry$ObfuscatedOpenTelemetry #7597

Flink with auto instrumentation throws NotSerializableException on GlobalOpenTelemetry$ObfuscatedOpenTelemetry #7597

Comments

MosheElisha commented Jan 17, 2023

trask commented Jan 18, 2023

MosheElisha commented Jan 18, 2023

trask commented Jan 18, 2023

laurit commented Jan 19, 2023

MosheElisha commented Jan 19, 2023

MosheElisha commented Feb 9, 2023

laurit commented Feb 9, 2023

MosheElisha commented Feb 9, 2023

laurit commented Feb 13, 2023

MosheElisha commented Feb 13, 2023