Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Failing Test]: Java Samza runner tests failing on Java11 #32208

Open
17 tasks
Abacn opened this issue Aug 15, 2024 · 3 comments
Open
17 tasks

[Failing Test]: Java Samza runner tests failing on Java11 #32208

Abacn opened this issue Aug 15, 2024 · 3 comments

Comments

@Abacn
Copy link
Contributor

Abacn commented Aug 15, 2024

What happened?

Error message:

org.apache.beam.sdk.Pipeline$PipelineExecutionException: org.apache.beam.sdk.util.UserCodeException: java.lang.RuntimeException: org.apache.beam.sdk.util.UserCodeException: java.lang.IncompatibleClassChangeError: Method 'void org.apache.samza.storage.kv.KeyValueStore.deleteAll(java.util.List)' must be InterfaceMethodref constant
	at app//org.apache.beam.runners.samza.SamzaPipelineResult.getStateInfo(SamzaPipelineResult.java:124)
...
Caused by: java.lang.IncompatibleClassChangeError: Method 'void org.apache.samza.storage.kv.KeyValueStore.deleteAll(java.util.List)' must be InterfaceMethodref constant
	at org.apache.samza.storage.kv.CachedStore.deleteAll(CachedStore.scala:253)
	at org.apache.samza.storage.kv.NullSafeKeyValueStore.deleteAll(NullSafeKeyValueStore.scala:72)
	at org.apache.samza.storage.kv.KeyValueStorageEngine$$anonfun$deleteAll$1.apply$mcV$sp(KeyValueStorageEngine.scala:92)
	at org.apache.samza.storage.kv.KeyValueStorageEngine$$anonfun$deleteAll$1.apply(KeyValueStorageEngine.scala:89)
	at org.apache.samza.storage.kv.KeyValueStorageEngine$$anonfun$deleteAll$1.apply(KeyValueStorageEngine.scala:89)
	at org.apache.samza.util.TimerUtil$class.updateTimer(TimerUtil.scala:37)
	at org.apache.samza.storage.kv.KeyValueStorageEngine.updateTimer(KeyValueStorageEngine.scala:40)
	at org.apache.samza.storage.kv.KeyValueStorageEngine.deleteAll(KeyValueStorageEngine.scala:89)
	at org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals$SamzaBagState.clear(SamzaStoreStateInternals.java:510)

Issue Failure

Failure: Test is continually failing

Issue Priority

Priority: 2 (backlog / disabled test but we think the product is healthy)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
@Abacn
Copy link
Contributor Author

Abacn commented Aug 15, 2024

tests also fails on M1 mac due to incompatible jni

Caused by: java.lang.UnsatisfiedLinkError: Can't load library: /var/folders/wg/hwmcqjwd4zz75mjs0r5z_3f400y2yj/T/librocksdbjni1775545785986054429.jnilib
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2633)
	at java.base/java.lang.Runtime.load0(Runtime.java:768)
	at java.base/java.lang.System.load(System.java:1837)
	at org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:79)
	at org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:57)
	at org.rocksdb.RocksDB.loadLibrary(RocksDB.java:69)
	at org.rocksdb.RocksDB.<clinit>(RocksDB.java:38)
	at org.rocksdb.Options.<clinit>(Options.java:24)
	at org.apache.samza.storage.kv.RocksDbOptionsHelper.options(RocksDbOptionsHelper.java:49)

Found that both are fixed on the latest Samza 1.8

@Abacn Abacn mentioned this issue Aug 15, 2024
3 tasks
@Abacn
Copy link
Contributor Author

Abacn commented Aug 15, 2024

Actually bump the version is not trivial , see #32209 (comment)

@Abacn
Copy link
Contributor Author

Abacn commented Nov 25, 2024

Some of Beam's depenencies has removed Java8 support, as a result pipelines using these component cannot run on Samza Runner, this includes ZetaSQL, see #33213

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant