Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] [Map OSS] org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem not found #7019

Closed
3 tasks done
zhouhoo opened this issue Jun 19, 2024 · 7 comments · Fixed by #7029
Closed
3 tasks done

[Doc] [Map OSS] org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem not found #7019

zhouhoo opened this issue Jun 19, 2024 · 7 comments · Fixed by #7029
Labels

Comments

@zhouhoo
Copy link
Contributor

zhouhoo commented Jun 19, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

when develop cluster by config map using aliyun oss fs.oss.credentials.provider: org.apache.hadoop.fs.aliyun.oss.AliyunCredentialsProvider
error occur : class org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem not found
微信截图_20240619091601

did I miss some thing? connector-file-oss-2.3.5.jar is in connectors folder.

SeaTunnel Version

2.3.5

SeaTunnel Config

map config in hazelcast.yaml
map:
    engine*:
       map-store:
         enabled: true
         initial-mode: EAGER
         factory-class-name: org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory
         properties:
           type: hdfs
           namespace: /tmp/seatunnel/imap
           clusterName: seatunnel-cluster
           storage.type: oss
           block.size: block size(bytes)
           oss.bucket: oss://bucket name/
           fs.oss.accessKeyId: OSS access key id
           fs.oss.accessKeySecret: OSS access key secret
           fs.oss.endpoint: OSS endpoint
           fs.oss.credentials.provider: org.apache.hadoop.fs.aliyun.oss.AliyunCredentialsProvider

Running Command

./bin/seatunnel-cluster.sh -d

Error Exception

java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2638) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3269) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3301) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.seatunnel.engine.imap.storage.file.IMapFileStorage.initialize(IMapFileStorage.java:156) ~[seatunnel-starter.jar:2.3.5]
	at org.apache.seatunnel.engine.imap.storage.file.IMapFileStorageFactory.create(IMapFileStorageFactory.java:41) ~[seatunnel-starter.jar:2.3.5]
	at org.apache.seatunnel.engine.server.persistence.FileMapStore.init(FileMapStore.java:49) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.MapStoreWrapper.init(MapStoreWrapper.java:87) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.mapstore.BasicMapStoreContext.callLifecycleSupportInit(BasicMapStoreContext.java:164) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.mapstore.BasicMapStoreContext.create(BasicMapStoreContext.java:137) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.mapstore.MapStoreContextFactory.createMapStoreContext(MapStoreContextFactory.java:48) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.MapContainer.<init>(MapContainer.java:148) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.MapServiceContextImpl.createMapContainer(MapServiceContextImpl.java:206) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.MapServiceContextImpl.lambda$createMapConstructor$0(MapServiceContextImpl.java:197) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.internal.util.ConcurrencyUtil.getOrPutSynchronized(ConcurrencyUtil.java:140) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.MapServiceContextImpl.getMapContainer(MapServiceContextImpl.java:288) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.PartitionContainer.createRecordStore(PartitionContainer.java:91) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.internal.util.ConcurrencyUtil.getOrPutSynchronized(ConcurrencyUtil.java:120) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.PartitionContainer.getRecordStore(PartitionContainer.java:153) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.MapServiceContextImpl.getRecordStore(MapServiceContextImpl.java:491) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.operation.MapChunk.getRecordStore(MapChunk.java:385) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.map.impl.operation.MapChunk.afterRunFinal(MapChunk.java:153) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.spi.impl.operationexecutor.OperationRunner.runDirect(OperationRunner.java:173) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.internal.partition.operation.MigrationOperation.runMigrationOperation(MigrationOperation.java:139) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.internal.partition.operation.MigrationOperation.doRun(MigrationOperation.java:115) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.internal.partition.operation.MigrationOperation.run(MigrationOperation.java:95) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123) ~[seatunnel-starter.jar:2.3.5]
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.5]
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2542) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2636) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5]
	... 37 more

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@zhouhoo zhouhoo added the bug label Jun 19, 2024
@Carl-Zhou-CN
Copy link
Member

Carl-Zhou-CN commented Jun 19, 2024

hi, @zhouhoo You must ensure seatunnel-hadoop3-3.1.4-uber.jar, aliyun-sdk-oss-3.4.1.jar, hadoop-aliyun-3.1.4.jar and jdom-1.1.jar in ${SEATUNNEL_HOME}/lib/ dir.

@zhouhoo
Copy link
Contributor Author

zhouhoo commented Jun 19, 2024

hi, @zhouhoo You must ensure seatunnel-hadoop3-3.1.4-uber.jar, aliyun-sdk-oss-3.4.1.jar, hadoop-aliyun-3.1.4.jar and jdom-1.1.jar in ${SEATUNNEL_HOME}/lib/ dir.

thanks , I download those jar and put into lib folder. but get another error .
rg.apache.seatunnel.engine.imap.storage.api.exception.IMapStorageException: Failed to get file system at org.apache.seatunnel.engine.imap.storage.file.IMapFileStorage.initialize(IMapFileStorage.java:159) ~[seatunnel-starter.jar:2.3.5] at org.apache.seatunnel.engine.imap.storage.file.IMapFileStorageFactory.create(IMapFileStorageFactory.java:41) ~[seatunnel-starter.jar:2.3.5] at org.apache.seatunnel.engine.server.persistence.FileMapStore.init(FileMapStore.java:49) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapStoreWrapper.init(MapStoreWrapper.java:87) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.mapstore.BasicMapStoreContext.callLifecycleSupportInit(BasicMapStoreContext.java:164) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.mapstore.BasicMapStoreContext.create(BasicMapStoreContext.java:137) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.mapstore.MapStoreContextFactory.createMapStoreContext(MapStoreContextFactory.java:48) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapContainer.<init>(MapContainer.java:148) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapServiceContextImpl.createMapContainer(MapServiceContextImpl.java:206) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapServiceContextImpl.lambda$createMapConstructor$0(MapServiceContextImpl.java:197) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.ConcurrencyUtil.getOrPutSynchronized(ConcurrencyUtil.java:140) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapServiceContextImpl.getMapContainer(MapServiceContextImpl.java:288) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.PartitionContainer.createRecordStore(PartitionContainer.java:91) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.ConcurrencyUtil.getOrPutSynchronized(ConcurrencyUtil.java:120) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.PartitionContainer.getRecordStore(PartitionContainer.java:153) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapServiceContextImpl.getRecordStore(MapServiceContextImpl.java:491) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.operation.MapChunk.getRecordStore(MapChunk.java:385) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.operation.MapChunk.afterRunFinal(MapChunk.java:153) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationexecutor.OperationRunner.runDirect(OperationRunner.java:173) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.partition.operation.MigrationOperation.runMigrationOperation(MigrationOperation.java:139) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.partition.operation.MigrationOperation.doRun(MigrationOperation.java:115) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.partition.operation.MigrationOperation.run(MigrationOperation.java:95) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.5] Caused by: java.io.IOException: org.apache.hadoop.fs.aliyun.oss.AliyunCredentialsProvider constructor exception. A class specified in fs.oss.credentials.provider must provide an accessible constructor accepting URI and Configuration, or an accessible default constructor. at org.apache.hadoop.fs.aliyun.oss.AliyunOSSUtils.getCredentialsProvider(AliyunOSSUtils.java:131) ~[hadoop-aliyun-3.1.4.jar:?] at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystemStore.initialize(AliyunOSSFileSystemStore.java:154) ~[hadoop-aliyun-3.1.4.jar:?] at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem.initialize(AliyunOSSFileSystem.java:350) ~[hadoop-aliyun-3.1.4.jar:?] at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.seatunnel.engine.imap.storage.file.IMapFileStorage.initialize(IMapFileStorage.java:156) ~[seatunnel-starter.jar:2.3.5] ... 29 more Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.fs.aliyun.oss.AliyunCredentialsProvider.<init>() at java.lang.Class.getConstructor0(Class.java:3082) ~[?:1.8.0_91] at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[?:1.8.0_91] at org.apache.hadoop.fs.aliyun.oss.AliyunOSSUtils.getCredentialsProvider(AliyunOSSUtils.java:125) ~[hadoop-aliyun-3.1.4.jar:?] at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystemStore.initialize(AliyunOSSFileSystemStore.java:154) ~[hadoop-aliyun-3.1.4.jar:?] at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem.initialize(AliyunOSSFileSystem.java:350) ~[hadoop-aliyun-3.1.4.jar:?] at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.seatunnel.engine.imap.storage.file.IMapFileStorage.initialize(IMapFileStorage.java:156) ~[seatunnel-starter.jar:2.3.5] ... 29 more

@Carl-Zhou-CN
Copy link
Member

@zhouhoo sorry,
3262108ef51e9108b47edc451addac8
Please try using these versions of the JAR files provided by the current users we are working with.

@Carl-Zhou-CN
Copy link
Member

@EricJoy2048 We need your support here

@zhouhoo
Copy link
Contributor Author

zhouhoo commented Jun 20, 2024

@zhouhoo sorry, 3262108ef51e9108b47edc451addac8 Please try using these versions of the JAR files provided by the current users we are working with.

ok , I tried, but another error occur, like missing jar or wrong jar version cause?
2024-06-19 16:21:58,359 ERROR [c.h.i.p.o.MigrationOperation ] [hz.main.partition-operation.thread-5] - [10.29.xx.xx]:5801 [seatunnel] [5.1] Error while executing replication operations MigrationInfo{uuid=8e78dd7b-db9d-4fa4-b6f7-e53e66efe6af, partitionId=5, source=[10.111.xx.xx]:5801 - 014d90cf-3a9f-41d6-af61-229a95aeccc4, sourceCurrentReplicaIndex=0, sourceNewReplicaIndex=3, destination=[10.29.xx.xx]:5801 - fc9d7cc0-8896-46f2-ab77-4da60e40be88, destinationCurrentReplicaIndex=-1, destinationNewReplicaIndex=0, master=[10.111.xx.xx]:5801, initialPartitionVersion=25026, partitionVersionIncrement=2, status=ACTIVE} java.lang.NoClassDefFoundError: org/apache/hadoop/util/Preconditions at org.apache.hadoop.fs.aliyun.oss.AliyunOSSUtils.longOption(AliyunOSSUtils.java:234) ~[hadoop-aliyun-3.3.6.jar:?] at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem.initialize(AliyunOSSFileSystem.java:343) ~[hadoop-aliyun-3.3.6.jar:?] at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.5] at org.apache.seatunnel.engine.imap.storage.file.IMapFileStorage.initialize(IMapFileStorage.java:156) ~[seatunnel-starter.jar:2.3.5] at org.apache.seatunnel.engine.imap.storage.file.IMapFileStorageFactory.create(IMapFileStorageFactory.java:41) ~[seatunnel-starter.jar:2.3.5] at org.apache.seatunnel.engine.server.persistence.FileMapStore.init(FileMapStore.java:49) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapStoreWrapper.init(MapStoreWrapper.java:87) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.mapstore.BasicMapStoreContext.callLifecycleSupportInit(BasicMapStoreContext.java:164) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.mapstore.BasicMapStoreContext.create(BasicMapStoreContext.java:137) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.mapstore.MapStoreContextFactory.createMapStoreContext(MapStoreContextFactory.java:48) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapContainer.<init>(MapContainer.java:148) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapServiceContextImpl.createMapContainer(MapServiceContextImpl.java:206) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapServiceContextImpl.lambda$createMapConstructor$0(MapServiceContextImpl.java:197) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.ConcurrencyUtil.getOrPutSynchronized(ConcurrencyUtil.java:140) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapServiceContextImpl.getMapContainer(MapServiceContextImpl.java:288) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.PartitionContainer.createRecordStore(PartitionContainer.java:91) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.ConcurrencyUtil.getOrPutSynchronized(ConcurrencyUtil.java:120) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.PartitionContainer.getRecordStore(PartitionContainer.java:153) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.MapServiceContextImpl.getRecordStore(MapServiceContextImpl.java:491) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.operation.MapChunk.getRecordStore(MapChunk.java:385) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.operation.MapChunk.afterRunFinal(MapChunk.java:153) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationexecutor.OperationRunner.runDirect(OperationRunner.java:173) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.partition.operation.MigrationOperation.runMigrationOperation(MigrationOperation.java:139) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.partition.operation.MigrationOperation.doRun(MigrationOperation.java:115) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.partition.operation.MigrationOperation.run(MigrationOperation.java:95) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.5]

@zhouhoo
Copy link
Contributor Author

zhouhoo commented Jun 20, 2024

finally,with the help of two master, @Carl-Zhou-CN @EricJoy2048 , It works ! thanks so much!
make sure lib foler have those jars:

aliyun-sdk-oss-3.13.2.jar

hadoop-aliyun-3.3.6.jar

jdom2-2.0.6.jar

netty-buffer-4.1.89.Final.jar 

netty-common-4.1.89.Final.jar

 seatunnel-hadoop3-3.3.6-uber.jar

@zhouhoo zhouhoo closed this as completed Jun 20, 2024
@zhouhoo zhouhoo changed the title [Bug] [Map OSS] org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem not found [Doc] [Map OSS] org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem not found Jun 20, 2024
@gzliu
Copy link

gzliu commented Oct 24, 2024

finally,with the help of two master, @Carl-Zhou-CN @EricJoy2048 , It works ! thanks so much! make sure lib foler have those jars:

aliyun-sdk-oss-3.13.2.jar

hadoop-aliyun-3.3.6.jar

jdom2-2.0.6.jar

netty-buffer-4.1.89.Final.jar 

netty-common-4.1.89.Final.jar

 seatunnel-hadoop3-3.3.6-uber.jar

how to get the 'seatunnel-hadoop3-3.3.6-uber.jar' ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants