java: Add spring-boot-starter-opendal #4756

Xuanwo · 2024-06-18T07:22:11Z

Spring is a crucial framework in the Java ecosystem. It would be beneficial for OpenDAL to develop a spring-boot-starter-opendal, enabling Java users to seamlessly integrate OpenDAL immediately.

Notes

We can place spring-boot-starter-opendal under integrations/spring-boot.

The text was updated successfully, but these errors were encountered:

hezhangjian · 2024-06-18T07:24:07Z

I can deep in that next week.

Xuanwo · 2024-06-18T07:24:40Z

I can deep in that next week.

Thanks a lot!

hezhangjian · 2024-08-04T00:55:54Z

@Xuanwo Sorry for my late, recently I was kind of busy. But I still want to work on this :). I think in the short term, we will not be putting the spring starter opendal into the Spring repository. Therefore, we are facing the issue of maintaining OpenDAL with multiple versions of Spring in the OpenDAL repository. I suggest the following strategy:

opendal-spring-xx will adapt to the corresponding Spring Boot version. For example, opendal-33 will adapt to Spring Boot 3.3:

<dependency>
  <groupId>org.apache.opendal</groupId>
  <artifactId>opendal-spring-33</artifactId>
  <version>${latest.version}</version>
</dependency>

<dependency>
  <groupId>org.apache.opendal</groupId>
  <artifactId>opendal-spring-27</artifactId>
  <version>${latest.version}</version>
</dependency>

opendal-spring-boot-starter will by default depend on the adapted latest version of Spring Boot. If you need to run it in a lower version of a Spring Boot project, you can replace it in the following way:

<dependency>
  <groupId>org.apache.opendal</groupId>
  <artifactId>opendal-spring-boot-starter</artifactId>
  <version>${latest.version}</version>
  <exclusions>
      <exclusion>
          <groupId>org.apache.opendal</groupId>
          <artifactId>opendal-spring-33</artifactId>
      </exclusion>
  </exclusions>
</dependency>
<dependency>
  <groupId>org.apache.opendal</groupId>
  <artifactId>opendal-spring-27</artifactId>
  <version>${latest.version}</version>
</dependency>

Xuanwo · 2024-08-04T03:59:29Z

Hi @shoothzj, thanks for your efforts. However, I feel targeting multiple Spring versions might be too complex for a starter proejct. I'm concerned about its future maintainability. Perhaps we could focus on the latest version and provide guidance for older versions?

hezhangjian · 2024-08-05T09:07:34Z

@Xuanwo I agree, we can focus on 3.3.x version.

hezhangjian · 2024-08-07T07:47:46Z

@Xuanwo, SpringBoot now supports two types of starters: reactive for asynchronous operations and non-reactive for synchronous operations. Let's look at a Redis example:

spring-data-redis contains the core logic.
spring-boot-starter-redis includes auto-configuration for Spring WebMVC (synchronous).
spring-boot-starter-redis-reactive includes auto-configuration for Spring WebFlux (asynchronous).

If a project is popular, the spring-boot-autoconfigure repository will include auto-configuration in their main repo, and spring-boot-starter-redis and spring-boot-starter-redis-reactive can be empty or non-existent.

I think we should first maintain these three sub-projects with the latest SpringBoot version in our repo.

Regarding the artifact ID, I prefer using "opendal" as the suffix, which is common in the Spring ecosystem. What do you think?

Opendal as Prefix

opendal-spring
opendal-spring-starter
opendal-spring-starter-reactor

Opendal as Suffix

spring-opendal
spring-starter-opendal
spring-starter-opendal-reactor

Xuanwo · 2024-08-07T08:38:05Z

I'm happy to continue using the common behavior in the ecosystem. spring-opendal looks fine to me.

hezhangjian · 2024-08-08T08:44:51Z

@Xuanwo I put some sample codes in https://gitcode.com/shoothzj/opendal-demo/overview. And here is some docs, would you please take a look?

Usage

Configuration

First, configure the application.yml file with the necessary OpenDAL settings:

spring:
  opendal:
    schema: "fs"
    conf:
      root: "/tmp"

Code Integration

Features

OpenDAL/OpenDALReactive Bean: Easily operate on data using OpenDAL/OpenDALReactive.
Auto Serialize/Deserialize Data: Leverage Jackson for automatic serialization and deserialization.

I plan to implement Auto Serialize/Deserialize Data in the next.

Implementation

Spring WebMVC

To use OpenDAL with Spring WebMVC:

Configure the application.yml file as shown above.
Autowire the OpenDAL bean in your controller.

import org.apache.opendal.Entry;
import org.apache.opendal.Metadata;
import org.apache.opendal.PresignedRequest;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.time.Duration;
import java.util.List;

@RestController
@RequestMapping("/api/v1/opendal")
public class OpenDALController {

    @Autowired
    private OpenDAL openDAL;

    @PostMapping("/write")
    public ResponseEntity<Void> write(@RequestParam String path, @RequestBody String content) {
        openDAL.write(path, content);
        return ResponseEntity.ok().build();
    }

    @PostMapping("/writeBytes")
    public ResponseEntity<Void> writeBytes(@RequestParam String path, @RequestBody byte[] content) {
        openDAL.write(path, content);
        return ResponseEntity.ok().build();
    }

    @PostMapping("/append")
    public ResponseEntity<Void> append(@RequestParam String path, @RequestBody String content) {
        openDAL.append(path, content);
        return ResponseEntity.ok().build();
    }

    @PostMapping("/appendBytes")
    public ResponseEntity<Void> appendBytes(@RequestParam String path, @RequestBody byte[] content) {
        openDAL.append(path, content);
        return ResponseEntity.ok().build();
    }

    @GetMapping("/stat")
    public ResponseEntity<Metadata> stat(@RequestParam String path) {
        Metadata metadata = openDAL.stat(path);
        return ResponseEntity.ok(metadata);
    }

    @GetMapping("/read")
    public ResponseEntity<byte[]> read(@RequestParam String path) {
        byte[] content = openDAL.read(path);
        return ResponseEntity.ok(content);
    }

    @GetMapping("/presignRead")
    public ResponseEntity<PresignedRequest> presignRead(@RequestParam String path, @RequestParam long duration) {
        PresignedRequest request = openDAL.presignRead(path, Duration.ofSeconds(duration));
        return ResponseEntity.ok(request);
    }

    @GetMapping("/presignWrite")
    public ResponseEntity<PresignedRequest> presignWrite(@RequestParam String path, @RequestParam long duration) {
        PresignedRequest request = openDAL.presignWrite(path, Duration.ofSeconds(duration));
        return ResponseEntity.ok(request);
    }

    @GetMapping("/presignStat")
    public ResponseEntity<PresignedRequest> presignStat(@RequestParam String path, @RequestParam long duration) {
        PresignedRequest request = openDAL.presignStat(path, Duration.ofSeconds(duration));
        return ResponseEntity.ok(request);
    }

    @DeleteMapping("/delete")
    public ResponseEntity<Void> delete(@RequestParam String path) {
        openDAL.delete(path);
        return ResponseEntity.ok().build();
    }

    @PostMapping("/createDir")
    public ResponseEntity<Void> createDir(@RequestParam String path) {
        openDAL.createDir(path);
        return ResponseEntity.ok().build();
    }

    @PostMapping("/copy")
    public ResponseEntity<Void> copy(@RequestParam String sourcePath, @RequestParam String targetPath) {
        openDAL.copy(sourcePath, targetPath);
        return ResponseEntity.ok().build();
    }

    @PostMapping("/rename")
    public ResponseEntity<Void> rename(@RequestParam String sourcePath, @RequestParam String targetPath) {
        openDAL.rename(sourcePath, targetPath);
        return ResponseEntity.ok().build();
    }

    @DeleteMapping("/removeAll")
    public ResponseEntity<Void> removeAll(@RequestParam String path) {
        openDAL.removeAll(path);
        return ResponseEntity.ok().build();
    }

    @GetMapping("/list")
    public ResponseEntity<List<Entry>> list(@RequestParam String path) {
        List<Entry> entries = openDAL.list(path);
        return ResponseEntity.ok(entries);
    }
}

Spring WebFlux

To use OpenDALReactive with Spring WebFlux:

Configure the application.yml file as shown above.
Autowire the OpenDALReactive bean in your controller.

import com.shoothzj.opendal.spring.OpenDALReactive;
import org.apache.opendal.Entry;
import org.apache.opendal.Metadata;
import org.apache.opendal.PresignedRequest;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

import java.time.Duration;

@RestController
@RequestMapping("/api/v1/opendal/reactive")
public class OpenDALReactiveController {

    @Autowired
    private OpenDALReactive<byte[]> openDALReactive;

    @PostMapping("/write")
    public Mono<ResponseEntity<Void>> write(@RequestParam("path") String path, @RequestBody String content) {
        return openDALReactive.write(path, content).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @PostMapping("/writeBytes")
    public Mono<ResponseEntity<Void>> writeBytes(@RequestParam("path") String path, @RequestBody byte[] content) {
        return openDALReactive.write(path, content).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @PostMapping("/append")
    public Mono<ResponseEntity<Void>> append(@RequestParam("path") String path, @RequestBody String content) {
        return openDALReactive.append(path, content).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @PostMapping("/appendBytes")
    public Mono<ResponseEntity<Void>> appendBytes(@RequestParam("path") String path, @RequestBody byte[] content) {
        return openDALReactive.append(path, content).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @GetMapping("/stat")
    public Mono<ResponseEntity<Metadata>> stat(@RequestParam("path") String path) {
        return openDALReactive.stat(path).map(ResponseEntity::ok);
    }

    @GetMapping("/read")
    public Mono<ResponseEntity<byte[]>> read(@RequestParam("path") String path) {
        return openDALReactive.read(path).map(ResponseEntity::ok);
    }

    @GetMapping("/presignRead")
    public Mono<ResponseEntity<PresignedRequest>> presignRead(@RequestParam("path") String path, @RequestParam("duration") long duration) {
        return openDALReactive.presignRead(path, Duration.ofSeconds(duration)).map(ResponseEntity::ok);
    }

    @GetMapping("/presignWrite")
    public Mono<ResponseEntity<PresignedRequest>> presignWrite(@RequestParam("path") String path, @RequestParam("duration") long duration) {
        return openDALReactive.presignWrite(path, Duration.ofSeconds(duration)).map(ResponseEntity::ok);
    }

    @GetMapping("/presignStat")
    public Mono<ResponseEntity<PresignedRequest>> presignStat(@RequestParam("path") String path, @RequestParam("duration") long duration) {
        return openDALReactive.presignStat(path, Duration.ofSeconds(duration)).map(ResponseEntity::ok);
    }

    @DeleteMapping("/delete")
    public Mono<ResponseEntity<Void>> delete(@RequestParam("path") String path) {
        return openDALReactive.delete(path).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @PostMapping("/createDir")
    public Mono<ResponseEntity<Void>> createDir(@RequestParam("path") String path) {
        return openDALReactive.createDir(path).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @PostMapping("/copy")
    public Mono<ResponseEntity<Void>> copy(@RequestParam("sourcePath") String sourcePath, @RequestParam("targetPath") String targetPath) {
        return openDALReactive.copy(sourcePath, targetPath).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @PostMapping("/rename")
    public Mono<ResponseEntity<Void>> rename(@RequestParam("sourcePath") String sourcePath, @RequestParam("targetPath") String targetPath) {
        return openDALReactive.rename(sourcePath, targetPath).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @DeleteMapping("/removeAll")
    public Mono<ResponseEntity<Void>> removeAll(@RequestParam("path") String path) {
        return openDALReactive.removeAll(path).then(Mono.just(ResponseEntity.ok().<Void>build()));
    }

    @GetMapping("/list")
    public Flux<Entry> list(@RequestParam("path") String path) {
        return openDALReactive.list(path);
    }
}

Testing with `curl`

Use the following curl commands to test the endpoints of your Spring WebMVC and WebFlux controllers:

Write String Content

curl -X POST "http://localhost:8080/api/v1/opendal/write?path=/example/path" -H "Content-Type: application/json" -d "This is a test content"

Write Byte Content

curl -X POST "http://localhost:8080/api/v1/opendal/writeBytes?path=/example/path

Append String Content

curl -X POST "http://localhost:8080/api/v1/opendal/append?path=/example/path" -H "Content-Type: application/json" -d "This is additional content"

Append Byte Content

curl -X POST "http://localhost:8080/api/v1/opendal/appendBytes?path=/example/path" -H "Content-Type: application/octet-stream" --data-binary @/path/to/your/file

Stat

curl -X GET "http://localhost:8080/api/v1/opendal/stat?path=/example/path"

Read

curl -X GET "http://localhost:8080/api/v1/opendal/read?path=/example/path"

Presign Read

curl -X GET "http://localhost:8080/api/v1/opendal/presignRead?path=/example/path&duration=3600"

Presign Write

curl -X GET "http://localhost:8080/api/v1/opendal/presignWrite?path=/example/path&duration=3600"

Presign Stat

curl -X GET "http://localhost:8080/api/v1/opendal/presignStat?path=/example/path&duration=3600"

Delete

curl -X DELETE "http://localhost:8080/api/v1/opendal/delete?path=/example/path"

Create Directory

curl -X POST "http://localhost:8080/api/v1/opendal/createDir?path=/example/path"

Copy

curl -X POST "http://localhost:8080/api/v1/opendal/copy?sourcePath=/example/source&targetPath=/example/target"

Rename

curl -X POST "http://localhost:8080/api/v1/opendal/rename?sourcePath=/example/source&targetPath=/example/target"

Remove All

curl -X DELETE "http://localhost:8080/api/v1/opendal/removeAll?path=/example/path"

List

curl -X GET "http://localhost:8080/api/v1/opendal/list?path=/example/path"

Xuanwo · 2024-08-08T12:55:37Z

Hi, @tisonkun, would you like to take a look? I'm not a Java expert like you are.

Xuanwo · 2024-08-08T12:59:35Z

Some ideas in my mind for now:

We don't need to seperate writeString and writeBytes, we only need to have write which accept bytes.
We should remove append, it's an option for write
I don't need we need to play with presign at current stage.

tisonkun · 2024-08-08T13:09:37Z

Hi, @tisonkun, would you like to take a look? I'm not a Java expert like you are.

Looks like a good start. Perhaps create a repo for this starter and we can collaborate as normal issues + PRs.

I don't think we need to include these code into the main repo since it would cause extra release burden. Let @shoothzj be the maintainer of the starters and modifying anything necessary in binding-java should be fine.

hezhangjian · 2024-08-08T14:23:53Z

We don't need to seperate writeString and writeBytes, we only need to have write which accept bytes.

We can only maintain writeBytes now, Java has powerful overload and generic, we can support write(Object obj) leverage jackson's serial.

We should remove append, it's an option for write

I need take a deep look, but why the java binding expose this method?

I don't need we need to play with presign at current stage.
Ok.

hezhangjian · 2024-08-08T14:24:50Z

Hi, @tisonkun, would you like to take a look? I'm not a Java expert like you are.

Looks like a good start. Perhaps create a repo for this starter and we can collaborate as normal issues + PRs.

I don't think we need to include these code into the main repo since it would cause extra release burden. Let @shoothzj be the maintainer of the starters and modifying anything necessary in binding-java should be fine.

I would be happy to help maintain the starters. :)

Xuanwo · 2024-08-15T07:34:49Z

Hi, @shoothzj, the subproject for spring-opendal has been started. Would you like to create a tracking issue for it?

hezhangjian · 2024-08-15T12:43:37Z

@Xuanwo #5015

Xuanwo · 2024-08-15T12:44:29Z

Thanks!

Xuanwo added the bindings/java label Jun 18, 2024

hezhangjian mentioned this issue Aug 9, 2024

feat(integrations/spring): add spring project module #4988

Merged

Xuanwo closed this as completed Aug 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

java: Add spring-boot-starter-opendal #4756

java: Add spring-boot-starter-opendal #4756

Xuanwo commented Jun 18, 2024

hezhangjian commented Jun 18, 2024

Xuanwo commented Jun 18, 2024

hezhangjian commented Aug 4, 2024

Xuanwo commented Aug 4, 2024

hezhangjian commented Aug 5, 2024

hezhangjian commented Aug 7, 2024

Xuanwo commented Aug 7, 2024

hezhangjian commented Aug 8, 2024

Xuanwo commented Aug 8, 2024

Xuanwo commented Aug 8, 2024

tisonkun commented Aug 8, 2024

hezhangjian commented Aug 8, 2024 •

edited

Loading

hezhangjian commented Aug 8, 2024

Xuanwo commented Aug 15, 2024

hezhangjian commented Aug 15, 2024

Xuanwo commented Aug 15, 2024

java: Add spring-boot-starter-opendal #4756

java: Add spring-boot-starter-opendal #4756

Comments

Xuanwo commented Jun 18, 2024

Notes

hezhangjian commented Jun 18, 2024

Xuanwo commented Jun 18, 2024

hezhangjian commented Aug 4, 2024

Xuanwo commented Aug 4, 2024

hezhangjian commented Aug 5, 2024

hezhangjian commented Aug 7, 2024

Opendal as Prefix

Opendal as Suffix

Xuanwo commented Aug 7, 2024

hezhangjian commented Aug 8, 2024

Usage

Configuration

Code Integration

Features

Implementation

Spring WebMVC

Spring WebFlux

Testing with curl

Xuanwo commented Aug 8, 2024

Xuanwo commented Aug 8, 2024

tisonkun commented Aug 8, 2024

hezhangjian commented Aug 8, 2024 • edited Loading

hezhangjian commented Aug 8, 2024

Xuanwo commented Aug 15, 2024

hezhangjian commented Aug 15, 2024

Xuanwo commented Aug 15, 2024

Testing with `curl`

hezhangjian commented Aug 8, 2024 •

edited

Loading