Skip to content

Commit

Permalink
format code.
Browse files Browse the repository at this point in the history
  • Loading branch information
yuqi1129 committed Jan 7, 2025
1 parent 6c1aac3 commit 44014d9
Show file tree
Hide file tree
Showing 4 changed files with 33 additions and 36 deletions.
20 changes: 8 additions & 12 deletions docs/hadoop-catalog-with-adls.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,13 @@ $ bin/gravitino-server.sh start
```
Once the server is up and running, you can proceed to configure the Hadoop catalog with ADLS.


```bash
$ bin/gravitino-server.sh start
```

## Create a Hadoop Catalog with ADLS

The rest of this document shows how to use the Hadoop catalog with ADLS in Gravitino with a full example.
## Configurations for creating a Hadoop catalog with ADLS

### Configuration for a ADLS Hadoop catalog
### Configuration for a ADLS Hadoop catalog

Apart from configurations mentioned in [Hadoop-catalog-catalog-configuration](./hadoop-catalog.md#catalog-properties), the following properties are required to configure a Hadoop catalog with ADLS:

Expand All @@ -41,18 +38,17 @@ Apart from configurations mentioned in [Hadoop-catalog-catalog-configuration](./
| `azure-storage-account-name ` | The account name of Azure Blob Storage. | (none) | Yes if it's a Azure Blob Storage fileset. | 0.8.0-incubating |
| `azure-storage-account-key` | The account key of Azure Blob Storage. | (none) | Yes if it's a Azure Blob Storage fileset. | 0.8.0-incubating |

### Configuration for a schema

Refer to [Schema operation](./manage-fileset-metadata-using-gravitino.md#schema-operations) for more details.
### Configurations for a schema

### Configuration for a fileset
Refer to [Schema configurations](./hadoop-catalog.md#schema-properties) for more details.

Refer to [Fileset operation](./manage-fileset-metadata-using-gravitino.md#fileset-operations) for more details.
### Configurations for a fileset

Refer to [Fileset configurations](./hadoop-catalog.md#fileset-properties) for more details.

## Using Hadoop catalog with ADLS
## Example of creating Hadoop catalog with ADLS

This section demonstrates how to use the Hadoop catalog with ADLS in Gravitino, with a complete example.
This section demonstrates how to create the Hadoop catalog with ADLS in Gravitino, with a complete example.

### Step1: Create a Hadoop catalog with ADLS

Expand Down
23 changes: 12 additions & 11 deletions docs/hadoop-catalog-with-gcs.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,20 @@ license: "This software is licensed under the Apache License version 2."
This document describes how to configure a Hadoop catalog with GCS.

## Prerequisites
To set up a Hadoop catalog with OSS, follow these steps:

In order to create a Hadoop catalog with GCS, you need to place [`gravitino-gcp-bundle-${gravitino-version}.jar`](https://mvnrepository.com/artifact/org.apache.gravitino/gravitino-gcp-bundle) in Gravitino Hadoop catalog classpath located
at `${GRAVITINO_HOME}/catalogs/hadoop/libs/`. After that, start Gravitino server with the following command:
1. Download the [`gravitino-gcp-bundle-${gravitino-version}.jar`](https://mvnrepository.com/artifact/org.apache.gravitino/gravitino-gcp-bundle) file.
2. Place the downloaded file into the Gravitino Hadoop catalog classpath at `${GRAVITINO_HOME}/catalogs/hadoop/libs/`.
3. Start the Gravitino server by running the following command:

```bash
$ bin/gravitino-server.sh start
```
Once the server is up and running, you can proceed to configure the Hadoop catalog with GCS.

## Create a Hadoop Catalog with GCS
## Configurations for creating a Hadoop catalog with GCS

The rest of this document shows how to use the Hadoop catalog with GCS in Gravitino with a full example.

### Configuration for a GCS Hadoop catalog
### Configurations for a GCS Hadoop catalog

Apart from configurations mentioned in [Hadoop-catalog-catalog-configuration](./hadoop-catalog.md#catalog-properties), the following properties are required to configure a Hadoop catalog with GCS:

Expand All @@ -31,15 +32,15 @@ Apart from configurations mentioned in [Hadoop-catalog-catalog-configuration](./
| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local`, for GCS, if we set this value, we can omit the prefix 'gs://' in the location. | `builtin-local` | No | 0.7.0-incubating |
| `gcs-service-account-file` | The path of GCS service account JSON file. | (none) | Yes if it's a GCS fileset. | 0.7.0-incubating |

### Configuration for a schema
### Configurations for a schema

Refer to [Schema operation](./manage-fileset-metadata-using-gravitino.md#schema-operations) for more details.
Refer to [Schema configurations](./hadoop-catalog.md#schema-properties) for more details.

### Configuration for a fileset
### Configurations for a fileset

Refer to [Fileset operation](./manage-fileset-metadata-using-gravitino.md#fileset-operations) for more details.
Refer to [Fileset configurations](./hadoop-catalog.md#fileset-properties) for more details.

## Using Hadoop catalog with GCS
## Example of creating Hadoop catalog with GCS

This section will show you how to use the Hadoop catalog with GCS in Gravitino, including detailed examples.

Expand Down
14 changes: 7 additions & 7 deletions docs/hadoop-catalog-with-oss.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ $ bin/gravitino-server.sh start
```
Once the server is up and running, you can proceed to configure the Hadoop catalog with OSS.

## Create a Hadoop Catalog with OSS
## Configurations for creating a Hadoop catalog with OSS

### Configuration for an OSS Hadoop catalog

Expand All @@ -35,19 +35,19 @@ In addition to the basic configurations mentioned in [Hadoop-catalog-catalog-con
| `oss-access-key-id` | The access key of the Aliyun OSS. | (none) | Yes if it's a OSS fileset. | 0.7.0-incubating |
| `oss-secret-access-key` | The secret key of the Aliyun OSS. | (none) | Yes if it's a OSS fileset. | 0.7.0-incubating |

### Configuration for a schema
### Configurations for a schema

To create a schema, refer to [Schema operation](./manage-fileset-metadata-using-gravitino.md#schema-operations).
To create a schema, refer to [Schema configurations](./hadoop-catalog.md#schema-properties).

### Configuration for a fileset
### Configurations for a fileset

For instructions on how to create a fileset, refer to [Fileset operation](./manage-fileset-metadata-using-gravitino.md#fileset-operations) for more details.
For instructions on how to create a fileset, refer to [Fileset configurations](./hadoop-catalog.md#fileset-properties) for more details.

## Using Hadoop catalog with OSS
## Example of creating Hadoop catalog/schema/fileset with OSS

This section will show you how to use the Hadoop catalog with OSS in Gravitino, including detailed examples.

### Create a Hadoop catalog with OSS
### Step1: Create a Hadoop catalog with OSS

First, you need to create a Hadoop catalog for OSS. The following examples demonstrate how to create a Hadoop catalog with OSS:

Expand Down
12 changes: 6 additions & 6 deletions docs/hadoop-catalog-with-s3.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ $ bin/gravitino-server.sh start
Once the server is running, you can proceed to create the Hadoop catalog with S3.


## Create a Hadoop Catalog with S3
## Configurations for creating a Hadoop catalog with S3

### Configuration for S3 Hadoop Catalog
### Configurations for S3 Hadoop Catalog

In addition to the basic configurations mentioned in [Hadoop-catalog-catalog-configuration](./hadoop-catalog.md#catalog-properties), the following properties are necessary to configure a Hadoop catalog with S3:

Expand All @@ -37,13 +37,13 @@ In addition to the basic configurations mentioned in [Hadoop-catalog-catalog-con
| `s3-access-key-id` | The access key of the AWS S3. | (none) | Yes if it's a S3 fileset. | 0.7.0-incubating |
| `s3-secret-access-key` | The secret key of the AWS S3. | (none) | Yes if it's a S3 fileset. | 0.7.0-incubating |

### Configuration for a schema
### Configurations for a schema

To learn how to create a schema, refer to [Schema operation](./manage-fileset-metadata-using-gravitino.md#schema-operations).
To learn how to create a schema, refer to [Schema configurations](./hadoop-catalog.md#schema-properties).

### Configuration for a fileset
### Configurations for a fileset

For more details on creating a fileset, Refer to [Fileset operation](./manage-fileset-metadata-using-gravitino.md#fileset-operations).
For more details on creating a fileset, Refer to [Fileset configurations](./hadoop-catalog.md#fileset-properties).


## Using the Hadoop catalog with S3
Expand Down

0 comments on commit 44014d9

Please sign in to comment.