Skip to content

Commit

Permalink
Docs Refresh: S3 Destination (#6704)
Browse files Browse the repository at this point in the history
  • Loading branch information
avaidyanatha authored Oct 4, 2021
1 parent ee05b31 commit fd2f96e
Showing 1 changed file with 10 additions and 11 deletions.
21 changes: 10 additions & 11 deletions docs/integrations/destinations/s3.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
# S3

## Overview

This destination writes data to S3 bucket.

The Airbyte S3 destination allows you to sync data to AWS S3/ Minio S3. Each stream is written to its own directory under the bucket.

## Sync Mode
## Features

| Feature | Support | Notes |
| :--- | :---: | :--- |
Expand All @@ -15,6 +9,12 @@ The Airbyte S3 destination allows you to sync data to AWS S3/ Minio S3. Each str
| Incremental - Deduped History || As this connector does not support dbt, we don't support this sync mode on this destination. |
| Namespaces || Setting a specific bucket path is equivalent to having separate namespaces. |

The Airbyte S3 destination allows you to sync data to AWS S3 or Minio S3. Each stream is written to its own directory under the bucket.

## Troubleshooting

**Issue: `Failed to parse XML document with handler class` with failed sync. [\[Solution\]](https://discuss.airbyte.io/t/destination-s3-failed-to-upload-files/113/2)**

## Configuration

| Parameter | Type | Notes |
Expand Down Expand Up @@ -348,15 +348,14 @@ These parameters are related to the `ParquetOutputFormat`. See the [Java doc](ht

Under the hood, an Airbyte data stream in Json schema is first converted to an Avro schema, then the Json object is converted to an Avro record, and finally the Avro record is outputted to the Parquet format. See the `Data schema` section from the [Avro output](#avro) for rules and limitations.

## Getting started
## Getting Started (Airbyte Open-Source / Airbyte Cloud)

### Requirements
#### Requirements

1. Allow connections from Airbyte server to your AWS S3/ Minio S3 cluster \(if they exist in separate VPCs\).
2. An S3 bucket with credentials.

### Setup guide

#### Setup Guide
* Fill up S3 info
* **S3 Endpoint**
* Leave empty if using AWS S3, fill in S3 URL if using Minio S3.
Expand Down

0 comments on commit fd2f96e

Please sign in to comment.