MeltanoLabs · Apr 6, 2022 · Apr 6, 2022 · Apr 6, 2022 · Apr 6, 2022 · Apr 6, 2022
diff --git a/.editorconfig b/.editorconfig
@@ -0,0 +1,14 @@
+root = true
+
+[*]
+end_of_line = lf
+insert_final_newline = true
+charset = utf-8
+indent_style = space
+
+[*.json]
+indent_size = 2
+
+[*.py]
+indent_style = space
+indent_size = 4
diff --git a/README.md b/README.md
@@ -7,12 +7,11 @@ Built with the [Meltano SDK](https://sdk.meltano.com) for Singer Taps and Target
 
 ## Capabilities
 
+* `state`
 * `sync`
 * `catalog`
 * `discover`
 
-Note: This tap currently does not support incremental state.
-
 ## Settings
 
 | Setting             | Required | Default | Description |
@@ -22,25 +21,52 @@ Note: This tap currently does not support incremental state.
 
 A full list of supported settings and capabilities is available by running: `tap-csv --about`
 
-The `config.json` contains an array called `files` that consists of dictionary objects detailing each destination table to be passed to Singer. Each of those entries contains: 
+The `config.json` contains an array called `files` that consists of dictionary objects detailing each destination table to be passed to Singer. Each of those entries contains:
 * `entity`: The entity name to be passed to singer (i.e. the table)
-* `path`: Local path to the file to be ingested. Note that this may be a directory, in which case all files in that directory and any of its subdirectories will be recursively processed
+* `path`: Local path to the file to be ingested. Note that this may be a directory, in which case all files in that directory be processed if they also match the `prefix`.
+* `prefix`: The required (case-insensitive) prefix of the filename for it to be processed.
 * `keys`: The names of the columns that constitute the unique keys for that entity
+* `start_from`: The replication key to start from. The format is `filename:rownumber` in lowercase.
+* `delimiter`: What separator is used in the CSV files.
+* `dialect`: What dialect is used in the CSV files. See <https://docs.python.org/3/library/csv.html#csv.list_dialects>
+* `encoding`: What encoding is used in the CSV files.
+* `quotechar`: What character is used for quoting in the CSV files.
+* `header`: In case some or all files are missing a header, it can be specified here. Rows which match the specified header are skipped. So files with or without headers can be included in the same batch.
+
+It is also possible to define a global `path` value which will be used as a fallback in case `path` is not specified for
+the individual files object.
 
 Example:
 
 ```json
 {
-	"files":	[ 	
-					{	"entity" : "leads",
-						"path" : "/path/to/leads.csv",
-						"keys" : ["Id"]
-					},
-					{	"entity" : "opportunities",
-						"path" : "/path/to/opportunities.csv",
-						"keys" : ["Id"]
-					}
-				]
+  "path": "/path/to/default/",
+  "files": [
+    {
+      "entity": "leads",
+      "path": "/path/to/leads/",
+      "prefix": "lead_con",
+      "delimiter": ",",
+      "dialect": "excel",
+      "encoding": "utf-8",
+      "quotechar": "\"",
+      "header": [
+        "Id",
+        "Column1",
+        "Column2"
+      ],
+      "keys": [
+        "Id"
+      ],
+      "start_from": "lead_contracts.csv:8"
+    },
+    {
+      "entity": "opportunities",
+      "keys": [
+        "Id"
+      ]
+    }
+  ]
 }
 ```
 
@@ -49,29 +75,35 @@ Optionally, the files definition can be provided by an external json file:
 **config.json**
 ```json
 {
+  "path": "/path/to/default/",
 	"csv_files_definition": "files_def.json"
 }
 ```
 
-
 **files_def.json**
 ```json
-[ 	
-	{	"entity" : "leads",
-		"path" : "/path/to/leads.csv",
-		"keys" : ["Id"]
-	},
-	{	"entity" : "opportunities",
-		"path" : "/path/to/opportunities.csv",
-		"keys" : ["Id"]
-	}
+[
+  {
+    "entity": "leads",
+    "path": "/path/to/leads.csv",
+    "keys": [
+      "Id"
+    ]
+  },
+  {
+    "entity": "opportunities",
+    "path": "opportunities.csv",
+    "keys": [
+      "Id"
+    ]
+  }
 ]
 ```
 
 ## Installation
 
 ```bash
-pipx install git+https://github.com/MeltanoLabs/tap-csv.git
+pipx install git+https://github.com/youcruit/tap-csv.git
 ```
 
 ## Usage
@@ -142,5 +174,5 @@ meltano elt tap-csv target-jsonl
 
 ### SDK Dev Guide
 
-See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to 
+See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to
 develop your own taps and targets.