Skip to content

Commit

Permalink
#394 Add documentation for the added feature
Browse files Browse the repository at this point in the history
  • Loading branch information
yruslan committed Jun 30, 2021
1 parent 1dfb523 commit 00ac5b5
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,17 @@ $ spark-shell --jars spark-cobol-assembly-2.2.2-SNAPSHOT.jar

## Other Features

### Loading several paths
Currently, specifying multiple paths in `load()` is not supported. Use the following syntax:
```scala
spark
.read
.format("cobol")
.option("copybook_contents", copybook)
.option("paths", inputPaths.mkString(","))
.load()
```

### Spark SQL schema extraction
This library also provides convenient methods to extract Spark SQL schemas and Cobol layouts from copybooks.

Expand Down Expand Up @@ -1075,6 +1086,7 @@ Again, the full example is available at

| Option (usage example) | Description |
| ------------------------------------------ |:----------------------------------------------------------------------------- |
| .option("paths", "/path1,/path2") | Allows loading data from multiple unrelated paths on the same filesystem. |
| .option("record_length", "100") | Overrides the length of the record (in bypes). Normally, the size is derived from the copybook. But explicitly specifying record size can be helpful for debugging fixed-record length files. |
| .option("file_start_offset", "0") | Specifies the number of bytes to skip at the beginning of each file. |
| .option("file_end_offset", "0") | Specifies the number of bytes to skip at the end of each file. |
Expand Down

0 comments on commit 00ac5b5

Please sign in to comment.