feat: CSV import (#255)

Feature properties

* Configurable CSV dialect (delimiter, doubleQuotes
* Parse CSV files with headings or fields declared.
* URI handling and customizable glossary title.
* Skip lines with error and proceed
* Group same name columns

Fixes

* fix: Be tolerant about additional content-type info (e.g. ;encoding=utf8)
* fix: schema config doc linking

Tests

* test: New test cases.
* test: New baseline.

Maintenance

* chore: replace deprecated substr() with substring()
* chore(docs): Update docs
* chore(docs): Indicate web links

.eslintrc.json

@@ -17,7 +17,7 @@
     "rules": {
         "semi": [ "error", "always" ]
         ,"quotes": [ "error", "double" ]
-        ,"indent": [ "error", 4 ]
+        ,"indent": [ "error", 4, { "SwitchCase": 1 } ]
         ,"dot-location": ["error", "property"]
         ,"dot-notation": ["error"]
         ,"linebreak-style": [ "error", "unix" ]

conf/README.md

@@ -1,97 +1,74 @@
-# Configuration
+# Configure
 
-[doc-readme]: ../README.md#install
+[doc-readme]: ../doc/install.md
+[doc-cli]: ../doc/cli.md
 
-- [Config Format v5](./v5/doc/schema.md).
 
-## Generate a config file with `--init`
+[All config options (Config Format v5)](./v5/doc/schema.md)
+~~~
+npx glossarify-md --config [file]
+~~~
 
-Use `--init` to write a *minimal* config to the console.
-- add `--more` to write a config with more options and default values
-- add `--local` to load the config schema from the `node_modules` directory
-- add `--new`  to write a config to `./glossarify-md.conf.json` and a glossary to `./docs/glossary.md`
+## Generate a config file
 
-Examples:
+~~~
+npx glossarify-md --init
+~~~
 
-Write config to console
+- add `--new`
+  - to write to a config file `./glossarify-md.conf.json`
+  - to create a glossary file `./docs/glossary.md`
+- add `--local` to load the config `$schema` from `./node_modules` ([see below](#local-schema))
+- add `--more` to write a verbose config with more options and their default values
+
+*Example: Writing console outputs to a file*
+~~~
+npx glossarify-md --init --local > my-own.conf.json
 ~~~
-npx glossarify-md --init --local
 
+
+*Example: Minimal configuration*
+~~~
 {
   "$schema": "./node_modules/glossarify-md/conf/v5/schema.json",
   "baseDir": "./docs",
   "outDir": "../docs-glossarified"
 }
 ~~~
 
-Write config from console to a file
-~~~
-npx glossarify-md --init --local > glossarify-md.conf.json
-~~~
 
-Write config with default filename and initialize a ./doc/ directory with a glossary file.
-~~~
-npx glossarify-md --init --local --new
-~~~
 
 > **ⓘ Paths**
 >
 > 1. `baseDir` and `$schema` are resolved relative to the config file or current working directory (when passed via CLI)
 > 1. all other paths  are resolved relative to `baseDir`
 > 1. `outDir` *must not* be in `baseDir`so, if relative, must step out of `baseDir`
-
+> 1. prefer relative paths over absolute paths in the configuration
 
 ## Editor Support
 
-Many IDEs and editors provide suggestions when editing JSON files that refer to a JSON Schema using a conventional `$schema` property. There are two ways to refer to a glossarify-md config schema.
+Many IDEs and editors provide suggestions for JSON files with a `$schema` property locating a JSON Schema.
 
-### Local Referencing with `--local` 
+### Local `$schema`
 
-Local referencing is the **recommended** way **when glossarify-md was [installed "locally"][doc-readme] to a project**. It references the config schema of the glossarify-md version installed to the `./node_modules/` folder of the project. This way your editor can suggest all the latest options supported by the installed version.
+... is **recommended** when glossarify-md was [installed][doc-readme] using `npm install` (*without* the `-g` switch). If supported, an editor loads the schema from the `./node_modules/` directory of your project, thus it can suggest you the config options matching your currently installed version of glossarify-md.
 
-*glossarify-md.conf.json*
-~~~
+~~~json
 {
   "$schema": "./node_modules/glossarify-md/conf/v5/schema.json"
 }
 ~~~
 
-### Web References
+### Remote `$schema`
 
-Web references  are intended to be used when glossarify-md was installed "globally" using `npm install -g`. They will be generated when omitting the `--local` option.
+...may be used when glossarify-md was installed using `npm install -g`. It requires editors downloading the schema from a remote location.
 
-*glossarify-md.conf.json*
-~~~
-{
-  "$schema": "https://raw.githubusercontent.com/about-code/glossarify-md/v5.1.0/conf/v5/schema.json"
-}
-~~~
-
-> **ⓘ** Note that the web reference contains a config *format version* `v5` but also a particular *release version* `v5.1.0`. The latter is the version by the glossarify-md release that generated the config. If you later install a newer version of glossarify-md (say `v5.2.0`) web references in your config files **will only get upgraded on breaking changes to the configuration format**. Thus your config files may keep on referring to `v5.1.0`. Due to backwards compatibility this isn't a problem but your editor won't suggest you *the latest* options that could be used with glossarify-md `v5.2.0`. For this to happen change the *release version* in the path, manually.
-
-
-
-<!--
-If you use `latest` release version your editor will suggest you options from
-the 'latest' tag. But there may be options not yet supported by the release
-you've installed, locally. Keep that in mind otherwise you're wasting time
-trying things that can't work until you update.
-
-*glossarify-md.conf.json*
-~~~
+~~~json
 {
-  "$schema": "https://raw.githubusercontent.com/about-code/glossarify-md/latest/conf/v5/schema.json"
+  "$schema": "https://raw.githubusercontent.com/about-code/glossarify-md/v6.1.0/conf/v5/schema.json"
 }
 ~~~
 
-> **Note:** If your editor doesn't validate against the latest version it may have cached an older version.
-
-IMPORTANT:
-When introducing a new config format version KEEP the previous format's /conf/v.. folder.
-Otherwise moving the 'latest' tag forward onto a new revision which misses the old folder would
-cause $schema-URLs onto the old path to break, although still widely in public use:
-
-https://raw.githubusercontent.com/about-code/glossarify-md/latest/conf/---BREAKING--/schema.json
-
-We may only remove versions after they phased out and will no longer be supported.
--->
+> **ⓘ Note** The remote `$schema` URL contains a config *format version* (`v5`) and a glossarify-md *release version* (`v6.1.0`). The release version is the glossarify-md version *which initially generated the config file*. When you install a newer release of glossarify-md (say `v6.2.0`) the URL *won't* be updated and keeps on referring to `v6.1.0`. Therefore, your editor or IDE won't suggest you *the latest* options available for `v6.2.0` when editing the config file. However, glossarify-md will inform you when it detects this situation.
+>
+> On breaking changes affecting the *format version* glossarify-md will attempt to provide assistance on upgrading your configuration files including the `$schema` URL. This should happen *rarely*, though.

conf/v5/doc/schema-defs-csvdialect-properties-delimiter.md

@@ -0,0 +1,7 @@
+## delimiter Default Value
+
+The default value is:
+
+```json
+";"
+```

conf/v5/doc/schema-defs-csvdialect-properties-escapechar.md

@@ -0,0 +1,7 @@
+## escapeChar Default Value
+
+The default value is:
+
+```json
+"\""
+```

conf/v5/doc/schema-defs-csvdialect-properties-quotechar.md

@@ -0,0 +1,7 @@
+## quoteChar Default Value
+
+The default value is:
+
+```json
+"\""
+```

conf/v5/doc/schema-defs-csvdialect.md

@@ -0,0 +1,67 @@
+## dialect Default Value
+
+The default value is:
+
+```json
+{
+  "delimiter": ";"
+}
+```
+
+# dialect Properties
+
+
+
+## delimiter
+
+A character sequence to use as the field separator.
+
+`delimiter`
+
+*   is required
+
+*   Type: `string`
+
+### delimiter Default Value
+
+The default value is:
+
+```json
+";"
+```
+
+## quoteChar
+
+A one-character string for surrounding field values in CSV data (no matter whether being a numeric value or an alphanumeric value or something else). Uses the quote character " by default. A CSV field whose field value is text containing quotes is required to embed the whole text value between two `quoteChar` as well as escaping the quotes in the text data using `escapeChar`. For example a raw value of ;This is "quoted" text; is expected to be encoded in CSV as ;"This is ""quoted"" text"; where the outer quotes are `quoteChars` and the inner quotes each are preceeded by an `escapeChar` (which is a quote character by default, either).
+
+`quoteChar`
+
+*   is optional
+
+*   Type: `string`
+
+### quoteChar Default Value
+
+The default value is:
+
+```json
+"\""
+```
+
+## escapeChar
+
+Specifies a one-character string to use as an escape character within a field value. Uses the quote " character as a default which requires encoding quotes in text data using two consecutive quotes (one being the escape character and one being the actual quote of the data). This is a very common encoding scheme. However, change this to an empty string to disable escaping, completely or use another character as needed.
+
+`escapeChar`
+
+*   is optional
+
+*   Type: `string`
+
+### escapeChar Default Value
+
+The default value is:
+
+```json
+"\""
+```

conf/v5/doc/schema-defs-csvtableschema-properties-fields-items.md

@@ -0,0 +1,21 @@
+# items Properties
+
+
+
+## name
+
+A name for this field.
+
+`name`
+
+*   is optional
+
+*   Type: `string`
+
+### name Default Value
+
+The default value is:
+
+```json
+"http://www.w3.org/2004/02/skos/core#"
+```

conf/v5/doc/schema-defs-csvtableschema-properties-fields.md

@@ -0,0 +1,3 @@
+## fields Constraints
+
+**minimum number of items**: the minimum number of items for this array is: `1`

conf/v5/doc/schema-defs-csvtableschema.md

@@ -0,0 +1,17 @@
+# schema Properties
+
+
+
+## fields
+
+
+
+`fields`
+
+*   is required
+
+*   Type: `object[]` ([Details](schema-defs-csvtableschemafield.md))
+
+### fields Constraints
+
+**minimum number of items**: the minimum number of items for this array is: `1`

conf/v5/doc/schema-defs-csvtableschemafield-properties-name.md

@@ -0,0 +1,7 @@
+## name Default Value
+
+The default value is:
+
+```json
+"http://www.w3.org/2004/02/skos/core#"
+```

conf/v5/doc/schema-defs-csvtableschemafield.md

@@ -0,0 +1,21 @@
+# items Properties
+
+
+
+## name
+
+A name for this field.
+
+`name`
+
+*   is optional
+
+*   Type: `string`
+
+### name Default Value
+
+The default value is:
+
+```json
+"http://www.w3.org/2004/02/skos/core#"
+```

conf/v5/doc/schema-defs-csvtableschemafieldname.md

@@ -0,0 +1,7 @@
+## name Default Value
+
+The default value is:
+
+```json
+"http://www.w3.org/2004/02/skos/core#"
+```

conf/v5/doc/schema-defs-glossaryfile-properties-import.md

@@ -4,7 +4,7 @@
 
 ## file
 
-The JSON file to import terms from.
+The file to import terms from. Supported file content types: 'application/json', 'application/ld+json', 'application/n-quads'.
 
 `file`
 
@@ -14,7 +14,7 @@ The JSON file to import terms from.
 
 ## context
 
-File path or URL to a custom JSON-LD context document. Expected to map attributes and type names of a custom import document format onto terms of the well-known W3C SKOS vocabulary.
+File path or URL to a custom JSON-LD context document (application/ld+json) mapping format terminology (attributes, type names) of a JSON data document ('application/json') onto well-known W3C SKOS terminology.
 
 `context`
 

conf/v5/doc/schema-defs-glossaryfileimport-if-properties-file.md

@@ -0,0 +1,9 @@
+## file Constraints
+
+**pattern**: the string must match the following regular expression: 
+
+```regexp
+^(.*).csv$
+```
+
+[try pattern](https://regexr.com/?expression=%5E\(.\*\).csv%24 "try regular expression with regexr.com")

conf/v5/doc/schema-defs-glossaryfileimport-if.md

@@ -0,0 +1,23 @@
+# if Properties
+
+
+
+## file
+
+
+
+`file`
+
+*   is optional
+
+*   Type: `string`
+
+### file Constraints
+
+**pattern**: the string must match the following regular expression: 
+
+```regexp
+^(.*).csv$
+```
+
+[try pattern](https://regexr.com/?expression=%5E\(.\*\).csv%24 "try regular expression with regexr.com")