feat!: streaming CAR packing (#148)

Features:

* Uses new JS UnixFS library to generate CARs.
* All CLI commands now accept input on `stdin` and write to `stdout` by default.
* In the CLI, CAR files streamed to disk are then updated to have correct root CID in header.
* Due to streaming generation this library is now around 50% faster than the previous version.
* Uses new https://github.com/web3-storage/car-block-validator library when unpacking.
     * Adds a `--no-verify` flag to disable
* Ability to work with CAR files with no root CID in the CAR header by searching for "implicit" root CIDs in the blocks it contains.

Removals:

* The programmatic API for unpacking a CAR has been removed.
    * There's not really one good way to do this right now and not really a pure streaming version (the current `unpackStream` function buffers in memory) which doesn't feel in the spirit of this library anymore. All `unpack` does currently is take a `CarReader`, wrap it in a blockstore interface and give it directly to `ipfs-unixfs-exporter`, which isn't adding loads of value...so I just, removed it and recommended that people just use `@ipld/car` and `ipfs-unixfs-exporter` modules instead. We can always add it back if folks are desperate for it...

Additional changes:

* Switched to using `sade` for CLI
* Updated CI to test on Node.js 18 & 19
* Switched to pure JS with no build/compile step
* ESM only
* Added coverage reporting and 100% line coverage with tests
* Consolidated license files into single file

BREAKING CHANGE: The programmatic API has changed significantly, see the README for new streaming API. The CLI "commands" like `--pack` have changed to `pack` (i.e. without dashes) but are largely very similar. In the CLI, CAR files written to stdout or piped to another program (i.e. not written to disk using `--output`) will not have a root CID in the CAR header. Minimum Node.js version for the CLI has changed to 18.

.github/workflows/main.yml

@@ -16,7 +16,6 @@ jobs:
         with:
           node-version: ${{matrix.node}}
       - uses: bahmutov/npm-install@v1
-      - run: npm run build
       - name: Lint
         run: npm run lint
   test-node:
@@ -25,23 +24,23 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [windows-latest, ubuntu-latest, macos-latest]
-        node: [14, 16]
+        os: [ubuntu-latest, macos-latest]
+        node: [18, 19]
     steps:
       - uses: actions/checkout@v2
       - uses: actions/setup-node@v2
         with:
           node-version: ${{matrix.node}}
       - uses: bahmutov/npm-install@v1
-      - name: Unit tests
+      - name: Test Node.js
         run: npm run test:node
   test-browser:
     needs: check
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2
       - uses: bahmutov/npm-install@v1
-      - name: Unit tests
+      - name: Test Browser
         run: npm run test:browser
   changelog:
     if: github.event_name == 'push' && github.ref == 'refs/heads/main'
@@ -71,8 +70,7 @@ jobs:
           registry-url: https://registry.npmjs.org/
           node-version: 16
       - uses: bahmutov/npm-install@v1
-      - run: npm run build
-      - name: NPM Publish
+      - name: Publish
         env:
           NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
         run: npm publish --access=public

.gitignore

@@ -1,107 +1,2 @@
-# Logs
-logs
-*.log
-npm-debug.log*
-yarn-debug.log*
-yarn-error.log*
-lerna-debug.log*
-
-# Diagnostic reports (https://nodejs.org/api/report.html)
-report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
-
-# Runtime data
-pids
-*.pid
-*.seed
-*.pid.lock
-
-# Directory for instrumented libs generated by jscoverage/JSCover
-lib-cov
-
-# Coverage directory used by tools like istanbul
+node_modules
 coverage
-*.lcov
-
-# nyc test coverage
-.nyc_output
-
-# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
-.grunt
-
-# Bower dependency directory (https://bower.io/)
-bower_components
-
-# node-waf configuration
-.lock-wscript
-
-# Compiled binary addons (https://nodejs.org/api/addons.html)
-build/Release
-
-# Dependency directories
-node_modules/
-jspm_packages/
-
-# TypeScript v1 declaration files
-typings/
-
-# TypeScript cache
-*.tsbuildinfo
-
-# Optional npm cache directory
-.npm
-
-# Optional eslint cache
-.eslintcache
-
-# Microbundle cache
-.rpt2_cache/
-.rts2_cache_cjs/
-.rts2_cache_es/
-.rts2_cache_umd/
-
-# Optional REPL history
-.node_repl_history
-
-# Output of 'npm pack'
-*.tgz
-
-# Yarn Integrity file
-.yarn-integrity
-
-# dotenv environment variables file
-.env
-.env.test
-
-# parcel-bundler cache (https://parceljs.org/)
-.cache
-
-# Next.js build output
-.next
-
-# Nuxt.js build / generate output
-.nuxt
-dist
-
-# Gatsby files
-.cache/
-# Comment in the public line in if your project uses Gatsby and *not* Next.js
-# https://nextjs.org/blog/next-9-1#public-directory-support
-# public
-
-# vuepress build output
-.vuepress/dist
-
-# Serverless directories
-.serverless/
-
-# FuseBox cache
-.fusebox/
-
-# DynamoDB Local files
-.dynamodb/
-
-# TernJS port file
-.tern-port
-
-# IDE files
-.idea/