-
Notifications
You must be signed in to change notification settings - Fork 134
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into fil/preview-exported-files
- Loading branch information
Showing
124 changed files
with
1,358 additions
and
261 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -65,7 +65,7 @@ const db2 = await DuckDBClient.of({base: FileAttachment("quakes.db")}); | |
| db2.queryRow(`SELECT COUNT() FROM base.events`) | ||
| ``` | ||
|
|
||
| For externally-hosted data, you can create an empty `DuckDBClient` and load a table from a SQL query, say using [`read_parquet`](https://duckdb.org/docs/guides/import/parquet_import) or [`read_csv`](https://duckdb.org/docs/guides/import/csv_import). DuckDB offers many affordances to make this easier (in many cases it detects the file format and uses the correct loader automatically). | ||
| For externally-hosted data, you can create an empty `DuckDBClient` and load a table from a SQL query, say using [`read_parquet`](https://duckdb.org/docs/guides/import/parquet_import) or [`read_csv`](https://duckdb.org/docs/guides/import/csv_import). DuckDB offers many affordances to make this easier. (In many cases it detects the file format and uses the correct loader automatically.) | ||
|
|
||
| ```js run=false | ||
| const db = await DuckDBClient.of(); | ||
|
|
@@ -105,3 +105,100 @@ const sql = DuckDBClient.sql({quakes: `https://earthquake.usgs.gov/earthquakes/f | |
| ```sql echo | ||
| SELECT * FROM quakes ORDER BY updated DESC; | ||
| ``` | ||
|
|
||
| ## Extensions <a href="https://github.com/observablehq/framework/releases/tag/v1.13.0" class="observablehq-version-badge" data-version="^1.13.0" title="Added in 1.13.0"></a> | ||
|
|
||
| [DuckDB extensions](https://duckdb.org/docs/extensions/overview.html) extend DuckDB’s functionality, adding support for additional file formats, new types, and domain-specific functions. For example, the [`json` extension](https://duckdb.org/docs/data/json/overview.html) provides a `read_json` method for reading JSON files: | ||
|
|
||
| ```sql echo | ||
| SELECT bbox FROM read_json('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson'); | ||
| ``` | ||
|
|
||
| To read a local file (or data loader), use `FileAttachment` and interpolation `${…}`: | ||
|
|
||
| ```sql echo | ||
| SELECT bbox FROM read_json(${FileAttachment("../quakes.json").href}); | ||
| ``` | ||
|
|
||
| For convenience, Framework configures the `json` and `parquet` extensions by default. Some other [core extensions](https://duckdb.org/docs/extensions/core_extensions.html) also autoload, meaning that you don’t need to explicitly enable them; however, Framework will only [self-host extensions](#self-hosting-of-extensions) if you explicitly configure them, and therefore we recommend that you always use the [**duckdb** config option](../config#duckdb) to configure DuckDB extensions. Any configured extensions will be automatically [installed and loaded](https://duckdb.org/docs/extensions/overview#explicit-install-and-load), making them available in SQL code blocks as well as the `sql` and `DuckDBClient` built-ins. | ||
|
|
||
| For example, to configure the [`spatial` extension](https://duckdb.org/docs/extensions/spatial/overview.html): | ||
|
|
||
| ```js run=false | ||
| export default { | ||
| duckdb: { | ||
| extensions: ["spatial"] | ||
| } | ||
| }; | ||
| ``` | ||
|
|
||
| You can then use the `ST_Area` function to compute the area of a polygon: | ||
|
|
||
| ```sql echo run=false | ||
| SELECT ST_Area('POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))'::GEOMETRY) as area; | ||
| ``` | ||
|
|
||
| To tell which extensions have been loaded, you can run the following query: | ||
|
|
||
| ```sql | ||
| FROM duckdb_extensions() WHERE loaded AND JSON '1'; | ||
| ``` | ||
|
|
||
| ```sql run=false | ||
| FROM duckdb_extensions() WHERE loaded; | ||
| ``` | ||
|
|
||
| <div class="warning"> | ||
|
|
||
| If the `duckdb_extensions()` function runs before DuckDB autoloads a core extension (such as `json`), it might not be included in the returned set. | ||
|
|
||
| </div> | ||
|
|
||
| ### Self-hosting of extensions | ||
|
|
||
| As with [npm imports](../imports#self-hosting-of-npm-imports), configured DuckDB extensions are self-hosted, improving performance, stability, & security, and allowing you to develop offline. Extensions are downloaded to the DuckDB cache folder, which lives in <code>.observablehq/<wbr>cache/<wbr>\_duckdb</code> within the source root (typically `src`). You can clear the cache and restart the preview server to re-fetch the latest versions of any DuckDB extensions. If you use an [autoloading core extension](https://duckdb.org/docs/extensions/core_extensions.html#list-of-core-extensions) that is not configured, DuckDB-Wasm [will load it](https://duckdb.org/docs/api/wasm/extensions.html#fetching-duckdb-wasm-extensions) from the default extension repository, `extensions.duckdb.org`, at runtime. | ||
|
|
||
| ## Configuring | ||
|
|
||
| The second argument to `DuckDBClient.of` and `DuckDBClient.sql` is a [`DuckDBConfig`](https://shell.duckdb.org/docs/interfaces/index.DuckDBConfig.html) object which configures the behavior of DuckDB-Wasm. By default, Framework sets the `castBigIntToDouble` and `castTimestampToDate` query options to true. To instead use [`BigInt`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt): | ||
|
|
||
| ```js run=false | ||
| const bigdb = DuckDBClient.of({}, {query: {castBigIntToDouble: false}}); | ||
| ``` | ||
|
|
||
| By default, `DuckDBClient.of` and `DuckDBClient.sql` automatically load all [configured extensions](#extensions). To change the loaded extensions for a particular `DuckDBClient`, use the **extensions** config option. For example, pass an empty array to instantiate a DuckDBClient with no loaded extensions (even if your configuration lists several): | ||
|
|
||
| ```js echo run=false | ||
| const simpledb = DuckDBClient.of({}, {extensions: []}); | ||
| ``` | ||
|
|
||
| Alternatively, you can configure extensions to be self-hosted but not load by default using the **duckdb** config option and the `load: false` shorthand: | ||
|
|
||
| ```js run=false | ||
| export default { | ||
| duckdb: { | ||
| extensions: { | ||
| spatial: false, | ||
| h3: false | ||
| } | ||
| } | ||
| }; | ||
| ``` | ||
|
|
||
| You can then selectively load extensions as needed like so: | ||
|
|
||
| ```js echo run=false | ||
| const geosql = DuckDBClient.sql({}, {extensions: ["spatial", "h3"]}); | ||
| ``` | ||
|
|
||
| In the future, we’d like to allow DuckDB to be configured globally (beyond just [extensions](#extensions)) via the [**duckdb** config option](../config#duckdb); please upvote [#1791](https://github.com/observablehq/framework/issues/1791) if you are interested in this feature. | ||
|
|
||
| ## Versioning | ||
|
|
||
| Framework currently uses [DuckDB-Wasm 1.29.0](https://github.com/duckdb/duckdb-wasm/releases/tag/v1.29.0) <a href="https://github.com/observablehq/framework/releases/tag/v1.13.0" class="observablehq-version-badge" data-version="^1.13.0" title="Added in 1.13.0"></a>, which aligns with [DuckDB 1.1.1](https://github.com/duckdb/duckdb/releases/tag/v1.1.1). You can load a different version of DuckDB-Wasm by importing `npm:@duckdb/duckdb-wasm` directly, for example: | ||
|
|
||
| ```js run=false | ||
| import * as duckdb from "npm:@duckdb/[email protected]"; | ||
| ``` | ||
|
|
||
| However, you will not be able to change the version of DuckDB-Wasm used by SQL code blocks or the `sql` or `DuckDBClient` built-ins, nor can you use Framework’s support for self-hosting extensions with a different version of DuckDB-Wasm. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.