Skip to content

Commit

Permalink
better SQL docs:
Browse files Browse the repository at this point in the history
* illustrates how to make sql fenced code blocks from a DuckDBClient isntance
* syncs the sql fenced code blocks with the mosaic db  (thanks, @tel!)
* gives better guidance for arbitrary query creation (with a safer approach for `xxx IN(${array})`)

supersedes #1849
  • Loading branch information
Fil committed Dec 3, 2024
1 parent 8efdcf4 commit 071a546
Show file tree
Hide file tree
Showing 3 changed files with 36 additions and 6 deletions.
13 changes: 12 additions & 1 deletion docs/lib/duckdb.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,11 @@ db.queryRow("SELECT count() AS count FROM gaia")

See the [DatabaseClient Specification](https://observablehq.com/@observablehq/database-client-specification) for more details on these methods.

Finally, the `DuckDBClient.sql` method <a href="https://github.com/observablehq/framework/releases/tag/v1.4.0" class="observablehq-version-badge" data-version="^1.4.0" title="Added in 1.4.0"></a> takes the same arguments as `DuckDBClient.of` and returns the corresponding `db.sql` tagged template literal. The returned function can be used to redefine the built-in [`sql` tagged template literal](../sql#sql-literals) and thereby change the database used by [SQL code blocks](../sql), allowing you to query dynamically-registered tables (unlike the **sql** front matter option).
## Custom setup

The `DuckDBClient.sql` method <a href="https://github.com/observablehq/framework/releases/tag/v1.4.0" class="observablehq-version-badge" data-version="^1.4.0" title="Added in 1.4.0"></a> takes the same arguments as `DuckDBClient.of` and returns the corresponding `db.sql` tagged template literal.

The returned function can be used to redefine the built-in [`sql` tagged template literal](../sql#sql-literals) and thereby change the database used by [SQL code blocks](../sql), allowing you to query dynamically-registered tables (unlike the **sql** front matter option).

```js
const feed = view(Inputs.select(new Map([["M4.5+", "4.5"], ["M2.5+", "2.5"], ["All", "all"]]), {label: "Earthquake feed"}));
Expand All @@ -106,6 +110,13 @@ const sql = DuckDBClient.sql({quakes: `https://earthquake.usgs.gov/earthquakes/f
SELECT * FROM quakes ORDER BY updated DESC;
```

The definition above is shorthand for:

```js run=false
const db = await DuckDBClient.of({quakes: `https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/${feed}_day.csv`});
const sql = db.sql.bind(db);
```

## Extensions <a href="https://github.com/observablehq/framework/releases/tag/v1.13.0" class="observablehq-version-badge" data-version="^1.13.0" title="Added in 1.13.0"></a>

[DuckDB extensions](https://duckdb.org/docs/extensions/overview.html) extend DuckDB’s functionality, adding support for additional file formats, new types, and domain-specific functions. For example, the [`json` extension](https://duckdb.org/docs/data/json/overview.html) provides a `read_json` method for reading JSON files:
Expand Down
1 change: 1 addition & 0 deletions docs/lib/mosaic.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ const db = await DuckDBClient.of({trips: FileAttachment("nyc-taxi.parquet")});
const coordinator = new vgplot.Coordinator();
coordinator.databaseConnector(vgplot.wasmConnector({duckdb: db._db}));
const vg = vgplot.createAPIContext({coordinator});
const sql = db.sql.bind(db);
```

The code below creates three views, coordinated by Mosaic’s [crossfilter](https://uwdata.github.io/mosaic/api/core/selection.html#selection-crossfilter) helper.
Expand Down
28 changes: 23 additions & 5 deletions docs/sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,18 +191,36 @@ The `sql` tag is available by default in Markdown. You can also import it explic
import {sql} from "npm:@observablehq/duckdb";
```

The `sql` tag is also useful for working around a current limitation of DuckDB-Wasm: prepared statements do not support array arguments. (Please upvote [#447](https://github.com/duckdb/duckdb-wasm/issues/447) if you run into this issue.) Instead of passing the array as a parameter, you can interpolate the array values directly into the SQL query.
For a more custom setup, see [DuckDBClient](./lib/duckdb#custom-setup).

<div class="tip">

DuckDB only supports interpolation of strings and numbers with `${…}`. To interpolate an array of values, such as a list of ids, serialize to JSON:

```js echo
const source_ids = [2028328031008716288n, 2076498116457016960n, 4315266827603868160n, 4123529214004874624n, 5312548578630777344n];
const ids = ["2028328031008716288", "2076498116457016960", "4315266827603868160", "4123529214004874624", "5312548578630777344"];
```

```sql echo
SELECT * FROM gaia
WHERE source_id::string IN JSON(${JSON.stringify(ids)});
```

If you need to create a query on the fly, say with inputs that drive the name of a field or table, you can call the `sql` tagged template literal directly.

```js echo
Inputs.table(await sql([`SELECT * FROM gaia WHERE source_id IN (${[source_ids]})`]))
const field = view(Inputs.select(["ra", "dec", "parallax"], {label: "field"}))
```

```js
display(extent);
```

<div class="warning">
```js echo
const query = `SELECT MIN(${field}) AS min, MAX(${field}) AS max FROM gaia;`;
const [extent] = await sql([query]);
```

When interpolating values into SQL queries, be careful to avoid [SQL injection](https://en.wikipedia.org/wiki/SQL_injection) by properly escaping or sanitizing user input. The example above is safe only because `source_ids` are known to be numeric.
Be careful to avoid [SQL injection](https://en.wikipedia.org/wiki/SQL_injection) by properly escaping or sanitizing user input.

</div>

0 comments on commit 071a546

Please sign in to comment.