Skip to content

Commit

Permalink
Blog: Update about namespace enforcement (#6753)
Browse files Browse the repository at this point in the history
  • Loading branch information
ajantha-bhat authored May 8, 2023
1 parent 0ab18bf commit fdb1f1c
Show file tree
Hide file tree
Showing 3 changed files with 51 additions and 7 deletions.
1 change: 1 addition & 0 deletions site/docs/blog/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
Highlights

* [Rolling upgrade issue to 0.26.0](incident-2022-05.md) (May 2022)
* [Namespace enforcement](namespace-enforcement.md) (May 2023)
13 changes: 13 additions & 0 deletions site/docs/blog/namespace-enforcement.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Namespace enforcement in Nessie

Starting from Nessie version 0.52.3, it is required to have existing Namespaces before creating or committing to tables.
In case of tables that have implicit namespaces, new commits will fail after the upgrade as this rule will be enforced.
Therefore, it is necessary to explicitly create the namespaces.

Namespaces can be created using any of these options:
* Using SQL command (for Spark, `CREATE NAMESPACE catalogName.namespaceName`)
* Using [Iceberg Catalog API](https://github.com/apache/iceberg/blob/3ab00171b48bb793a3b71845eb12d5077ba892f1/nessie/src/main/java/org/apache/iceberg/nessie/NessieCatalog.java#L262).
* Using [content generator tool](https://github.com/projectnessie/nessie/blob/main/tools/content-generator/README.md) to batch create the missing namespaces with `create-missing-namespaces` option.

[More info from the spec](https://github.com/projectnessie/nessie/blob/main/api/NESSIE-SPEC-2-0.md#200-beta1).
[Issue background](https://github.com/projectnessie/nessie/issues/6244).
44 changes: 37 additions & 7 deletions tools/content-generator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,43 @@ Functionality supported:
for "expected duration" divided by "number of commits" (default: no sleep)
* Generate content for N tables (default: 1 table)
* Generate content for any content type (default: Iceberg-table)
* Read references, commits and content objects
* Delete content objects
* Create missing namespaces


```
$ java -jar tools/content-generator/target/nessie-content-generator-0.12.2-SNAPSHOT.jar help generate
$ java -jar nessie-content-generator-0.58.1.jar -h
Usage: nessie-content-generator [-hV] [-u=<uri>] [COMMAND]
-h, --help Show this help message and exit.
-u, --uri=<uri> Nessie API endpoint URI, defaults to http://localhost:
19120/api/v2.
-V, --version Print version information and exit.
Commands:
generate Generate commits
commits Read commits
refs Read references
content Read content objects
content-refresh Get and Put content objects without changes to
refresh their storage model
delete Delete content objects
create-missing-namespaces Creates missing namespaces for content keys at
branch HEADs.
help Display help information about the specified
command.
```


Usage: nessie-content-generator generate [-hV] [-b=<branchCount>]
[-d=<runtimeDuration>] [-D=<defaultBranchName>] [-n=<numCommits>]
[-t=<numTables>] [-T=<newTagProbability>] [--type=<contentType>]
[-u=<uri>]
```
$ java -jar nessie-content-generator-0.58.1.jar help generate
Usage: nessie-content-generator generate [-hvV] [--continue-on-error]
[-b=<branchCount>] [-d=<runtimeDuration>] [-D=<defaultBranchName>]
[--key-pattern=<keyPattern>] [-n=<numCommits>]
[--puts-per-commit=<putsPerCommit>] [-t=<numTables>]
[-T=<newTagProbability>] [--type=<contentType>] [-u=<uri>]
Generate commits
-b, --num-branches=<branchCount>
Number of branches to use.
--continue-on-error
-d, --duration=<runtimeDuration>
Runtime duration, equally distributed among the
number of commits to create. See java.time.
Expand All @@ -30,8 +55,12 @@ Generate commits
Name of the default branch, uses the server's
default branch if not specified.
-h, --help Show this help message and exit.
--key-pattern=<keyPattern>
-n, --num-commits=<numCommits>
Number of commits to create.
--puts-per-commit=<putsPerCommit>
-t, --num-tables=<numTables>
Number of table names, each commit chooses a
random table (content-key).
Expand All @@ -42,6 +71,7 @@ Generate commits
ICEBERG_TABLE. Possible values: ICEBERG_TABLE,
ICEBERG_VIEW, DELTA_LAKE_TABLE
-u, --uri=<uri> Nessie API endpoint URI, defaults to http:
//localhost:19120/api/v1.
//localhost:19120/api/v2.
-v, --verbose Produce verbose output (if possible)
-V, --version Print version information and exit.
```

0 comments on commit fdb1f1c

Please sign in to comment.