docs: Update ingest self-hosted postgres to use the console #30516
Conversation
kay-kim
commented
Nov 16, 2024
kay-kim
commented
- Added the console instructions to the Ingest data > Postgres > self-hosted page.
- Since I already had the pictures, and the Console pages have been getting some traction, reused the content to make the Console > Create New page.
- Added an ingest-data page content since I needed a common place to send people before creating their source. Very bare-bones for now.
| @@ -261,7 +261,40 @@ scenarios, we recommend separating your workloads into multiple clusters for | |||
| [resource isolation](https://materialize.com/docs/sql/create-cluster/#resource-isolation). | |||
| {{< /note >}} | |||
|
|
|||
| {{% postgres-direct/create-a-cluster %}} | |||
|  | |||
There was a problem hiding this comment.
There was a problem hiding this comment.
The create-a-cluster shortcut adds additional context that is now being discarded and/or pigeonholed into only one of the options (unless that's your intention). We could instead just swap the cluster creation-specific bit by using a tab view that provides both options (Materialize console / SQL) and defaults to Materialize console.
There was a problem hiding this comment.
Will tab -- wasn't sure how strongly we wanted users to go the console route.
| @@ -0,0 +1,65 @@ | |||
| --- | |||
| title: "Ingest data" | |||
There was a problem hiding this comment.
https://preview.materialize.com/materialize/30516/ingest-data/
I needed a centralized place to send people from the create a new source -- since they would need to configure upstream before they can create the connection and source. It really is like jenga.
Later on, will need to also add landing pages to those folders where we don't have any. But, they weren't needed for this pull.
There was a problem hiding this comment.
👍 -- left the menu specification in the config.toml (meaning, when people click ingest-data folder from the left-hand toc, it'll only expand but not land on the page. people can still land if they do ..../ingest-data/). When we get Steffen's real landing page, will move that to the frontmatter of the page so that when people click on the folder, the page content displays.
| identifier: console-create-new | ||
| --- | ||
|
|
||
| From the Console, you can create new [clusters](/concepts/clusters/ "Isolated |
There was a problem hiding this comment.
kay-kim
force-pushed
the
docs-console-create-connection
branch
from
f52bbcc
to
d9f2e5b
Compare
| | **Name** | A name for the connection. | | ||
| | **Schema** | A Materialize database and schema for the connection. | |
There was a problem hiding this comment.
Could we separate these out into the different sections, like in the console? It's pretty confusing otherwise, since these two options refer to Materialize, and the options below to PostgreSQL.
There was a problem hiding this comment.
Also, not sure how useful it is to document each field in the documentation. If it's unclear what the field refers to, it should be annotated in the UI itself, which can then point to the CREATE CONNECTION reference documentation. This also makes it more cumbersome to make changes to the source creation workflow in the console, and unnecessarily creates multiple definitions for the same concepts we already document elsewhere for the underlying commands.
There was a problem hiding this comment.
Personally agree with you on limited usefulness on each field (and definitely, difficult to maintain + again with you 💯 on it should be in the UI). While I have found in the past that some readers do seem to prefer having these, I'll see if I can make it less specific fields in the patch (since, later on, as we get more users -- if it turns out we need it here, we can add it back here).
| Before it starts consuming the replication stream, Materialize takes a snapshot of the relevant tables in your publication. Until this snapshot is complete, Materialize won't have the same view of your data as your PostgreSQL database. | ||
|
|
||
| 
There was a problem hiding this comment.
Oh wow! That did not show up on mine (albeit, my 2 tables had like 10 rows or whatnot) -- but looking at my screen recording did not show this. Eh.
| From the [**Database object explorer**](/console/data/) in the [Materialize | ||
| console], you can see the status of the source you created. | ||
|
|
||
| Before it starts consuming the replication stream, Materialize takes a snapshot of the relevant tables in your publication. Until this snapshot is complete, Materialize won't have the same view of your data as your PostgreSQL database. |
There was a problem hiding this comment.
"Materialize won't have the same view of your data as your PostgreSQL database" suggests that the source can be queried while it's snapshotting — it can't. So we should adapt this sentence (in all guides) to instead make it clear that snapshotting must complete before you can query the source.
Co-authored-by: Marta Paes <[email protected]>
