Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc error: publication inheritance was added in PG 13, not PG 14 #520

Closed
james-johnston-thumbtack opened this issue Apr 10, 2023 · 3 comments

Comments

@james-johnston-thumbtack

The documentation suggests that publication inheritance was added in PG 14. Examples where PG 14 is referenced:

| publications | 14 | 10,11,12,13 |

-- NOTE: Native publication inheritance is only supported on PG14+

-- NOTE: Native publication inheritance is only supported on PG14+

-- NOTE: Native publication inheritance is only supported on PG14+

-- Fix default partition not being added to logical replication publication when using template table for PostgreSQL versions before 14. It is recommended to use the built in publication inheritance for PG14+.

Actually, this was introduced in PG 13, not PG 14: https://www.postgresql.org/docs/release/13.0/

Allow partitioned tables to be logically replicated via publications (Amit Langote)

Previously, partitions had to be replicated individually. Now a partitioned table can be published explicitly, causing all its partitions to be published automatically. Addition/removal of a partition causes it to be likewise added to or removed from the publication. The CREATE PUBLICATION option publish_via_partition_root controls whether changes to partitions are published as their own changes or their parent's.

I tested something like this on a PG 13 instance, just to be sure. It seems to work, so I think the pg_partman documentation is mistaken in identifying the feature as PG 14 only.

BEGIN;
CREATE TABLE native_inheritance (
    row_time TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT (current_timestamp),
    name text default '1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ'
) PARTITION BY RANGE (row_time);
CREATE PUBLICATION native_inheritance_pub FOR TABLE native_inheritance WITH (publish = 'insert');
SELECT partman.create_parent(
    p_parent_table => 'public.native_inheritance',
    p_control => 'row_time',
    p_type => 'native',
    p_interval => 'daily',
    p_premake => 7
    -- NOTE:  Not setting p_publications here!!!
);
COMMIT;

And then to check replication, I run on one psql instance:

select pg_create_logical_replication_slot('native_inheritance_slot', 'pgoutput');

Then, on a different psql instance, insert a row:

insert into native_inheritance values (current_timestamp);

Read the logs:

SELECT * FROM pg_logical_slot_get_binary_changes('native_inheritance_slot', NULL, NULL, 'proto_version', '1', 'publication_names', 'native_inheritance_pub');

If you do a quick decoding of the result from hex to ASCII you'll see the inserted row as expected. I also tried publish_via_partition_root = true on the publication and it also works as expected: the logs show insertions into the root partition table.

One more thing: my example avoids setting the p_publications parameter because I didn't see that it would actually create a publication on the root table and take advantage of this new PG 13 feature. Instead it still seems to add individual partitions to the publication. However, the documentation doesn't really make any mention of these aspects. If I was a new pg_partman user starting out on PG 15, I'd probably do the wrong thing here and not take advantage of this new PG 13 feature (or at least get confused without specific guidance here).

I think the documentation should point out that the parameter should not be used with native partitions on PG 13+. Alternatively, pg_partman itself could have some added special logic here.

@keithf4
Copy link
Collaborator

keithf4 commented Apr 11, 2023

Thanks for the update. For pg_partman 5.x that I'm working on right now, the minimum version requirement is now PG14 and I've also completely removed the p_publications parameter, so anything publication related is all handled natively now. There still is a need to run a subscription refresh on the subscriber side, so that is still part of maintenance.

If you want to double-check the new docs to see if things look correct, I'd appreciate it - https://github.com/pgpartman/pg_partman/tree/5.0.0-beta

If I was still keeping the 4.x series maintained outside of critical bug fixes, I'd definitely go back and update the docs as you've suggested, but I think since things are moving forward with a new minimum PG version required, I'll let things be as they are for 4.x for now.

Thanks again!

@james-johnston-thumbtack
Copy link
Author

Ah, that makes sense & I agree that removing the parameter entirely is the right move.

I took a quick look at the docs directory in the 5.0 beta tree you linked. I didn't see hardly any mention at all about publications any more. It might be worth a brief mention that (1) pg_partman doesn't manage publications at all, (2) one should create the publication yourself, (3) a reminder that PG will automatically add partitions to the publication.

@keithf4 keithf4 self-assigned this Apr 28, 2023
@keithf4 keithf4 added this to the 5.0 milestone Apr 28, 2023
@keithf4
Copy link
Collaborator

keithf4 commented Sep 28, 2023

Just noting that version 5.0.0 has been released and that I also had to remove the subscription refresh feature unfortunately. Seems some changes that were also in PG13 do not allow that to be done in an explicit transaction anymore. Will see if there some other possibility, maybe with a background worker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants