Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Working on funcs to move default partition data to new partitions #2547

Merged
merged 17 commits into from
Jan 25, 2021
Merged

Working on funcs to move default partition data to new partitions #2547

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
d6a6c4d
WIP working on funcs to move default partition data to new partitions
Red-HAP Dec 10, 2020
6bb9026
Re-engineer into func + class
Red-HAP Dec 10, 2020
eba1a24
Basic unit test passing
Red-HAP Dec 10, 2020
cd9d4e3
Added tests
Red-HAP Dec 14, 2020
a83cb52
Fix text comparison
Red-HAP Dec 14, 2020
fd41aea
Merge branch 'master' into move_default_partition_data
Red-HAP Dec 14, 2020
fdfa230
migration to reapply db func changes
Red-HAP Dec 16, 2020
736f7f7
Merge branch 'master' into move_default_partition_data
Red-HAP Dec 16, 2020
b2aa167
migration bump
Red-HAP Dec 16, 2020
614c6a5
merge master into branch
Red-HAP Dec 17, 2020
e024488
Merge branch 'master' into move_default_partition_data
Red-HAP Jan 22, 2021
a25f2e4
fixes buggy date parse
Red-HAP Jan 23, 2021
377f244
add extra test condition
Red-HAP Jan 25, 2021
ffb6818
fixes migration to process default partition data
Red-HAP Jan 25, 2021
fe71393
Merge branch 'master' into move_default_partition_data
Red-HAP Jan 25, 2021
944d5cf
Merge branch 'master' into move_default_partition_data
Red-HAP Jan 25, 2021
ff9fc54
Merge branch 'master' into move_default_partition_data
Red-HAP Jan 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions db_functions/clone_schema.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,9 +183,10 @@ BEGIN
EXECUTE 'CREATE TABLE IF NOT EXISTS ' || dest_obj || ' (LIKE ' || source_obj || ' INCLUDING ALL) ;';
END IF;

IF copy_data OR
(object_rec.table_name ~ 'partitioned_tables') OR
(object_rec.table_name ~ 'django_migrations')
IF (copy_data OR
(object_rec.table_name ~ 'partitioned_tables') OR
(object_rec.table_name ~ 'django_migrations')) AND
(object_rec.table_kind = 'r')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to fix a bug copying partitioned table data.

THEN
IF _verbose
THEN
Expand Down
60 changes: 36 additions & 24 deletions db_functions/partitioned_tables_manage_trigger_function.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,40 +26,52 @@ DECLARE
BEGIN
IF ( TG_OP = 'DELETE' )
THEN
alter_stmt = 'ALTER TABLE ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.partition_of_table_name) ||
' DETACH PARTITION ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.table_name) ||
' ;';
IF ( OLD.active )
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The multiple active checks here prevent unnecessary (and unwanted) action when updating the partitiion_parameters when the record is inactive.

THEN
alter_stmt = 'ALTER TABLE ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.partition_of_table_name) ||
' DETACH PARTITION ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.table_name) ||
' ;';
END IF;
action_stmt = 'DROP TABLE IF EXISTS ' || quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.table_name) || ' ;';
table_name = quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.partition_of_table_name);
alter_msg = 'DROP PARTITION ' || quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.table_name);
ELSIF ( TG_OP = 'UPDATE' )
THEN
alter_stmt = 'ALTER TABLE ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.partition_of_table_name) ||
' DETACH PARTITION ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.table_name)
|| ' ;';
action_stmt = 'ALTER TABLE ' ||
/* If the partition was active, then detach it */
if ( OLD.active )
THEN
alter_stmt = 'ALTER TABLE ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.partition_of_table_name) ||
' ATTACH PARTITION ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.table_name) || ' ';
IF ( (NEW.partition_parameters->>'default')::boolean )
' DETACH PARTITION ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.table_name)
|| ' ;';
END IF;

/* If we are going to active or are still active, then attach the partition */
if ( NEW.active )
THEN
alter_stmt = alter_stmt || 'DEFAULT ;';
action_msg = 'DEFAULT';
ELSE
alter_stmt = alter_stmt || 'FOR VALUES FROM ( ' ||
quote_literal(NEW.partition_parameters->>'from') || '::date ) TO (' ||
quote_literal(NEW.partition_parameters->>'to') || '::date ) ;';
action_msg = 'FOR VALUES FROM ( ' ||
quote_literal(NEW.partition_parameters->>'from') || '::date ) TO (' ||
quote_literal(NEW.partition_parameters->>'to') || '::date )';
action_stmt = 'ALTER TABLE ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.partition_of_table_name) ||
' ATTACH PARTITION ' ||
quote_ident(OLD.schema_name) || '.' || quote_ident(OLD.table_name) || ' ';
IF ( (NEW.partition_parameters->>'default') = 'true' )
THEN
action_stmt = action_stmt || 'DEFAULT ;';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes malformed statement build

action_msg = 'DEFAULT';
ELSE
action_stmt = action_stmt || 'FOR VALUES FROM ( ' ||
quote_literal(NEW.partition_parameters->>'from') || '::date ) TO (' ||
quote_literal(NEW.partition_parameters->>'to') || '::date ) ;';
action_msg = 'FOR VALUES FROM ( ' ||
quote_literal(NEW.partition_parameters->>'from') || '::date ) TO (' ||
quote_literal(NEW.partition_parameters->>'to') || '::date )';
END IF;
END IF;

table_name = quote_ident(NEW.schema_name) || '.' || quote_ident(NEW.partition_of_table_name);
action_msg = 'UPDATE PARTITION ' || quote_ident(NEW.schema_name) || '.' || quote_ident(NEW.table_name) ||
action_msg = 'ALTER PARTITION ' || quote_ident(NEW.schema_name) || '.' || quote_ident(NEW.table_name) ||
' ' || action_msg;
ELSIF ( TG_OP = 'INSERT' )
THEN
Expand Down
2 changes: 1 addition & 1 deletion grafana/Dockerfile-grafana
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ FROM grafana/grafana:latest
USER root
RUN apk add sqlite
USER grafana
COPY --chown=grafana:grafana grafana.db.sql /var/lib/grafana/grafana.db.sql
COPY --chown=grafana:0 grafana.db.sql /var/lib/grafana/grafana.db.sql
Red-HAP marked this conversation as resolved.
Show resolved Hide resolved
RUN cat /var/lib/grafana/grafana.db.sql | sqlite3 /var/lib/grafana/grafana.db && rm /var/lib/grafana/grafana.db.sql
Loading