Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow microsecond precision for epoch #659

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/pg_partman.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ RETURNS boolean
+ *\<interval\>* - Any valid value for the interval data type. Do not type cast the parameter value, just leave as text.
+ *\<integer\>* - For ID based partitions, the integer value range of the ID that should be set per partition. Enter this as an integer in text format ('100' not 100). If the interval is >=2, then the `p_type` must be `range`. If the interval equals 1, then the `p_type` must be `list`. Also note that while numeric values are supported for id-based partitioning, the interval must still be a whole number integer.
* `p_type` - the type of partitioning to be done. Currently only **range** and **list** are supported. See `p_interval` parameter for special conditions concerning type.
* `p_epoch` - tells `pg_partman` that the control column is an integer type, but actually represents and epoch time value. Valid values for this option are: 'seconds', 'milliseconds', 'nanoseconds', and 'none'. The default is 'none'. All table names will be time-based. In addition to a normal index on the control column, be sure you create a functional, time-based index on the control column (to_timestamp(controlcolumn)) as well so this works efficiently.
* `p_epoch` - tells `pg_partman` that the control column is an integer type, but actually represents and epoch time value. Valid values for this option are: 'seconds', 'milliseconds', 'microseconds', 'nanoseconds', and 'none'. The default is 'none'. All table names will be time-based. In addition to a normal index on the control column, be sure you create a functional, time-based index on the control column (to_timestamp(controlcolumn)) as well so this works efficiently.
* `p_premake` - is how many additional partitions to always stay ahead of the current partition. Default value is 4. This will keep at minimum 5 partitions made, including the current one. For example, if today was Sept 6th, and `premake` was set to 4 for a daily partition, then partitions would be made for the 6th as well as the 7th, 8th, 9th and 10th. Note some intervals may occasionally cause an extra partition to be premade or one to be missed due to leap years, differing month lengths, etc. This usually won't hurt anything and should self-correct (see **About** section concerning timezones and non-UTC). If partitioning ever falls behind the `premake` value, normal running of `run_maintenance()` and data insertion should automatically catch things up.
* `p_start_partition` - allows the first partition of a set to be specified instead of it being automatically determined. Must be a valid timestamp (for time-based) or positive integer (for id-based) value. Be aware, though, the actual parameter data type is text. For time-based partitioning, all partitions starting with the given timestamp up to CURRENT_TIMESTAMP (plus `premake`) will be created. For id-based partitioning, only the partition starting at the given value (plus `premake`) will be made. Note that for subpartitioning, this only applies during initial setup and not during ongoing maintenance.
* `p_default_table` - boolean flag to determine whether a default table is created. Defaults to true.
Expand Down
9 changes: 9 additions & 0 deletions sql/functions/create_partition_time.sql
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ END IF;
v_partition_expression := CASE
WHEN v_epoch = 'seconds' THEN format('to_timestamp(%I)', v_control)
WHEN v_epoch = 'milliseconds' THEN format('to_timestamp((%I/1000)::float)', v_control)
WHEN v_epoch = 'microseconds' THEN format('to_timestamp((%I/1000000)::float)', v_control)
WHEN v_epoch = 'nanoseconds' THEN format('to_timestamp((%I/1000000000)::float)', v_control)
ELSE format('%I', v_control)
END;
Expand Down Expand Up @@ -236,6 +237,14 @@ FOREACH v_time IN ARRAY p_partition_times LOOP
, v_partition_name
, EXTRACT('epoch' FROM v_partition_timestamp_start)::bigint * 1000
, EXTRACT('epoch' FROM v_partition_timestamp_end)::bigint * 1000);
ELSIF v_epoch = 'microseconds' THEN
EXECUTE format('ALTER TABLE %I.%I ATTACH PARTITION %I.%I FOR VALUES FROM (%L) TO (%L)'
, v_parent_schema
, v_parent_tablename
, v_parent_schema
, v_partition_name
, EXTRACT('epoch' FROM v_partition_timestamp_start)::bigint * 1000000
, EXTRACT('epoch' FROM v_partition_timestamp_end)::bigint * 1000000);
ELSIF v_epoch = 'nanoseconds' THEN
EXECUTE format('ALTER TABLE %I.%I ATTACH PARTITION %I.%I FOR VALUES FROM (%L) TO (%L)'
, v_parent_schema
Expand Down
1 change: 1 addition & 0 deletions sql/functions/partition_data_time.sql
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ SELECT partition_tablename INTO v_last_partition FROM @[email protected]_partition
v_partition_expression := CASE
WHEN v_epoch = 'seconds' THEN format('to_timestamp(%I)', v_control)
WHEN v_epoch = 'milliseconds' THEN format('to_timestamp((%I/1000)::float)', v_control)
WHEN v_epoch = 'microseconds' THEN format('to_timestamp((%I/1000000)::float)', v_control)
WHEN v_epoch = 'nanoseconds' THEN format('to_timestamp((%I/1000000000)::float)', v_control)
ELSE format('%I', v_control)
END;
Expand Down
1 change: 1 addition & 0 deletions sql/functions/run_maintenance.sql
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ LOOP
v_partition_expression := CASE
WHEN v_row.epoch = 'seconds' THEN format('to_timestamp(%I)', v_row.control)
WHEN v_row.epoch = 'milliseconds' THEN format('to_timestamp((%I/1000)::float)', v_row.control)
WHEN v_row.epoch = 'microseconds' THEN format('to_timestamp((%I/1000000)::float)', v_row.control)
WHEN v_row.epoch = 'nanoseconds' THEN format('to_timestamp((%I/1000000000)::float)', v_row.control)
ELSE format('%I', v_row.control)
END;
Expand Down
2 changes: 2 additions & 0 deletions sql/functions/show_partition_info.sql
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,8 @@ IF v_control_type = 'time' OR (v_control_type = 'id' AND v_epoch <> 'none') THEN
child_start_time := to_timestamp(v_start_string::double precision);
ELSIF v_epoch = 'milliseconds' THEN
child_start_time := to_timestamp((v_start_string::double precision) / 1000);
ELSIF v_epoch = 'microseconds' THEN
child_start_time := to_timestamp((v_start_string::double precision) / 1000000);
ELSIF v_epoch = 'nanoseconds' THEN
child_start_time := to_timestamp((v_start_string::double precision) / 1000000000);
END IF;
Expand Down
2 changes: 2 additions & 0 deletions sql/functions/show_partitions.sql
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ ELSIF v_control_type = 'id' AND v_epoch <> 'none' THEN
v_epoch_divisor := 1;
ELSIF v_epoch = 'milliseconds' THEN
v_epoch_divisor := 1000;
ELSIF v_epoch = 'microseconds' THEN
v_epoch_divisor := 1000000;
ELSIF v_epoch = 'nanoseconds' THEN
v_epoch_divisor := 1000000000;
END IF;
Expand Down
1 change: 1 addition & 0 deletions sql/functions/undo_partition.sql
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ END IF;
v_partition_expression := CASE
WHEN v_epoch = 'seconds' THEN format('to_timestamp(%I)', v_control)
WHEN v_epoch = 'milliseconds' THEN format('to_timestamp((%I/1000)::float)', v_control)
WHEN v_epoch = 'microseconds' THEN format('to_timestamp((%I/1000000)::float)', v_control)
WHEN v_epoch = 'nanoseconds' THEN format('to_timestamp((%I/1000000000)::float)', v_control)
ELSE format('%I', v_control)
END;
Expand Down
2 changes: 1 addition & 1 deletion sql/tables/tables.sql
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ CREATE FUNCTION @[email protected]_epoch_type (p_type text) RETURNS boolean
DECLARE
v_result boolean;
BEGIN
SELECT p_type IN ('none', 'seconds', 'milliseconds', 'nanoseconds') INTO v_result;
SELECT p_type IN ('none', 'seconds', 'milliseconds', 'microseconds', 'nanoseconds') INTO v_result;
RETURN v_result;
END
$$;
Expand Down