Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add dynamic parameter expression #780

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions proto/substrait/algebra.proto
Original file line number Diff line number Diff line change
Expand Up @@ -957,6 +957,7 @@ message Expression {
Cast cast = 11;
Subquery subquery = 12;
Nested nested = 13;
DynamicParameter dynamic_parameter = 14;

// deprecated: enum literals are only sensible in the context of
// function arguments, for which FunctionArgument should now be
Expand Down Expand Up @@ -1611,6 +1612,16 @@ message Expression {
}
}

// Expression that represents a dynamic parameter.
// Dynamic parameters are identified by a surrogate key within a plan.
message DynamicParameter {
// The type of the dynamic parameter.
Type type = 1;

// The surrogate key used within a plan to reference a specific parameter binding.
uint32 parameter_reference = 2;
}

// The description of a field to sort on (including the direction of sorting and null semantics)
message SortField {
Expression expr = 1;
Expand Down
15 changes: 15 additions & 0 deletions proto/substrait/plan.proto
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ message Plan {
// unused. In many cases, a consumer may be able to work with a plan even if
// one or more message types defined here are unknown.
repeated string expected_type_urls = 5;

// An optional list of bindings for dynamic parameters used in this plan.
// Each binding maps a parameter_anchor to its corresponding runtime value.
repeated DynamicParameterBinding parameter_bindings = 7;
}

// This message type can be used to deserialize only the version of a Substrait
Expand Down Expand Up @@ -80,3 +84,14 @@ message Version {
// created the plan.
string producer = 5;
}

// Represents a binding for a dynamic parameter.
message DynamicParameterBinding {
// The parameter anchor that identifies the dynamic parameter reference.
uint32 parameter_anchor = 1;

// The literal value assigned to the parameter at runtime.
jcamachor marked this conversation as resolved.
Show resolved Hide resolved
// The type of the literal needs to match the type of the corresponding
// DynamicParameter expression in the plan.
Expression.Literal value = 2;
}
1 change: 1 addition & 0 deletions site/docs/expressions/_config
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ arrange:
- table_functions.md
- user_defined_functions.md
- embedded_functions.md
- dynamic_parameters.md
12 changes: 12 additions & 0 deletions site/docs/expressions/dynamic_parameters.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Dynamic Parameter Expression

The dynamic parameter expression represents a placeholder within an expression whose value is determined at runtime.
This is particularly useful for parameterized queries where certain values are not known until execution.
Additionally, using dynamic parameters can enable other use cases, such as sharing execution plans without embedding sensitive information.

A dynamic parameter expression includes the following properties:

| Property | Description | Required |
|-----------------------|-------------------------------------------------------------------------------|----------|
| `type` | Specifies the expected data type of the dynamic parameter. | Yes |
| `parameter_reference` | A surrogate key used within a plan to reference a specific parameter binding. | Yes |
8 changes: 8 additions & 0 deletions site/docs/relations/basics.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,14 @@ A guarantee that data output from this operation is provided with a sort order.
| Clustered | Ensures that all equal values are coalesced (but no ordering between values is defined). E.g. for values 1,2,3,1,2,3, output could be any of the following: 1,1,2,2,3,3 or 1,1,3,3,2,2 or 2,2,1,1,3,3 or 2,2,3,3,1,1 or 3,3,1,1,2,2 or 3,3,2,2,1,1. | N/A, may appear anywhere but will be coalesced. |


## Dynamic Parameters

Dynamic parameters act as placeholders within expressions, with their values determined at runtime.
This is particularly useful for *parameterized queries*, *query reuse*, and *execution plan sharing*, where specific values are not known ahead of time or need to be replaced dynamically during execution.

### Dynamic Parameter Bindings
In a plan's relational tree, dynamic parameters are referenced in expressions without predefined values.
Their values are supplied via parameter bindings, which can either be included in the plan's optional bindings field or supplied separately at execution time.



Expand Down
Loading