[pkg/ottl] Add support for optional parameters in functions

[evan-bradley]

Component(s)

pkg/ottl

Is your feature request related to a problem? Please describe.

Currently there is no support for optional parameters in OTTL functions. Optional parameters have been proposed as a solution to allow functions to take on more functionality while keeping function calls cleaner in the general case, for example with ConvertCase: #18084.

Describe the solution you'd like

See prior discussion here: #18822 (comment).

The approach introduced in #14712 opens the possibility to take advantage of the additional factory structure to annotate arguments as optional. One approach for this could be to annotate the fields on the Arguments struct as being optional. I think this would need two components to allow both the parser and factory to handle optional parameters:

1. An ottloptional struct tag on the field representing the optional parameter. This would signal to the parser that an argument doesn't necessarily need to be passed in an invocation.
2. An ottl.OptionalParameter[T any] type for optional parameters that don't have a sensible default value. Go's use of zero-values makes it difficult to tell whether an argument has been set or not, so we would need some way of easily specifying this. We could also do this with pointers, but I think a proper type would be a cleaner solution.

An example of what this could look like with a type where we want to check whether a parameter has been passed:

type ConvertCaseArguments[K any] struct {
	Target ottl.StringGetter[K] `ottlarg:"0"`
	ToCase string               `ottlarg:"1"`
	ToDelimited ottl.OptionalParameter[string] `ottlarg:"1",ottloptional`
}

Checking whether ToDelimited was passed in an invocation could then be done with a ToDelimited.IsEmpty() or something similar.

For parameters with default values, here's a contrived example:

type LimitArguments[K any] struct {
	Target       ottl.PMapGetter[K] `ottlarg:"0"`
	PriorityKeys []string           `ottlarg:"1"`
	Limit        int64              `ottlarg:"2",ottloptional`
}

func (f LimitFactory[K]) CreateDefaultArguments() ottl.Arguments {
	return &LimitArguments[K]{ Limit: 10 }
}

Describe alternatives you've considered

We could also add support for keyword/named arguments, e.g. delete_key(target="attributes", key="test") which would allow argument reordering and would make having multiple or mutually exclusive optional parameters much easier. However, this would be a larger change and provides an open door for "do everything" functions.

Additional context

No response

[evan-bradley]

cc codeowners @bogdandrutu @TylerHelmuth @kentquirk

[TylerHelmuth]

Hype!

[bogdandrutu]

I am confused about the "ottloptional". We need to ensure that only one optional argument is defined correct? Otherwise from foo("a", "b") (and if there are 2 optional params { first "optional", second, third "optional" }) you will not be able to know which of them are set (first or third).

Do we need ottlarg? I think it complicates, we just use the order defined in the struct.

An alternative to this optional thing we can consider some other options:

• Use python style named arguments
• Use order to always say that last arguments are optional. For example if you have 5 arguments and first 2 are "required" then you can call with first 2 or first 3 or first 4 or all 5.

[TylerHelmuth]

Do we need ottlarg? I think it complicates, we just use the order defined in the struct.

While not require, we discussed in the SIG meeting and it was agreed that a struct tag removes any ambiguity caused by relying on only the struct's field order. I personally like the tag a lot more.

[TylerHelmuth]

@evan-bradley In your ConvertCaseArguments example should the optional arg have ottlarg:2 ?

I agree that optional args should always be after required args.

[evan-bradley]

I am confused about the "ottloptional". We need to ensure that only one optional argument is defined correct?

Use order to always say that last arguments are optional. For example if you have 5 arguments and first 2 are "required" then you can call with first 2 or first 3 or first 4 or all 5.

My goal with ottloptional was to clearly annotate the arguments with which ones are optional. We would need to do a runtime check to ensure that required arguments aren't defined after optional arguments, but it should be straightfoward outside that. I agree with the behavior in your example.

Do we need ottlarg? I think it complicates, we just use the order defined in the struct.

While not require, #14712 (comment) and it was agreed that a struct tag removes any ambiguity caused by relying on only the struct's field order.

I made it a requirement after the SIG discussion for the sake of consistency, and struct fields without the tag will cause an error during parsing. Personally I'm okay with relying on the struct order, but I understand that it may not be immediately clear that field ordering is guaranteed in Go.

Use python style named arguments

I would support this, I think it removes a lot of ambiguity.

@evan-bradley In your ConvertCaseArguments example should the optional arg have ottlarg:2 ?

Thanks for catching this, I've edited the code to correct that.

[evan-bradley]

I have a working prototype of this that uses named arguments in the style of Python's keyword arguments. I'm aiming to have a draft PR ready for design discussion by sometime next week.

[rnishtala-sumo]

@evan-bradley would you be able to share the working prototype as a draft PR?

[evan-bradley]

@rnishtala-sumo sure thing, here it is: #24456.

[evan-bradley]

Resolved via #24456