spec: infer type arguments from assignments of generic functions (reverse type inference)

[griesemer]

This proposal suggests a generalization of type inference where type parameters of generic function values are inferred from assignments of those values to variables. Similar ideas have been suggested in the past, e.g. see #53138 and related issues mentioned in that issue.

Background

At the moment, and even with the proposed reformulation of type inference (#58650), type inference determines the type arguments of a generic function based on (partially) provided type arguments and function arguments passed to that generic function.

We propose that missing type arguments of generic functions be inferred when such generic function values are assigned to variables of function type. That is, rather than inferring the type arguments of the function being called (and values being passed to), we infer the type arguments of the function being passed from the function being called (hence reverse type inference for lack of a better term).

For instance, the generic function g

func g[P, Q any](x P) Q { ... }

may be assigned to a variable of function type

var f func(int) bool = g

and type inference will infer that P must be int and Q must be bool. The function g may also be partially instantiated, so this assignment (after f is declared) would be allowed as well

f = g[int]

More importantly, passing a generic function as a function value to another (possibly generic) function may infer the type arguments of the function value (and possibly of the generic function being called). Given

func less[P Ordered](x, y P) bool { return x < y }
func sort[Q any](list []Q, less func(x, y Q) bool) { ... }

we can call

sort(list, less)

and the type argument Q of sort will be inferred from the list argument, and the type argument P of less will be inferred from passing less to sort.

Proposal

A generic function may not be fully instantiated when its (function) value is assigned to a variable of matching function type. In that case, type inference will infer any unknown type arguments, if possible.

For the purpose of this proposal, initialization expressions (to fully typed variables) in variable declarations, assignments to redeclared (and thus fully typed) variables in short variable declarations, returning results to function result parameters, and passing values to (user-defined) functions in function calls are considered assignments where this form of type inference will be applicable.

This is the entire proposal.
(Together with @ianlancetaylor.)

Implementation

We have a partial implementation (currently disabled in the dev branch) that implements significant aspects of this proposal (assignments and return statements, except passing arguments to generic functions) so that we can explore the ramifications. If this proposal is accepted, we hope to make the feature available for Go 1.21.

[zigo101]

This is cool!

BTW, shouldn't the type constraint in sort be also Ordered instead of any?

[griesemer]

@go101 The sort example is hypothetical, but here I meant Ordered to mean types that support <. So while the list elements supplied to sort must be ordered in some way, that way is entirely defined by the supplied less function argument (the elements could be structs that have some order defined on them, but < is not permitted on structs). If sort is called with say a []int list, than we infer P (of the supplied less function) to be int which satisfies the Ordered constraint. (And if we called sort with a []struct{...} we would get an error if we were to use the provided less function, because struct{...} does not satisfy Ordered. In that case we'd provide a different function whose constraint would not be Ordered). So in short, no, the constraint in sort does not have to be Ordered, at least not the Ordered that implies < is permitted.

[zigo101]

OK, I get it now. (BTW, it looks the type of list should be []Q instead. Otherwise, it doesn't compile).)

[earthboundkid]

OK, I get it now. (BTW, it looks the type of list should be []Q instead. Otherwise, it doesn't compile).)

Yeah:

func less[P Ordered](x, y P) bool { return x < y }
func sort[Q any, QS ~[]Q](list QS, less func(x, y Q) bool) { ... }

But []Q works too, since the QS type isn't returned.

[leaxoy]

Can this proposal infer type indirectly. For example:

type Values[V any] struct {
	inner *vector.Vector[V]
}

func (m HashMap[K, V]) Values() Values[V] {
        // vector type param V should infer to V by Values returns Values[V] lazily
        // or interred by values.Append(v)
	values := vector.New()
	for _, v := range m.inner {
		values.Append(v)
	}
	return Values{inner: values}
}

[timothy-king]

FWIW I think if I came across the partial instantiation f = g[int] in the wild, I would [potentially incorrectly] conclude g takes one type parameter.

What is the benefit of allowing partial instantiations, e.g. f = g[int]? (Compared to supporting both f = g and f = g[int, bool].) Does someone have an example where partial instantiations would help readability?

[ianlancetaylor]

@leaxoy No, that is not this proposal.

@timothy-king I would say that the benefit of allowing partial instantiations is consistency with other forms of type inference. If we omitted them some people would ask why that case does not work.

[kalexmills]

I would say that the benefit of allowing partial instantiations is consistency with other forms of type inference. If we omitted them some people would ask why that case does not work.

It does seem weird to me as well, at least as far as the syntax goes, but maybe I just need to get used to it.

If I'm reading it correctly then what we're really proposing is adding inference which works on all of the following assignments:

func g[P, Q any](x P) Q { ... }
var f func(int) bool
f = g            // infers P = int, Q = bool
f = g[int]       // infers Q = bool
f = g[int, bool] // infers nothing

On a related note, would this edge case also be covered by the proposal?

func g[P, Q any](x P) Q { ... }

func h[P any](x P) {
  var f func(P) bool

  f = g[P]       // infers Q = bool
  f = g[P, bool] // infers nothing
}

I don't foresee any issues with admitting the above but that doesn't mean they might not be lurking in a dark corner.

[ianlancetaylor]

@kalexmills Yes, that example should work.

[mdempsky]

Overall seems reasonable.

I'm curious about some corner cases though. Is there an easy way to experiment with the prototype?

For example, given:

func f[T any](T) (_ int) { return }
func g[T any](int) (_ T) { return }
func h[T any](T, T)      {}

then currently h(f[int], g[int]) type checks. Will this proposal allow h(f, g) to be inferred?

[rogpeppe]

Would it be reasonable to do the same thing for generic struct literals?

For example:

type Pair[A, B any] struct {A A, B B}

var P Pair[int, string] = Pair{12, "hello"}

[kalexmills]

@rogpeppe fair warning; that might well need to be a separate proposal depending on the current compiler implementation. I'm sure a maintainer will be able to comment further.

In your example, why not go further? There's enough type information to infer the type of at least the second type parameter on the LHS of the =. Per the convention setup in this proposal, with enough type inference we may one day be able to write...

type Pair[A, B any] struct {A A; B B}

var P Pair[int, string] = Pair{12, "hello"}
var Q Pair[int] = Pair{12, "hello"} // ...this...
R := Pair{12, "hello"}              // ... or even this!

That last one might raise some concerns, but I don't think it's much worse than the below, which is already possible in Go.

x := 12

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[ianlancetaylor]

@rogpeppe Thanks, that is a different proposal.

To me it also seems less useful. Not to say that we shouldn't do it. But the impetus for this proposal is passing generic functions to other functions, such as a generic Less function. That kind of case seems less common for a generic struct.

[griesemer]

@mdempsky In your example, I would think that h(f, g) should be permitted and infer the correct types.

Currently, this kind of inference for parameter passing is not yet prototyped. But it is there for ordinary assignments and returns. To play with it, you could enable it in the compiler by setting the flag EnableReverseTypeInference when setting up the respective types2.Config.

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

[gopherbot]

Change https://go.dev/cl/492515 mentions this issue: go/types, types2: make Checker.renameTParams work on any type

[gopherbot]

Change https://go.dev/cl/491797 mentions this issue: go/types, types2: consider generic functions in inference simplify step

[gopherbot]

Change https://go.dev/cl/491798 mentions this issue: go/types, types2: remove Config.EnableReverseTypeInference flag

[gopherbot]

Change https://go.dev/cl/494115 mentions this issue: go/types, types2: explicitly look for nil type arguments in infer

[gopherbot]

Change https://go.dev/cl/494118 mentions this issue: go/types, types2: control type inference in Checker.funcInst via infer argument

[gopherbot]

Change https://go.dev/cl/494116 mentions this issue: go/types, types2: permit partially instantiated functions as function arguments

[gopherbot]

Change https://go.dev/cl/499282 mentions this issue: doc/go1.21: document type inference changes

[aclements]

Marking as a release-blocker to track the pending changes to the spec document. This does not have to block any RCs.

[griesemer]

This has been described in the spec a while back.
(What is currently missing in the spec is the more up-to-day description of unification, which is in progress. But unification is used for inference in general and is not specific to this issue.)
Closing.