proposal: allow a slice of an impl. type to be the same as a slice of the interface

[luca-moser]

Problem

One function has to be programmed which works by iterating over a slice of the interface type AccessControlledEntity which tells whether all objects pass the access flag criteria:

func PassAll(entities []AccessControlledEntity, flag AccessFlag) bool {
....
}

When having a type A which implements an interface I, a slice of type A can not be passed as a slice of type I to a function. In my use case I have a type Document which holds an ACL and implements the interface type AccessControlledEntity. Another type called Folder also implements the interface type AccessControlledEntity.

Go does does not allow a slice of []Document or []Folder to be passed to the function. The wiki article here describes the problem but makes the assumption that one should actually manually create a copy of the slice which is the concrete type []AccessControlledEntity (or whatever the type might be).

This language restraint therefore wants me to write for each different concrete type a separate function, which does the exact same thing:

type Documents []*Document

func (docs *Documents) AsAccessControlledEntity() []AccessControlledEntity {
	list := []AccessControlledEntity{}
	for x := range *docs {
		list = append(list, (*docs)[x])
	}
	return list
}

type Folders []*Folder

func (folders *Folders) AsAccessControlledEntity() []AccessControlledEntity {
	list := []AccessControlledEntity{}{}
	for x := range *folders {
		list = append(list, (*folders)[x])
	}
	return list
}

In my humble opinion a programming language should try to reduce redundancy in code but this language spec. is the exact contrary. The compiler acts as if it almost knows that it is wrong, since converting a *Folder to an AccessControlledEntity works without any problems. The last piece in the puzzle would be the slice conversion to work seamlessly.

This becomes even worse when a function returns a slice of the interface type:

func FilterBy(objs []AccessControlledEntity{}, flag AccessFlag) []AccessControlledEntity{} {
...
}

Now each type for which we want to use the function, has to re-implement the same "unpack" function:

func (docs *Documents) Unpack(objs []AccessControlledEntity) {
	for x := range objs {
		*docs = append(*docs, objs[x].(*Folder))
	}
}

func (folders *Folders) Unpack(objs []AccessControlledEntity) {
	for x := range objs {
		*folders = append(*folders, objs[x].(*Folder))
	}
}

If I understand the wiki article correctly the runtime overhead is always there, whether I manually write X times the same for loops or if the compiler auto generates the same code under the hood. However one is manual work with no benefits and the other is the work a compiler should do anyways. In countless other programming languages this is a non existing problem and it hinders me to really enjoy Go, because this restraint seems to be so implausible.

Proposal

Note: This "proposal" does not explain how to solve the problem but just describes in a declarative way, what should be fixed.

Allow in the Go language, that a slice of an implementing type can be passed as a slice of the implemented interface type to a function. Manual writing of for loops which copy the slice to or from the other type should not be necessary.

This should work:

type Documents []*Document
type Folders []*Folder

docs := Documents{}
PassAll(docs, flag.READ)

or

docs := Documents{}
folders := Folders{}
filteredDocuments, isType := Filter(docs, flag.READ).(Documents)
filteredFolders, isType := Filter(folders, flag.READ).(Folders)

[dsnet]

One major problem is that []T has an entirely different memory layout than []I, and so you can't just shoe-horn the former into the later. Supposing, the runtime hid this fact from the programmer, what happens when you try and write into []I with an object of type R that also happens to satisfies interface I?

The problem you are seeing is a widely studied topic called the covariance and contravariance of types and leads to many subtle and non-intuitive outcomes.

In countless other programming languages this is a non existing problem

This is not entirely accurate. Other languages have subtle restrictions and side-effects when allowing this. Go simply chose to avoid the problem altogether (i.e., []I and []T are invariant with each other). It's more programmer pain, but less subtleties.

This problem is heavily related to the topic of generics, and we will most likely not address this until Go2, and even then, what happens here is going to be heavily driven by what (if anything) happens with generics.

[mdlayher]

Sorry, hit the wrong button.

[ziflex]

I like the proposal.
I understand the reasons behind the limitation, but at the same time, I feel that the limitation breaks the abstraction and it's just an implementation details which can be changed behind the hood.

From logical perspective, as a developer, I expect this code to work:

type (
    Person interface {
        GetName() string
    }

    type User struct {
        name string
    }

    type Admin struct {
        User
    }
)

func (u *User) GetName() string {
    return u.name
}

func printNames(people []Person) {
    for _, person := range people {
        fmt.Println(person.GetName())
    }
}

func main() {
    printNames([]Person {
        &User{ name: "foo" },
        &User{ name: "bar" }
    })

    printNames([]Person {
        &Admin{ name: "foo" },
        &Admin{ name: "bar" }
    })
}

And since, it does not work, means that we have classic leaking abstraction and less flexible interfaces.

[randall77]

@ziflex: That code does work fine, modulo a few typos and issue #9859.

package foo

import "fmt"

type (
    Person interface {
        GetName() string
    }

    User struct {
        name string
    }

    Admin struct {
        User
    }
)

func (u *User) GetName() string {
    return u.name
}

func printNames(people []Person) {
    for _, person := range people {
        fmt.Println(person.GetName())
    }
}

func main() {
    printNames([]Person {
        &User{ name: "foo" },
        &User{ name: "bar" },
    })

    printNames([]Person {
        &Admin{ User{name: "foo"} },
        &Admin{ User{name: "bar"} },
    })
}

[randall77]

... and I just learned that putting "go" after the triple backquote enables Go syntax highlighting!

[ziflex]

@randall77 my bad! too much of copy paste.
Here is an updated version:

package foo

import "fmt"

type (
    Person interface {
        GetName() string
    }

    User struct {
        name string
    }

    Admin struct {
        User
    }
)

func (u *User) GetName() string {
    return u.name
}

func printNames(people []Person) {
    for _, person := range people {
        fmt.Println(person.GetName())
    }
}

func main() {
    printNames([]*User {
        &User{ name: "foo" },
        &User{ name: "bar" },
    })

    printNames([]*Admin {
        &Admin{ User{name: "foo"} },
        &Admin{ User{name: "bar"} },
    })
}

Even though, these types both implement the interface due to technical details you cannot use them in a such way.

[randall77]

Right, the issue is that using []*User { instead of []Person { doesn't work. A *User can be cast to a Person but a []*User can't be cast to a []Person.
@dsnet's comment explains the problem. I don't see any way around that problem.

[ziflex]

@randall77 so, that's what I'm talking about - it's a pretty common use case of interfaces utilization.

The most straightforward workaround is to create []Person, iterate over []*User and []*Admin and populate the array with their values. Which is not very convenient and requires more repetitive code.

I saw the comment and read the explanation before, but still not convinced that it's ok.
To me it's a broken abstraction, because as I said earlier - the reason why we have this behavior is limitation of current implementation, not the concept (of interfaces). That's why I think improving the implementation is worth to be a part of Go2.

[dsnet]

Are you suggesting that Go automatically create a copy of []*User as []Person and subtly use the copy instead of the original []*User? What happens when someone mutates the copy? The original will not be modified at all. This introduces both two subtleties: 1) function calls can become surprisingly expensive. 2) mutations of slice argument sometimes have side-effects and sometimes don't.

[dsnet]

the reason why we have this behavior is limitation of current implementation, not the concept (of interfaces)

As I pointed out in my first comment. This is not true. Covariance and contravariance are real topics in type theory, and you can create a mapping of how those theories applies to Go's type system (especially in regard to concrete types and interfaces).

[luca-moser]

A []Person slice consisting of []*User should indicate, that mutations inside []Person directly affect the underyling User the pointer points to. You are merely copying the pointers.

If your []Person consists out of []User, you are actually having a copy.

Are you suggesting that Go automatically creates a copy of []*User as []Person and subtly use the copy instead of the original []*User? What happens when someone mutates the copy?
Again, it's better when the compiler does the "copying"/creation of the slice, instead of having me write the same loop over and over again. It feels highly unnatural writing this stupid loops by myself.

[dsnet]

If it's implemented by pointer copying, there are still issues:

func SwapPersons(ps []Person, i, j int) {
    ps[i], ps[j] = ps[j], ps[i]
}

This function would not work with the pointer copying solution since []Person is implicitly a *User under the hood and not a **User.

And copying pointers is certainly cheaper than copying the entire struct, but it is still an expensive operation since it is O(n) regardless of the size of each element.

[dsnet]

It feels highly unnatural writing this stupid loops by myself.

I understand the annoyance of doing this. I have struggled with this myself this past weekend, but I'm not convinced the type system is where this should be "fixed".

#15209 is probably a better solution to this problem. Since a *User is assignable to a Person, you would be able to do:

printNames(append([]Person{}, users...)) // With #15209
printNames([]Person{users...})           // With #15209 and #19218

This is more verbose than having an "implicit conversion", but at least it makes it obvious an allocation is occurring and avoids magic.

[metakeule]

@dsnet @ziflex

you may write the following (is shorter an works right now):

package foo

import "fmt"

type (
    Person interface {
        GetName() string
    }

    User struct {
        name string
    }

    Admin struct {
        User
    }
)

func (u *User) GetName() string {
    return u.name
}

func printNames(people ...Person) {
    for _, person := range people {
        fmt.Println(person.GetName())
    }
}

func main() {
    printNames(
        &User{ name: "foo" },
        &User{ name: "bar" },
    )

    printNames(
        &Admin{ User{name: "foo"} },
        &Admin{ User{name: "bar"} },
    )
}

[ianlancetaylor]

Adding covariance to the language requires more discussion than this. Currently function calls work by assignment, so this proposal is presumably also changing assignment. It is a change that affects several aspects of the language. It presumably introduces hidden loops and raises the problem mentioned above, of changing elements in a copied slice. This proposal can not be fully analyzed as is, and is unlikely to be accepted in any case, so closing.