Do tuples need a shared non-Object superclass?

[lrhn]

The proposal for tuples have a Record class which acts as a common supertype of tuple types.

Whether that is necessary or desirable depends on perspective and features.

The main reason for separate types to have a shared supertype is either having a shared interface or other shared affordances.
For example function types have a common supertype Function which abstracts over their ability to be called (they have a "dynamic call" member), and people are asking for a supertype of enums, which have a shared int get index; interface member.

So, the question is whether tuples should have any shared functionality, and if not, whether they should have a supertype anyway, for good measure.

The proposal suggests that the Record class have methods which reflect on the tuple. Those work on all tuples, and only on tuples, and therefore need an argument type of Record.
I would prefer to not add such functions outside of dart:mirrors, and then they would be part of a RecordMirror, not something which works on the reified tuple itself. If they exist outside of mirrors, the functions could also take any object as argument, and treat non-tupes as if they were one-tuples containing the value (I also argue that there should be no one-tuples because the product type T^1 is just T, so that would be consistent).

There is no other shared functionality between tuples, those two reflective functions are the only ones which work on tuples independently of the tuple "width"/"arity"/"structure". There is no shared interface members between (int, int) and ({int x}), so without those two method, there is no advantage in knowing that something is a tuple, because you still don't know which tuple structure it is (and there is a countably infinite number of those if you try to do an exhaustive search). You're better off casting to dynamic if you want to explore the members of an arbitrary tuple, than by trying to cast to a tuple type.

If there are no empty tuples (or even no singleton-tuples), the Record type also feels incomplete (to me). It's true that it contains all tuples, but with that view, maybe a single object is a one-tuple, so why is Object not a subtype of Record?

Still, people like to have categories and be able to express those categories in code. Saying that a variable is a Record r; is documentation that is statically and dynamically enforced (whereas Object tuple; is just text, no enforcement).

(Betteridge's law says no.)

[eernstg]

class C<X extends Record> {...} could enforce the property that class C has a type argument which is a tuple type, and that might be useful from a conceptual point of view (it outlines how C should be understood) even if it doesn't make much difference technically.

For the shared properties, maybe the dynamic type of a Record could reveal the structure (as much as it must already, for a boxed value of that type). This could be offered to developers as instance getters int get numberOfPositionalFields.

This won't allow for the same level of introspection as the record proposal's positionalFields and namedFields, but it would allow a similar functionality at a lower level if a record r with 3 (or more ;-) positional fields would make r is (Object?, Object?, Object?) true.

We could also consider a getter Set<Symbol> get namedFields, but that's less promising: The named fields could already be accessed using a dynamic invocation, and the ability to get a set of symbols at run time won't help much.

[munificent]

I would prefer to not add such functions outside of dart:mirrors, and then they would be part of a RecordMirror, not something which works on the reified tuple itself.

Even if we do that, I still think they should have a more precise type that they accept beyond just Object, so that callers can better catch mistakes attempting to pass non-record types to it.

If they exist outside of mirrors, the functions could also take any object as argument, and treat non-tupes as if they were one-tuples containing the value (I also argue that there should be no one-tuples because the product type T^1 is just T, so that would be consistent).

Records as I've proposed them aren't really like strict product types. That model falls down when you consider that a record with a single named field is still clearly distinct from the field's value. (This is part of the reason I call them "records" and not "tuples".) I think it's better to think of them as immutable heterogeneous collections. From that perspective, there's no reason not to support a one-positional-field record.

There is no shared interface members between (int, int) and ({int x}), so without those two method, there is no advantage in knowing that something is a tuple, because you still don't know which tuple structure it is (and there is a countably infinite number of those if you try to do an exhaustive search).

Record types may not have any members in common now, but that doesn't mean that users might not want to define extensions shared across all record types. I admit the body of that extension doesn't have a ton of useful affordances on this, but I think users could and likely will still find them useful to define.

Basically, I think providing Record as a tag interface is relatively harmless and potentially useful, so I lean towards just doing it.

[lrhn]

I would prefer to not add such functions outside of dart:mirrors, and then they would be part of a RecordMirror, not something which works on the reified tuple itself.

Even if we do that, I still think they should have a more precise type that they accept beyond just Object, so that callers can better catch mistakes attempting to pass non-record types to it.

Nobody would be passing tuples into anything. You'd do reflect(myTuple) and the result would be a TupleMirror instead of (or in addition to) an InstanceMirror. You can then check that it is a tuple mirror, and use the tuple specific methods.
Mirrors work by reflecting, then working on the mirrors, not providing introspection directly on the reified objects.

[munificent]

In 98341cd, I removed the reflective methods on Record. This means that it's currently an empty interface with no members, so we could remove it if we wanted to. However, I've left it in for now because I suspect it might be somewhat useful as a tag interface.

There are places where users write functions that accept one of a handful of known types. In the absence of overloading, they usually just accept Object and then type test each of the allowed types in the body. I expect that code like that will sometimes happen where all of the allowed types are record types, and having a Record supertype gives users at least a marginally more precise type to use.

[natebosch]

Is there risk in experimenting, or even shipping, without the Record type? Can we wait until we see the case where it is useful to provide it?

[munificent]

Is there risk in experimenting, or even shipping, without the Record type? Can we wait until we see the case where it is useful to provide it?

Yes, I think we could do that. We essentially did the same thing with Enum, which was added as the supertype of enums in 2.14. But... I think the fact that we ended up adding Enum is also a hint that we will end up wanting a supertype for record types too.

I think it's fairly harmless to add it eagerly, and I do think it's useful and something users will occasionally want. Also, all of our other structural types have a corresponding supertype: Enum for enum types, Function for function types.

[lrhn]

Out existing supertypes have funcitonality. Enum has int get index; and Function has a magic call. Record likely will not have any interface members. (It might have abstract == and hashCode declaration for documentation purposes.)

The interface methods are not the only reason these types to exist. I use Function just as a reminder that whatever is stored in this polymorphic function, at least I know it's a function of some sort. It's not necessary, but it is slightly helping readability. I'd expect people to want Record for the same reason.

For example, if we rewrite noSuchMethod/Invocation and Function.apply to use records for the arguments (which we should eventually), the type of Function.apply would be dynamic apply(Function function, Record arguments);, and that's actually the best we can do.
(Well, unless we can use record types to abstract over function type parameter list, like make Function generic as Function<A extends Record, R>, so that int Function(int, int) extends Function<(int, int), int>, then we can do R apply<A, R>(Function<A, R> function, A arguments)!)

[munificent]

I use Function just as a reminder that whatever is stored in this polymorphic function, at least I know it's a function of some sort. It's not necessary, but it is slightly helping readability. I'd expect people to want Record for the same reason.

Yes, I think this is the primary motivation for having a Record type even if it's (currently) just an empty interface. Since we don't have overloading or polymorphism over record arity, I'm certain users will sometimes write functions like:

/// Does some useful thing with the given [tuple], which should be a record
/// with only positional fields up to arity ten.
doStuffWithTuple(tuple) {
  switch (tuple) {
    case (f1, f2): ...
    case (f1, f2, f3): ...
    case (f1, f2, f3, f4): ...
    ...
  }
}

Having a Record supertype gives users a type that is at least marginally more precise than Object for that tuple parameter's type.

I'm going to leave Record in the proposal and close this, but feel free to re-open if any of you disagree.

[Levi-Lesches]

@munificent, the proposal says

A built-in class Record with no members except those inherited from Object

meaning that Record is a subtype of Object. But given that a) Record cannot be extended or implemented and b) any Object can be packaged in a 1-element record, would there be any reason not to make Record a supertype of Object?

It's true that you would lose the ability to say "this function accepts records only", but any amount of arguments can be packaged into a corresponding record (including a 1-element positional record), so I don't see the need for that distinction. On the other hand, there may be valid reasons for wanting an object over a record, like Expando. You would also not need a new, specialized error for extending Record since it isn't an Object in the first place. I list a few more reasons why this would be beneficial in this comment. Thoughts?

[lrhn]

If Record is a (proper) supertype of Object?, then every object is a record (which fair, it's a one-positional-element record, and Record is really Object?*, using the Kleene star). We'd have (), Object?, Object?×Object?, etc as immediate subtypes of Record (with further covariant subtypes on each position of the record).
I would love that! (Even more if we could make Null be the empty record, but I don't believe that's ever going to work.)

It's not how it's modelled, though. You can nest records! (One of my many prototypes of records tried to disallow that, making (int, (int, int)) isomorphic with (int, int, int). It's just not how people actually work with data.)

The problem with having any proper supertype of Object? is that a lot of existing Dart code assumes that every object can be assigned to Object?, and every non-null value can be assigned to Object. With this, (1, 2) is not an Object?.

It's not entirely new, we did it with Object?. Before that, everything was a subtype of Object. But that was a hard migration (and will be for a long time yet).

But surely that's fine, because no existing code uses records yet, so they can just change when migrating and starting to use tuples, right?
Sorry, but a lot of generic classes have an (implicit or explicit) bound of Object?. That would mean that you can't make a List<(int, int)>. Pretty restrictive.

But the, we could just say that "no bound" means a bound of Record instead of Object?, right?
Yes, except that if any implementation actually wrote <E extends Object?>, it would now be an invalid implementation.
That might be too potentially breaking.

But we could automatically change all Object? bounds to Record? bounds, as an automatic code migration?
Yes. It would most likely "work". It's a big migration, though, even if it's automatable, simply figuring out the order to migrate libraries in is going to be a huge job. Landing all the migrated packages, updating dependencies, of very library in the world (or at least O(n) of them).

We also need to consider interaction with older code. If we introduced records in, say, Dart 2.20.0, the what is Dart 2.19.0 code going to do if it sees a record.
If a record is-an Object, then it will just be "Hey, that's some weird object that I don't know the class of. Must be Tuesday."
If not, ... well, we're darn certainly going to be treating it as an Object anyway, because Dart 2.19.0 does not have any non-Null values which are not Objects. What if a Dart 2.19.0 gets (int, int) as a type argument (for a type variable with bound Object). Is that going to be a proper supertype of Object? Probably not, we're going to have to lie to spare the poor outdated code's feelings. Is a List<(int, int)> a subtype of List<Object> in Dart 2.19? Lying gets very difficult here.

I think we can make Record a supertype of Object?, but it's going to be expensive in migration and legacy-code-interaction management.

We'd also probably want another supertype, say Value, so that structs can be a subtype of Value, at the same level as Record (it's not a record, it's not an object).

Choosing to make Record a subtype of Object is not necessarily optimal, but it's realistic. We can get there from here.

(Will we want a type which means "non-Record object"? Say ReferenceValue <: Object, so that every non-record value is a subtype of ReferenceValue, but records are not? Or just Ref for short. And structs will also not be reference values. And we can decide whether to move num/bool/String out of Ref too (Null is already not an object), the types which do not work with Expando, so the type of Expando becomes Expando<T extends Ref>. We can add such a type at any time, but it seems relevant here.)

[eernstg]

If Record is a (proper) supertype of Object? ..

Then the test r is Record is indeed useless, so we don't need the type Record in the first place. What are the other benefits? ;-)

[Levi-Lesches]

I definitely understand how massive a change it would be to replace Object? as the top type, and I'm certainly not advocating for that to happen anytime soon. This is more just thinking about what could be, following the lines of "what if it had always been this way?" Maybe it'll be relevant in some future Dart 3.0, maybe not.

What are the other benefits?

In this comment I position Struct and Object as follows:

     Record 
       | 
     Value
       |
  +----+----+
Struct   Object

I'll copy from there what I think the benefits could be:

1. Answers a few questions flying around:
   i. Solves Records: What are the semantics of identity? #2390, just use Object when you need to rely on identity.
   ii. Solves What generated methods should structs provide? #2372, since structs won't inherit from Object.
   iii. Solves What does toString on a record do? #2389, since records won't inherit Object.toString.
   iv. Solves Do tuples need a shared non-Object superclass? #1290, since Record would be a supertype to Object.
2. All Object/Object? code would still work, just not accept any structs or records. This protects any code that relies on identity, like Expando, and doesn't need to worry about toString.
3. Inherently represents that structs cannot extend or be extended by objects without the need for "magic" errors.
4. You can use Value as the new Object/Object? (null can be a Value) to specify that you don't care about the differences in the identity of the value, just like how Object doesn't discriminate on the type.
5. Since all Values are Records, you can simply pass any Value as an argument to represent a 1-element Record. No need for myFunc( (value,) ). This is nice since records are similar to parameter lists in the first place.
6. You could specifically request a Struct, which is useful for optimizations like StructList and when writing struct-specific extensions. Maybe there could be macros that would generate code specifically for structs that might depend on how structs cannot be extended.

And it also answers this question about a non-Object type:

• Should Record be a subtype of Struct? (Possibly, or they should share a common "not a real object" supertype.)

It sounds like "not a real object" would mean it's more useful to ask, "is this an Object?" than "is this a Record or a Struct?" In that case, it wouldn't matter how Record and Struct are related, just that they cannot be used where an Object is specifically requested (which means they should not be subtypes of Object). In my proposal above, you could distinguish between Object and Value.

[lrhn]

If we go fully hypothetical, then your hierarchy makes sense. I wouldn't call the top Record, maybe Values (plural).

The question is what the other subtypes of Record/Records is, other than Value.

If the subtypes are Value0 (which I'd also call Null, we only need one singleton type),
Value, Value2, etc. (all the record shapes with Value as field type), then it doesn't allow nested records. Can't do ((1, 2), (3, 4)).
That's one, probably acceptable, design, where ((int, int), (int, int)) is considered isomorphic with (int, int, int, int). It's just four adjacent values in a specific order, and you can slice them any way you want. Records don't nest.
That's very likely to confuse a lot of people who think of records as entities in their own right. Mathematically sound, though.

If the subtypes are (), Value, Values2, etc., with Values as field type of the records, which allows nesting records, but has no singleton record type, then maybe it can still work. It starts looking weird that you can't nest singleton records. No ((1, ),). It's an exception to the otherwise general structure, and it might bleed through in some situations. Say, we want to use records/values to represent argument lists in noSuchMethod. Then you can't distinguish a single ((int, int) p) argument from an (int x, int y)argument list. So, singleton records will be needed. And then we lose the advantage of makingRecorda supertype ofObject, because the (Object,)type is different fromObject`.

The way I see things today is that Object? is just a misnamed Any. It's a supertype of Object and Null, and because it's the least supertype of both, it's also what you get when you add ? to Object. We need to make records a subtype of Object?, because all existing generic code assumes all values are Object?s. Too much code would break if we don't make it a subtype of Object, because the code also assumes everything is either Null or Object. When records are subtypes of Object, and record fields are subtypes of Object?, we either need real singleton records, different from just its content, or we need to have no singleton records at all. (The latter will come back to haunt us if we ever want records to represent parameter lists, because then we can't distinguish receiving a single pair as argument, or receiving two arguments.)

Pragmatism for the lose (and win!)