Revamp treatment of optional values (esp records)

[isaksky]

The current record constructors are problematic, because if you add a field to it, everywhere that uses it is broken. Adding a field to a record is therefore often not backwards compatible, which is a big downside, and leads to some authors avoiding records altogether if they may be exposed externally.

This is a shame, because records and the record construction syntax are otherwise fantastic parts of F#.

Solution: Better record construction syntax

To solve this problem, I propose we add a new record construction syntax that is more clear, more tooling friendly, more concise, and less brittle. The current record constructors are problematic, because if you add a field

Summary:

1. Introduce %RecordName { ... } syntax for constructing records (stolen from elixir)
2. Allow omitting fields with obvious "zero" values (such as None for option)

• null for reference types does not count

3. In addition to 2, or instead of 2: Allow omitting fields for fields that have a default defined in the record definition

Examples:

type Person = { fullName: string; email: string option; }

// A symbol like `%` (stolen from elixir) to say you want to start creating the record
// We leave out the optional field "email" here, and it still compiles. It will be set to None.
let person1 = %Person { fullName = "Don Syme";  }

// This does NOT compile, because fullName is not an option, and we don't want to encourage null values
let person3 = %Person { }

// Some other cases to consider
type InventoryRow = { name: string; weight: float; tags: string list; quantity: int }
let row1 = %InventoryRow { name =  "Bungie" }
// This would be equivalent to:
let row2 = { name = "Bungie"; weight = 0.0f; tags = []; quantity = 0 }

For the alternative version:

type InventoryRow = 
  { name: string = "<unnamed>"; 
    weight: float; 
    tags: string list = ["unlabeled"]; 
    quantity: int = 1 }

let row1 = %InventoryRow { weight = 1.0f }  // allowed

// no weight specified, a field that has no default specified.
// Allowed only if we go with AND for part 3 of the suggestion
let row2 = %InventoryRow {  }  

Pros and Cons

• Adding an optional field to a record will no longer lead to tons of compilation errors all over your codebase (everywhere you used record construction syntax)
• When typing your code, you type your intent to make a new X record at the very beginning, which leads to better / easier to implement editor support
• Faster type checking
• Better error messages
• Increased clarity

The disadvantages of making this adjustment to F# are ...

• There would be more than 1 way to construct records, adding confusion.

Extra information

For point 2 of the summary, some implicit defaults to consider for types:

• Option : None
• Collections: Empty collections
• Numbers : 0
• Built in non-numbers that have a .MinValue property. for example TimeSpan, DateTime.

Somewhat related: Kotlin data classes.

Estimated cost (XS, S, M, L, XL, XXL): Unknown.

Related suggestions: (put links to related suggestions here)

Affidavit (please submit!)

Please tick this by placing a cross in the box:

• This is not a question (e.g. like one you might ask on stackoverflow) and I have searched stackoverflow for discussions of this issue
• I have searched both open and closed suggestions on this site and believe this is not a duplicate
• This is not something which has obviously "already been decided" in previous versions of F#. If you're questioning a fundamental design decision that has obviously already been taken (e.g. "Make F# untyped") then please don't submit it.

Please tick all that apply:

• This is not a breaking change to the F# language design
• I or my company would be willing to help implement and/or test this

[cartermp]

Overall I think this is a good idea. Two questions:

1. You listed better error messages as a benefit. What did you have in mind w.r.t improvements?
2. Off the top of your head, what would you count as having an obvious zero value? Not looking for a detailed breakdown, just curious what your thoughts are here.

[Rickasaurus]

Did you know you can specify the type of the record currently in other ways like:
let person1 : Person = { fullName = "Don Syme"; email = None }
or
let person2 = { fullName = "Don Syme"; email = None } : Person

Maybe another way to go about the options as optional thing would be with a keyword like

let person3 = optionsBeGone { fullName = "Don Syme" }

As it would mirror the way struct tuples and such look.

[isaksky]

@cartermp

1. In some cases now there is ambiguity from guessing the type the user is trying to construct based only on the field names. For example, imagine if you were trying to construct the A type here. :

The error message assumes you meant to construct B.

2. Just off the top of my head, I would consider:

• Option : None
• Collections: Empty collections
• Numbers : 0
• Built in non-numbers that have a .MinValue property. for example TimeSpan, DateTime.

[isaksky]

@Rickasaurus I've seen that, and I think for the former, the problem is it may be far away from the actual {...} syntax at the end of the expression, which means editor support cannot be as good.

For example, imagine you wanted to make a feature that automatically filled added fullName: <cursor here now > } after you typed %Person {. One could not do that when typing { with the let a : Person = .. syntax, because you do not know if the expression being typed will end up in a return position.

For the latter variation, the problem is it comes after, which is also problematic for tooling support.

[cartermp]

@isaksky Makes sense! And w.r.t your tooling point, this does indeed make it easier. Eventually, we'll spec out what IntelliSense for F# needs to look like, and it's likely that we'll take a similar approach that Roslyn did where there are multiple providers. This would be a case for another IntelliSense provider and fits snugly with what I think to be a good approach to IntelliSense.

[theprash]

Regarding inference of the correct record type, I often use the fully qualified field name for the first field as a type hint:

let person1 = { Person.fullName = "Don Syme"; email = None }

Regarding default values, this seems to go against the philosophy of F# in other places, where it avoids implicit default values like those in C#, which are more useful with mutation. None as a default for option is fine, but why should the default of int be zero, and why should the default of int list be []? I think I'd prefer those defaults to be made explicit in code:

let makePerson fullName = { Person.fullName = fullName; email = None; productivityFactor = 1.0 }

let person1 = { makePerson "Don Syme" with productivityFactor = 10.0 }

Using a function, we can specify the mandatory values and this also gives you the type inference.

[isaksky]

@theprash A slight variation that is slightly more explicit would be:

type InventoryRow = 
  { name: string = "<unnamed>"; 
    weight: float; 
    tags: string list = ["unlabeled"]; 
    quantity: int = 1 }

let row1 = %InventoryRow { weight = 1.0f }  // allowed

// no weight specified, a field that has no default specified. Allowed? [0]
let row2 = %InventoryRow {  }  

So allow supplying default values in the record definition that will be used when nothing is specified for the field during initial construction.

This way everything would (or could, depending on [0]) be explicit, but we'd still have the benefits discussed earlier.

I think most people wouldn't be too shocked if [] was the default value for a list, and 0 for a number, but I don't have any evidence to point to.

[isaksky]

One of the goals of this is to prevent needing to make record constructors private because of how brittle they are. As things stand now, you cannot expose them in libraries or non-trivial programs, because if you need to add a field, you just broke a ton of code. Even if none of that code cares about your field at all.

Yes, you can make a factory type function that you expose instead, but then it is no longer first class, and people have to know about it.

[isaacabraham]

The "setting defaults" worries me. You add a new field to a record and won't know that you've forgotten to initialise it - it'll just be set to an empty list or 0.0 or whatever.

[enricosada]

I dont like the initial proposal % to default optional/part of fields.
I have some issues with that:

• seems like default initialization of field of c#. While is ok in c# lang, i dont think is a feature i like to have in F#.
• is hard to define default in compiler side. you need to restrict it to some types, like options/int? is the zero/empty? what about types without that, like DateTime? why i need to care about few cases? for classes? what about user types? dev need to know what is the default, so rules to learn (in .NET , default of classes object is null. That as example string, the default is null, not the empty string).
• personally, i like that when i add a new field, the whole program doesnt compile, so i see exactly where i used that. and the new value for that field maybe is not the same, depends on where is used.
• if i want to create a default for other consumers, is often domain specific or for some use case so i add a value (State.initial, Array.empty, etc) or function (mkUser name)
• creating a real default value for a type, is also faster, because can be reused/optimized/inlined. but that can be done intenrally by compiler (but more work to implement)

That said, the second variation you proposed is better.

type InventoryRow = 
  { name: string = "<unnamed>"; 
    weight: float; 
    tags: string list = ["unlabeled"]; 
    quantity: int = 1 }

but that mean the values are harder to bind. what is going to be allowed?

  name: string = "<unnamed>".Trim(); 
  name: string = f "<unnamed>"; 
  name: string = String.empty; 
  name: string = myInitialValue; 

[isaksky]

@isaacabraham @enricosada

F# already has this for classes:

and InventoryRow()=
  member val name : string = "<unnamed>" with get, set
  member val weight : float = 0. with get, set
  member val tags: string list = ["unlabeled"] with get, set

Do you think this is a problematic feature of F#? If not, what is so different about this case (especially the second variation)?

Keep in mind - if you want to be explicit, and have code that breaks when you add a new field, you already have that option, and will continue to have it even if this alternate syntax is added.

@enricosada Also, we definitely would not want null to ever be a default - that would go against the goals of F#. If we go with the version that adds an implicit default value for some types, it would only be for ones that have a sensible default value, like None for option.

To make it easier on people just starting to read this, I'll update the main post with the other variation too.

[dsyme]

I think this suggestion could only be part of a more systematic look at how F# treats default values and optionality throughout a number of constructs, not just record types

[voronoipotato]

There's nothing stopping you from making a static member called Create that has optional arguments and returns a record. I personally would be abundantly careful of adding anything that puts more foot-guns on the desk. Defaults of any kind can be deceptively dangerous and are frequently a source of bugs.

[0x53A]

I think this would be really useful with anonymous records for Fable Typescript interop.
JS APIs often contain a large number of optional parameters.

When I have a typescript function

function f(o : {a? : number, b? : string, c : boolean}) {
    // ....
}

then I can call it like this:

f({a: 1, c: false}); // b is omitted and undefined

Currently I have three options for the Fable bindings:

1. Named record

type FArgs = { a : float option; b : string option; c : bool }

[<Import("...")>]
let f(o:FArgs) = jsnative

f({a = Some 1.; b = None; c = false})

2. Anonymous record

[<Import("...")>]
let f(o:{| a : float option; b : string option; c : bool |}) = jsnative

f({|a = Some 1.; b = None; c = false|})

3. Union

type FArgs =
| A of float
| B of string
| C of bool

let f(o:FArgs list) =
    let o2 = keyValueList Fable.Core.CaseRules.LowerFirst o
    // ...

let o = 
f [ A 1.; C false |

The union has the advantage that I don't need to name the optional members (which are often a lot), but makes the whole thing a bit awkward.

I would really like to be able to use an Anonymous Record and omit the optional members.

2. Anonymous record

[<Import("...")>]
let f(o:{| a : float option; b : string option; c : bool |}) = jsnative

// same as 2), just the 'b = None' omitted
f({|a = Some 1.; c = false|})

IMO I would even restrict it to Anonymous Records only, because they are already a bit more loose than normal Records.

[daniellittledev]

The idea of default values seems very dangerous to me. Regarding collections, numbers and especially dates these values do not have a safe default because it depends so much on context. Defaults like DateTime.MinValue are never a great default. If someone does want all values to have a default, a default instance and the with keyword fills that gap.

One properties I like the most about records is they're more like constructors than a bag of properties. When you add a property to a record, there is an error at each location it's constructed. You can't forget to add a property.

I think the only case we should consider a default is for explicitly optional properties (like with Option<>) which default to None.

One case where I think this would be valuable is when calling a function with a config style object, this is common as a js/ts style pattern but I think it's much more flexible than 12 overloads, especially since fsharp functions don't allow overloading.

Consider the following HttpRequest style record where most of the properties are optional. While the function using this object may use certain default values, they are not defaults of this object.

type HttpRequest = {
    url: string
    method: Method option
    baseURL: string option
    headers: Map<string, string> option
    data: obj option
}

[isaksky]

One properties I like the most about records is they're more like constructors than a bag of properties. When you add a property to a record, there is an error at each location it's constructed. You can't forget to add a property.

I consider that a language design problem, since it leads to people not being able to this language feature if they do not inflict breaking changes on consumers of the code. I see that many F# people disagree, and love this part of F#.

I find the trivial compile errors really problematic. One time when I needed to add a property to a record, I actually introduced bugs because I got so bored updating the 50+ callsites that didn't care about by new optional property that I stopped being careful.

Regarding collections, numbers and especially dates these values do not have a safe default because it depends so much on context.

I'm sure it is very contextual for many data types, but that weakness doesn't apply to the alternate syntax I suggested, where you can specify what the default is, similar to C#.

[voronoipotato]

I think standard objects already exist for when you have default values and better suit your described scenario where you don't want the code to turn red. There's nothing wrong with using objects when they fit the bill. I think if you have an optional field in your record you should take an option type for that field. You can even add a "getAge" function in the module for that record that returns 21 when the value is None. Being said if you don't need to check your 50+ call sites perhaps a record is too rigorous for what you're using it for. Records are really great because they encourage fully disjunctive reasoning, but sometimes you just don't have the time. I love strict, strong, functional style, but objects are not a crime.

[abelbraaksma]

Being said if you don't need to check your 50+ call sites perhaps a record is too rigorous for what you're using it for

Or you are actually protecting yourself from making mistakes. Getting bored for fixing your code is not a good excuse for introducing bugs. Referential integrity is a great thing, and records protect you from making changes that aren't compatible with your code.

As a version between objects and records, you can always add a Create method to the record, if 90% of your call sites don't need to the new optional value, or if most values are optional, and are typically not set.

[isaksky]

I think standard objects already exist for when you have default values and better suit your described scenario where you don't want the code to turn red. There's nothing wrong with using objects when they fit the bill. I think if you have an optional field in your record you should take an option type for that field. You can even add a "getAge" function in the module for that record that returns 21 when the value is None. Being said if you don't need to check your 50+ call sites perhaps a record is too rigorous for what you're using it for. Records are really great because they encourage fully disjunctive reasoning, but sometimes you just don't have the time. I love strict, strong, functional style, but objects are not a crime.

@voronoipotato That would only be a useful suggestion if I didn't care about any of the other features of F# records: like structural equality, creating new objects from existing ones, etc. Why should I have to give up those things just because I don't want my constructor calls breaking needlessly?

Or you are actually protecting yourself from making mistakes. Getting bored for fixing your code is not a good excuse for introducing bugs. Referential integrity is a great thing, and records protect you from making changes that aren't compatible with your code.

@abelbraaksma if having a property be defaulted would lead to errors, you would just not specify a default when creating the type definition for the record. And people don't need to pass whatever test you made up for what a 'good excuse' is. What matters is how these language constructs affects what people do in practice, not what you think they should have done. Humans are not perfectly conscientious and inexhaustible robots.

As a version between objects and records, you can always add a Create method to the record, if 90% of your call sites don't need to the new optional value, or if most values are optional, and are typically not set.

So I could do it some other way that involves writing a lot of boilerplate? I can do that in Java too. By that logic, there was no point adding string interpolation to F# either, right? After all, you could do it this other way...

[abelbraaksma]

So I could do it some other way that involves writing a lot of boilerplate? I can do that in Java too. By that logic, there was no point adding string interpolation to F# either, right?

That's a fair point, for a long time, there was a strong sentiment not to implement string interpolation. And the net result is not the same as C# (mainly nested support, but also, F# adds mixin with sprintf style).

I like the integrity with records, I wouldn't like throwing that away. You can break the integrity with Create, and that's what we used on a large codebase, only to remove it later, because the optional items turned into a PITA. We went back to using straight record syntax with explicit optionals, and removed all other POCO objects that could be represented as records. It became easier to reason about the code and the small price for specifying None specifically turned out to be a big benefit on readability: no need to jump into constructors and other code to find out what a missing argument meant.

I'm not saying your proposal is without merit, I can see some use cases, but I'd prefer an addition only if it doesn't break the integrity of records as it stands.

Btw, I don't agree that using Create adds a lot of boilerplate. More the opposite. You write your default record construction once, instead of on each call site, and you use the with syntax to update anything you want to be non default. In my experience, it isn't always clearer, but it certainly is much, much less code to write.

[NinoFloris]

Another way to do this in F# - which isn't always a better fit than a constructor - is to have your different starting points represented as regular values like so

type Bobble = { Ding: int; NewProp: int option }
let dingBobble = { Ding = 0; NewProp = None }

let code ding = { dingBobble with Ding = ding } 

[isaksky]

@abelbraaksma What you say makes sense, my only objection would be that I think programmers can be trusted to just not provide a default value for a field if it the value can come as a surprise (or cause integrity problems as you say). Also, keep in mind this would only be for the alternate record construction syntax, so it wouldn't affect what is in use now.

@NinoFloris I like that way, I think that is one of the best ways to do it today. My only issue with it is that it is another name, and another thing that will not necessarily exist for a record. So let's say I know a type name, Foo - how do I know where the default Instance for it is? There is no way to know, except if you establish some convention. In that way, it is similar to the static Create method.

For that strategy, if there were a way register a default type for a record, so that I could just write this, I think it would be a lot better (and would also solve my problem) :

let code ding = { default<Bobble> with Ding = ding } 

[abelbraaksma]

how do I know where the default Instance for it is

I'd put it behind a static property on the type for easy discovery.

[davidglassborow]

how do I know where the default Instance for it is

I'd put it behind a static property on the type for easy discovery.

I often have a static property for my record types called Empty or Initial, depending on what it represents, where it makes sense

[voronoipotato]

The term proto, or prototype might be a good word.

[charlesroddie]

The need for disambiguation isn't related to optional arguments. You can work around the ambiguity problem with let r:MyRecord = { Field1 = x, ... } and, when https://github.com/fsharp/fslang-design/blob/master/RFCs/FS-1073-record-constructors.md is implemented, you can use let r = MyRecord(x,...). Having a third record construction syntax isn't a good idea.

Making constructor arguments automatically optional is a bad idea, but making them explicitly optional could work and has precedent with POCO constructors. To be consistent with these you would have:

type MyRecord = {
    X:int
    ?Y:int
    ?Z:int
    }
let r1 = { X = 0; Z = Some 3}
let r3 = MyRecord(0, Z=3)

There is a difference from POCO constructors, as you probably wouldn't have Z = and ?Z = distinguished in the {} constructor syntax. But this difference is small.

[isaksky]

The need for disambiguation isn't related to optional arguments.

If we take over the other existing syntax, it would:

1. cause backward incompatible language changes
2. remove the ability to have all your constructor sites break when you add a field, which some consider to be a feature

That is why it is not done that way (using the existing syntax) in this proposal.

Having a third record construction syntax isn't a good idea.

That is not the intent here, this is from way before that became a thing.

[Tarmil]

1. cause backward incompatible language changes

If optional fields must be explicitly declared as such, then I don't think so.

2. remove the ability to have all your constructor sites break when you add a field, which some consider to be a feature

Mandatory fields would still break all your constructor sites. Optional fields wouldn't, but that's the whole point of optional fields :P

[uxsoft]

This would be super useful when working with Fable and interfacing with Javascript components!