C# Design Notes for Aug 18, 2015

[MadsTorgersen]

C# Design Notes for Aug 18, 2015

Agenda

A summary of the design we (roughly) landed on in #5031 was put out on GitHub as #5032, and this meeting further discussed it.

1. Array creation
2. Null checking operator
3. Generics

Array creation with non-nullable types

For array creation there is the question whether to allow (big hole) or disallow (big nuisance?) on non-nullable referance types. We'll leave it at allow for now, but may reconsider.

Null checking operator

Casting to a non-nullable reference type would not, and should not, do a runtime null check. Should we, however, have an operator for checking null, throwing if the value is null, resulting in the non-null value if it isn't?

This seems like a good idea. The operator is postfix !, and it should in fact apply to values of nullable value types as well as reference types. It "upgrades" the value to non-nullable, by throwing if it is null.

if (person.Kind == Student) 
{
  List<Course> courses = person.Courses!; // I know it's not null for a student, but the compiler doesn't.
  ...
}

The ! operator naturally leads to x!.y, which is great! Although !. is two operators, it will feel as a cousin of ?. (which is one operator). While the latter is conditional on null, the former just plows through. Naively, it implies two redundant null checks, one by ! and one by ., but we'll optimize that of course.

if (person.Kind == Student) 
{
  var passed = !person.Courses!.Any(c => c.Grade == F);
  ...
}

Technically this would allow x!?.y, which comes quite close to swearing. We should consider warning when you use ?. on non-null things.

VB may have a problem with post-fix !. We'll cross that bridge when we get there.

Generics and nullability

Is it too heavyhanded to require ? on constraints to allow nullable type arguments?

Often, when you have a constraint it is because you want to operate on instances. So it's probably good that the default is not nullable.

It may feel a bit egregious to require it on all the constraints of a type parameter, though. Should we put any ?'s on the type parameter declaration instead of in the constraints? No, that is too weird and different. The case of multiple nullable constraints is probably sufficiently rare that it is reasonable to ask folks to put a ? on each. In fact we should disallow having ? on only some, since those question marks won't have an effect: they'll be cancelled by the non-nullable fellow constraints.

The proposal talks about allowing ? on the use of type parameters to explicitly override their nullness. Maybe we should have an explicit ! as well, to explicitly override in the other direction: non-nullable. Think for instance of a FirstNonNull method.

T! FirstNonNull<T>(IList<T> list) { ... }
T? FirstOrDefault<T>(IList<T> list) { ... }

This means complexity slowly creeps into the proposal, thanks to generics. However, it seems those overrides are relatively rare, yet really useful when you need them.

T! would only be allowed on type parameters, and only when they are not already non-null by constraint.

[AdamSpeight2008]

VB may have a problem with post-fix ! . We'll cross that bridge when we get there.

That bridge is around, how it'll work with the dictionary lookup literal which is also !

[orthoxerox]

What about using a single static readonly immutable instance of a non-nullable class to populate an array? Yes, it will be slower than just XORing the pointers with themselves, but since that default object would always have the same address during the whole life of the program, pasting that address all over the array should be reasonably fast.

[SolalPirelli]

In the second example, it should be person! rather than !person, right?

---

EDIT: Nope, I was wrong. It does show the confusion potential with this operator, though.

[antiufo]

In the second example, it should be person! rather than !person , right?

It's a boolean negation

[YuvalItzchakov]

@SolalPirelli Thank you for that question, because I was just about to comment that the ! operator will cause confusion with regards to the NOT operation. The example if a student that received an F in any of his course. This is definitely confusing to the naked eye.

I think a different operator symbol should be considered.

[govert]

Please don't use the ! for any of these cases.

Can you explain why cast should not do a null check?
Otherwise, what does this do:
string x = (string)null;

Can the 'as' operator check for null?

For the generics, co/contravariance uses keywords already. Maybe a slightly more verbose indicator would be OK (even as an extra type constraint or something:
`S FirstNonNull(IEnumerable list) where S : notnull T { ... }

I vote for verbose on the edge cases, rather than more operators.

[jeffanders]

I am not sure what opportunity members of the language design team get to look at the large numbers of proposals on GitHub but I thought I would highlight my proposal that I believe specifically addresses all of these issues. Please see #4443. I will give some examples addressing the scenarios above.

Array creation with non-nullable types. I proposed a new array creation expressions that takes a single non null expression that will initialise all elements of the array with that one value. For example:

string![] words1 = new string![10] "hello"; // creates a 10 element array with all elements initialised to "hello"

Null checking operator. I proposed a conditional assignment operator which would alter definite assignment rules:

string! x; // not definitely assigned
string y = "hello";
if (x ?= y)
{
  // do something with x as it is now definitely assigned and non null
}
// x is no longer definitely assigned

Generics and Nullability. I propose a scheme that allows you to differentiate between generic types or methods that can only take nullable or non nullable references or may take either and does so in such a way that the compiler can enforce the scheme with appropriate errors rather than warnings or analyzers. Basically, generic type parameters as written today would erase the non-nullability of type arguments. This provides complete compatibility with existing code. For example the definition of GetFirstOrDefault would not need to change at all and would remain exactly as is as it makes sense that even when the type argument is non-nullable that the result may be nullable. So it would remain:

public static TSource FirstOrDefault<TSource>(this IEnumerable<TSource> source) { ... }
IEnumerable<string!> values = ...; // initialise to some appropriate value
string value = values.FirstOrDefault(); // return value is nullable

Now consider another overload of FirstOrDefault that took an extra parameter representing the default value to return if there is no first element. It makes sense that this should work with both nullable and non nullable types and the return value should preserve the nullability of the parameters. This is opt in similar to how interface/delegate variance was added to C# 4.0 in a highly compatible manner. This does apply an annotation/modifier (!?) to the type parameter which the design team notes as too weird above but it could be expressed in constraints if this is undesirable.

public static TSource FirstOrDefault<TSource!?>(this IEnumerable<TSource> source, TSource defaultValue) 
{ 
  foreach (var value in source)
    return value;
  return defaultValue;
}
IEnumerable<string> nullableValues = ...; // initialise to some appropriate value
IEnumerable<string!> nonNullableValues = ...; // initialise to some appropriate value
string nullableValue = nullableValues.FirstOrDefault("hello"); // return value is nullable
string! nonNullableValue = nonNullableValues.FirstOrDefault("hello"); // return value is non-nullable

As an added bonus here is a definition of the ToArray extension method since it combines both of the array and generic in one concise example:

public static TSource[] ToArray<TSource!?>(this IEnumerable<TSource> source)
{
  int count = source.Count();
  TSource[] result = null;
  int index = 0;
  foreach (var value in source)
  {
    if (result == null)
      result = new TSource[count] value;
    else
      result[index] = value;
    index++;
  }
  return result ?? new TSource[0];
}
string[] nullableElements = nullableValues.ToArray(); // return value is string[]
string![] nonNullableElements = nonNullableValues.ToArray(); // return value is string![]

I would appreciate any feedback anyone has on the proposal as it goes far beyond these scenarios and I have attempted to address non nullability in combination with all language features from C# 1.0 to C# 6.0.

[qrli]

null check operator can be done by a simple extension method:

public static T NotNull(this T obj) where T : class
{
  if (obj == null) throw ...
  return obj;
}

var passed = !person.Courses.NotNull().Any(c => c.Grade == F);

If the operator will not be frequently used, I think the extension method solution is good enough, given the ! symbol is somehow confusing.

[GeirGrusom]

if(string foo ?= nullableString)
{
  WriteLine(foo);
}
WriteLine(foo) // Compile error: foo is not in scope.

[paulomorgado]

@jeffanders, leaving aside the questionable syntax for array initialization, using only strings to reason about reference types is dangerous because string instances are immutable.

As some operations on arrays are already only possible through static methods of the Array class, I would prefer to have an Array.Create method that would take a value or a value factory function:

T![] Array.Create<T!>(int size, T! value);
T![] Array.Create<T!>(int size, Func<T!> valueFactory);

[craigkovatch]

Should we, however, have an operator for checking null, throwing if the value is null, resulting in the non-null value if it isn't?

Please do educate me here (I'm sure I'm missing something) -- but this seems the behavior we already have with NREs: thrown if a dereference is null, no problem otherwise. What does the postfix ! operator give us?

[gafter]

@craigkovatch That is exactly right. The hypothetical postfix ! operator could very well be thought of as the dereference operator, as it performs the null check and does nothing else. There is no existing operator in the language that performs only the null check.

[craigkovatch]

How is that useful? Allowing us to throw earlier?

[gafter]

@craigkovatch If we have nullable reference types where it would be an error to just . off them, this would be a way to express the intent that the exception is desired if it is null, otherwise the operator returns a value that could not be null.

[craigkovatch]

@gafter If I'm understanding then, it's a way of foregoing the compiler protection, rather than adding any kind of new runtime protection. Is that right?

[gafter]

@craigkovatch No, that's not right, because the postfix ! operator could be used without being immediately followed by ..

[FrankBakkerNl]

@craigkovatch, I imagine it would allow you to use a Nullable refence where a nun-nullable reference is required, like in

void Foo(Person! person)
{ 
    person.Name; // will not throw
}

Person person = GetPerson(); // we 'know' this will (of should) not be null, but the compiler does not.
Foo(person!); // will throw here if person was null

[alrz]

T! would only be allowed on type parameters

I think ! would make sense on concrete types as well, meaning "not nullable". for example in

string! F() { ... }

string! will assure you that this method will not return null. this would be something like checked exceptions in Java, and will spread over all methods.

string G() { ... }
string! H() { ... }
string! F() {
    return null; // error
    return G(); // error
    return H(); // OK
}

This can take a step further and produce a warning if a method is not returning null:

// warning: this can return a string!
// make it string? to explicitly define it as nullable,
// even if it doesn't return null currently
string G() { 
    return "";
}

EDIT: I think #227 addressed this.

[qrli]

@alrz That notation was one of the original proposals. The major benefit of it is that it provides better compatibility with source code, because non-nullability is opt-in on parameter/return value level. The drawbacks are:

1. nullability will have different syntax for value type and reference type, which brings more confusions (as you must know whether it is a class or struct to understand the code) and refactor troubles;
2. because non-nullable is more common than nullable, source code will be polluted by lots of !s.

[gafter]

Design notes have been archived at https://github.com/dotnet/roslyn/blob/future/docs/designNotes/2015-08-18%20C%23%20Design%20Meeting.md but discussion can continue here.

[shadowfoxish]

I watched the video https://channel9.msdn.com/Blogs/Seth-Juarez/Looking-Ahead-to-C-7-with-Mads-Torgersen which towards the end, discusses the nullability specifier for types to disallow a ref type from being null.

Rather than implementing more punctuation, what if we did a keyword similar to readonly so you would declare a property or variable like this:

public notnull string FirstName { get; set; }

I think this is much more readable than this (traditionally ! means 'not', so really, what the heck am I doing?):

public string! FirstName { get; set; }

Of course, if I tried to do something like this.FirstName = null; I would get a compiler error or an exception.

You'd probably have to deal with return types as well, so that you can eliminate the ambiguity and let the compiler help you be 'safe'.

public notnull string GetName() {
   notnull string temp = "a string"; //notnull keyword here should probably be optional
   return temp;
}
public notnull string GetName2() {
   return null; //Compiler error
}
...
this.FirstName = GetName();

notnull types should also coalesce directly to a nullable type (or normal type) so if I had a property

public string LastName { get; set; }

I could assign a notnull string to it without problem. This might make the whole thing opt-in for those who care to use it.

You could also do this with arrays perhaps, but you'd have to deal with the initial value of every element (D)...

//A
notnull SpecialType[] arrA = new SpecialType[10]; //Array itself cannot be null
arrA[0] = null; //OK
arrA = null; //Error
//B
SpecialType notnull[] arrB = ...; //Maybe implies each element must not be null?
arrB[0] = //Error
arrB = null; //OK
//C
(notnull SpecialType)[] arrC = ...; //Less ambiguous version of B
arrC[0] = null; //Error
arrC = null; //OK
//D
notnull (notnull SpecialType)[] arrD = new SpecialType[10] => new SpecialType();
arrD[0] = null; //Error
arrD = null; //Error

A, B, C are ideas to address the ambiguity of is arr not allowed to be null, or are the elements not allowed to be null? D is an approach to set the initial value of every element using the lambda operator. The collection initializer should also be allowed.

I suppose the alternative to putting notnull where you reference the type might be to decorate the class itself as being notnull, but that removes flexibility, unless you could inherit and apply the notnull-ness. I think I like this less.

public class Abc { }
public notnull class AbcN : Abc { }

/spitballing