-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Help compiler enforce nullability annotations #32090
Conversation
@@ -221,7 +221,7 @@ public ImmutableHashSet<T> Remove(T item) | |||
[Pure] | |||
public bool TryGetValue(T equalValue, out T actualValue) | |||
{ | |||
int hashCode = _equalityComparer.GetHashCode(equalValue); | |||
int hashCode = _equalityComparer.GetHashCode(equalValue!); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 should we remove the [DisallowNull]
on that GetHashCode
, or add [DisallowNull]
here?
(same question below) #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general IEqualityComparer<T>.GetHashCode
(e.g. StringComparer.GetHashCode throws ArgumentNullException for null) even if some implementations do, so we can't remove [DisallowNull]
. We could consider adding [DisallowNull]
to this T equalValue
, but then we'd be saying that null wasn't allowed even if the developer provided a comparer implementation that did actually allow null. This is an example where we lack sufficient expressivity. I guess for now it'd be best to add [DisallowNull]
, and then see what kind of issues that causes; we can remove [DisallowNull]
later on, but it'll be harder to add it later on. #Resolved
@@ -140,10 +140,12 @@ internal virtual bool TryGetExports(ImportDefinition definition, [NotNullWhen(tr | |||
if (multipleExports != null) | |||
{ | |||
multipleMatches = multipleExports; | |||
// TODO2 singleMatch dould be null when returning true | |||
singleMatch = null!; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 the [NotNullWhen(...)]
annotation on method seems incorrect #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please remove it. Thanks. #Resolved
@@ -172,14 +172,15 @@ private static bool IsLazyGenericType(Type genericType) | |||
|
|||
private static bool TryGetCastFunction(Type genericType, bool isOpenGeneric, Type[] arguments, [NotNullWhen(true)] out Func<Export, object>? castFunction) | |||
{ | |||
castFunction = null; | |||
castFunction = null!; // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 the [NotNullWhen(...)]
annotation on method seems incorrect #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please remove it. The dangers of the compiler not having checked those annotations ;) #Resolved
{ | ||
if (this.IsGeneric()) | ||
{ | ||
singleMatch = null; | ||
singleMatch = null!; // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 the [NotNullWhen(...)]
annotation on method seems incorrect #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto #Resolved
src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/Interpreter/LightCompiler.cs
Show resolved
Hide resolved
{ | ||
if (d1.Index > d2.Index) return 1; | ||
if (d1!.Index > d2!.Index) return 1; // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 IComparer
declares that it can compare null
inputs, but this implementation doesn't handle. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's only ever used with non-null values: https://source.dot.net/#System.Linq.Expressions/System/Linq/Expressions/Interpreter/LightCompiler.cs,233 #Resolved
@@ -25,7 +28,9 @@ public virtual void Fail(string? message, string? detailMessage) | |||
} | |||
WriteAssert(stackTrace, message, detailMessage); | |||
FailCore(stackTrace, message, detailMessage, "Assertion failed."); | |||
#nullable disable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 without this suppression, the compiler points out that this method could return normally. Looking at the implementation of FailCore
, I think it has the same problem.
We need to discuss what is the best way to suppress this warning. Maybe a more fined-grained suppression on a specific diagnostic would be better than the coarse #nullable disable
I used... #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're nullable disabling/enabling around the closing brace?
Regardless, yes, it's possible for someone to implement Fail in a way that still returns, but that goes against the purpose of the method. If we held true to the possibility that someone did that, then we'd need to remove [DoesNotReturnIf(false)]
from Debug.Assert and all such methods, which would be terrible. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't we add DoesNotReturn
to FailCore
? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@safern FailCore
has return
statements, so the same warning "method should not return" would be produced there if we add [DoesNotReturn]
. #Resolved
{ | ||
if (Name != other.Name) | ||
if (Name != other!.Name) // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 not sure what to do here #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why did we need to make it nullable? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an internal type, and presumably it's not expecting Equals to be used with null, even though it implements an interface that allows it. Seems like the right change is to remove this ! and ? and suppress the warning on the Equals method signature. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To implement IEquatable
you must be compatible with the signature bool Equals([AllowNull] T other);
#Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To implement IEquatable you must be compatible with the signature bool Equals([AllowNull] T other);
I understand that. So I'm saying use #pragma warning disable whatever
to suppress the warning. #Resolved
src/libraries/System.Runtime.Extensions/ref/System.Runtime.Extensions.cs
Show resolved
Hide resolved
@@ -639,7 +639,7 @@ private static bool IsSupersetOf(IEnumerable<T> other, MutationInput origin) | |||
private static MutationResult Add(T item, MutationInput origin) | |||
{ | |||
OperationResult result; | |||
int hashCode = origin.EqualityComparer.GetHashCode(item); | |||
int hashCode = origin.EqualityComparer.GetHashCode(item!); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But now it seems like that would actually not just be for TryGetValue, but really all operations on the hash set, at which point we'd be better off using a notnull constraint? Maybe instead of changing the annotations, open an issue about it and keep the ! changes you have here. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(And actually, the right answer is probably to do a null check prior to calling GetHashCode, as we do in HashSet<T>
.) #Resolved
@@ -15,7 +16,7 @@ private ReferenceEqualityComparer() | |||
{ | |||
} | |||
|
|||
public bool Equals(T x, T y) | |||
public bool Equals([AllowNull] T x, [AllowNull] T y) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since T
is constraint to class, can we instead annotate T
as T?
? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed. I'd missed that. Thanks #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hum, it looks like this file is compiled into more than one project. One with nullability enabled and one without. Reverted to use the attribute. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use the annotation and just add #nullable enable
at the top of the file as well? That's what we've done in other cases like this, where we've not yet gotten to every project that includes it. #Resolved
...braries/System.Linq.Parallel/src/System/Linq/Parallel/QueryOperators/Unary/ForAllOperator.cs
Show resolved
Hide resolved
@@ -303,7 +303,8 @@ public void Add<T>(T value) | |||
|
|||
public void Add<T>(T value, IEqualityComparer<T>? comparer) | |||
{ | |||
Add(comparer != null ? comparer.GetHashCode(value) : (value?.GetHashCode() ?? 0)); | |||
// TODO2 | |||
Add(comparer != null ? comparer.GetHashCode(value!) : (value?.GetHashCode() ?? 0)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As with the earlier examples with ImmutableHashSet, we should probably fix this implementation. e.g. today if you wrote code like:
#nullable enable
using System;
class Program
{
static void Main(string[] args)
{
HashCode c = default;
c.Add(null, StringComparer.Ordinal);
}
}
it will blow up. Seems like this line should instead be:
Add(value is null ? 0 :
comparer != null ? comparer.GetHashCode(value) :
value.GetHashCode());
or something like that. #Resolved
@@ -307,7 +307,7 @@ public static bool TryGetArray<T>(ReadOnlyMemory<T> memory, out ArraySegment<T> | |||
{ | |||
TManager? localManager; // Use register for null comparison rather than byref | |||
manager = localManager = memory.GetObjectStartLength(out _, out _) as TManager; | |||
return localManager != null; | |||
return localManager != null; // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's wrong here? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The compiler doesn't know that manager == localManager
, so it looks like we may be returning with a null
manager when this method returns true
.
We could fix that by doing return manager != null;
, but that defeats the purpose indicated by above comment ("Use register for null comparison rather than byref"). #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, so this TODO was about improving the compiler :)
For the purposes of this change, just suppress the warning with whatever syntax is least obtrusive. #Resolved
@@ -329,7 +329,7 @@ public static bool TryGetArray<T>(ReadOnlyMemory<T> memory, out ArraySegment<T> | |||
|
|||
Debug.Assert(length >= 0); | |||
|
|||
if (localManager == null) | |||
if (localManager == null) // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's wrong here? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto #Closed
using System.Runtime.CompilerServices; | ||
#nullable enable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: mind moving this above the usings? #Resolved
@@ -221,7 +221,7 @@ public ImmutableHashSet<T> Remove(T item) | |||
[Pure] | |||
public bool TryGetValue(T equalValue, out T actualValue) | |||
{ | |||
int hashCode = _equalityComparer.GetHashCode(equalValue); | |||
int hashCode = equalValue is object ? _equalityComparer.GetHashCode(equalValue) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We've generally not used "is object" as a null check.
int hashCode = equalValue is object ? _equalityComparer.GetHashCode(equalValue) : 0; | |
int hashCode = equalValue != null ? _equalityComparer.GetHashCode(equalValue) : 0; | |
``` #Resolved |
@@ -639,7 +639,7 @@ private static bool IsSupersetOf(IEnumerable<T> other, MutationInput origin) | |||
private static MutationResult Add(T item, MutationInput origin) | |||
{ | |||
OperationResult result; | |||
int hashCode = origin.EqualityComparer.GetHashCode(item); | |||
int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; | |
int hashCode = item != null ? origin.EqualityComparer.GetHashCode(item) : 0; | |
``` #Resolved |
@@ -658,7 +658,7 @@ private static MutationResult Add(T item, MutationInput origin) | |||
private static MutationResult Remove(T item, MutationInput origin) | |||
{ | |||
var result = OperationResult.NoChangeRequired; | |||
int hashCode = origin.EqualityComparer.GetHashCode(item); | |||
int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; | |
int hashCode = item != null ? origin.EqualityComparer.GetHashCode(item) : 0; | |
``` #Resolved |
@@ -680,7 +680,7 @@ private static MutationResult Remove(T item, MutationInput origin) | |||
/// </summary> | |||
private static bool Contains(T item, MutationInput origin) | |||
{ | |||
int hashCode = origin.EqualityComparer.GetHashCode(item); | |||
int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; | |
int hashCode = item != null ? origin.EqualityComparer.GetHashCode(item) : 0; | |
``` #Resolved |
{ | ||
if (second == null) | ||
{ | ||
Debug.Assert(value is object); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How do we know this assert is valid? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The two callers get their values from ComposablePartDefinition.TryGetExports
. That method can return with
- both
singleMatch
andmultipleMatches
being null (return false;
, but thenFastAppendToListAllowNulls
won't be called because of calling pattern, copied below), - or
multipleMatches
being set and returning true, - or
singleMatch
being set and returning true.
singleMatch
corresponds to parameter value
and multipleMatches
corresponds to parameter second
here.
Calling pattern:
if (part.TryGetExports(definition, out Tuple<ComposablePartDefinition, ExportDefinition>? singleMatch, out IEnumerable<Tuple<ComposablePartDefinition, ExportDefinition>>? multipleMatches))
{
exports = exports.FastAppendToListAllowNulls(singleMatch, multipleMatches);
}
In reply to: 378614963 [](ancestors = 378614963)
@@ -86,7 +86,10 @@ public static int ListHashCode<T>(this ReadOnlyCollection<T> list) | |||
int h = 6551; | |||
foreach (T t in list) | |||
{ | |||
h ^= (h << 5) ^ cmp.GetHashCode(t); | |||
if (t is object) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (t is object) | |
if (t != null) | |
``` #Resolved |
{ | ||
Debug.Assert(d1 is object && d2 is object); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Debug.Assert(d1 is object && d2 is object); | |
Debug.Assert(d1 != null && d2 != null); | |
``` #Resolved |
@@ -43,7 +43,8 @@ public bool Equals(Wrapper<T> x, Wrapper<T> y) | |||
public int GetHashCode(Wrapper<T> x) | |||
{ | |||
Debug.Assert(_comparer != null); | |||
return _comparer.GetHashCode(x.Value); | |||
T value = x.Value; | |||
return value is null ? 0 : _comparer.GetHashCode(value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return value is null ? 0 : _comparer.GetHashCode(value); | |
return value != null ? 0 : _comparer.GetHashCode(value); | |
``` #Resolved |
src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/MemoryMarshal.cs
Show resolved
Hide resolved
@@ -19,7 +19,6 @@ public static void ThrowArgumentException_DeserializeWrongType(Type type, object | |||
throw new ArgumentException(SR.Format(SR.DeserializeWrongType, type, value.GetType())); | |||
} | |||
|
|||
[DoesNotReturn] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am curious, how did you catch this typo?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A new version of the compiler checks method bodies to see that they honor the nullability attributes that they declare in their API. This method has return
statements, so it was flagged ;-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!
I believe this is ready to merge/squash if I can get the required sign-off(s). Thanks |
Thanks, Julien. |
Thanks! |
TL;DR: the goal for this PR is to confirm that some upcoming compiler changes for VS 16.6 are okay.
This PR is intended for discussion but not to merge as-is :-)Tagging @stephentoub @safern for discussion.
Background:
I have been working on compiler changes which adds some warnings as a result of nullability attributes.
For instance:
[MaybeNull] string parameter
would start with a maybe-null state.[MaybeNull] override T M()
cannot overridevirtual T M()
[MaybeNullWhen(true)] string parameter
[DoesNotReturn]
indeed don't returnJared recommended that I test the change on the
runtime
repo, and it indeed produced more warnings. Here's a PR that addresses some of the issues flagged and tags some others.Issues identified so far:
GetHashCode
methods are marked as[DisallowNull]
and that's causing problemsFailFast
is marked with[DoesNotReturn]
, but it looks like it could return in some cases[Maybe/NotNullWhen(...)]
attributes seem incorrectly implemented