Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce unconditional attributes within method bodies #40789

Merged
merged 13 commits into from
Jan 18, 2020

Conversation

jcouv
Copy link
Member

@jcouv jcouv commented Jan 7, 2020

Addresses part of #36039
MaybeNullWhen and NotNullWhen are treated unconditionally to produce fewer warnings.
DoesNotReturn and DoesNotReturnIf are not yet enforced.
OHI will come later.

Fixes #40139 ([DisallowNull] T parameter has not-null initial value)
Fixes #39922

Relates to discussion in LDM on Jan 6th 2020.

@jcouv jcouv added this to the 16.5 milestone Jan 7, 2020
@jcouv jcouv self-assigned this Jan 7, 2020
@jcouv jcouv marked this pull request as ready for review January 8, 2020 01:53
@jcouv jcouv requested a review from a team as a code owner January 8, 2020 01:53
@jcouv jcouv requested a review from cston January 8, 2020 23:18
@@ -6782,11 +6827,6 @@ private Symbol VisitMemberAccess(BoundExpression node, BoundExpression receiverO
}
}

if (_expressionIsRead)
{
ReportMaybeNullFromTypeParameterValueIfNeeded(node, resultType, memberAnnotations);
Copy link
Member Author

@jcouv jcouv Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 removed empty method #Resolved

@jcouv
Copy link
Member Author

jcouv commented Jan 13, 2020

@dotnet/roslyn-compiler @cston for review. Thanks

I'll add a note to breaking changes. #Closed

@@ -3459,7 +3465,7 @@ private FlowAnalysisAnnotations GetParameterAnnotations(ParameterSymbol paramete
/// Fix a TypeWithAnnotations based on Allow/DisallowNull annotations prior to a conversion or assignment.
/// Note this does not work for nullable value types, so an additional check with <see cref="CheckDisallowedNullAssignment"/> may be required.
/// </summary>
private static TypeWithAnnotations ApplyLValueAnnotations(TypeWithAnnotations declaredType, FlowAnalysisAnnotations flowAnalysisAnnotations)
internal static TypeWithAnnotations ApplyLValueAnnotations(TypeWithAnnotations declaredType, FlowAnalysisAnnotations flowAnalysisAnnotations)
Copy link
Member

@cston cston Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internal [](start = 8, length = 8)

Is this change needed? #Closed

// [NotNull]TNullable -> X
// [NotNull]TStruct? -> X
// [NotNull]TOpen -> X
return type is null || (type.Kind == SymbolKind.TypeParameter && !type.IsReferenceType) || type.IsNullableTypeOrTypeParameter();
Copy link
Member

@cston cston Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type is null [](start = 23, length = 12)

When do we hit this, and is it necessary to report a warning in that case? #Resolved

@@ -6099,6 +6129,21 @@ static FlowAnalysisAnnotations getPropertyAnnotations(SourcePropertySymbol prope
}
}

FlowAnalysisAnnotations ToInwardAnnotations(FlowAnalysisAnnotations outwardAnnotations)
Copy link
Member

@cston cston Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

private static #Closed

@@ -5439,10 +5439,10 @@ To remove the warning, you can use /reference instead (set the Embed Interop Typ
<value>Argument cannot be used as an output for parameter due to differences in the nullability of reference types.</value>
</data>
<data name="WRN_DisallowNullAttributeForbidsMaybeNullAssignment" xml:space="preserve">
<value>A possible null value may not be assigned to a target marked with the [DisallowNull] attribute</value>
<value>A possible null value may not be assigned to a target marked with the [DisallowNull] attribute or output to a target marked with the [NotNull] attribute</value>
Copy link
Member

@cston cston Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

output to a target marked with [](start = 109, length = 30)

Is this qualifier necessary? Do we need to differentiate "assigned to" and "output to", or can we find one phrase that captures both?

How about: "A possible null value may not be used for a type marked with [NotNull] or [DisallowNull]"? #Closed

[DisallowNull] string? P1 { get; set; } = """";
[AllowNull] string P2 { get; set; } = """";
[MaybeNull] string P3 { get; set; } = """";
[NotNull] string? P4 { get; set; } = """";
Copy link
Member

@cston cston Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear we're that the attributes affect the property bodies since there are no warnings. Should the assignments be changed to = null' to at least test two of the four? #Closed

Copy link
Member

@sharwell sharwell Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 The previous test has good coverage of these, so a starting point could model after it. #Resolved


[Fact]
[WorkItem(36039, "https://github.com/dotnet/roslyn/issues/36039")]
public void NontNullOutParameterWithVariousTypes()
Copy link
Member

@cston cston Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nont [](start = 20, length = 4)

Typo #Closed

x.ToString(); // 1
}

void M2([DisallowNull]T x)
Copy link
Member

@sharwell sharwell Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Should have a test showing that x = default is allowed within a method like this. #Closed

Copy link
Member Author

@jcouv jcouv Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sharwell I disagree with this one. x = default is disallowed (ie. a warning) in absence of [DisallowNull], so should still be disallowed. #Closed

Copy link
Member

@sharwell sharwell Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense to me. Consider adding a line showing this diagnostic is reported for the case. #Resolved

Copy link
Member Author

@jcouv jcouv Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that case is already covered. See DefaultLiteralInConditional #Resolved

[DisallowNull] string? P1 { get; set; } = """";
[AllowNull] string P2 { get; set; } = """";
[MaybeNull] string P3 { get; set; } = """";
[NotNull] string? P4 { get; set; } = """";
Copy link
Member

@sharwell sharwell Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 The previous test has good coverage of these, so a starting point could model after it. #Resolved

Comment on lines 28525 to 28527
// (4,43): warning CS8607: A possible null value may not be assigned to a target marked with the [DisallowNull] attribute or output to a target marked with the [NotNull] attribute
// [AllowNull, NotNull]public TOpen P1 { get; set; } = default!;
Diagnostic(ErrorCode.WRN_DisallowNullAttributeForbidsMaybeNullAssignment, "get;").WithLocation(4, 43),
Copy link
Member

@sharwell sharwell Jan 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😟 This makes sense if you have { get; }, but I don't see why this would be a warning if the property has a setter and has [AllowNull], especially since it uses the suppression in default!. #Closed

Copy link
Member Author

@jcouv jcouv Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We analyze a body that says return backingField;. But that backing field is a TOpen and lacks attribute annotations. So we warn.
I'll take a look to see if we can refine that. #Closed

Copy link
Member Author

@jcouv jcouv Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find out that some warnings involving backing fields and other compiler-generated values were suppressed. I removed that, which not only keeps this warning on P1, but also adds a warning on P3.
@sharwell @cston Let me know what you think about those. In my opinion, those are bad usages of the attributes, and so I think it's actually desirable to warn. #Resolved

Copy link
Member

@sharwell sharwell Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both cases can be resolved (in user code) by removing the unnecessary initializing expression. If users find this confusing, we can iterate on it. #Resolved

Copy link
Member Author

@jcouv jcouv Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From discussion with @cston I'm going to restore the logic to skip analyzing bodies of auto-prop getters and setters. We'll lose some warning about misused attributes, but that's okay.

One reason is that there is a scenario where that would produce an unwanted warning: [MaybeNull, AllowNull] T { get; set; } which the user can't fix. #Resolved

Diagnostic(ErrorCode.WRN_DisallowNullAttributeForbidsMaybeNullAssignment, "get;").WithLocation(4, 32),
// (14,35): warning CS8607: A possible null value may not be assigned to a target marked with the [DisallowNull] attribute or output to a target marked with the [NotNull] attribute
// [NotNull]public TStruct? P5 { get; set; }
Diagnostic(ErrorCode.WRN_DisallowNullAttributeForbidsMaybeNullAssignment, "get;").WithLocation(14, 35)
Copy link
Member

@sharwell sharwell Jan 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 It would be good to have a test showing that the value can be assigned in a constructor to eliminate the warning (may already exist as a test). #Resolved

Copy link
Member

@sharwell sharwell Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ I didn't see a test that used the following in a constructor:

P5 = new TStruct();

Unlike the newly-added NotNull_Property_InitializedInConstructor cases, I would expect the above to not produce a warning and resolve the warning reported on the property definition in the two tests here. #Pending

@@ -3909,6 +3908,11 @@ private void CheckDisallowedNullAssignment(TypeWithState state, FlowAnalysisAnno

static bool hasNoNonNullableCounterpart(TypeSymbol type)
{
if (type is null)
Copy link
Member Author

@jcouv jcouv Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 not sure if we can hit this. Just included for safety. #Resolved

if (((annotations & FlowAnalysisAnnotations.DisallowNull) != 0) && state.Type.IsNullableTypeOrTypeParameter() && state.MayBeNull)
if (boundValueOpt is object && boundValueOpt.WasCompilerGenerated)
{
// We need to skip `return backingField;` in auto-prop getters
Copy link
Member

@cston cston Jan 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// We need to skip return backingField; in auto-prop getters [](start = 16, length = 62)

Can we simply avoid running NullableWalker on auto-property accessors? #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, but I'd prefer to contain the scope of this PR given timeline.
Also, even if we skip auto-prop accessors, we'll still need some logic to skip compiler-generated nodes because of field = initializerValue; that is injected to constructor body and analyzed.


In reply to: 367228167 [](ancestors = 367228167)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider logging a bug to skip auto-property accessors.


In reply to: 367632535 [](ancestors = 367632535,367228167)

Copy link
Member Author

@jcouv jcouv Jan 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #41017 #Resolved

{
if (((annotations & FlowAnalysisAnnotations.DisallowNull) != 0) && state.Type.IsNullableTypeOrTypeParameter() && state.MayBeNull)
if (boundValueOpt is object && boundValueOpt.WasCompilerGenerated)
Copy link
Member

@cston cston Jan 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

boundValueOpt is object && boundValueOpt.WasCompilerGenerated [](start = 16, length = 61)

Consider using ?. #Resolved

@jcouv
Copy link
Member Author

jcouv commented Jan 16, 2020

@dotnet/roslyn-compiler for a second review please.
Thanks @cston! #Resolved

@@ -3883,12 +3894,40 @@ private VisitArgumentResult VisitArgumentEvaluate(BoundExpression argument, RefK
Debug.Assert(!this.IsConditionalState);
}

private void CheckDisallowedNullAssignment(TypeWithState state, FlowAnalysisAnnotations annotations, Location location)
private void CheckDisallowedNullAssignment(TypeWithState state, FlowAnalysisAnnotations annotations, Location location, BoundExpression boundValueOpt = null)
Copy link
Contributor

@RikkiGibson RikkiGibson Jan 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider naming CheckDisallowNullAssignment to more strongly denote that this is about DisallowNullAttribute #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that "check disallowed null assignment" rings better as an action. I'll keep as-is


In reply to: 368161189 [](ancestors = 368161189)

@@ -6099,6 +6135,21 @@ static FlowAnalysisAnnotations getPropertyAnnotations(SourcePropertySymbol prope
}
}

private static FlowAnalysisAnnotations ToInwardAnnotations(FlowAnalysisAnnotations outwardAnnotations)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ToInwardAnnotations [](start = 47, length = 19)

Consider naming ToInboundAnnotations to resemble similar terminology used in VisitArguments

// MaybeNull and MaybeNullWhen count as MaybeNull
annotations |= FlowAnalysisAnnotations.AllowNull;
}
if ((outwardAnnotations & FlowAnalysisAnnotations.NotNull) == FlowAnalysisAnnotations.NotNull)
Copy link
Contributor

@RikkiGibson RikkiGibson Jan 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style between this and the above if feels inconsistent #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's on purpose. MaybeNull is two bits, either of which should count (see comment above).
NotNull is also two bits, but we are lenient (we ignore NotNullWhenTrue and NotNullWhenFalse on their own here).


In reply to: 368161275 [](ancestors = 368161275)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add a comment


In reply to: 368183067 [](ancestors = 368183067,368161275)

@@ -119937,12 +120682,12 @@ class Program
// (7,71): warning CS8601: Possible null reference assignment.
// static void F3<T>( [AllowNull]T x, [DisallowNull]ref T y) { y = x; } // 3
Diagnostic(ErrorCode.WRN_NullReferenceAssignment, "x").WithLocation(7, 71),
// (8,71): warning CS8601: Possible null reference assignment.
// static void F4<T>( [AllowNull]T x, [MaybeNull]ref T y) { y = x; }
Diagnostic(ErrorCode.WRN_NullReferenceAssignment, "x").WithLocation(8, 71),
// (9,71): warning CS8601: Possible null reference assignment.
// static void F5<T>( [AllowNull]T x, [NotNull]ref T y) { y = x; } // 4
Copy link
Contributor

@RikkiGibson RikkiGibson Jan 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

T x [](start = 55, length = 3)

I think this is maybe innate to the problem, but it feels odd to have a different message here depending on whether AllowNull was used. In F5, T is "definitely maybe null", but in FA, T is "possibly maybe null". #WontFix

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could have the new check (from CheckDisallowedNullAssignment) report the same diagnostic as the regular checks that VisitConversion can report. But since we have more context/information here, I figured out that some hint would be helpful, when possible. But it is a bit inconsistent, I agree.


In reply to: 368176797 [](ancestors = 368176797)

@RikkiGibson
Copy link
Contributor

RikkiGibson commented Jan 17, 2020

static void F2<T>(   [AllowNull]T x,    [AllowNull]out T y) { y = x; } // 2

It feels like AllowNull/DisallowNull on out parameters is something that could never be what the user intended. Related to #36073 #Resolved


Refers to: src/Compilers/CSharp/Test/Semantic/Semantics/NullableReferenceTypesTests.cs:119870 in 501855b. [](commit_id = 501855b, deletion_comment = True)

@RikkiGibson
Copy link
Contributor

RikkiGibson commented Jan 17, 2020

static void F8<T>(              T x, [DisallowNull]T y) { y = x; }

It feels like if assigning maybe-null to a [NotNull]T parameter gives a warning, assigning maybe-null to a [DisallowNull]T parameter inside the method should give a warning. #WontFix


Refers to: src/Compilers/CSharp/Test/Semantic/Semantics/NullableReferenceTypesTests.cs:119847 in 501855b. [](commit_id = 501855b, deletion_comment = True)

@RikkiGibson
Copy link
Contributor

RikkiGibson commented Jan 17, 2020

static void FA<T>(              T x,      [NotNull]T y) { y = x; }

Also consider a // 1 comment here.


Refers to: src/Compilers/CSharp/Test/Semantic/Semantics/NullableReferenceTypesTests.cs:120557 in 501855b. [](commit_id = 501855b, deletion_comment = False)

@RikkiGibson
Copy link
Contributor

RikkiGibson commented Jan 17, 2020

[return: NotNull] public override T F1<T>(T t) => t;

Consider adding a // 1 comment here #Resolved


Refers to: src/Compilers/CSharp/Test/Semantic/Semantics/NullableReferenceTypesTests.cs:33885 in 501855b. [](commit_id = 501855b, deletion_comment = False)

@"using System.Diagnostics.CodeAnalysis;
public class COpen<TOpen> where TOpen : new()
{
[NotNull]public TOpen P1 { get; set; }
Copy link
Contributor

@RikkiGibson RikkiGibson Jan 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just additional coverage, or did your change affect behavior of these scenarios? It doesn't seem like enforcing unconditional attributes inside the members using the attributes affects this? #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a while, this PR include analyzing the body of the auto-prop accessors. So those tests were useful.
Since I added the tests, I'd rather keep them even though they are less useful now :-)


In reply to: 368177028 [](ancestors = 368177028)

@@ -30937,11 +31496,11 @@ public void MaybeNull_ReturnValue_02_WithNotNull()
@"using System.Diagnostics.CodeAnalysis;
class Program
{
[return: MaybeNull, NotNull] static T F1<T>(T t) => t;
[return: MaybeNull, NotNull] static T F1<T>(T t) => t; //
Copy link
Contributor

@RikkiGibson RikkiGibson Jan 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// [](start = 59, length = 2)

Was typing // -1 a bridge too far? :) #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not bring myself to do it :-P


In reply to: 368177057 [](ancestors = 368177057)

@@ -28877,6 +29417,9 @@ static void M1<T>(T t1)
}";
var comp = CreateCompilation(new[] { DisallowNullAttributeDefinition, source }, options: WithNonNullTypesTrue());
comp.VerifyDiagnostics(
// (11,12): warning CS8607: A possible null value may not be assigned to a target marked with the [DisallowNull] attribute
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Consider updating the diagnosic message in this comment. I had started to comment about how this message could be better but it turned out you already changed it for the better :)

Copy link
Contributor

@RikkiGibson RikkiGibson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@jcouv
Copy link
Member Author

jcouv commented Jan 18, 2020

static void F2<T>(   [AllowNull]T x,    [AllowNull]out T y) { y = x; } // 2

Per our discussion and spec, NotNull is definitely useful on inputs.
I can't think of a scenario where AllowNull or DisallowNull would be useful on an output. But I don't want to produce individual warnings for those situations until we've decided that we want to warn on misused attributes as a whole (there are many more cases that could warrant a warning).


In reply to: 575833532 [](ancestors = 575833532)


Refers to: src/Compilers/CSharp/Test/Semantic/Semantics/NullableReferenceTypesTests.cs:119870 in 501855b. [](commit_id = 501855b, deletion_comment = True)

@jcouv
Copy link
Member Author

jcouv commented Jan 18, 2020

FYI @cston, I took Rikki's suggestion so that void M([DisallowNull]T x) { x = null; } now produces a warning and void M([AllowNull]string x) { x = null; } no longer does.

@jcouv
Copy link
Member Author

jcouv commented Jan 18, 2020

FYI @cston, I took Rikki's suggestion so that void M([DisallowNull]T x) { x = null; } now produces a warning and void M([AllowNull]string x) { x = null; } no longer does.

Never mind, that won't work because of void M([DisallowNull] ref T x) { x = null; } scenario. I'll revert.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants