Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LoggingGenerator #51064

Merged
merged 123 commits into from
Apr 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
71cf1eb
Rejigger project names
Nov 17, 2020
afd85ab
Update namespaces
Nov 17, 2020
fe21a95
Nuke the temporary .Attributes namespace
Nov 17, 2020
b6281b5
Add error checking to prevent multiple logging messages from using th…
Nov 17, 2020
ca3ead4
Use ISyntaxReceiver to be more IDE friendly
jaredpar Nov 17, 2020
470b160
Finish implementation of exception support
Nov 17, 2020
515f029
Make the generated type have the same access modifiers as the input i…
Nov 17, 2020
6affdd8
Enforce that logging methods must return void
Nov 17, 2020
7f15797
Cleanup how semantic models are handled
Nov 18, 2020
8b3cdaa
Prevent generic interfaces or methods.
Nov 18, 2020
1c0e756
Improve efficiency of the generated code for log messages without tem…
Nov 18, 2020
5893090
Various cleanup items
Nov 18, 2020
56943dd
Few small changes (#3)
davidfowl Nov 18, 2020
056e81c
Minor refactoring
Nov 18, 2020
b72ec2d
More refactoring
Nov 18, 2020
4ab2acc
Support partial methods (#4)
davidfowl Nov 19, 2020
0718293
Optimize some code gen.
Nov 19, 2020
d065c90
Simplify the model. You can now only annotate partial methods
Nov 19, 2020
d99cec0
Improve code generation by statically defining delegate types
Nov 20, 2020
b31bca8
Simplify generated code
Nov 20, 2020
1a4e336
Revamp code gen to shrink jitted size
Nov 21, 2020
128f5d2
Bunch of improvements.
Nov 22, 2020
3740b68
Enable localization of error messages
Nov 22, 2020
e275551
Substantially reduce the size of the generated code
Nov 22, 2020
fc8ce48
Add unit tests for Microsoft.Extensions.Logging.Attributes
Nov 22, 2020
951f5cc
Renamed the *Attributes assembly to *Extras
Nov 22, 2020
bf36548
Remove an allocation that snuck in during the last batch of optimizat…
Nov 22, 2020
b773ce8
Add a few more tests
Nov 22, 2020
6b25016
Use proper code to get fully-qualified type names
Nov 22, 2020
1eb4e9f
Add support for message strings containing linefeeds or quotes
Nov 22, 2020
bb45336
Add support for logging messages containing carriage returns.
Nov 22, 2020
e979d5f
Use static analysis consistently
Nov 22, 2020
c459d32
More tests, and a fix for string formatting of large cardinality log …
Nov 22, 2020
14530de
Add code coverage for logging generator error paths
Nov 22, 2020
d0ebfb2
Readme update
Nov 22, 2020
6354e8f
Add support for early termination via the cancellation token
Nov 23, 2020
8ece920
Improvements in the code generator's performance
Nov 23, 2020
0167f30
Produce an error when [LoggerMessage] is applied to a non-partial met…
Nov 23, 2020
9915bbd
Add support for generic parameters for logging methods
Nov 23, 2020
43f671c
Add support for 'protected internal' and 'protected private' logging …
Nov 23, 2020
66310a1
Minor cleanup
Nov 23, 2020
58709eb
Improved code gen for log levels, and add level unit tests
Nov 23, 2020
11b5ef1
Add support for ToString in the log state
Nov 23, 2020
da8099b
Introduce an analyzer.
Nov 25, 2020
dd39eba
Combine the fixer into the analyzer assembly.
Nov 25, 2020
a93b5c0
Introduce analyzer and fixer functionality
Nov 27, 2020
17d3a5b
More stuff.
Nov 28, 2020
561dbf3
More tests, and ensuing bug fixes.
Nov 28, 2020
8f94d10
Tests and fixes
Nov 29, 2020
34d2237
Finish support for nullable logging args, and extension logging methods
Nov 30, 2020
6fabebc
Add license file and a bit of info in the README
Nov 30, 2020
438f2bb
Improve generator's perf to minimize the impact on the IDE
Dec 1, 2020
3fd97a1
Refactor the main generator class to allow unit testing of the produc…
Dec 1, 2020
9c6c08b
More test coverage, more resulting fixes
Dec 1, 2020
71fd60a
Make the source generator robust to malformed source
Dec 2, 2020
4424c8b
More testing and tweaking
Dec 2, 2020
bb33d1c
More coverage, more fixes
Dec 3, 2020
c89bb35
Relocate using statements
Dec 9, 2020
755f3ff
Cleanup analyzer story
Dec 11, 2020
225b63d
Refactoring and cleanup
Dec 26, 2020
139e5bb
Get coverage to 100%
Dec 29, 2020
e8db38a
A few improvements
Mar 8, 2021
5e3d279
Variety of improvements.
Mar 14, 2021
00cac9b
Various improvements.
Mar 15, 2021
9e150d9
Add support for logging methods as instance methods
Mar 15, 2021
83eb36e
Minor cleanup
Mar 15, 2021
cc44192
Various improvements
Mar 16, 2021
b611bbd
Rename LogStateHolder to LogValues
Mar 16, 2021
60fef62
Improve test coverage
Mar 16, 2021
79dc2d4
Use the Invariant culture when formatting log messages
Mar 17, 2021
055cf3f
Improve the error message around id reuse
Mar 27, 2021
abb5010
A few improvements
Mar 27, 2021
9087427
Emit GenerateCodeAttribute instead of CompilerGeneratedAttribute
Mar 30, 2021
58b3369
Minor cleanup items
Mar 31, 2021
82f1611
Added preliminary support for an alternate code gen strategy
Mar 31, 2021
bb63032
Overhaul of the emitter.
Apr 9, 2021
5f210fc
Address more feedback and bugs
Apr 10, 2021
fe54581
Two nit fixes
maryamariyan Apr 10, 2021
407a9e8
Merge pull request #2 from maryamariyan/nit-fixes
maryamariyan Apr 10, 2021
b1f5364
API Review feedback: LoggerMessageAttribute (#4)
maryamariyan Apr 10, 2021
3afda5f
Merge branch 'logging-generator' of /Users/maryam/CodeHub/LoggingGene…
maryamariyan Apr 10, 2021
b7b2928
- Update header license
maryamariyan Apr 10, 2021
c41c463
Minor renames, also added project to solution file
maryamariyan Apr 10, 2021
3ef2083
- Add LoggerMessageAttribute - ref/src
maryamariyan Apr 11, 2021
a7eba74
- Add localization and packaging
maryamariyan Apr 11, 2021
ba71c47
- ActiveIssue for Mono
maryamariyan Apr 11, 2021
4cb1466
Complete one TODO: use available SYSLIBXXXX values
maryamariyan Apr 11, 2021
daf1a7a
Fix path to pick up root folder
maryamariyan Apr 11, 2021
3a97c73
- Add versions to versions.props
maryamariyan Apr 13, 2021
946f764
- Switch ActiveIssue from closed to dupe one.
maryamariyan Apr 13, 2021
80840e6
- Move RoslynTestUtils to common folder
maryamariyan Apr 13, 2021
5637b69
Remove suppressed warnings
maryamariyan Apr 13, 2021
0cc8316
Added ActiveIssue for roslyn issue
maryamariyan Apr 13, 2021
1297f11
Rename DiagDescriptors to DiagnosticDescriptors
maryamariyan Apr 13, 2021
2add7ce
- Remove LangVersions from csproj
maryamariyan Apr 13, 2021
13631d2
- Remove Moq
maryamariyan Apr 13, 2021
d9970f7
- Skip on browser Cant load Microsoft.CodeAnalysis
maryamariyan Apr 13, 2021
d809dad
- Add CLSCompliant on gen csproj
maryamariyan Apr 13, 2021
dd51c53
- Add record of SYSLIBXXXX in the md file
maryamariyan Apr 14, 2021
70f24ef
- Rename md file to list-of-diagnostics.md
maryamariyan Apr 14, 2021
afb43d7
- Add a set of baseline files to test against
maryamariyan Apr 14, 2021
2d41c6d
- Fix new line issue when comparing baselines
maryamariyan Apr 14, 2021
7e68b1c
Add Generator and Resources to the Logging package
ericstj Apr 14, 2021
9200131
Refactor part 1
maryamariyan Apr 14, 2021
6a71553
Fixup whitespaces after removing try/finally
maryamariyan Apr 14, 2021
7547f38
Refactor part 2
maryamariyan Apr 15, 2021
6eab93d
- Remove duplication in the resource string values
maryamariyan Apr 15, 2021
cc0bb07
Replace operations using StringComparison.Ordinal
maryamariyan Apr 15, 2021
e56d447
Include generator project to src.proj
maryamariyan Apr 15, 2021
b970fdd
Upgrade generator package versions
maryamariyan Apr 15, 2021
bd93ccf
- Can't have a log level set twice
maryamariyan Apr 15, 2021
a6ecab3
Add three more diagnostics
maryamariyan Apr 15, 2021
35a584c
Use wildcard, include gen projects under NonNetCoreApp
maryamariyan Apr 15, 2021
0ec16c4
Renames only
maryamariyan Apr 15, 2021
e81015d
- Use other Define overload using skipEnabledCheck
maryamariyan Apr 15, 2021
1b7a158
Revert "Upgrade generator package versions"
maryamariyan Apr 15, 2021
d94e42f
Block range for logging in MD file
maryamariyan Apr 15, 2021
a2f18d9
enable nullable on test project
maryamariyan Apr 16, 2021
e110684
Add missing "global::" to types
maryamariyan Apr 16, 2021
ba9b77f
Fix up triple slash comments for LoggerMessageAttribute
maryamariyan Apr 16, 2021
843c0f3
lock strings
maryamariyan Apr 16, 2021
65b1f51
- Feedback on RoslynTestUtils
maryamariyan Apr 16, 2021
c0c22ce
- Enable nullable on csproj
maryamariyan Apr 16, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,32 @@ Currently the identifiers `SYSLIB0001` through `SYSLIB0999` are carved out for o
| __`SYSLIB0011`__ | `BinaryFormatter` serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for recommended alternatives. |
| __`SYSLIB0012`__ | Assembly.CodeBase and Assembly.EscapedCodeBase are only included for .NET Framework compatibility. Use Assembly.Location instead. |
| __`SYSLIB0013`__ | Uri.EscapeUriString can corrupt the Uri string in some cases. Consider using Uri.EscapeDataString for query string components instead. |
| __`SYSLIB0015`__ | DisablePrivateReflectionAttribute has no effect in .NET 6.0+ applications. |
| __`SYSLIB0015`__ | DisablePrivateReflectionAttribute has no effect in .NET 6.0+ applications. |

### Analyzer warnings (`SYSLIB1001` - `SYSLIB1999`)
maryamariyan marked this conversation as resolved.
Show resolved Hide resolved
| Diagnostic ID | Description |
| :---------------- | :---------- |
| __`SYSLIB1001`__ | Logging method names cannot start with _ |
| __`SYSLIB1002`__ | Don't include log level parameters as templates in the logging message |
| __`SYSLIB1003`__ | InvalidLoggingMethodParameterNameTitle |
| __`SYSLIB1004`__ | Logging class cannot be in nested types |
| __`SYSLIB1005`__ | Could not find a required type definition |
| __`SYSLIB1006`__ | Multiple logging methods cannot use the same event id within a class |
| __`SYSLIB1007`__ | Logging methods must return void |
| __`SYSLIB1008`__ | One of the arguments to a logging method must implement the Microsoft.Extensions.Logging.ILogger interface |
| __`SYSLIB1009`__ | Logging methods must be static |
| __`SYSLIB1010`__ | Logging methods must be partial |
| __`SYSLIB1011`__ | Logging methods cannot be generic |
| __`SYSLIB1012`__ | Redundant qualifier in logging message |
| __`SYSLIB1013`__ | Don't include exception parameters as templates in the logging message |
| __`SYSLIB1014`__ | Logging template has no corresponding method argument |
| __`SYSLIB1015`__ | Argument is not referenced from the logging message |
| __`SYSLIB1016`__ | Logging methods cannot have a body |
| __`SYSLIB1017`__ | A LogLevel value must be supplied in the LoggerMessage attribute or as a parameter to the logging method |
| __`SYSLIB1018`__ | Don't include logger parameters as templates in the logging message |
| __`SYSLIB1019`__ | Couldn't find a field of type Microsoft.Extensions.Logging.ILogger |
| __`SYSLIB1020`__ | Found multiple fields of type Microsoft.Extensions.Logging.ILogger |
| __`SYSLIB1021`__ | Can't have the same template with different casing |
| __`SYSLIB1022`__ | Can't have malformed format strings (like dangling {, etc) |
| __`SYSLIB1023`__ | Generating more than 6 arguments is not supported |
| __`SYSLIB1029`__ | *_Blocked range `SYSLIB1024`-`SYSLIB1029` for logging._* |
6 changes: 6 additions & 0 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,11 @@
<!-- Targeting packs are only patched in extreme cases. -->
<ProjectServicingConfiguration Include="Microsoft.NETCore.App.Ref" PatchVersion="0" />
</ItemGroup>
<PropertyGroup>
<!-- For source generator support we need to target a pinned version in order to be able to run on older versions of Roslyn -->
<MicrosoftCodeAnalysisCSharpWorkspacesVersion>3.8.0</MicrosoftCodeAnalysisCSharpWorkspacesVersion>
<MicrosoftCodeAnalysisVersion>3.8.0</MicrosoftCodeAnalysisVersion>
</PropertyGroup>
<PropertyGroup>
<MicrosoftCodeAnalysisNetAnalyzersVersion>6.0.0-preview3.21168.1</MicrosoftCodeAnalysisNetAnalyzersVersion>
<MicrosoftCodeAnalysisCSharpCodeStyleVersion>3.9.0-5.final</MicrosoftCodeAnalysisCSharpCodeStyleVersion>
Expand Down Expand Up @@ -154,6 +159,7 @@
<XUnitRunnerVisualStudioVersion>2.4.2</XUnitRunnerVisualStudioVersion>
<CoverletCollectorVersion>1.3.0</CoverletCollectorVersion>
<NewtonsoftJsonVersion>12.0.3</NewtonsoftJsonVersion>
<SQLitePCLRawbundle_greenVersion>2.0.4</SQLitePCLRawbundle_greenVersion>
<MoqVersion>4.12.0</MoqVersion>
<FsCheckVersion>2.14.3</FsCheckVersion>
<!-- Docs -->
Expand Down
293 changes: 293 additions & 0 deletions src/libraries/Common/tests/SourceGenerators/RoslynTestUtils.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,293 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CodeActions;
using Microsoft.CodeAnalysis.CodeFixes;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.Diagnostics;
using Microsoft.CodeAnalysis.Text;
using Xunit;

namespace SourceGenerators.Tests
{
internal static class RoslynTestUtils
maryamariyan marked this conversation as resolved.
Show resolved Hide resolved
{
/// <summary>
/// Creates a canonical Roslyn project for testing.
/// </summary>
/// <param name="references">Assembly references to include in the project.</param>
/// <param name="includeBaseReferences">Whether to include references to the BCL assemblies.</param>
public static Project CreateTestProject(IEnumerable<Assembly>? references, bool includeBaseReferences = true)
{
string corelib = Assembly.GetAssembly(typeof(object))!.Location;
string runtimeDir = Path.GetDirectoryName(corelib)!;

var refs = new List<MetadataReference>();
if (includeBaseReferences)
{
refs.Add(MetadataReference.CreateFromFile(corelib));
refs.Add(MetadataReference.CreateFromFile(Path.Combine(runtimeDir, "netstandard.dll")));
refs.Add(MetadataReference.CreateFromFile(Path.Combine(runtimeDir, "System.Runtime.dll")));
}

if (references != null)
{
foreach (var r in references)
{
refs.Add(MetadataReference.CreateFromFile(r.Location));
}
}

return new AdhocWorkspace()
.AddSolution(SolutionInfo.Create(SolutionId.CreateNewId(), VersionStamp.Create()))
.AddProject("Test", "test.dll", "C#")
.WithMetadataReferences(refs)
.WithCompilationOptions(new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary).WithNullableContextOptions(NullableContextOptions.Enable));
}

public static Task CommitChanges(this Project proj, params string[] ignorables)
{
Assert.True(proj.Solution.Workspace.TryApplyChanges(proj.Solution));
return AssertNoDiagnostic(proj, ignorables);
}

public static async Task AssertNoDiagnostic(this Project proj, params string[] ignorables)
{
foreach (Document doc in proj.Documents)
{
SemanticModel? sm = await doc.GetSemanticModelAsync(CancellationToken.None).ConfigureAwait(false);
Assert.NotNull(sm);

foreach (Diagnostic d in sm!.GetDiagnostics())
{
bool ignore = ignorables.Any(ig => d.Id == ig);

Assert.True(ignore, d.ToString());
}
}
}

private static Project WithDocuments(this Project project, IEnumerable<string> sources, IEnumerable<string>? sourceNames = null)
{
int count = 0;
Project result = project;
if (sourceNames != null)
{
List<string> names = sourceNames.ToList();
foreach (string s in sources)
result = result.WithDocument(names[count++], s);
}
else
{
foreach (string s in sources)
result = result.WithDocument($"src-{count++}.cs", s);
}

return result;
}

public static Project WithDocument(this Project proj, string name, string text)
{
return proj.AddDocument(name, text).Project;
}

public static Document FindDocument(this Project proj, string name)
{
foreach (Document doc in proj.Documents)
{
if (doc.Name == name)
{
return doc;
}
}

throw new FileNotFoundException(name);
}

/// <summary>
/// Looks for /*N+*/ and /*-N*/ markers in a string and creates a TextSpan containing the enclosed text.
/// </summary>
public static TextSpan MakeSpan(string text, int spanNum)
{
int start = text.IndexOf($"/*{spanNum}+*/", StringComparison.Ordinal);
if (start < 0)
{
throw new ArgumentOutOfRangeException(nameof(spanNum));
}

start += 6;

int end = text.IndexOf($"/*-{spanNum}*/", StringComparison.Ordinal);
if (end < 0)
{
throw new ArgumentOutOfRangeException(nameof(spanNum));
}

end -= 1;

return new TextSpan(start, end - start);
}

/// <summary>
/// Runs a Roslyn generator over a set of source files.
/// </summary>
public static async Task<(ImmutableArray<Diagnostic>, ImmutableArray<GeneratedSourceResult>)> RunGenerator(
ISourceGenerator generator,
IEnumerable<Assembly>? references,
IEnumerable<string> sources,
AnalyzerConfigOptionsProvider? optionsProvider = null,
bool includeBaseReferences = true,
CancellationToken cancellationToken = default)
{
Project proj = CreateTestProject(references, includeBaseReferences);

proj = proj.WithDocuments(sources);

Assert.True(proj.Solution.Workspace.TryApplyChanges(proj.Solution));

Compilation? comp = await proj!.GetCompilationAsync(CancellationToken.None).ConfigureAwait(false);

CSharpGeneratorDriver cgd = CSharpGeneratorDriver.Create(new[] { generator }, optionsProvider: optionsProvider);
GeneratorDriver gd = cgd.RunGenerators(comp!, cancellationToken);

GeneratorDriverRunResult r = gd.GetRunResult();
return (r.Results[0].Diagnostics, r.Results[0].GeneratedSources);
}

/// <summary>
/// Runs a Roslyn analyzer over a set of source files.
/// </summary>
public static async Task<IList<Diagnostic>> RunAnalyzer(
DiagnosticAnalyzer analyzer,
IEnumerable<Assembly> references,
IEnumerable<string> sources)
{
Project proj = CreateTestProject(references);

proj = proj.WithDocuments(sources);

await proj.CommitChanges().ConfigureAwait(false);

ImmutableArray<DiagnosticAnalyzer> analyzers = ImmutableArray.Create(analyzer);

Compilation? comp = await proj!.GetCompilationAsync().ConfigureAwait(false);
return await comp!.WithAnalyzers(analyzers).GetAllDiagnosticsAsync().ConfigureAwait(false);
}

/// <summary>
/// Runs a Roslyn analyzer and fixer.
/// </summary>
public static async Task<IList<string>> RunAnalyzerAndFixer(
DiagnosticAnalyzer analyzer,
CodeFixProvider fixer,
IEnumerable<Assembly> references,
IEnumerable<string> sources,
IEnumerable<string>? sourceNames = null,
string? defaultNamespace = null,
string? extraFile = null)
{
Project proj = CreateTestProject(references);

int count = sources.Count();
proj = proj.WithDocuments(sources, sourceNames);

if (defaultNamespace != null)
{
proj = proj.WithDefaultNamespace(defaultNamespace);
}

await proj.CommitChanges().ConfigureAwait(false);

ImmutableArray<DiagnosticAnalyzer> analyzers = ImmutableArray.Create(analyzer);

while (true)
{
Compilation? comp = await proj!.GetCompilationAsync().ConfigureAwait(false);
ImmutableArray<Diagnostic> diags = await comp!.WithAnalyzers(analyzers).GetAllDiagnosticsAsync().ConfigureAwait(false);
if (diags.IsEmpty)
{
// no more diagnostics reported by the analyzers
break;
}

var actions = new List<CodeAction>();
foreach (Diagnostic d in diags)
{
Document? doc = proj.GetDocument(d.Location.SourceTree);

CodeFixContext context = new CodeFixContext(doc!, d, (action, _) => actions.Add(action), CancellationToken.None);
await fixer.RegisterCodeFixesAsync(context).ConfigureAwait(false);
}

if (actions.Count == 0)
{
// nothing to fix
break;
}

ImmutableArray<CodeActionOperation> operations = await actions[0].GetOperationsAsync(CancellationToken.None).ConfigureAwait(false);
Solution solution = operations.OfType<ApplyChangesOperation>().Single().ChangedSolution;
Project? changedProj = solution.GetProject(proj.Id);
if (changedProj != proj)
{
proj = await RecreateProjectDocumentsAsync(changedProj!).ConfigureAwait(false);
}
}

var results = new List<string>();

if (sourceNames != null)
{
List<string> l = sourceNames.ToList();
for (int i = 0; i < count; i++)
{
SourceText s = await proj.FindDocument(l[i]).GetTextAsync().ConfigureAwait(false);
results.Add(s.ToString().Replace("\r\n", "\n", StringComparison.Ordinal));
}
}
else
{
for (int i = 0; i < count; i++)
{
SourceText s = await proj.FindDocument($"src-{i}.cs").GetTextAsync().ConfigureAwait(false);
results.Add(s.ToString().Replace("\r\n", "\n", StringComparison.Ordinal));
}
}

if (extraFile != null)
{
SourceText s = await proj.FindDocument(extraFile).GetTextAsync().ConfigureAwait(false);
results.Add(s.ToString().Replace("\r\n", "\n", StringComparison.Ordinal));
}

return results;
}

private static async Task<Project> RecreateProjectDocumentsAsync(Project project)
{
foreach (DocumentId documentId in project.DocumentIds)
{
Document? document = project.GetDocument(documentId);
document = await RecreateDocumentAsync(document!).ConfigureAwait(false);
project = document.Project;
}

return project;
}

private static async Task<Document> RecreateDocumentAsync(Document document)
{
SourceText newText = await document.GetTextAsync().ConfigureAwait(false);
return document.WithText(SourceText.From(newText.ToString(), newText.Encoding, newText.ChecksumAlgorithm));
}
}
}
Loading