Support BeginScope

[dingmeng-xue]

Is your feature request related to a problem? Please describe.

We expect newrelic-dotnet-agent can support scope information

Feature Description

.NET logger supports BeginScope

For instance, below is my code which is part of sample WeatherForecast sample app. I only logger related 2 lines.

            using (_logger.BeginScope("Begin scope of WeatherForecast log"))
            {
                _logger.LogInformation("start to WeatherForecast info level log");
                return Enumerable.Range(1, 5).Select(index => new WeatherForecast
                {
                    Date = DateTime.Now.AddDays(index),
                    TemperatureC = Random.Shared.Next(-20, 55),
                    Summary = Summaries[Random.Shared.Next(Summaries.Length)]
                })
                .ToArray();
            }

and enable scope of logging in appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    },
    "Console": { "IncludeScopes": true }
  },
  "AllowedHosts": "*"
}

When I execute it locally, I can see console output including scope message

info: DotnetCoreWebApp.Controllers.WeatherForecastController[0]
      => SpanId:a7cea0cf48f720a0, TraceId:81b002e5aa0e630b351f47558882c621, ParentId:0000000000000000 => ConnectionId:0HN1HC881GI7M => RequestPath:/WeatherForecast RequestId:0HN1HC881GI7M:00000003 => DotnetCoreWebApp.Controllers.WeatherForecastController.Get (DotnetCoreWebApp) => Begin scope of WeatherForecast log   
      start to WeatherForecast info level log

However, I cannot see scope message on logs page in newrelic portal.

Priority

Must Have

[workato-integration]

https://new-relic.atlassian.net/browse/NR-230177

[Shilpapagare]

https://new-relic.atlassian.net/browse/NR-230177
Thanks @dingmeng-xue
We are encountering the same issue, and it's currently a blocker for migrating from Sumo Logic to New Relic. What is the timeline for resolving this @amweiss @mrickard @PhenX @jasonrclark

[nr-ahemsath]

Thank you for your bug report, @dingmeng-xue and @Shilpapagare.

Could both of you please clarify what logging provider you are using along with Microsoft.Extensions.Logging?

[Shilpapagare]

@nr-ahemsath I am using Serilog;

[dingmeng-xue]

@nr-ahemsath , Serilog or console

[nr-ahemsath]

@dingmeng-xue and @Shilpapagare: thank you for responding.

Right now, the agent handles the data passed as arguments to BeginScope(...) as context data. Context data forwarding is not enabled by default, but it can be enabled by following the instructions here: https://docs.newrelic.com/docs/logs/logs-context/net-configure-logs-context-all/#1-agent (scroll down a bit to the section that starts with "Beginning with agent version 10.4.0, the agent also supports capturing and forwarding context data (custom attributes) from log events."

When configured using Serilog as the underlying logging provider, and the context data forwarding featured enabled, the following code:

            using (_logger.BeginScope("Begin scope of Weatherforecast log"))
            {
                _logger.LogInformation("Get Weather Forecast");
                return Enumerable.Range(1, 5).Select(index => new WeatherForecast
                {
                    Date = DateTime.Now.AddDays(index),
                    TemperatureC = Random.Shared.Next(-20, 55),
                    Summary = Summaries[Random.Shared.Next(Summaries.Length)]
                })
                .ToArray();
            }

results in log data being sent to New Relic that looks like:

{
  "context.ActionId": "a9f0cc14-987b-49c0-a1ca-3f80347fec70",
  "context.ActionName": "MyApplication.Controllers.WeatherForecastController.Get (SyntheticsHeaderTest)",
  "context.ConnectionId": "0HN1J7K6CA0KP",
  "context.RequestId": "0HN1J7K6CA0KP:00000002",
  "context.RequestPath": "/weatherforecast",
  "context.Scope": "[\"Begin scope of Weatherforecast log\"]",
  "context.SourceContext": "MyApplication.Controllers.WeatherForecastController",
  "entity.guid": "<redacted>",
  "entity.guids": "<redacted>",
  "entity.name": "MyAppName",
  "hostname": "MyHostname",
  "level": "INFORMATION",
  "message": "Get Weather Forecast",
  "newrelic.logPattern": "Get Weather Forecast",
  "newrelic.source": "logs.APM",
  "span.id": "7eebbad0881730d6",
  "timestamp": 1708561648879,
  "trace.id": "c9ccc1f02354d9c5c7ee61e0c0ab5e7c"
}

The context.Scope property can be added to the Logs view as an additional column, so you would see it without having to drill down into every single log event individually:

Unfortunately, when using the built-in console logger for Microsoft.Extensions.Logging, there does appear to be a bug in the agent where the scope data is not being picked up correctly. I will be creating that as a separate issue and will reference it here. As far as a timeline for a fix, we can't say anything until we've done some initial investigation as to the root cause.

[nr-ahemsath]

@dingmeng-xue @Shilpapagare Good news, the bug that was preventing context data capture from working when using the built-in console logger with Microsoft.Extensions.Logging has been fixed (thank you @lowell-trimble !!) so this issue should be resolved now. Please let us know if you're still having an issue.