[BUG] [AzureMonitorExporter] Null Reference Exceptions at exporting logs

[pocki]

Library name and version

Azure.Monitor.OpenTelemetry.Exporter 1.0.0-beta.7

Describe the bug

Null Reference Exceptions somewhere in exporting logs when running ASP.NET Core 7 web app.

OpenTelemetry-AzureMonitor-Exporter - EventId: [2], EventName: [WriteError], Message: [FailedToExport - System.NullReferenceException: Object reference not set to an instance of an object.
   at Azure.Monitor.OpenTelemetry.Exporter.Internals.LogsHelper.ExtractProperties(String& message, IDictionary`2 properties, IReadOnlyCollection`1 stateDictionary)
   at Azure.Monitor.OpenTelemetry.Exporter.Internals.LogsHelper.GetMessageAndSetProperties(LogRecord logRecord, IDictionary`2 properties)
   at Azure.Monitor.OpenTelemetry.Exporter.Models.MessageData..ctor(Int32 version, LogRecord logRecord)
   at Azure.Monitor.OpenTelemetry.Exporter.Internals.LogsHelper.OtelToAzureMonitorLogs(Batch`1 batchLogRecord, AzureMonitorResource resource, String instrumentationKey)
   at Azure.Monitor.OpenTelemetry.Exporter.AzureMonitorLogExporter.Export(Batch`1& batch)]

Expected behavior

Running WebApp without errors in logs

Actual behavior

Null Reference Exception during running ASP.NET Core v7 web apps.

OpenTelemetry-AzureMonitor-Exporter - EventId: [2], EventName: [WriteError], Message: [FailedToExport - System.NullReferenceException: Object reference not set to an instance of an object.
   at Azure.Monitor.OpenTelemetry.Exporter.Internals.LogsHelper.ExtractProperties(String& message, IDictionary`2 properties, IReadOnlyCollection`1 stateDictionary)
   at Azure.Monitor.OpenTelemetry.Exporter.Internals.LogsHelper.GetMessageAndSetProperties(LogRecord logRecord, IDictionary`2 properties)
   at Azure.Monitor.OpenTelemetry.Exporter.Models.MessageData..ctor(Int32 version, LogRecord logRecord)
   at Azure.Monitor.OpenTelemetry.Exporter.Internals.LogsHelper.OtelToAzureMonitorLogs(Batch`1 batchLogRecord, AzureMonitorResource resource, String instrumentationKey)
   at Azure.Monitor.OpenTelemetry.Exporter.AzureMonitorLogExporter.Export(Batch`1& batch)]

Reproduction Steps

ASP.NET Core 7 web app

Adding latest Azure.Monitor.OpenTelemetry.Exporter 1.0.0-beta.7 and OpenTelemetry nuget packages:

    <PackageVersion Include="OpenTelemetry" Version="1.4.0-rc.3" />
    <PackageVersion Include="OpenTelemetry.Contrib.Preview" Version="1.0.0-beta2" />
    <PackageVersion Include="OpenTelemetry.Exporter.Console" Version="1.4.0-rc.3" />
    <PackageVersion Include="OpenTelemetry.Exporter.Jaeger" Version="1.4.0-rc.3" />
    <PackageVersion Include="OpenTelemetry.Extensions.Hosting" Version="1.4.0-rc.3" />
    <PackageVersion Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.0.0-rc9.12" />
    <PackageVersion Include="OpenTelemetry.Instrumentation.Http" Version="1.0.0-rc9.12" />
    <PackageVersion Include="OpenTelemetry.Instrumentation.Quartz" Version="1.0.0-alpha.1" />
    <PackageVersion Include="OpenTelemetry.Instrumentation.SqlClient" Version="1.0.0-rc9.12" />

Adding OpenTelemetry and AzureMonitorLogExporter to ConfigureLogging

            return Host.CreateDefaultBuilder(args)
                .ConfigureWebHostDefaults(webBuilder => webBuilder.UseStartup<Startup>())
                .ConfigureLogging((context, builder) =>
                {
                    builder.AddOpenTelemetry(options =>
                    {
                        options.IncludeScopes = true;
                        options.ParseStateValues = true;
                        options.IncludeFormattedMessage = true;
                        options.AttachLogsToActivityEvent();
                        options.AddAzureMonitorLogExporter(o => o.ConnectionString = context.Configuration.GetValue<string>("ApplicationInsights:ConnectionString"));
                    });
                });

Run Webapp

Environment

Visual Studio 17.5 Preview 6
Visual Studio 17.4.4

.NET SDK:
 Version:   7.0.200-preview.22628.1
 Commit:    76bbaffad6

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22621
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.200-preview.22628.1\

Host:
  Version:      7.0.2
  Architecture: x64
  Commit:       d037e070eb

.NET SDKs installed:
  6.0.301 [C:\Program Files\dotnet\sdk]
  6.0.302 [C:\Program Files\dotnet\sdk]
  7.0.102 [C:\Program Files\dotnet\sdk]
  7.0.200-preview.22628.1 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  arm64 [C:\Program Files\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\arm64\InstallLocation]
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[SaurabhSharma-MSFT]

@pocki We are looking into it and get back to you for any additional information.

[TimothyMothra]

Hi @pocki, I wasn't able to reproduce this. Can you share a minimal repro (zip)?

An NRE can be thrown from the Azure.Monitor.OpenTelemetry.Exporter.Internals.LogsHelper.ExtractProperties() method and I'm working to fix these instances before the next beta.
If you can upload a repro, I would love to verify a fix.
Thanks,

[pocki]

I tried to build a minimal repro but the exception don't happen there.
The Exception is only thrown in a bigger project and I have to check which part or configuration causes the problem.
It will take some time and I will come back later.

Thank you very much

[SaurabhSharma-MSFT]

@pocki Thank you. We will now proceed to close this thread.

[CSymes]

@SaurabhSharma-MSFT, I was experiencing the same issue, and managed to form a small reproducable project, attached.
log-test.zip

The bug originates (for me at least) when using structured logging with a variable that is null. e.g.

string a = null;
logger.LogInformation("A null: {NullValue}", a);

I've updated to the new beta.8 release of the library that released this morning, and it does indeed look like it addresses the issue. Thanks @TimothyMothra!