Publishing default ASP.Net Core Web app template to azure doesn't modify the log location in web.config, resulting in no logs being produced #540

[moozzyk]

With the project created from templates the log path is set to be .\logs\stdout. If the user turns on logging but does not modify the path or delete the attribute the path will be used as is (publish-iis does not change path as it might have been set by the user). When publishing this project for Azure the path won't be correct and no logs are written.

More details:
See: https://github.com/aspnet/Tooling-Internal/issues/540

Need to decide what we want to do.

[muratg]

@moozzyk is the change in publish-iis command or the templates?

@Eilon @DamianEdwards do we want to fix this for RC2?

cc @danroth27 @glennc

[moozzyk]

I don't think it can be done in templates (or only in templates). When the application is being created you don't know whether it's going to be published for Azure or IIS. When publish-iis is invoked we know if we are publishing for Azure or not. This is when we have a chance to react. The question is how we should know whether to react. One option is to not put stdoutPath to the web.config by default (this would requires changes in templates) at all. In this case publish-iis will set the path accordingly. Another way is just to leave the templates alone as today and have publish-iis check the value - if the stdoutPath has the default value it would be overwritten otherwise it would be not touched to not overwrite something the user set. In this case the changes are needed only in publish-iis tool.

[Tratcher]

The other alternative is to have the templates set the log path to %LOG_PATH% like we do for the app and args. Then it publish-iis could check for this default and replace it for normal or azure publish.

[DamianEdwards]

If we can make a very simple fix (aka extremely-low-risk) to the templates ASAP then I'm OK with this for RC2. Otherwise, we should revisit this in RTM.

[davidebbo]

Today, when git deploying an app to Azure, you end up with this in web.config:

<aspNetCore processPath="dotnet" arguments=".\AspNetCoreRC2.dll" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" forwardWindowsAuthToken="false" />

To enable logging (including streaming), you need to change it to:

<aspNetCore processPath="dotnet" arguments=".\AspNetCoreRC2.dll" stdoutLogEnabled="true" stdoutLogFile="%home%\LogFiles\dotnetcore" forwardWindowsAuthToken="false" />

But Azure Web App already has an existing mechanism to enable logging, and it would be good to just honor it. It works as follows:

• there is a file called settings.json in %home%\site\diagnostics
• it contains something like this: {"AzureDriveEnabled":true,"AzureDriveTraceLevel":"Error"}
• if AzureDriveEnabled is true, it logs to files in %home%\LogFiles\Application

[moozzyk]

The setting.json based logging mechanism uses .NET tracing - so it's tracing from the application. Here we are talking about IIS capturing whatever was written to standard output and dumping it to a file. Very useful for troubleshooting issues (e.g. startup) but probably not as much for application logging. Application can configure logging and decide to log elsewhere in which case these entries won't show up in stdOutLog. We will hover need to verify if the setting.json based Azure logging mechanism can be used to write application logs.

[moozzyk]

We decided to overwrite the outpath always to "?%home%\LogFiles" when publishing for Antares since the stdoutLog should be used for diagnostic purposes and the folder always exists.

[davidebbo]

%home%\LogFiles\Application might be a better choice. By default, when looking at log streams in VS, that's the only folder that gets looked at. So if you don't write there, most people will get confused (VS allows changing this, but it's an advanced setting).

[davidebbo]

@moozzyk please discuss with @glennc, as he's working on a separate logging provider which would likely conflict with this. His approach will grow better into supporting storage logging and more advanced features.

[moozzyk]

@davidebbo - as I said before - stdoutLogOutput is not for logging it is just for diagnosing (mostly startup) issues. It's especially useful to troubleshoot issues which happen even before we have a logger at all. Configuring a console logger and using stdoutLogOutput to log to a file is not a substitute for a file logger.
As for the path - we have always used %home%\LogFiles\ - feels a bit late to change this and I don't think it works out of the box. I just checked on one of my website on Azure and there is no Application subfolder in LogFiles:

> ls
D:\home\LogFiles
eventlog.xml
kudu

IIS does not create a folder for logging so you set stdoutLogEnabled and nothing is created and you are hitting this: projectkudu/kudu#2014 (comment)

[davidebbo]

I'm not sure I quite buy this argument. The fact is that by default apps are configured with:

loggerFactory.AddConsole(Configuration.GetSection("Logging"));

Such that anything they log using ILogger will go there, for the lifetime of the console app (and not just the startup sequence).

Point is that if this is enabled and so is @glennc's logger, we will end up writing the same content under D:\home\LogFiles twice. And if someone turns on log streaming at the root (which is D:\home\LogFiles), every log line will be streamed twice.

[moozzyk]

Why would applications in production log to the Console? How do you even read it when you run in Antares (let's say AspNetCoreModule does not have stdoutLog feature)?

stdoutLog is a IIS/AspNetCoreModule feature and not AspNet's and you only turn it on when you need to diagnose things - it's unstructured and you may and will get additional spew not related to your application at all (e.g. when the application starts we Console.WriteLine some things, when there is a misplaced Console.WriteLine in the app it will get it).

If you use stdoutLog only for diagnostic purposes you will get (especially when application starts) some additional content that logger can't get and you should not run your application with stdoutLog enabled otherwise (this is one of the reasons why it is disabled by default).

[davidebbo]

As long as users keep that off in Azure Web Apps and primarily use the new logger, I guess we're fine. And this will probably be the guidance, with the exception of diagnosing startup condition. If some does turn it on, then they will get duplicate stream, but hopefully this will be rare.

[moozzyk]

Agreed.

[moozzyk]

Fixed in 31a6a12

[moozzyk]

aspnet/Logging#441

[Sharpeli]

Is there any update on enabling logging to azure blob?

[guardrex]

@Sharpeli There is no built-in logging implementation. There are implementations though with Azure sinks. Serilog has DocumentDB and Table Storage sinks. Log4Net has an Azure Blob piece.

https://github.com/serilog/serilog/wiki/Provided-Sinks
http://stemarie.github.io/log4net.Azure/

[Sharpeli]

@guardrex Thanks GuardRex, looks like the log4net is incompatible with .net core. Is there any workround to make the built-in “ILoggerFactory” or "Diagnostics.Trace" can log to azure blob for asp.net core app ?

[guardrex]

@PureKrome says on Slack to check out NLog. He says they use it quite a lot with great success, and it has an Azure Blob Storage sink.

• https://github.com/nlog/nlog/wiki/Targets
• http://stackoverflow.com/questions/23518078/store-nlog-generated-logs-to-azure-blob-storage-in-seperate-columns
• http://stackoverflow.com/questions/35058029/how-to-integrate-nlog-to-write-log-to-azure-streaming-log
  https://aspnetcore.slack.com/messages/general/details/

[PureKrome]

Disclaimer:

• I've never logged to Azure Blob storage before.
• I love using nLog
• I now use Serilog with an nLog sink (with various targets as required) :)

[Eilon]

I believe that's what https://github.com/aspnet/AzureIntegration/ does - it wires up logging along with Serilog and pushes log data to tables/blobs/wherever.

[guardrex]

There isn't much in the way of guidance tho for the new package. cc/ @Rick-Anderson @danroth27 @spboyer

[EDIT] I just see it in the annoucement. https://blogs.msdn.microsoft.com/webdev/2016/10/25/announcing-asp-net-core-1-1-preview-1/

[Sharpeli]

Thanks all, I've found an simple trick for azure app ,see https://github.com/aspnet/Logging/tree/dev/src , just add the package "Microsoft.Extensions.Logging.AzureAppServices": "1.0.0-preview1-final" and update related dependencies,
add the azure diagnostics in startup.cs like this :
loggerFactory.AddAzureWebAppDiagnostics(); // for default setting.
or
loggerFactory.AddAzureWebAppDiagnostics(new AzureAppServicesDiagnosticsSettings( ...)); // add custom setting.
And enable the diagnostics log on azure, both logging on blob and file are work well. No need for any
extra configuration. :)