GitVersion JsonReader null object crash on Azure Pipelines agent in dotnet build (5.0.0-beta2 and 5.0.0-beta3-29)

[teqdruid]

I'm getting the below crash when running GitVersionTask from "dotnet publish". I only notice this on my Azure Pipelines agent when running the DotNetCoreCLI task. It works locally.

C:\Users\csifplsa\.nuget\packages\gitversiontask\5.0.0-beta3-29\build\functionality\GitVersionBuild.targets(6,5): error MSB4018: The "WriteVersionInfoToBuildLog" task failed unexpectedly. [F:\vstsbuilder\_work\28\s\HWBuildLib\HWBuildLib.csproj]
C:\Users\csifplsa\.nuget\packages\gitversiontask\5.0.0-beta3-29\build\functionality\GitVersionBuild.targets(6,5): error MSB4018: Newtonsoft.Json.JsonReaderException: Error reading JObject from JsonReader. Current JsonReader item is not an object: Null. Path '', line 1, position 4. [F:\vstsbuilder\_work\28\s\HWBuildLib\HWBuildLib.csproj]
C:\Users\csifplsa\.nuget\packages\gitversiontask\5.0.0-beta3-29\build\functionality\GitVersionBuild.targets(6,5): error MSB4018:    at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader, JsonLoadSettings settings) [F:\vstsbuilder\_work\28\s\HWBuildLib\HWBuildLib.csproj]
C:\Users\csifplsa\.nuget\packages\gitversiontask\5.0.0-beta3-29\build\functionality\GitVersionBuild.targets(6,5): error MSB4018:    at NuGetUtils.MSBuild.Exec.TaskProxy.ExecuteAsync(IBuildEngine be) [F:\vstsbuilder\_work\28\s\HWBuildLib\HWBuildLib.csproj]
C:\Users\csifplsa\.nuget\packages\gitversiontask\5.0.0-beta3-29\build\functionality\GitVersionBuild.targets(6,5): error MSB4018:    at NuGetUtils.MSBuild.Exec.TaskProxy.Execute(IBuildEngine be) [F:\vstsbuilder\_work\28\s\HWBuildLib\HWBuildLib.csproj]
C:\Users\csifplsa\.nuget\packages\gitversiontask\5.0.0-beta3-29\build\functionality\GitVersionBuild.targets(6,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [F:\vstsbuilder\_work\28\s\HWBuildLib\HWBuildLib.csproj]
C:\Users\csifplsa\.nuget\packages\gitversiontask\5.0.0-beta3-29\build\functionality\GitVersionBuild.targets(6,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [F:\vstsbuilder\_work\28\s\HWBuildLib\HWBuildLib.csproj]

I've got package references in a few different csprojs which target netstandard2.0, net465, and netcoreapp2.2. Example:

<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="CsvHelper" Version="12.1.2" />
    <PackageReference Include="GitVersionTask" Version="5.0.0-beta3-29">
      <!-- <PrivateAssets>all</PrivateAssets> -->
    </PackageReference>

This may not be related, but I also get a warning:
WARN [05/31/19 20:11:01:23] Could not determine assembly version: LibGit2Sharp.LockedFileException: the index is locked; this might be due to a concurrent or crashed process just before I see this error. Here is my pipeline task config:

- task: DotNetCoreCLI@2
  displayName: Build
  inputs:
    command: publish
    publishWebProjects: false
    projects: '*.sln'
    feedsToUse: config
    nugetConfigPath: nuget.config    # Relative to root of the repository
    arguments: '--configuration Release' # Update this to match your need

- task: DotNetCoreCLI@2
  displayName: Build x64
  inputs:
    command: publish
    publishWebProjects: false
    projects: '*.sln'
    feedsToUse: config
    nugetConfigPath: nuget.config    # Relative to root of the repository
    arguments: '--configuration Release -r win-x64' # Update this to match your nee

This seems to appear in builds deterministically, but not consistently. Some configurations of csprojs with the projectreference enabled, but not others. Some fail in the second build when the first build succeeded.

[teqdruid]

This appears to be a parallelism issue. When I switched to msbuild and limited the parallelism, this problem went away:

- task: DotNetCoreCLI@2
  displayName: Build
  inputs:
    command: custom
    custom: msbuild
    publishWebProjects: false
    projects: '*.sln'
    feedsToUse: config
    nugetConfigPath: nuget.config    # Relative to root of the repository
    arguments: '-property:Configuration=Release /restore -maxCpuCount:1' # Update this to match your need

- task: DotNetCoreCLI@2
  displayName: Build x64
  inputs:
    command: custom
    custom: msbuild
    publishWebProjects: false
    projects: '*.sln'
    feedsToUse: config
    nugetConfigPath: nuget.config    # Relative to root of the repository
    arguments: '-property:Configuration=Release;RuntimeIdentifier=win-x64 /restore -maxCpuCount:1' # Update this to match your need

Obviously not ideal since now my solution takes a lot longer to build, but it works

[dazinator]

@teqdruid I remember discovering issues with parallelism as well, I logged some details here: #1381

[teqdruid]

Duplicate of #1381. I believe the root cause of these are the same

[teqdruid]

I think this one should be merged into #1381, but I can't seem to figure out how...

[dazinator]

The problem I found was that, each csproj was being built in parallel (this used to be opt in behaviour in previous versions of msbuild and legacy .net projects, but now it appears opt out behaviour) and this means the gitversion msbuild tasks also run in parallel and they end up using libgit2sharp concurrently and that hits the exception. If libgit2sharp doesn't support concurrent usage then the solution to this would seem to be that gitversion task would have to wrap libgit2sharp usage in it's own locking mechanism - a mechanism that works across process boundaries..

[teqdruid]

So the opt-out option is only in dotnet restore, not build or publish which I'm using. I've had to resort to msbuild -maxCpuCount:1

[dazinator]

Ok thanks.
Note: we should document the restore, build, and publish parallelism workarounds until we can find a proper fix for this in the future.