Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Very Minor] Concurrency Issue with Parallel Project Builds #73

Closed
Sewer56 opened this issue May 2, 2018 · 5 comments
Closed

[Very Minor] Concurrency Issue with Parallel Project Builds #73

Sewer56 opened this issue May 2, 2018 · 5 comments
Labels
bug origin
Milestone
1.7.3

Comments

@Sewer56
Copy link

Sewer56 commented May 2, 2018 •
edited
Loading

How to reproduce

Compile a Visual Studio solution containing a high amount of DllExport powered projects on fast storage and CPU.

Reproduction of this is quite difficult and rare, unless stresstesting with unreasonable amounts of projects.
If the bug is reproduced, a compilation error is thrown, although very rarely:

Image

What version and selected configuration ?

Version 1.6, latest at the time of writing.

Can you provide detailed log of your build ?

No detailed or diagnostics level (note: Now provided, see below) due to it slowing down the build process, making the issue not reproducible.

Workarounds

  • Reduce parallel builds to 1 in Tools > Options > Build & Run.
  • Simply build solution again.

Extra Notes:

The issue was first found and identified in this personal project of mine Reloaded Mod Loader
@3F 3F added the unclear label May 5, 2018
@3F
Copy link
Owner

3F commented May 5, 2018

Well, I'll try to reproduce this anyway. Thanks. Any additional information is very welcomed.

Just for checking, did you see same blocking without Fody ? I'm also seeing FodyWeavers.xml from your repo:

<Weavers>
  <Costura>
    <Unmanaged32Assemblies>
      FASM.NET
    </Unmanaged32Assemblies>
  </Costura>
</Weavers>

@3F 3F removed the unclear label May 5, 2018
@Sewer56
Copy link
Author

Sewer56 commented May 5, 2018 •
edited
Loading

No, it'd happen to be unrelated to Costura.Fody and/or Fody, the only assemblies using DllExport in my project are actually individual modifications (Reloaded-Mod-Samples directory), which (apart from the template) do not use Fody at all. For reproducing this, best bet is just compiling that solution directory alone in parallel.

I've also managed to get hold of a log in detailed level, it's slightly too big for pastebin so a direct file upload will have to do: https://my.mixtape.moe/jtctxv.txt

Here's a snippet of the output around the error (line ~8982), in this sample here specifically; 0 of the projects referenced Fody:

8>  Comparison path is "C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test".
8>Done executing task "FindUnderPath".
8>Task "FindUnderPath"
8>  Comparison path is "..\..\build\Release\Default-Mods\WPF-Demo\".
8>Done executing task "FindUnderPath".
8>Task "FindUnderPath"
8>  Comparison path is "obj\Release\".
8>Done executing task "FindUnderPath".
8>Task "RemoveDuplicates"
8>Done executing task "RemoveDuplicates".
8>Done building target "_CleanGetCurrentAndPriorFileWrites" in project "WPF-Demo.csproj".
8>Target "IncrementalClean" in file "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets" from project "C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\WPF-Demo.csproj" (target "CoreBuild" depends on it):
8>Task "FindUnderPath"
8>  Comparison path is "..\..\build\Release\Default-Mods\WPF-Demo\".
8>Done executing task "FindUnderPath".
8>Task "FindUnderPath"
8>  Comparison path is "obj\Release\".
8>Done executing task "FindUnderPath".
8>Task "Delete"
8>Done executing task "Delete".
8>Task "RemoveDuplicates"
8>Done executing task "RemoveDuplicates".
8>Task "WriteLinesToFile" skipped, due to false condition; ('@(_CleanUnfilteredPriorFileWrites)'!='@(_CleanUniqueRemainingFileWritesAfterIncrementalClean)') was evaluated as ('C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\Banner.png;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\Config.json;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\main.dll;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\main.pdb;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\WPF-Demo.csprojResolveAssemblyReference.cache;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\WPF-Demo.csproj.CoreCompileInputs.cache;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\WPF-Demo.csproj.CopyComplete;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\main.dll;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\main.pdb'!='C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\Banner.png;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\Config.json;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\main.dll;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\main.pdb;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\WPF-Demo.csprojResolveAssemblyReference.cache;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\WPF-Demo.csproj.CoreCompileInputs.cache;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\WPF-Demo.csproj.CopyComplete;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\main.dll;C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\obj\Release\main.pdb').
8>Done building target "IncrementalClean" in project "WPF-Demo.csproj".
8>Target "GetFrameworkPaths" skipped. Previously built successfully.
8>Target "DllExportMod" in file "C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\packages\DllExport.1.6.0\tools\net.r_eg.DllExport.targets" from project "C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\WPF-Demo.csproj" (target "PostBuildEvent" depends on it):
8>Using "DllExportAppDomainIsolatedTask" task from assembly "C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\packages\DllExport.1.6.0\tools\RGiesecke.DllExport.MSBuild.dll".
8>Task "DllExportAppDomainIsolatedTask"
8>  Cannot find lib.exe in 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\bin'.
8>  calling 'C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools\ildasm.exe' with /quoteallnames /unicode /nobar "/out:C:\Users\Arianrhod\AppData\Local\Temp\tmp9D33\main.il" "C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\build\Release\Default-Mods\WPF-Demo\main.dll"
8>  ildasm 'C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools\ildasm.exe' returned gracefully.
8>C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\packages\DllExport.1.6.0\tools\net.r_eg.DllExport.targets(56,5): error : The process cannot access the file 'main.il' because it is being used by another process.
8>     at System.IO.Directory.DeleteHelper(String fullPath, String userPath, Boolean recursive, Boolean throwOnTopLevelDirectoryNotFound, WIN32_FIND_DATA& data)
8>     at System.IO.Directory.Delete(String fullPath, String userPath, Boolean recursive, Boolean checkHost)
8>     at RGiesecke.DllExport.Utilities.<>c.<CreateTempDirectory>b__9_0(String dir) in D:\prg\projects\DllExport\DllExport\RGiesecke.DllExport\Utilities.cs:line 95
8>     at RGiesecke.DllExport.ValueDisposable`1.<>c__DisplayClass4_0.<.ctor>b__0() in D:\prg\projects\DllExport\DllExport\RGiesecke.DllExport\ValueDisposable.cs:line 18
8>     at RGiesecke.DllExport.GenericDisposable.Dispose(Boolean disposing) in D:\prg\projects\DllExport\DllExport\RGiesecke.DllExport\GenericDisposable.cs:line 41
8>     at RGiesecke.DllExport.GenericDisposable.Dispose() in D:\prg\projects\DllExport\DllExport\RGiesecke.DllExport\GenericDisposable.cs:line 30
8>     at RGiesecke.DllExport.DllExportWeaver.Run() in D:\prg\projects\DllExport\DllExport\RGiesecke.DllExport\DllExportWeaver.cs:line 93
8>     at RGiesecke.DllExport.MSBuild.ExportTaskImplementation`1.Execute() in D:\prg\projects\DllExport\DllExport\RGiesecke.DllExport.MSBuild\RGiesecke.DllExport.MSBuild\ExportTaskImplementation.cs:line 461
8>Done executing task "DllExportAppDomainIsolatedTask" -- FAILED.
8>Done building target "DllExportMod" in project "WPF-Demo.csproj" -- FAILED.
8>Target "_CleanGetCurrentAndPriorFileWrites" skipped. Previously built successfully.
8>Target "_CleanRecordFileWrites" in file "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets" from project "C:\Users\Arianrhod\Desktop\Github\Reloaded-Mod-Loader\Reloaded-Mod-Samples\WPF-Test\WPF-Demo.csproj" (target "CoreBuild" depends on it):
8>Task "RemoveDuplicates"
8>Done executing task "RemoveDuplicates".
8>Task "MakeDir"
8>Done executing task "MakeDir".
8>Task "WriteLinesToFile"
8>Done executing task "WriteLinesToFile".
8>Done building target "_CleanRecordFileWrites" in project "WPF-Demo.csproj".
8>
8>Build FAILED.

Edit: As seen in VS image

@3F
Copy link
Owner

3F commented May 5, 2018 via email

@Sewer56
Copy link
Author

Sewer56 commented May 5, 2018 •
edited
Loading

It also came to my mind and I thought it'd be worth mentioning:

All of the assemblies in question that were compiled again and again until I reproduced it share the same assembly name of main, I.e. they all compile to main.dll in different target directories.

This is probably a matter of DllExport using a common directory for some of its procedures. Being busy with study at the moment, I did not go through the DllExport source down the stack trace and am not entirely familiar with the inner workings at the moment to tell exactly.

As for my own project where I initially found this bug, it is essentially a glorified DLL injector, the mod loader's modifications are all set to compile to Any CPU. After compilation, I use a build event to rename the resultant 32bit and 64bit DLLs to main32 and main64 respectively - the mod loader chooses which to inject based on the target process' architecture on startup. I've come across this minor bug by luck I guess :), it was probably inevitable as I added more and more sample mods.

@3F 3F added the bug label Jul 14, 2018
@3F 3F mentioned this issue Jul 3, 2019
Closed
3F added a commit that referenced this issue Jun 9, 2020
@3F
re-implemented logic of CreateTempDirectory and processing intermed…
91f2445 
…iate files when RunIlAsm

related issue: #73
@3F
Copy link
Owner

3F commented Jun 9, 2020

Well, after 2y, I think we can try to review this issue for 1.7.3 :)

Unfortunately I can't personally reproduce this problem and I will talk about it using only stack trace info above.

I noticed that the original code uses really problematic logic for CreateTempDirectory (3rd frame from your trace) through ValueDisposable<T> wrapper that looks more like some helper for IDisposable pattern etc.

And nothing else. That's why I just re-implemented similar logic from scratch: 91f2445

I close this issue as resolved due to 2y of inactivity and finally new logic for this part.
But please feel free to open this again or create new. You can also test solution using related CI build.

@3F 3F closed this as completed Jun 9, 2020
@3F 3F added the origin label Jun 9, 2020
@3F 3F added this to the 1.7.3 milestone Jun 9, 2020
3F added a commit that referenced this issue Jun 12, 2020
@3F
Public release. .NET DllExport 1.7.3
9a4bc51 
* FIXED: Fixed syntax error at token '-' in: IL_002d: ldc.r8 -nan(ind) with "return Double.NaN"
         Through "Single + Double NaN token patching" option. Issue #158.
            ```
            .field = float32(-nan(ind)) -> 0xFFC00000
            .field = float64(-nan(ind)) -> 0xFFF8000000000000

            ldc.r8 -nan(ind) -> 00 00 00 00 00 00 F8 FF
            ldc.r4 -nan(ind) -> 00 00 C0 FF
            ```

* FIXED: NuGet packages are not visible as installed when DLLExport enabled for project.
         Issue #152.

* FIXED: Concurrency Issue with Parallel Project Builds. Issue #73.

* FIXED: Fixed the first phase of the build when restoring. Related issue #159.

* FIXED: Fixed VS NuGet PM possible error:
         "You cannot call a method on a null-valued expression."

* CHANGED: Improves package integration. PR  #161.
            - Updated logic for packages through `-pkg-link` key.
            - Removed `-force` key from the project files
              if it was pushed together with other commands for manager.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug origin
Projects
None yet
Milestone
1.7.3
Development

No branches or pull requests
2 participants
@3F @Sewer56