`dotnet test` does not work for projects with custom output path #445

TheRealPiotrP · 2017-02-06T05:47:24Z

Moved from https://github.com/dotnet/cli/issues/5582 on behalf of @alecor191

Steps to reproduce

Use VS 2017 with .csproj and create a .NET Core test project using XUnit
Make sure to add XUnit runner package to project:

<PackageReference Include="xunit" Version="2.2.0-beta5-build3474" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0-beta5-build1225" />

Configure the .csproj to use a custom output path:

<BaseIntermediateOutputPath>..\..\artifacts\obj\$(MSBuildProjectName)\$(Configuration)\$(ProcessorArchitecture)\</BaseIntermediateOutputPath>
<OutputPath>..\..\artifacts\test\$(MSBuildProjectName)\$(Configuration)\$(ProcessorArchitecture)\</OutputPath>

Run the XUnit tests from VS2017 IDE
Run the XUnit tests using dotnet test

Expected behavior

Both VS2017 IDE and dotnet test should execute the XUnit tests in the test project.

Actual behavior

VS2017 IDE is able to execute the tests successfully. However, dotnet test fails with the following warning:

Warning: The path '[...]\artifacts\test\<test_project_name>\Debug\net461' specified in the 'TestAdapterPath' does not contain any test adapters, provide a valid path and try again.

Additional info

I noticed the following:

My guess is that the execution with dotnet test fails as I can't find xunit.runner.visualstudio assemblies in the output path (xunit.runner.visualstudio.testadapter.dll is missing).
If I remove the custom output path settings (i.e. remove both <BaseIntermediateOutputPath> and <OutputPath> from the .csproj file, then dotnet test succeeds. In fact, I see in the default bin directory the files from xunit.runner.visualstudio package (xunit.runner.visualstudio.testadapter.dll).
Note that other NuGet package assemblies are properly copied to the custom output directory. I assume those are the dependencies that are also referenced in code; whereas the XUnit runner/testadapter assemblies are not referenced anywhere in the solution.

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-rc3-004530)

Product Information:
Version: 1.0.0-rc3-004530
Commit SHA-1 hash: 0de3338607

Runtime Environment:
OS Name: Windows
OS Version: 10.0.14393
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\1.0.0-rc3-004530

The text was updated successfully, but these errors were encountered:

TheRealPiotrP · 2017-02-06T05:50:38Z

@livarcocc and I hit this last week in the https://github.com/dotnet/sdk when trying to port their tests to dotnet test. This should be an easy fix, looking for places where $(OutputPath) and $(IntermediateOutputPath)'s values are being assumed and using the property instead. It may even be a nice issue to mark as up for grabs in case @alecor191 wants to take a shot at the fix :)

TheRealPiotrP · 2017-02-08T09:31:27Z

This is actually not a test SDK issue. @alecor191 is actually hitting dotnet/sdk#802. @livarcocc and I hit the same thing. That issue causes the targets included in the test SDK nuget packages to not get picked up, and thereby the Main method to not get generated [amongst other issues].

I think this is something that the in-box test components may be able to check for and provide some diagnosability, though I'm not sure if that's necessary once dotnet/sdk#802 is fixed. I'll leave it to the team to decide once they have a chance to think through the issue.

StefH · 2017-03-20T12:45:28Z

I've the same issue when using NUnit (without any custom output path setting).

Error message is like:

C:\xxx\SimMetrics.Net\tests\SimMetrics.Net.Tests>dotnet test
Build started, please wait...
Build completed.

Test run for C:\xxx\SimMetrics.Net\tests\SimMetrics.Net.Tests\bin\Debug\net45\SimMetrics.Net.Tests.dll(.NETFramework,Version=v4.5)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

The path 'C:\xxx\SimMetrics.Net\tests\SimMetrics.Net.Tests\bin\Debug\net45' specified in the 'TestAdapterPath' does not contain any test adapters, provide a valid path and try again.

Starting test execution, please wait...
No test discoverer is registered to perform discovery of test cases. Register a test discoverer and try again.

The csproj looks like:

<Project Sdk="Microsoft.NET.Sdk">

  <ItemGroup Condition=" '$(TargetFramework)' == 'net45' ">
    <ProjectReference Include="..\..\src\SimMetrics.Net\SimMetrics.Net.csproj" />
  </ItemGroup>

  <ItemGroup>
    <Reference Include="System" />
    <Reference Include="Microsoft.CSharp" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
    <PackageReference Include="NFluent" Version="2.0.0-alpha" />
    <PackageReference Include="NUnit" Version="3.6.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.7.0" />
  </ItemGroup>

</Project>

jaymedavis · 2017-03-29T23:36:27Z

@StefH I have the exact same problem. Did you figure this out?

codito · 2017-03-30T02:29:46Z

@StefH @jaymedavis does test project output path contain any *.TestAdapter.dll assemblies? e.g. C:\xxx\SimMetrics.Net\tests\SimMetrics.Net.Tests\bin\Debug\net45

TheAngryByrd · 2017-03-31T14:12:50Z

I'm also experiencing the same issue with xunit on osx.

The path '/xxx/Chessie.Hopac/tests/Chessie.Hopac.Tests/bin/Release/net45' specified in the 'TestAdapterPath' does not contain any test adapters, provide a valid path and try again.

fsproj:

<Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>netcoreapp1.1;net45</TargetFrameworks>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="**/*.fs" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="FSharp.Core" Version="4.1.*" />
    <PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
    <PackageReference Include="xunit" Version="2.2.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
    <ProjectReference Include="../../src/Chessie.Hopac/Chessie.Hopac.fsproj" />
    <DotNetCliToolReference Include="Microsoft.DotNet.Watcher.Tools" Version="1.0.0" />
  </ItemGroup>
</Project>

jaymedavis · 2017-03-31T15:02:30Z

I am using xunit as well. I believe the issue is it's looking at the x86 version of the sdk. I only installed the 64bit version originally. When I installed the x86 sdk, that took care of it.

I did a lot of things working on this problem yesterday, so let me know if this solves it 👍

codito · 2017-03-31T17:34:51Z

I think NUnit test projects are probably hitting nunit/nunit3-vs-adapter#296. Please try this workaround:

<PropertyGroup>
  <DebugType Condition="'$(TargetFramework)' != '' AND '$(TargetFramework)' != 'netcoreapp1.0'">Full</DebugType>
</PropertyGroup>

@TheAngryByrd XUnit tests seem to have a requirement on net452 (from https://xunit.github.io/docs/getting-started-desktop.html). Can you please try with net452?

TheAngryByrd · 2017-03-31T18:35:09Z

The path '/xxx/Chessie.Hopac/tests/Chessie.Hopac.Tests/bin/Debug/net452' specified in the 'TestAdapterPath' does not contain any test adapters, provide a valid path and try again.

<Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>netcoreapp1.1;net452</TargetFrameworks>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="**/*.fs" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="FSharp.Core" Version="4.1.*" />
    <PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
    <PackageReference Include="xunit" Version="2.2.0" />
    <PackageReference Include="xunit.core" Version="2.2.0" />
    <PackageReference Include="xunit.extensibility.execution" Version="2.2.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
    <ProjectReference Include="../../src/Chessie.Hopac/Chessie.Hopac.fsproj" />
    <DotNetCliToolReference Include="Microsoft.DotNet.Watcher.Tools" Version="1.0.0" />
  </ItemGroup>
</Project>

With some debug flags turned on:

Test run for xxx/Chessie.Hopac/tests/Chessie.Hopac.Tests/bin/Debug/net452/Chessie.Hopac.Tests.dll(.NETFramework,Version=v4.5.2)
VSTest: Starting vstest.console...
VSTest: Arguments: dotnet exec "/usr/local/share/dotnet/sdk/1.0.1/vstest.console.dll" --framework:".NETFramework,Version=v4.5.2" "xxx/Chessie.Hopac/tests/Chessie.Hopac.Tests/bin/Debug/net452/Chessie.Hopac.Tests.dll" --testAdapterPath:"xxx/Chessie.Hopac/tests/Chessie.Hopac.Tests/bin/Debug/net452"
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

The path 'xxx/Chessie.Hopac/tests/Chessie.Hopac.Tests/bin/Debug/net452' specified in the 'TestAdapterPath' does not contain any test adapters, provide a valid path and try again.

And the output of the build

bin/Debug/net452/
├── Chessie.Hopac.Tests.dll
├── Chessie.Hopac.dll
├── Chessie.Hopac.pdb
├── Chessie.dll
├── FSharp.Core.dll
├── Hopac.Core.dll
├── Hopac.Platform.dll
├── Hopac.dll
├── System.ValueTuple.dll
├── xunit.abstractions.dll
├── xunit.assert.dll
├── xunit.core.dll
├── xunit.execution.desktop.dll
├── xunit.runner.reporters.net452.dll
├── xunit.runner.utility.net35.dll
└── xunit.runner.visualstudio.testadapter.dll

jaymedavis · 2017-03-31T18:42:14Z

@TheAngryByrd I believe your issue on osx is net452 doesn't exist there. You should only target netcoreapp1.1, as that's compatible with your mac

TheAngryByrd · 2017-03-31T18:51:50Z

Figured it's some mono issue. I can get it to build with mono with this https://github.com/dotnet/netcorecli-fsc/wiki/.NET-Core-SDK-1.0.1#using-net-framework-as-targets-framework-the-osxunix-build-fails just fine, but can't run the tests.

TheAngryByrd · 2017-03-31T19:43:18Z

For sake of completion. I also get this error:

Unhandled Exception: System.InvalidOperationException: StandardError has not been redirected.
   at System.Diagnostics.Process.get_StandardError()
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.<>c__DisplayClass9_0.<SetupChannel>b__0(Process process)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Helpers.ProcessHelper.<>c__DisplayClass0_0.<LaunchProcess>b__0(Object sender, EventArgs args)
   at System.Diagnostics.Process.OnExited()
   at System.Diagnostics.Process.RaiseOnExited()
   at System.Diagnostics.Process.CompletionCallback(Object context, Boolean wasSignaled)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(Object state, Boolean timedOut)
Abort trap: 6

I'd be willing to look into making this work for mono. Any pointers you have would be appreciated. cc: @nosami @mrward

codito · 2017-04-01T01:31:41Z

@TheAngryByrd the adapters are correctly getting copied to output directory. This error is coming because vstest.console.dll is trying to read stderr of testhost.*.exe process (it would have launched this process to actually run the tests). More diagnostic information should be in log*.txt files generated with dotnet test -d:log.txt.

Possibly testhost.*.exe process is not getting launched in mono (since vstest.console just invokes testhost.exe without mono prefix, since it is not aware of mono runtime)? We can confirm it from log.txt file.

Created #679 to track support running tests on mono.

Update: added observations on ubuntu with mono in #679 (comment).

codito · 2017-07-11T08:07:30Z

Closing this issue since mono support is tracked in #679.

michael-freidgeim-webjet · 2017-07-11T13:37:05Z

@codito, the error is not mono specific, why did you close it due to mono support?

TheAngryByrd · 2017-07-11T14:36:49Z

Yeah I shouldn't have hijacked this thread, similar error message different cause.

codito · 2017-07-12T06:26:45Z

@michael-freidgeim-webjet my bad, reopening it. Thanks for correcting me.

Faizan2304 · 2017-07-12T08:57:59Z

@alecor191 The issue is with msbuild and sdk and here is the solution of your problem dotnet/sdk#980 (comment)

You have to update your csproj similar like

<Project >
  <PropertyGroup>
	<BaseIntermediateOutputPath>..\..\artifacts\obj\$(MSBuildProjectName)\$(Configuration)\$(ProcessorArchitecture)\</BaseIntermediateOutputPath>
    <OutputPath>..\..\artifacts\test\$(MSBuildProjectName)\$(Configuration)\$(ProcessorArchitecture)\</OutputPath>
  </PropertyGroup>

  <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />

  <PropertyGroup>
    <TargetFramework>netcoreapp1.1</TargetFramework>
  </PropertyGroup>
  
  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
    <PackageReference Include="xunit" Version="2.2.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
  </ItemGroup>
  
  <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
</Project>

Faizan2304 · 2017-07-12T09:00:12Z

@StefH the issue is with the Nunit nuget package NUnit3TestAdapter (Version="3.7.0"). I tried locally and it has been fixed with NUnit3TestAdapter (Version="3.8.0-alpha1")

Faizan2304 · 2017-07-12T09:01:32Z

@michael-freidgeim-webjet Please do let me know if your issue is not getting resolve with above two comment.

michael-freidgeim-webjet · 2017-07-12T12:27:30Z

For my MSTest project (.NETFramework,Version=v4.6.1)
dotnet test
caused

Warning: The path <test_project_name>\bin\Debug' specified in the 'TestAdapterPath' does not contain any test adapters, provide a valid path and try again

I was able to successfully run the tests after installing:

Install-Package MSTest.TestAdapter
Install-Package MSTest.TestFramework

Faizan2304 · 2017-07-18T06:03:31Z

@michael-freidgeim-webjet : dotnet restore should do the package installation part.

Did you tried command in this order

dotnet restore
dotnet test

MiYanni · 2018-06-30T02:23:20Z

@Faizan2304 Why was this closed? The OP reported this issue using the xunit runner. And, I've tried the latest preview, 2.4.0-beta.2.build4010, and this issue still exists. Is it an xunit issue or a general testrunner issue?

singhsarab · 2019-02-27T13:00:14Z

@MiYanni I tried the repro steps mentioned in the original issue filed . It seems to be working as expected.

Use VS 2017 with .csproj and create a XUnit .NET Core test project
Make sure to add XUnit runner package to project:

<PackageReference Include="xunit" Version="2.4.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.0" />

Configure the .csproj to use a custom output path:

<BaseIntermediateOutputPath>..\..\artifacts\obj\$(MSBuildProjectName)\$(Configuration)\$(ProcessorArchitecture)\</BaseIntermediateOutputPath>
<OutputPath>..\..\artifacts\test\$(MSBuildProjectName)\$(Configuration)\$(ProcessorArchitecture)\</OutputPath>

Run the XUnit tests from VS2017 IDE
Run the XUnit tests using dotnet test

I was able to run the tests with both VS IDE and dotnet test verb. Can you please confirm ?

VS version : 15.9.4
dotnet version : 3.0.100-preview-010184

MiYanni · 2019-02-27T19:01:55Z

@singhsarab At the time that I commented on this, I was working on our build process for Azure PowerShell related to using Azure DevOps. I had tried a lot of different ways to modify our build. Unfortunately, I didn't even put enough context for myself to understand what I was referring to.

That said, I simply modified our test projects' output folder. I ran our build and it worked appropriately. For now, I'd close this issue. If I get back to modifying our process and run into this again, I'd give more details.

singhsarab · 2019-02-28T07:13:31Z

Thanks @MiYanni I am closing this issue.

codito added exp: cli up-for-grabs labels Feb 8, 2017

angularsen mentioned this issue Jun 5, 2017

Migrate to VS2017 /MSBuild15 / .NET Standard / .NET Core angularsen/UnitsNet#261

Merged

7 tasks

TheAngryByrd mentioned this issue Jun 7, 2017

Build on linux giraffe-fsharp/Giraffe#57

Closed

codito closed this as completed Jul 11, 2017

codito reopened this Jul 12, 2017

Faizan2304 closed this as completed Jul 18, 2017

smadala reopened this Aug 6, 2018

smadala self-assigned this Aug 6, 2018

cltshivash added enhancement and removed up-for-grabs labels Sep 11, 2018

smadala assigned singhsarab Oct 22, 2018

singhsarab closed this as completed Feb 28, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`dotnet test` does not work for projects with custom output path #445

`dotnet test` does not work for projects with custom output path #445

TheRealPiotrP commented Feb 6, 2017

TheRealPiotrP commented Feb 6, 2017

TheRealPiotrP commented Feb 8, 2017

StefH commented Mar 20, 2017

jaymedavis commented Mar 29, 2017

codito commented Mar 30, 2017

TheAngryByrd commented Mar 31, 2017 •

edited

Loading

jaymedavis commented Mar 31, 2017

codito commented Mar 31, 2017

TheAngryByrd commented Mar 31, 2017 •

edited

Loading

jaymedavis commented Mar 31, 2017

TheAngryByrd commented Mar 31, 2017

TheAngryByrd commented Mar 31, 2017

codito commented Apr 1, 2017 •

edited

Loading

codito commented Jul 11, 2017

michael-freidgeim-webjet commented Jul 11, 2017

TheAngryByrd commented Jul 11, 2017

codito commented Jul 12, 2017

Faizan2304 commented Jul 12, 2017

Faizan2304 commented Jul 12, 2017

Faizan2304 commented Jul 12, 2017

michael-freidgeim-webjet commented Jul 12, 2017

Faizan2304 commented Jul 18, 2017

MiYanni commented Jun 30, 2018 •

edited

Loading

singhsarab commented Feb 27, 2019

MiYanni commented Feb 27, 2019

singhsarab commented Feb 28, 2019

dotnet test does not work for projects with custom output path #445

dotnet test does not work for projects with custom output path #445

Comments

TheRealPiotrP commented Feb 6, 2017

Moved from https://github.com/dotnet/cli/issues/5582 on behalf of @alecor191

Steps to reproduce

Expected behavior

Actual behavior

Additional info

Environment data

TheRealPiotrP commented Feb 6, 2017

TheRealPiotrP commented Feb 8, 2017

StefH commented Mar 20, 2017

jaymedavis commented Mar 29, 2017

codito commented Mar 30, 2017

TheAngryByrd commented Mar 31, 2017 • edited Loading

jaymedavis commented Mar 31, 2017

codito commented Mar 31, 2017

TheAngryByrd commented Mar 31, 2017 • edited Loading

jaymedavis commented Mar 31, 2017

TheAngryByrd commented Mar 31, 2017

TheAngryByrd commented Mar 31, 2017

codito commented Apr 1, 2017 • edited Loading

codito commented Jul 11, 2017

michael-freidgeim-webjet commented Jul 11, 2017

TheAngryByrd commented Jul 11, 2017

codito commented Jul 12, 2017

Faizan2304 commented Jul 12, 2017

Faizan2304 commented Jul 12, 2017

Faizan2304 commented Jul 12, 2017

michael-freidgeim-webjet commented Jul 12, 2017

Faizan2304 commented Jul 18, 2017

MiYanni commented Jun 30, 2018 • edited Loading

singhsarab commented Feb 27, 2019

MiYanni commented Feb 27, 2019

singhsarab commented Feb 28, 2019

`dotnet test` does not work for projects with custom output path #445

`dotnet test` does not work for projects with custom output path #445

TheAngryByrd commented Mar 31, 2017 •

edited

Loading

TheAngryByrd commented Mar 31, 2017 •

edited

Loading

codito commented Apr 1, 2017 •

edited

Loading

MiYanni commented Jun 30, 2018 •

edited

Loading