"cannot merge projects" error #1585

KristofferC · 2020-01-03T10:35:45Z

Testing the package "DisplayAs" gives the error

ERROR: can not merge projects
Stacktrace:
 [1] pkgerror(::String) at /workspace/srcdir/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:54
 [2] (::Pkg.Operations.var"#84#88"{Pkg.Operations.var"#93#94"{Bool,Cmd,Cmd,Nothing,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,Nothing,String})(::String) at /workspace/srcdir/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:1346
 [3] mktempdir(::Pkg.Operations.var"#84#88"{Pkg.Operations.var"#93#94"{Bool,Cmd,Cmd,Nothing,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,Nothing,String}, ::String; prefix::String) at ./file.jl:673
 [4] mktempdir(::Function, ::String) at ./file.jl:671 (repeats 2 times)

This type of error message seems similar to an assertion error so is it a bug in Pkg? If not, the error message should probably be expanded to say what is actually wrong and how to fix it. And this package passes on 1.3 so what's up with that?

The text was updated successfully, but these errors were encountered:

00vareladavid · 2020-01-04T08:16:50Z

Yeah that is a pretty bad error message. It happens when the active manifest and the test manifest both have fixed nodes that disagree (for example the active manifest and the test manifest both are tracking the same UUID by path but are tracking it a different paths). Pkg has to preserve fixed nodes but it doesn't know which one to keep.

tkf · 2020-01-06T03:16:29Z

Most of my projects check-in test/Manifest.toml that is a full environment that includes the main environment simply via dev mode installation with relative path (path = ".."). This is possible as I am using a custom testing entry point https://github.com/tkf/Run.jl and not Pkg.test in CI. This has been working quite well especially for supporting Juila < 1.2 and using unregistered helper packages.

It would be great if I can keep using this workflow. Can Pkg just use test/Manifest.toml if all the required packages are in test/Manifest.toml?

fredrikekre · 2020-01-08T10:06:21Z

Pkg has to preserve fixed nodes but it doesn't know which one to keep.

IMO Manifest.toml should take precedent over test/Manifest.toml.

00vareladavid · 2020-01-08T18:15:54Z

Yeah, that would make sense. Probably better than erroring out

tkf · 2020-01-08T19:17:24Z

IMO Manifest.toml should take precedent over test/Manifest.toml.

Why not the other way around? Shouldn't project /test be a superset of project / (in terms of importable projects)? If so, shouldn't /test/Manifest.toml be used as-is, ignoring /Manifest.toml? The former is guaranteed to contain compatible dependencies while the latter may not, due to test-only dependencies.

fredrikekre · 2020-01-08T22:06:22Z

I mean, what we really want is #1233.

tkf · 2020-01-08T23:08:11Z

I'm not so sure. I commented my concern in #1233 (comment)

tkf · 2020-01-16T07:20:21Z

FYI, I moved test/Manifest.toml to test/environments/main/Manifest.toml in most of my packages. Hopefully it works well in next NewPkgEval.

laborg · 2020-01-24T15:20:38Z

Is there a workaround for this? I'm getting the error in test, where the test project and the project under test depend on a local package that is tracked with the same UUID and an absolute path.
Why not bail out with this this kind of pseudo code?

# In Operations.jl line1348 do something like
if haskey(working_manifest, uuid) && ( uuid referenced entries are really not equal)  
....

fredrikekre · 2020-01-24T15:52:10Z

Is there a workaround for this?

Don't use thefunctionality of test/Project.tomls which is pretty beta (and documented as such) 🤷‍♂

cscherrer · 2020-04-20T20:21:12Z

Could this be made more clear? There's nothing I can find that says this is beta, only that functionality may change in the future. This seems to advocate this approach as the "modern" way of doing things, without any hint that it might be buggy:

https://julialang.github.io/Pkg.jl/v1/creating-packages/#Test-specific-dependencies-in-Julia-1.2-and-above-1

rana · 2020-09-22T02:31:31Z

Looking forward to a solution 😀

Using Test-specific dependencies in Julia 1.0 and 1.1 in the mean time.

racinmat · 2021-01-04T14:49:54Z

To me this happens even when I have completely identical Manifest.toml and test/Manifest.toml files, no paths there, everything from git or package server.

I had to include them in the global Project.toml as extras, because I ran into JuliaLang/Pkg.jl#1585. This also deletes the test/Project.toml.

dhanak · 2021-05-17T10:15:29Z

To me this happens even when I have completely identical Manifest.toml and test/Manifest.toml files, no paths there, everything from git or package server.

Yes, this happens also when both the package and the test code tries to use the exact same dependency from a URL (i.e., not from a registry).

This used to work well with Julia 1.3, but it broke in 1.4, and has not been fixed since then. Pretty annoying.

CameronBieganek · 2021-06-07T23:11:30Z

Yes, this happens also when both the package and the test code tries to use the exact same dependency from a URL

Yup, I just ran into this again. My Project.toml and test/Project.toml are both tracking the same private github repo.

dpinol · 2021-07-07T15:06:24Z

I solved the issue removing the tested package from [deps] at my test/Project.toml

QuantumControlBase extends QuantumPropagators, but QuantumPropagators does not depend on QuantumControlBase except for testing... Thus the dependency should be only in test/Project.toml and docs/Project.toml, but not in the main Project.toml. This also makes it necessary to not use Pkg.test() for local development: There's just no way to dev-install a local test-dependency without that dependency being in the main Project.toml. So anything that's only a test-dependency, but not a dependency of the package itself, will break. That's actually how the circular dependency got into Project.toml in the first place. On CI, we don't have to worry about this as much: the testing will modify the main Project.toml, but we throw away the checkout after the CI run finishes, so it works. All of these problems are because Pkg.test() is has some buggy behavior if there is a test/Manifest.toml (which would be the appropriate place to record the local dev-installation of test dependencies): JuliaLang/Pkg.jl#1585 JuliaLang/Pkg.jl#1233

The "ERROR: can not merge projects" seems impossible to fix when using Packages from custom repos (my patched Gnuplot.jl). See JuliaLang/Pkg.jl#1585. Therefore, we remove the test project and have all dependencies for tests added into the top-level project.

We don't use a separate test project, because with it, we get the error "ERROR: can not merge projects", which seems impossible to fix when using packages from custom repos (my patched Gnuplot.jl). See JuliaLang/Pkg.jl#1585.

864: Move test env to integration_tests, add test/Project.toml r=charleskawczynski a=charleskawczynski I think that this should (hopefully) fix our environment issues. This PR moves our `test/` env to `integration_tests/`, which means that users will need to use `julia --project=integration_tests`. We can change `integration_tests` to a shorter name if people prefer, or just move everything from there into `driver/`-- I have no strong opinions about where these things should live. Right now, if users try to `julia --project; ] test`, they'll see the error: ``` (TurbulenceConvection) pkg> test Testing TurbulenceConvection ERROR: can not merge projects ``` This [is](JuliaLang/Pkg.jl#1714) [not](https://discourse.julialang.org/t/testing-package-where-the-test-depends-on-local-deved-package-fails-in-julia-1-4-2-with-error-can-not-merge-projects/41611) [an](JuliaLang/julia#41819) [uncommon](JuliaLang/Pkg.jl#1788) [issue](JuliaLang/Pkg.jl#1585). This PR has the implications that our unit/package tests, or rather tests that are run with `Pkg.test()`, will not be quite as reproducible as those in our integration tests with `julia --project=integration_tests test/runtests.jl`. But I think that this is okay since the only places it seems we rely on running tests this way is - in GHA CI, which we could customize to run via `julia --project=integration_tests test/runtests.jl` anyway, and - codecov, which I'm not sure how else to correctly pass the `coverage` argument And bors doesn't depend on either of these, so they'll just be a soft test. ```julia help?> Pkg.test Pkg.test(; kwargs...) ... • coverage::Bool=false: enable or disable generation of coverage statistics. ``` Co-authored-by: Charles Kawczynski <[email protected]>

kobussch · 2022-04-07T22:08:05Z

I just ran into this problem with Julia 1.6 on Windows. The suggestion of not using the project/manifest in the test folder was not satisfactory to me, simply because all my other projects are tested this way. I found an easy workaround though. It is somewhat similar to dpinol's suggestion, which did not work for me.

The problem is well described so I will not repeat all of it. To explain the workaround, I will refer to two packages, MyworkA and MyworkB, both local packages tracked only by folder path. MyworkA uses MyworkB and I also want to use MyworkB in the runtests.jl file associated with MyworkA. The workaround is to not add MyworkB to the Project.toml of the test folder. If it already there, use Pkg.remove to get rid of it. In the runtests.jl file, there will be a problematic uses clause that contains at least the following:

using Test, MyworkA, MyworkB

but this gives an error because MyworkB was removed from the test project scope. Instead, use the following:

using Test, MyworkA, MyworkA.MyworkB

Now everything works without any complaints from Julia (at least for me)

schlichtanders · 2022-11-06T21:12:03Z

I also stumbled into this because I wanted to have a nice VSCode support for working interactively within my test environment. Hence I added the parent repo via dev ... Now after having completed some interactive part, I wanted to run the tests, just to see that it is broken.

It would be really nice if it possible to have both

a clean self-descriptive test-environment
as well as Pkg.test() working at the same time

it seems like the only thing needed is that some logic is added which checks for same path names (after making them absolute) so that the error "cannot merge projects" can be circumvented.

tfiers · 2023-01-15T18:39:52Z

Until #3276 is merged and released, and if you removed your package from test/Project.toml as per the above,
you might find something like the following useful,
if you want to run individual test files without going through (MyPackage) pkg> test:
https://github.com/tfiers/PkgGraph.jl/blob/main/test/activate_standalone.jl

ERROR: can not merge projects see JuliaLang/Pkg.jl#1585

KristofferC added this to the 1.4 milestone Jan 3, 2020

00vareladavid mentioned this issue Feb 28, 2020

better merging of test project file #1707

Open

cscherrer mentioned this issue Apr 20, 2020

Clarify current fragility of test/Project.toml approach #1788

Open

KristofferC mentioned this issue Sep 1, 2020

New error: can not merge projects #1714

Closed

stevengogogo mentioned this issue Dec 14, 2020

Testenv stevengogogo/FindSteadyStates.jl#27

Merged

felixcremer mentioned this issue Jan 15, 2021

RFC: Add wrap keyword to cubefromshape JuliaDataCubes/EarthDataLab.jl#237

Closed

DilumAluthge modified the milestones: 1.4, 1.6 May 8, 2021

DrChainsaw mentioned this issue Aug 7, 2021

Incompatible test/Manifest.toml gives unhelpful error JuliaLang/julia#41819

Closed

IanButterworth modified the milestones: 1.6, 1.8 Jan 2, 2022

jondeuce mentioned this issue Feb 17, 2022

Conditional Revise support using @requires Roger-luo/FromFile.jl#36

Merged

filchristou mentioned this issue Mar 2, 2022

add reference tests based on examples in docs MakieOrg/GraphMakie.jl#22

Closed

charleskawczynski mentioned this issue Mar 5, 2022

Move test env to integration_tests, add test/Project.toml CliMA/TurbulenceConvection.jl#864

Merged

frankier mentioned this issue Sep 14, 2022

Woes with Pkg.test() JuliaPsychometricsBazaar/ComputerAdaptiveTesting.jl#52

Open

KristofferC mentioned this issue Dec 13, 2022

try improve the merging of sandbox manifest and parent manifest #3276

Merged

gdalle mentioned this issue Feb 28, 2023

Clean up tests to diagnose Frank-Wolfe linear solver error JuliaDecisionFocusedLearning/InferOpt.jl#39

Merged

jlapeyre mentioned this issue Apr 6, 2023

Fail with error "trying to activate test environment of an unnamed project" JuliaTesting/ReTestItems.jl#34

Closed

ablaom mentioned this issue May 24, 2023

Remove test/Project.toml and put back into [extras] JuliaAI/MLJBase.jl#907

Merged

alexandrebouchard added a commit to Julia-Tempering/Pigeons.jl that referenced this issue May 29, 2023

This is broken - error message:

388b632

ERROR: can not merge projects see JuliaLang/Pkg.jl#1585

IanButterworth closed this as completed in #3276 Jul 9, 2023

sethaxen mentioned this issue Sep 10, 2023

Add tests for BLAS.dot, BLAS.dotc, and BLAS.dotu EnzymeAD/Enzyme.jl#842

Merged

timholy mentioned this issue Sep 28, 2023

Extension-specific "cannot merge projects" #3636

Closed

jishnub mentioned this issue Nov 25, 2023

Aqua.jl JuliaArrays/FillArrays.jl#304

Merged

soldasim mentioned this issue Feb 23, 2024

Add ParamTests.jl dependency soldasim/BOSS.jl#17

Open

gdalle mentioned this issue Apr 16, 2024

Switch to ADTypes.jl v1.0 adrhill/SparseConnectivityTracer.jl#16

Merged

gdalle mentioned this issue Jun 17, 2024

[documentation] Explain coloring and detector kwargs in sparse backends JuliaSmoothOptimizers/ADNLPModels.jl#249

Merged

MilesCranmer added a commit to nmheim/DynamicExpressions.jl that referenced this issue Jul 26, 2024

fix issue due to JuliaLang/Pkg.jl#1585

86d2096

gdalle mentioned this issue Aug 7, 2024

Set up test env fully in CI script JuliaDiff/DifferentiationInterface.jl#404

Closed

KristianHolme mentioned this issue Nov 12, 2024

Restrict python version in CI workflow sisl/Crux.jl#17

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"cannot merge projects" error #1585

"cannot merge projects" error #1585

KristofferC commented Jan 3, 2020

00vareladavid commented Jan 4, 2020

tkf commented Jan 6, 2020 •

edited

Loading

fredrikekre commented Jan 8, 2020

00vareladavid commented Jan 8, 2020

tkf commented Jan 8, 2020

fredrikekre commented Jan 8, 2020

tkf commented Jan 8, 2020

tkf commented Jan 16, 2020

laborg commented Jan 24, 2020

fredrikekre commented Jan 24, 2020

cscherrer commented Apr 20, 2020

rana commented Sep 22, 2020 •

edited

Loading

racinmat commented Jan 4, 2021

dhanak commented May 17, 2021

CameronBieganek commented Jun 7, 2021

dpinol commented Jul 7, 2021

kobussch commented Apr 7, 2022

schlichtanders commented Nov 6, 2022

tfiers commented Jan 15, 2023 •

edited

Loading

"cannot merge projects" error #1585

"cannot merge projects" error #1585

Comments

KristofferC commented Jan 3, 2020

00vareladavid commented Jan 4, 2020

tkf commented Jan 6, 2020 • edited Loading

fredrikekre commented Jan 8, 2020

00vareladavid commented Jan 8, 2020

tkf commented Jan 8, 2020

fredrikekre commented Jan 8, 2020

tkf commented Jan 8, 2020

tkf commented Jan 16, 2020

laborg commented Jan 24, 2020

fredrikekre commented Jan 24, 2020

cscherrer commented Apr 20, 2020

rana commented Sep 22, 2020 • edited Loading

racinmat commented Jan 4, 2021

dhanak commented May 17, 2021

CameronBieganek commented Jun 7, 2021

dpinol commented Jul 7, 2021

kobussch commented Apr 7, 2022

schlichtanders commented Nov 6, 2022

tfiers commented Jan 15, 2023 • edited Loading

tkf commented Jan 6, 2020 •

edited

Loading

rana commented Sep 22, 2020 •

edited

Loading

tfiers commented Jan 15, 2023 •

edited

Loading