Package fails to extract on win-64 due to symlinks - Filesystem error on package extraction not fully forwarded to CLI.

[stellarpower]

I get the following error when trying to install a particular package on win-64:

warning  libmamba Invalid package cache, file 'C:\Users\User\MicromambaRoot\pkgs\llvm-tools-14.0.3-hf00eed6_0\Library\bin\bugpoint.exe' is missing
error    libmamba Cannot find a valid extracted directory cache for 'llvm-tools-14.0.3-hf00eed6_0.tar.bz2'
critical libmamba Package cache error.

using micromamba 0.23.2 and 0.23.3:

(Bug) PS C:\Users\User> micromamba info

                                           
            __   __ __   / /  __ 
         / / / /  ` /  `/  __ \/ __ \/ __ /
        / // / / / / / / // / / / / / / // / // /
       / .__// // //_,// // //.__/_,/
      /_/


            environment : Bug (active)
           env location : C:\Users\User\MicromambaRoot\envs\Bug
      user config files : C:\Users\User.mambarc
 populated config files :
       libmamba version : 0.23.1
     micromamba version : 0.23.2
           curl version : libcurl/7.83.0-DEV Schannel zlib/1.2.12
     libarchive version : libarchive 3.5.2 zlib/1.2.12 liblzma/5.2.5 bz2lib/1.0.8 liblz4/1.9.3 libzstd/1.5.2
       virtual packages : win=0=0
                          archspec=1=x86_64
               channels :
       base environment : C:\Users\User\MicromambaRoot
               platform : win-64

The file mentioned is in fact present in the package tarball. This particular package includes symbolic links, and trying to extract it manually (using 7-zip), I get a warning back from the OS: Can't create symbolic link, required privilege not held by client.

The inclusion of symlinks in the package may be an error here, however, diagnosing the root cause would have been much easier had this message been forwarded from the OS - "Invalid package cache" would lead me to start cleaning the caches or think that the package has not downloaded properly, and "file is missing" somewhay implies a "no such file or directory" errr, but in reality this is more to do with the extraction process beforehand.

I would like to suggest that if some file-related error occurs during the extraction process, more details should propogate their way up to the CLI, to help in reporting what sort of cause for the issue we have (permissions issues, files/directories not found, filesystem-based errors as here, ...)

The original bug with the package feedstock itself is also open here.

Cheers

[wolfv]

@stellarpower can you check that the file is there or missing? I see it in the package (just extracted it locally).
You may be able to remove the extracted package and just rerun ...

Maybe previous operation was cancelled or interrupted, which lead to the package not being fully extracted? Or the disk was full?

[wolfv]

I just tried and it worked fine on Windows. Maybe removing the extracted folder will fix it...

Then we should make sure that micromamba does that automatically in the future.

[stellarpower]

I don't think that's it. I've cleaned the full cache several times, and tried installing multiple different versions of the package. I have also rm -red that directory, and the tarball. And I have just pointed micromamba to a brand new root directory, still the same scenario. Can I get more verbose output?

[stellarpower]

Trying to extract the tarball manually results in a message about missing privileges - it appears to be down to handling of symbolic links. I cannot find a consistent answer online that works about why this is not enabled or how I can change it, but in any case, if you agree with the root cause, then this bug should probably be changed to forwarding the underlying error from the OS so that the issue causing the extraction failure is displayed properly.

[wolfv]

Thanks for testing further! Indeed, symbolic links are tricky on Windows -- they need some sort of admin privileges.

It's generally good to avoid them in packages as far as I can tell!

It might be possible to replace the symbolic link with a hard link.

[stellarpower]

I agree; I don't know what would be a suitable workaround (copying files could be a poor idea), but presumably there must be some solution (juntion points?) that will work reliably, as mamba must be linking the expanded packages into the environment somehow. Symlinks are generally the right solution, but it feels to me like it could be asking for trouble on MS. Whatever version I was using, I cannot find any way to give myself the permission explicitly, and I'd rather not run the whole root environment with admin perms.

[wolfv]

there are many solutions -- you could use a .bat file that calls the right .exe file and forward arguments.

Or you can use a hardlink instead of a symlink...

But yeah, that needs to be fixed on the package level (Isuru reopened the issue).

But we should also show the error to the user, indeed.