PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.11/site-packages/licensedcode/data/cache/license_index'

[eclipseo]

Hello,

OK so now that I have made some packages and installed them systemwide, I have a new issue:

$ scancode -h
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.11/site-packages/licensedcode/data/cache'

Same with extractcode.

I've looked through the existing issue, and supposedly that issue was fixed in v3. I suppose I could sudo it but this is not user friendly.

[eclipseo]

So we have /usr/lib/python3.11/site-packages/scancode_config.py

if _SCANCODE_DEV_MODE:
    # in dev mode the cache and temp files are stored exclusively under the
    # scancode_root_dir
    scancode_cache_dir = join(scancode_root_dir, '.cache')
else:
    # In other usage modes (as a CLI or as a library, regardless of how
    # installed) the cache dir goes to the home directory and is different for
    # each version
    user_home = abspath(expanduser('~'))
    __env_cache_dir = os.getenv('SCANCODE_CACHE')
    std_scancode_cache_dir = join(user_home, '.cache', 'scancode-tk', __version__)
    scancode_cache_dir = (__env_cache_dir or std_scancode_cache_dir)

which supposes that I am in dev mode?

[eclipseo]

works now. idk why. Sorry for the noise.

[eclipseo]

Yeah now this doesn't work afterward either:

scancode -clpieu --classify --consolidate --summary --json-pp output5.json musescore-4.1.1-1.fc38.src.rpm-extract/
Setup plugins...
Removing temporary files...done.
Usage: scancode [OPTIONS] <OUTPUT FORMAT OPTION(s)> <input>...
Try the 'scancode --help' option for help on options and arguments.

Error: Invalid value: ERROR: failed to setup plugin: scan:licenses:
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/scancode/cli.py", line 741, in run_scan
    plugin.setup(**requested_options)
  File "/usr/lib/python3.11/site-packages/licensedcode/plugin_license.py", line 149, in setup
    populate_cache()
  File "/usr/lib/python3.11/site-packages/licensedcode/cache.py", line 419, in populate_cache
    _LICENSE_CACHE = LicenseCache.load_or_build(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/licensedcode/cache.py", line 102, in load_or_build
    create_dir(idx_cache_dir)
  File "/usr/lib/python3.11/site-packages/commoncode/fileutils.py", line 79, in create_dir
    os.makedirs(location)
  File "<frozen os>", line 225, in makedirs
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.11/site-packages/licensedcode/data/cache/license_index'

In /usr/lib/python3.11/site-packages/scancode_config.py:

# we pre-build the index and bundle this with the the deployed release
# therefore we use package data
# .... but we accept this to be overriden with and env variable
std_license_cache_dir = join(scancode_src_dir, 'licensedcode', 'data', 'cache')
__env_license_cache_dir = os.getenv('SCANCODE_LICENSE_INDEX_CACHE')
licensedcode_cache_dir = (__env_license_cache_dir or std_license_cache_dir)

_create_dir(licensedcode_cache_dir)
_create_dir(scancode_cache_dir)

scancode_src_dir is in /usr/lib/python3.11/site-packages which is obviously not writable.

[eclipseo]

Something like this would work:

if _SCANCODE_DEV_MODE:
    # in dev mode the cache and temp files are stored exclusively under the
    # scancode_root_dir
    scancode_cache_dir = join(scancode_root_dir, '.cache')
    std_license_cache_dir = join(scancode_src_dir, 'licensedcode', 'data', 'cache')
else:
    # In other usage modes (as a CLI or as a library, regardless of how
    # installed) the cache dir goes to the home directory and is different for
    # each version
    user_home = abspath(expanduser('~'))
    __env_cache_dir = os.getenv('SCANCODE_CACHE')
    std_scancode_cache_dir = join(user_home, '.cache', 'scancode-tk', __version__)
    scancode_cache_dir = (__env_cache_dir or std_scancode_cache_dir)
    std_license_cache_dir = join(std_scancode_cache_dir, 'licensedcode', 'data', 'cache')

# we pre-build the index and bundle this with the the deployed release
# therefore we use package data
# .... but we accept this to be overriden with and env variable
__env_license_cache_dir = os.getenv('SCANCODE_LICENSE_INDEX_CACHE')
licensedcode_cache_dir = (__env_license_cache_dir or std_license_cache_dir)

[pombredanne]

This is a bit messy and needs cleanup. We just discussed this in our weekly call.... SCANCODE_CACHE is there as an envt. variable to set where the cache should be written for now. But we need to revamp this alright. @AyanSinhaMahapatra ping

[shymega]

I've tried setting SCANCODE_CACHE on the version of Scancode packaged under NixOS to no avail - is that fix still valid, @pombredanne?

[pombredanne]

@shymega

I've tried setting SCANCODE_CACHE on the version of Scancode packaged under NixOS to no avail - is that fix still valid, @pombredanne?

It should be.... can you provide a few more details on your setup and the messages you saw?

[shymega]

@pombredanne As it turned out, through reading of the scancode_config.py module, I had to set the following [env] variables:

• SCANCODE_LICENSE_INDEX_CACHE
• SCANCODE_CACHE

The command I was using was scancode -clpieu --json-pp - ., but after setting these variables, I was then able to scan the repo I'm working with.