[Workaround] Unable to install remnux.python3-packages RECORD/METADATA file not found

[seanthegeek]

[email protected]
remnux-version: v2023.45.1

Attempted to upgrade to v2023.51.1

Update: @digitalsleuth provided a workaround

pip_|-remnux-python3-packages-olefile3_|-olefile_|-installed:
    __id__: remnux-python3-packages-olefile3
    __run_num__: 209
    __sls__: remnux.python3-packages.olefile
    changes: {}
    comment: "Failed to install packages: olefile. Error: Requirement already satisfied:\
      \ olefile in /usr/local/lib/python3.8/dist-packages (0.46)\nCollecting olefile\n\
      \  Using cached olefile-0.47-py2.py3-none-any.whl.metadata (9.7 kB)\nUsing cached\
      \ olefile-0.47-py2.py3-none-any.whl (114 kB)\nInstalling collected packages:\
      \ olefile\n  Attempting uninstall: olefile\n    Found existing installation:\
      \ olefile 0.46 WARNING: Error parsing requirements for olefile: [Errno 2] No\
      \ such file or directory: '/usr/local/lib/python3.8/dist-packages/olefile-0.46.dist-info/METADATA'\n\
      WARNING: Error parsing requirements for protobuf: [Errno 2] No such file or\
      \ directory: '/usr/local/lib/python3.8/dist-packages/protobuf-3.20.3.dist-info/METADATA'\n\
      WARNING: Error parsing requirements for typing-extensions: [Errno 2] No such\
      \ file or directory: '/usr/local/lib/python3.8/dist-packages/typing_extensions-4.8.0.dist-info/METADATA'\n\
      \    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages\n\
      ERROR: Cannot uninstall olefile 0.46, RECORD file not found. You might be able\
      \ to recover from this via: 'pip install --force-reinstall --no-deps olefile==0.46'."
    duration: 1460.999
    name: olefile
    result: false
    start_time: '11:23:05.052171'

pip_|-remnux-python3-package-virustotal-api_|-git+https://github.com/doomedraven/VirusTotalApi_|-installed:
    __id__: remnux-python3-package-virustotal-api
    __run_num__: 331
    __sls__: remnux.python3-packages.virustotal-api
    changes: {}
    comment: "Failed to install packages: git+https://github.com/doomedraven/VirusTotalApi.\
      \ Error: Collecting git+https://github.com/doomedraven/VirusTotalApi\n  Cloning\
      \ https://github.com/doomedraven/VirusTotalApi to /tmp/pip-req-build-c3w31t5c\n\
      \  Resolved https://github.com/doomedraven/VirusTotalApi to commit 86f8590d80df8e76b6ccac671f25cce3ee00fbec\n\
      \  Preparing metadata (setup.py): started\n  Preparing metadata (setup.py):\
      \ finished with status 'done'\nRequirement already satisfied: requests>=2.5.0\
      \ in /usr/local/lib/python3.8/dist-packages (from vt==4.0.0.0a4) (2.31.0)\n\
      Requirement already satisfied: python-dateutil>=1.5 in /usr/local/lib/python3.8/dist-packages\
      \ (from vt==4.0.0.0a4) (2.8.2)\nRequirement already satisfied: olefile>=0.42\
      \ in /usr/local/lib/python3.8/dist-packages (from vt==4.0.0.0a4) (0.46)\nRequirement\
      \ already satisfied: texttable in /usr/local/lib/python3.8/dist-packages (from\
      \ vt==4.0.0.0a4) (1.6.4)\nRequirement already satisfied: HTMLParser in /usr/local/lib/python3.8/dist-packages\
      \ (from vt==4.0.0.0a4) (0.0.2)\nRequirement already satisfied: six in /usr/local/lib/python3.8/dist-packages\
      \ (from vt==4.0.0.0a4) (1.16.0)   Running command git clone --filter=blob:none\
      \ --quiet https://github.com/doomedraven/VirusTotalApi /tmp/pip-req-build-c3w31t5c\n\
      ERROR: Could not install packages due to an OSError: [Errno 2] No such file\
      \ or directory: '/usr/local/lib/python3.8/dist-packages/olefile-0.46.dist-info/METADATA'"
    duration: 1424.658
    name: git+https://github.com/doomedraven/VirusTotalApi
    result: false
    start_time: '11:23:55.358699'

pip_|-remnux-python3-packages-protobuf-install_|-protobuf_|-installed:
    __id__: remnux-python3-packages-protobuf-install
    __run_num__: 382
    __sls__: remnux.python3-packages.protobuf
    changes: {}
    comment: "Failed to install packages: protobuf. Error: Requirement already satisfied:\
      \ protobuf in /usr/local/lib/python3.8/dist-packages (3.20.3)\nCollecting protobuf\n\
      \  Using cached protobuf-4.25.1-cp37-abi3-manylinux2014_x86_64.whl.metadata\
      \ (541 bytes)\nUsing cached protobuf-4.25.1-cp37-abi3-manylinux2014_x86_64.whl\
      \ (294 kB)\nInstalling collected packages: protobuf\n  Attempting uninstall:\
      \ protobuf\n    Found existing installation: protobuf 3.20.3 WARNING: Error\
      \ parsing requirements for olefile: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/olefile-0.46.dist-info/METADATA'\n\
      WARNING: Error parsing requirements for protobuf: [Errno 2] No such file or\
      \ directory: '/usr/local/lib/python3.8/dist-packages/protobuf-3.20.3.dist-info/METADATA'\n\
      WARNING: Error parsing requirements for typing-extensions: [Errno 2] No such\
      \ file or directory: '/usr/local/lib/python3.8/dist-packages/typing_extensions-4.8.0.dist-info/METADATA'\n\
      \    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages\n\
      ERROR: Cannot uninstall protobuf 3.20.3, RECORD file not found. You might be\
      \ able to recover from this via: 'pip install --force-reinstall --no-deps protobuf==3.20.3'."
    duration: 1627.751
    name: protobuf
    result: false
    start_time: '11:24:34.526058'

pip_|-typing-extensions>=4.5.0_|-typing-extensions>=4.5.0_|-installed:
    __id__: typing-extensions>=4.5.0
    __run_num__: 360
    __sls__: remnux.python3-packages.typing-extensions
    changes: {}
    comment: "Failed to install packages: typing-extensions>=4.5.0. Error: Requirement\
      \ already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.8/dist-packages\
      \ (4.8.0)\nCollecting typing-extensions>=4.5.0\n  Using cached typing_extensions-4.9.0-py3-none-any.whl.metadata\
      \ (3.0 kB)\nUsing cached typing_extensions-4.9.0-py3-none-any.whl (32 kB)\n\
      Installing collected packages: typing-extensions\n  Attempting uninstall: typing-extensions\n\
      \    Found existing installation: typing-extensions 4.8.0 WARNING: Error parsing\
      \ requirements for olefile: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/olefile-0.46.dist-info/METADATA'\n\
      WARNING: Error parsing requirements for protobuf: [Errno 2] No such file or\
      \ directory: '/usr/local/lib/python3.8/dist-packages/protobuf-3.20.3.dist-info/METADATA'\n\
      WARNING: Error parsing requirements for typing-extensions: [Errno 2] No such\
      \ file or directory: '/usr/local/lib/python3.8/dist-packages/typing_extensions-4.8.0.dist-info/METADATA'\n\
      \    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages\n\
      ERROR: Cannot uninstall typing-extensions 4.8.0, RECORD file not found. You\
      \ might be able to recover from this via: 'pip install --force-reinstall --no-deps\
      \ typing-extensions==4.8.0'."
    duration: 1494.132
    name: typing-extensions>=4.5.0
    result: false
    start_time: '11:24:23.443332'

[seanthegeek]

Updated issue name and description

[digitalsleuth]

Hi @seanthegeek , you should be able to fix this with the instructions below:

sudo touch /usr/local/lib/python3.8/dist-packages/protobuf-3.20.3.dist-info/METADATA
sudo touch /usr/local/lib/python3.8/dist-packages/protobuf-3.20.3.dist-info/RECORD
sudo touch /usr/local/lib/python3.8/dist-packages/typing_extensions-4.8.0.dist-info/METADATA
sudo touch /usr/local/lib/python3.8/dist-packages/typing_extensions-4.8.0.dist-info/RECORD
sudo touch /usr/local/lib/python3.8/dist-packages/olefile-0.46.dist-info/METADATA
sudo touch /usr/local/lib/python3.8/dist-packages/olefile-0.46.dist-info/RECORD

Once you do this, you can try the install process again and it should work. This seems to be a compatibility issue with pip, setuptools, and wheel that we've been fighting with for a while, but once past the initial issue (this one), everything should be fine in the future.

[seanthegeek]

That worked. Thanks! I've updated the issue title and will leave it open so that others can find the workaround easily.

[seanthegeek]

In case anyone is Googling I recently had to do this for the 4.3.1 release of yara-python.

sudo touch /usr/local/lib/python3.8/dist-packages/yara_python-4.3.1.dist-info/METADATA
sudo touch /usr/local/lib/python3.8/dist-packages/yara_python-4.3.1.dist-info/RECORD

[digitalsleuth]

@lennyzeltser Perhaps we can pin this issue until it's permanently resolved upstream?

[lennyzeltser]

Yes, great idea.