Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import error despite having satisfactory version of requests #59

Closed
kingkupps opened this issue Jan 9, 2019 · 9 comments
Closed

Import error despite having satisfactory version of requests #59

kingkupps opened this issue Jan 9, 2019 · 9 comments
Assignees
Labels
type: question Request for information or clarification. Not an issue.

Comments

@kingkupps
Copy link

kingkupps commented Jan 9, 2019

Using a virtualenv for installing dependencies through pip. I notice that pip installs the following versions of requests and google-resumable-media:

requests-2.21.0
google-resumable-media-0.3.2

But I'm still met with the following import error:

Traceback (most recent call last):
  File "some/proprietary/file.py", line X, in <module>
  File "/tmp/pip-install-itP2Ai/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
  File "another/proprietary/file.py", line Y, in <module>
  File "/tmp/pip-install-itP2Ai/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
  File "site-packages/google/cloud/storage/__init__.py", line 39, in <module>
  File "/tmp/pip-install-itP2Ai/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
  File "site-packages/google/cloud/storage/blob.py", line 44, in <module>
  File "/tmp/pip-install-itP2Ai/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
  File "site-packages/google/resumable_media/requests/__init__.py", line 673, in <module>
  File "site-packages/six.py", line 737, in raise_from
ImportError: ``requests >= 2.18.0`` is required by the ``google.resumable_media.requests`` subpackage.
It can be installed via
    pip install google-resumable-media[requests].

Even after adding google-resumable-media[requests] to the set of pip dependencies being installed, the same message appears.

@tseaver
Copy link
Contributor

tseaver commented Jan 9, 2019

Pip has a bug with extras which is likely affecting you here: pypa/pip#4957

@kingkupps
Copy link
Author

kingkupps commented Jan 9, 2019

Thanks for the heads up! I think this is a separate issue though since I can verify through pkg_resources.working_set that requests version 2.21.0 is being installed correctly.

Since google-resumable-media only requires >= 2.18.0 this seems like a bug.

Just for completeness I separated my pip install call into separate calls, which according to this comment should resolve the extras issue, and added google-resumable-media[requests] but I'm still seeing this same issue.

@tseaver
Copy link
Contributor

tseaver commented Jan 11, 2019

I cannot reproduce in a fresh virtual environment:

$ python3.6 -m venv /tmp/grmp/59
$ /tmp/grmp/59/bin/pip install --upgrade setuptools pip
...
Successfully installed pip-18.1 setuptools-40.6.3
$ /tmp/grmp/59/bin/pip install google-resumable-media[requests]
...
Successfully installed certifi-2018.11.29 chardet-3.0.4 google-resumable-media-0.3.2 idna-2.8 requests-2.21.0 six-1.12.0 urllib3-1.24.1
$ /tmp/grmp/59/bin/pip freeze
-f file:///XXX/.pip/wheels
certifi==2018.11.29
chardet==3.0.4
google-resumable-media==0.3.2
idna==2.8
requests==2.21.0
six==1.12.0
urllib3==1.24.1
$ /tmp/grmp/59/bin/python -c "from google import resumable_media; print(resumable_media.__path__)"
['/tmp/grmp/59/lib/python3.6/site-packages/google/resumable_media']

@kingkupps
Copy link
Author

Ahh shoot my bad this looks to be a pyinstaller specific issue as requests wasn't being included in the resulting binary. Issues are resolved now!

For others that might stumble across this issue, adding a hook called hooks-google_resumable_media.requests.py with the following should fix issues:

from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('requests')

@tseaver thanks for looking into this!

@tseaver tseaver added type: question Request for information or clarification. Not an issue. and removed question labels Jan 16, 2019
tcibinan added a commit to epam/cloud-pipeline that referenced this issue Apr 3, 2019
Add requests package hidden import in order to bypass the importing issue
described in: googleapis/google-resumable-media-python#59
tcibinan added a commit to epam/cloud-pipeline that referenced this issue Apr 3, 2019
Add pyinstaller hook in order to bypass the importing issue
described in: googleapis/google-resumable-media-python#59
@Kryptonh4x0r
Copy link

I'm facing the same problem and i also tried to add the hook in the hooks folder as said by you and also copied your code. if u found the solution please help me ! its still giving me the same error please help me

@shahanaz873
Copy link

shahanaz873 commented Aug 26, 2019

I am facing the same issue when I am running the exe, Please help me on how to avoid this issue, this is my code

def list_blobs(bucket_name):
"""Lists all the blobs in the bucket."""
storage_client = storage.Client()
blobs = storage_client.list_blobs(bucket_name)
for blob in blobs:
    print(blob.name)


def download_blob(bucket_name, source_blob_name, destination_file_name):
"""Downloads a blob from the bucket."""
unzip_loc = "C:\\Users\\Downloads\\Install\\"
storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name)
blob = bucket.blob(source_blob_name)
blob.download_to_filename(destination_file_name)
print('Blob {} downloaded to {}.'.format(
    source_blob_name,
    destination_file_name))

ResumableMediaError

@randolfo75
Copy link

Same problem here, I already included the hook as said and it didn't solve the problem.

@Kryptonh4x0r
Copy link

This is how i fixed the error :

Made a hook(python file) named it

hook-google.resumable_media.requests

and the content of it is as follow :

from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('requests')

Then after just past this file to the hooks folder inside pyinstaller where all the other hooks are.

Hope it fixed the error for you.

@randolfo75
Copy link

Thank you very much @Kryptonh4x0r for trying to help, the file name I was using indeed was wrong, but nevertheless the problem stills. Notice that in my log the hook is processed but I am still receiving the error when run the exe.

(cap) D:\dev\code\deepstockwebcapture>pyinstaller --distpath D:\dev\install capture.spec --workpath D:\dev\install\build
62 INFO: PyInstaller: 3.5
63 INFO: Python: 3.7.4
63 INFO: Platform: Windows-10-10.0.18362-SP0
66 INFO: UPX is not available.
109 INFO: Extending PYTHONPATH with paths
['D:\dev\code\deepstockwebcapture']
111 INFO: checking Analysis
111 INFO: Building Analysis because Analysis-00.toc is non existent
112 INFO: Initializing module dependency graph...
118 INFO: Initializing module graph hooks...
167 INFO: Analyzing base_library.zip ...
3304 INFO: Analyzing hidden import 'requests'
3465 INFO: Processing pre-safe import module hook urllib3.packages.six.moves
4862 INFO: running Analysis Analysis-00.toc
4866 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
required by c:\users\rando\anaconda3\envs\cap\python.exe
5260 INFO: Caching module hooks...
5266 INFO: Analyzing capture.py
5773 INFO: Processing pre-find module path hook distutils
6832 INFO: Processing pre-find module path hook site
6834 INFO: site: retargeting to fake-dir 'c:\users\rando\anaconda3\envs\cap\lib\site-packages\PyInstaller\fake-modules'
7832 INFO: Processing pre-safe import module hook setuptools.extern.six.moves
12081 INFO: Processing pre-safe import module hook six.moves
17301 INFO: Loading module hooks...
17302 INFO: Loading module hook "hook-certifi.py"...
17304 INFO: Loading module hook "hook-cv2.py"...
17354 INFO: Loading module hook "hook-distutils.py"...
17357 INFO: Loading module hook "hook-encodings.py"...
17540 INFO: Loading module hook "hook-google.api_core.py"...
17556 INFO: Loading module hook "hook-google.cloud.py"...
17613 INFO: Loading module hook "hook-google.cloud.storage.py"...
17617 INFO: Loading module hook "hook-google.resumable_media.requests.py"...
17659 INFO: Loading module hook "hook-google.resumable_media.requests.py"...
17711 INFO: Loading module hook "hook-httplib2.py"...
17789 INFO: Loading module hook "hook-lib2to3.py"...
17813 INFO: Loading module hook "hook-numpy.core.py"...
18061 INFO: MKL libraries found when importing numpy. Adding MKL to binaries
18065 INFO: Loading module hook "hook-numpy.py"...
18111 INFO: Loading module hook "hook-pandas.py"...
19367 INFO: Loading module hook "hook-pkg_resources.py"...
19825 INFO: Processing pre-safe import module hook win32com
Traceback (most recent call last):
File "", line 2, in
ModuleNotFoundError: No module named 'win32com'
19934 INFO: Processing pre-safe import module hook win32com
Traceback (most recent call last):
File "", line 2, in
ModuleNotFoundError: No module named 'win32com'
20340 INFO: Loading module hook "hook-pydoc.py"...
20341 INFO: Loading module hook "hook-pytz.py"...
20402 INFO: Loading module hook "hook-setuptools.py"...
21143 INFO: Loading module hook "hook-sqlite3.py"...
21282 INFO: Loading module hook "hook-sysconfig.py"...
21284 INFO: Loading module hook "hook-xml.dom.domreg.py"...
21286 INFO: Loading module hook "hook-xml.py"...
21402 INFO: Looking for ctypes DLLs
21446 INFO: Analyzing run-time hooks ...
21455 INFO: Including run-time hook 'pyi_rth_certifi.py'
21460 INFO: Including run-time hook 'pyi_rth_pkgres.py'
21499 INFO: Including run-time hook 'pyi_rth_multiprocessing.py'
21521 INFO: Looking for dynamic libraries
21647 WARNING: lib not found: msmpi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_msmpi_ilp64.dll
21775 WARNING: lib not found: msmpi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_msmpi_lp64.dll
22034 WARNING: lib not found: pgc14.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_pgi_thread.dll
22092 WARNING: lib not found: pgf90rtl.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_pgi_thread.dll
22151 WARNING: lib not found: pgf90.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_pgi_thread.dll
22267 WARNING: lib not found: tbb.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_tbb_thread.dll
22374 WARNING: lib not found: impi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_intelmpi_ilp64.dll
22583 WARNING: lib not found: mpich2mpi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_mpich2_ilp64.dll
22647 WARNING: lib not found: impi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_intelmpi_lp64.dll
22803 WARNING: lib not found: mpich2mpi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_mpich2_lp64.dll
23439 WARNING: lib not found: api-ms-win-downlevel-shlwapi-l1-1-0.dll dependency of C:\Users\rando\Anaconda3\envs\cap\Library\bin\opencv_videoio342.dll
23963 INFO: Looking for eggs
23963 INFO: Using Python library c:\users\rando\anaconda3\envs\cap\python37.dll
23963 INFO: Found binding redirects:
[]
23980 INFO: Warnings written to D:\dev\install\build\capture\warn-capture.txt
24225 INFO: Graph cross-reference written to D:\dev\install\build\capture\xref-capture.html
24279 INFO: Appending 'binaries' from .spec
24298 INFO: Appending 'datas' from .spec
24306 INFO: checking PYZ
24306 INFO: Building PYZ because PYZ-00.toc is non existent
24374 INFO: Building PYZ (ZlibArchive) D:\dev\install\build\capture\PYZ-00.pyz
25779 INFO: Building PYZ (ZlibArchive) D:\dev\install\build\capture\PYZ-00.pyz completed successfully.
25815 INFO: checking PKG
25841 INFO: Building PKG because PKG-00.toc is non existent
25842 INFO: Building PKG (CArchive) PKG-00.pkg
25912 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
25915 INFO: Bootloader c:\users\rando\anaconda3\envs\cap\lib\site-packages\PyInstaller\bootloader\Windows-64bit\run_d.exe
25915 INFO: checking EXE
['D:\dev\code\deepstockwebcapture']
144 INFO: checking Analysis
144 INFO: Building Analysis because Analysis-00.toc is non existent
145 INFO: Initializing module dependency graph...
150 INFO: Initializing module graph hooks...
152 INFO: Analyzing base_library.zip ...
3314 INFO: Analyzing hidden import 'requests'
3471 INFO: Processing pre-safe import module hook urllib3.packages.six.moves
4871 INFO: running Analysis Analysis-00.toc
4874 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
required by c:\users\rando\anaconda3\envs\cap\python.exe
5225 INFO: Caching module hooks...
5231 INFO: Analyzing capture.py
5732 INFO: Processing pre-find module path hook distutils
6853 INFO: Processing pre-find module path hook site
6854 INFO: site: retargeting to fake-dir 'c:\users\rando\anaconda3\envs\cap\lib\site-packages\PyInstaller\fake-modules'
7959 INFO: Processing pre-safe import module hook setuptools.extern.six.moves
12631 INFO: Processing pre-safe import module hook six.moves
17415 INFO: Loading module hooks...
17416 INFO: Loading module hook "hook-certifi.py"...
17417 INFO: Loading module hook "hook-cv2.py"...
17418 INFO: Loading module hook "hook-distutils.py"...
17420 INFO: Loading module hook "hook-encodings.py"...
17594 INFO: Loading module hook "hook-google.api_core.py"...
17596 INFO: Loading module hook "hook-google.cloud.py"...
17661 INFO: Loading module hook "hook-google.cloud.storage.py"...
17674 INFO: Loading module hook "hook-httplib2.py"...
17714 INFO: Loading module hook "hook-lib2to3.py"...
17719 INFO: Loading module hook "hook-numpy.core.py"...
17938 INFO: MKL libraries found when importing numpy. Adding MKL to binaries
17942 INFO: Loading module hook "hook-numpy.py"...
17943 INFO: Loading module hook "hook-pandas.py"...
19287 INFO: Loading module hook "hook-pkg_resources.py"...
19686 INFO: Processing pre-safe import module hook win32com
Traceback (most recent call last):
File "", line 2, in
ModuleNotFoundError: No module named 'win32com'
19771 INFO: Processing pre-safe import module hook win32com
Traceback (most recent call last):
File "", line 2, in
ModuleNotFoundError: No module named 'win32com'
20123 INFO: Loading module hook "hook-pydoc.py"...
20125 INFO: Loading module hook "hook-pytz.py"...
20187 INFO: Loading module hook "hook-setuptools.py"...
20826 INFO: Loading module hook "hook-sqlite3.py"...
21015 INFO: Loading module hook "hook-sysconfig.py"...
21017 INFO: Loading module hook "hook-xml.dom.domreg.py"...
21075 INFO: Loading module hook "hook-xml.py"...
21077 INFO: Loading module hook "hook-google.resumable_media.requests.py"...
21208 INFO: Looking for ctypes DLLs
21306 INFO: Analyzing run-time hooks ...
21340 INFO: Including run-time hook 'pyi_rth_certifi.py'
21404 INFO: Including run-time hook 'pyi_rth_pkgres.py'
21472 INFO: Including run-time hook 'pyi_rth_multiprocessing.py'
21565 INFO: Looking for dynamic libraries
21707 WARNING: lib not found: mpich2mpi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_mpich2_ilp64.dll
21908 WARNING: lib not found: impi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_intelmpi_ilp64.dll
22011 WARNING: lib not found: mpich2mpi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_mpich2_lp64.dll
22124 WARNING: lib not found: msmpi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_msmpi_ilp64.dll
22432 WARNING: lib not found: tbb.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_tbb_thread.dll
22790 WARNING: lib not found: pgc14.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_pgi_thread.dll
22862 WARNING: lib not found: pgf90rtl.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_pgi_thread.dll
22921 WARNING: lib not found: pgf90.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_pgi_thread.dll
23043 WARNING: lib not found: msmpi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_msmpi_lp64.dll
23101 WARNING: lib not found: impi.dll dependency of c:\users\rando\anaconda3\envs\cap\Library\bin\mkl_blacs_intelmpi_lp64.dll
23846 WARNING: lib not found: api-ms-win-downlevel-shlwapi-l1-1-0.dll dependency of C:\Users\rando\Anaconda3\envs\cap\Library\bin\opencv_videoio342.dll
24163 INFO: Looking for eggs
24163 INFO: Using Python library c:\users\rando\anaconda3\envs\cap\python37.dll
24164 INFO: Found binding redirects:
[]
24180 INFO: Warnings written to D:\dev\install\build\capture\warn-capture.txt
24483 INFO: Graph cross-reference written to D:\dev\install\build\capture\xref-capture.html
24539 INFO: Appending 'binaries' from .spec
24550 INFO: Appending 'datas' from .spec
24557 INFO: checking PYZ
24559 INFO: Building PYZ because PYZ-00.toc is non existent
24559 INFO: Building PYZ (ZlibArchive) D:\dev\install\build\capture\PYZ-00.pyz
26086 INFO: Building PYZ (ZlibArchive) D:\dev\install\build\capture\PYZ-00.pyz completed successfully.
26127 INFO: checking PKG
26151 INFO: Building PKG because PKG-00.toc is non existent
26152 INFO: Building PKG (CArchive) PKG-00.pkg
26233 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
26238 INFO: Bootloader c:\users\rando\anaconda3\envs\cap\lib\site-packages\PyInstaller\bootloader\Windows-64bit\run_d.exe
26239 INFO: checking EXE
26239 INFO: Building EXE because EXE-00.toc is non existent
26240 INFO: Building EXE from EXE-00.toc
26240 INFO: Appending archive to EXE D:\dev\install\build\capture\Capture Service.exe
26258 INFO: Building EXE from EXE-00.toc completed successfully.
26328 INFO: checking COLLECT
26328 INFO: Building COLLECT because COLLECT-00.toc is non existent
26329 INFO: Building COLLECT COLLECT-00.toc
39131 INFO: Building COLLECT COLLECT-00.toc completed successfully.

Running exe:

[17132] LOADER: Running capture.py
Traceback (most recent call last):
File "capture.py", line 16, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\rando\anaconda3\envs\cap\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.dict)
File "site-packages\google\cloud\storage_init
.py", line 39, in
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "c:\users\rando\anaconda3\envs\cap\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.dict)
File "site-packages\google\cloud\storage\blob.py", line 44, in
File "", line 983, in find_and_load
File "", line 967, in find_and_load_unlocked
File "", line 677, in load_unlocked
File "c:\users\rando\anaconda3\envs\cap\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.dict)
File "site-packages\google\resumable_media\requests_init
.py", line 663, in
File "site-packages\pkg_resources_init
.py", line 900, in require
File "site-packages\pkg_resources_init.py", line 786, in resolve
pkg_resources.DistributionNotFound: The 'google-resumable-media[requests]' distribution was not found and is required by the application
[17132] Failed to execute script capture
[17132] LOADER: OK.
[17132] LOADER: Cleaning up Python interpreter.

evgeniimv pushed a commit to evgeniimv/cloud-pipeline that referenced this issue Oct 14, 2019
Add requests package hidden import in order to bypass the importing issue
described in: googleapis/google-resumable-media-python#59
evgeniimv pushed a commit to evgeniimv/cloud-pipeline that referenced this issue Oct 14, 2019
Add pyinstaller hook in order to bypass the importing issue
described in: googleapis/google-resumable-media-python#59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

5 participants