analysis pipeline - no module named 'future'

[dev-rod]

Hi @shankari ,
I'm Rodrigue, Transway company, France, we test e-mission-server and e-mission-phone to detect bicycle trips
(I also have contact with Patrick Gendre and the french fabmob project)

Actually I have a problem with analysis pipeline

(emission) root@vps753733:/home/emission/web/e-mission-server-pve# python --version
Python 3.6.1

we use miniconda installed with Miniconda3-4.4.10-Linux-x86_64.sh

(emission) root@vps753733:/var/log/emission# cd /home/emission/web/e-mission-server-pve && /opt/miniconda3/bin/python3.6 bin/intake_multiprocess.py 3 >> /var/log/emission/pipeline/cron_intake.stdinout 2>&1
in the log file I have :
Traceback (most recent call last):
File "bin/intake_multiprocess.py", line 5, in
from future import standard_library
ModuleNotFoundError: No module named 'future'

I don't understand because i'm in emission conda environment, have you an idea ? Thanks a lot

[shankari]

@dev-rod thank you for clarifying this for others as well.

being in the emission conda environment primarily ensures that the right python is used. But then you use the full path to the python environment, which removes the effect of being in the environment.

The following examples may help clarify.

(1) In the environment, running using the standard script, works

(emission) $ ./e-mission-py.bash bin/debug/intake_single_user.py
Connecting to database URL localhost

(2) Hardcoding the base anaconda python, doesn't find the modules in the emission environment

(emission) $ ~/OSS/anaconda/bin/python3.6 bin/debug/intake_single_user.py
Traceback (most recent call last):
  File "bin/debug/intake_single_user.py", line 5, in <module>
    from future import standard_library
ModuleNotFoundError: No module named 'future'

(3) Hardcoding the anaconda python from the enviroment, finds future, but not the emission.* modules

(emission) $ ~/OSS/anaconda/envs/emission/bin/python3.6 bin/debug/intake_single_user.py
Traceback (most recent call last):
  File "bin/debug/intake_single_user.py", line 15, in <module>
    import emission.pipeline.intake_stage as epi
ModuleNotFoundError: No module named 'emission'

(4) Hardcoding the anaconda python from the enviroment and using the right PYTHONPATH makes everything work again

(emission) $ PYTHONPATH=. ~/OSS/anaconda/envs/emission/bin/python3.6 bin/debug/intake_single_user.py
Connecting to database URL localhost

I generally recommend using the first option since the e-mission-py.bash script sets the PYTHONPATH correctly and you can run it directly in the environment, but of course you can launch as per your preference.

If you are running this in docker, I have a docker-compose that sets up a container and runs the pipeline using devcron, which should make this much easier. But since I am busy, I am waiting for somebody to request that I check it in so that I can service the request lazily 😄

[shankari]

If nothing further is required, please close the issue.

[dev-rod]

Thanks @shankari , as we use also supervisor, /opt/miniconda3/envs/emission/bin/python is the correct path for us indeed. I close the issue