update-dependencies

[keenangraham]

Hey @jmschrei.

Maybe I'm doing something wrong but when I try to install Avocado in a clean conda environment I get following error (by the way I'm a little confused if I should be using Python 2 or 3 but I get same problem when I try on Python 2):

Create environment:

kiene:avocado keenan$ conda create --name avo3 python=3.6
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.5.11
  latest version: 4.6.8

Please update conda by running

    $ conda update -n base conda



## Package Plan ##

  environment location: /Users/keenan/anaconda/envs/avo3

  added / updated specs: 
    - python=3.6


The following NEW packages will be INSTALLED:

    ca-certificates: 2019.3.9-hecc5488_0  conda-forge
    certifi:         2019.3.9-py36_0      conda-forge
    libcxx:          7.0.0-h2d50403_1     conda-forge
    libffi:          3.2.1-1              bioconda   
    llvm-meta:       7.0.0-0              conda-forge
    ncurses:         6.1-h0a44026_1002    conda-forge
    openssl:         1.1.1b-h1de35cc_1    conda-forge
    pip:             19.0.3-py36_0        conda-forge
    python:          3.6.7-h8dc6b48_1004  conda-forge
    readline:        7.0-hcfe32e1_1001    conda-forge
    setuptools:      40.8.0-py36_0        conda-forge
    sqlite:          3.26.0-h1765d9f_1001 conda-forge
    tk:              8.6.9-ha441bb4_1000  conda-forge
    wheel:           0.33.1-py36_0        conda-forge
    xz:              5.2.4-h1de35cc_1001  conda-forge
    zlib:            1.2.11-h1de35cc_1004 conda-forge

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate avo3
#
# To deactivate an active environment, use
#
#     $ conda deactivate

kiene:avocado keenan$ conda activate avo3

Install Avocado:

avo3) kiene:avocado keenan$ pip install avocado-epigenome
Collecting avocado-epigenome
Collecting keras>=2.0.8 (from avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/5e/10/aa32dad071ce52b5502266b5c659451cfd6ffcbf14e6c8c4f16c0ff5aaab/Keras-2.2.4-py2.py3-none-any.whl
Collecting pandas>=0.23.4 (from avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/99/12/bf4c58eea94cea4f91ff931f284146337814fb8546e6eb0b52584446fd52/pandas-0.24.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting tqdm>=4.24.0 (from avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/6c/4b/c38b5144cf167c4f52288517436ccafefe9dc01b8d1c190e18a6b154cd4a/tqdm-4.31.1-py2.py3-none-any.whl
Collecting theano>=1.0.1 (from avocado-epigenome)
Collecting numpy>=1.14.2 (from avocado-epigenome)
  Downloading https://files.pythonhosted.org/packages/93/0e/30aaa357c3065957344b240482818eef31d4080f73dfa5f1ef7dcd8744d2/numpy-1.16.2-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (13.9MB)
    100% |████████████████████████████████| 13.9MB 3.0MB/s 
Collecting keras-applications>=1.0.6 (from keras>=2.0.8->avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/90/85/64c82949765cfb246bbdaf5aca2d55f400f792655927a017710a78445def/Keras_Applications-1.0.7-py2.py3-none-any.whl
Collecting keras-preprocessing>=1.0.5 (from keras>=2.0.8->avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/c0/bf/0315ef6a9fd3fc2346e85b0ff1f5f83ca17073f2c31ac719ab2e4da0d4a3/Keras_Preprocessing-1.0.9-py2.py3-none-any.whl
Collecting six>=1.9.0 (from keras>=2.0.8->avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting pyyaml (from keras>=2.0.8->avocado-epigenome)
Collecting h5py (from keras>=2.0.8->avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/03/21/1cdf7fa7868528b35c1a08a770eb9334279574a8b5f1d7a2966dcec14e42/h5py-2.9.0-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting scipy>=0.14 (from keras>=2.0.8->avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/f4/b1/6e0c995d7646b12112a721e93dc6316409cc4827c10ee309747f45ea6bfd/scipy-1.2.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting python-dateutil>=2.5.0 (from pandas>=0.23.4->avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting pytz>=2011k (from pandas>=0.23.4->avocado-epigenome)
  Using cached https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl
Installing collected packages: numpy, six, h5py, keras-applications, keras-preprocessing, pyyaml, scipy, keras, python-dateutil, pytz, pandas, tqdm, theano, avocado-epigenome
Successfully installed avocado-epigenome-0.3.0 h5py-2.9.0 keras-2.2.4 keras-applications-1.0.7 keras-preprocessing-1.0.9 numpy-1.16.2 pandas-0.24.1 python-dateutil-2.8.0 pytz-2018.9 pyyaml-3.13 scipy-1.2.1 six-1.12.0 theano-1.0.4 tqdm-4.31.1

Import Avocado:

(avo3) kiene:avocado keenan$ python
Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 02:16:08) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from avocado import Avocado
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/keenan/Desktop/projects/github/avocado/avocado/__init__.py", line 11, in <module>
    from .model import Avocado
  File "/Users/keenan/Desktop/projects/github/avocado/avocado/model.py", line 11, in <module>
    from .io import sequential_data_generator
  File "/Users/keenan/Desktop/projects/github/avocado/avocado/io.py", line 2, in <module>
    from numba import jit
ModuleNotFoundError: No module named 'numba'

Once I install numba I get additional error:

(avo3) kiene:avocado keenan$ python
Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 02:16:08) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from avocado import Avocado
Using TensorFlow backend.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/keenan/Desktop/projects/github/avocado/avocado/__init__.py", line 11, in <module>
    from .model import Avocado
  File "/Users/keenan/Desktop/projects/github/avocado/avocado/model.py", line 16, in <module>
    import keras
  File "/Users/keenan/anaconda/envs/avo3/lib/python3.6/site-packages/keras/__init__.py", line 3, in <module>
    from . import utils
  File "/Users/keenan/anaconda/envs/avo3/lib/python3.6/site-packages/keras/utils/__init__.py", line 6, in <module>
    from . import conv_utils
  File "/Users/keenan/anaconda/envs/avo3/lib/python3.6/site-packages/keras/utils/conv_utils.py", line 9, in <module>
    from .. import backend as K
  File "/Users/keenan/anaconda/envs/avo3/lib/python3.6/site-packages/keras/backend/__init__.py", line 89, in <module>
    from .tensorflow_backend import *
  File "/Users/keenan/anaconda/envs/avo3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 5, in <module>
    import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'

Not sure if these are exact versions you want but once I add these to setup.py and pip install -e . things seem to work:

(avo) kiene:avocado keenan$ python
Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 02:16:08) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from avocado import Avocado
Using TensorFlow backend.
>>> 

[jmschrei]

Howdy

Thanks for pointing out that issue. It looks like I added numba afterwards to speed up extraction, but didn't add it to the dependencies. As for the TensorFlow error, you can use either TensorFlow or Theano to run keras models. I personally use Theano because I find it easier to install (and it's in the dependencies), but it's left up to the user. I should clarify this, though. I'm not sure how to set keras to default to Theano when TensorFlow isn't present, though.

[keenangraham]

I'm not sure how to set keras to default to Theano when TensorFlow isn't present, though.

It looks like Keras sets backend to TensorFlow by default:

https://github.com/keras-team/keras/blob/c24d16af155e20976bdf61e468ba760408e676ff/keras/backend/load_backend.py#L27

However it looks like you can also override with an environmental variable:

https://github.com/keras-team/keras/blob/c24d16af155e20976bdf61e468ba760408e676ff/keras/backend/load_backend.py#L76

Haven't tried but you can probably add something like this to your __init__.py:

import os
import pkgutil


if not pkgutil.find_loader('tensorflow'):
    os.environ['KERAS_BACKEND'] = 'theano'

I think this should take care of the clean install problem, since you are including Theano as dependency. If someone already has TF installed it will just default to the backend specified in their ~/.keras config.