Fail: No module named debops

[thomasnal]

Hi,

Running debops on a new host host1 results in failure of debops.pki.

TASK [debops.pki : DebOps pre_tasks hook] **************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: No module named debops
fatal: [aditi1]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}

PLAY RECAP *********************************************************************
host1                     : ok=45   changed=0    unreachable=0    failed=1

Any idea what causes error of out of box run?

$ ansible --version
ansible 2.2.2.0

$ echo $PYTHONPATH
/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages:/usr/local/Cellar/ansible/2.2.2.0/libexec/vendor/lib/python2.7/site-packages:

Ansible has been installed using homebrew on an osx box. The PYTHONPATH had to be added. Due No module named ansible error. This error sounds similar, however, I don't see any python modules named debops in /usr/local/Cellar or other locations.

$ debops -vvv
TASK [debops.pki : DebOps pre_tasks hook] **************************************
task path: /Users/thomas/Library/Application Support/debops/debops-playbooks/roles/debops.pki/tasks/main.yml:4
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 443, in _execute
    include_file = templar.template(include_file)
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/template/__init__.py", line 383, in template
    disable_lookups=disable_lookups,
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/template/__init__.py", line 584, in do_template
    res = j2_concat(rf)
  File "<template>", line 8, in root
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/jinja2/runtime.py", line 196, in call
    return __obj(*args, **kwargs)
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/template/__init__.py", line 493, in _lookup
    instance = self._lookup_loader.get(name.lower(), loader=self._loader, templar=self)
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/plugins/__init__.py", line 339, in get
    self._module_cache[path] = self._load_module_source('.'.join([self.package, name]), path)
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/plugins/__init__.py", line 324, in _load_module_source
    module = imp.load_source(name, path, module_file)
  File "/Users/thomas/Library/Application Support/debops/debops-playbooks/playbooks/lookup_plugins/task_src.py", line 33, in <module>
    from debops import *
ImportError: No module named debops

[drybjed]

It seems that you installed Ansible through Homebrew. Since Homebrew and pip environments are separate, try uninstalling Ansible and installing it through pip, this should let it find the debopsPython module correctly.

[thomasnal]

That worked.

I have seen confusing messages out on different websites and went with recommendation to use brewed ansible instead of pip-ed one.

[nodje]

This issue should probably be closed.
But I'm having this same issue with a pip installed ansible lastest, 2.4.1.0 on macOS, and just updated debops-playbooks.

TASK [debops.pki : DebOps pre_tasks hook] *******************************************************************************************************************************************************
task path: /Users/myuser/Library/Application Support/debops/debops-playbooks/roles/debops.pki/tasks/main.yml:4
The full traceback is:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 125, in run
    res = self._execute()
  File "/usr/local/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 456, in _execute
    include_file = templar.template(include_file)
  File "/usr/local/lib/python2.7/site-packages/ansible/template/__init__.py", line 450, in template
    disable_lookups=disable_lookups,
  File "/usr/local/lib/python2.7/site-packages/ansible/template/__init__.py", line 672, in do_template
    res = j2_concat(rf)
  File "<template>", line 11, in root
  File "/usr/local/lib/python2.7/site-packages/jinja2/runtime.py", line 260, in call
    return __obj(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/ansible/template/__init__.py", line 579, in _lookup
    instance = self._lookup_loader.get(name.lower(), loader=self._loader, templar=self)
  File "/usr/local/lib/python2.7/site-packages/ansible/plugins/loader.py", line 363, in get
    self._module_cache[path] = self._load_module_source(name, path)
  File "/usr/local/lib/python2.7/site-packages/ansible/plugins/loader.py", line 339, in _load_module_source
    module = imp.load_source(full_name, path, module_file)
  File "/Users/myuser/Library/Application Support/debops/debops-playbooks/playbooks/lookup_plugins/task_src.py", line 33, in <module>
    from debops import *
ImportError: No module named debops

What could be missing? Obviously it should be a path problem.
I installed ansible with a brew installed pip & python.
Do you thing it could cause this problem to occur?

$ ls -la $(which pip2)
lrwxr-xr-x  1 myuser  admin  32 26 Sep 11:39 /usr/local/bin/pip2 -> ../Cellar/python/2.7.14/bin/pip2

[drybjed]

The clue is in the last few lines:

  File "/Users/jean/Library/Application Support/debops/debops-playbooks/playbooks/lookup_plugins/task_src.py", line 33, in <module>
    from debops import *
ImportError: No module named debops

This role uses some custom lookup plugins which depend on external code provided by the debops PyPI package. You should make sure that you are using a pip Ansible version, and not the brew one as well.

[nodje]

As I was saying, I'm using a pip installed Ansible and NOT a brew one.
So this doesn't explain the error.

I was just mentioning that pip itself was installed using brew, but not ansible

[drybjed]

Great. In that case, pip install debops as well.

[nodje]

OK, thanks for your help, I realise I was using debops installed by the system pip instead of the brew intalled pip. Quite confusing.

I bumped into another issue during debops.pki install, I'll post it in an independent entry.

[drybjed]

This issue might be due to the project moving to the monorepo which is now used by the scripts, starting with v0.6.0. Best best option for you for now, if you don't want to use the full project, would be to clone it anyway for example via debops-update, and symlink interesting roles where Ansible can find them. In the future I plan to export DebOps roles to their own GitHub repositories, so that they can be installed independently via Ansible Galaxy, but that's not implemented yet.

[nodje]

Right, I was on 0.5.0, and figured I had to run debops-update to get the macOS /Users/myuser/Library/Application Support/debops/debops monorepo cloned.
I don't quite get the difference with the previous structure though.
What do you mean by "symlink interesting roles where Ansible can find them"?

I mean the debops roles are all in their own repo already aren't they?

[drybjed]

I'm not sure if you are using the project as a whole, or you want to only use specific roles. In any case, DebOps roles in the monorepo are located in ansible/roles/ subdirectory.

If you use the debops command, and consequently the project as a whole, then you shouldn't see any changes in its use. Just run the debops command nomrally and it should, after an update and download of the monorepo, switch to the new directory structure normally.

If you only use certain roles, instead of downloading them manually, you can now just clone the DebOps monorepo, and either create symlinks of the DebOps roles to where Ansible can find them (for example /etc/ansible/roles/), or in your ansible.cfg, add the path to the DebOps roles in the roles_path configuration parameter.

[nodje]

I now understand what you mean.
The new monorepo contains all the roles that use to be downloaded in debops-playbook, which can now be deleted. I can only take a guess at why you did this rather big change, but I'm sure you have good reasons ;)

In fact, my original idea was to use only a couple of roles from the project only (which I stumbled upon while searching galaxy for some roles). Then I got interested in this already robust project, and started to think I could use it as a base and build on it rather than to build my own server base.

It looks great, but to be able to use it for my own servers, I need to make it mine somehow, and to get to know the structure and scripts well enough that I understand to a time good extend how the server is setup. Quite some work to be done, but eh, I'm sure I'll learn a lot on the way. Need to find the time though...

[drybjed]

You can read the reasons and discussion behind the DebOps monorepo in the DebOps mailing list archives. It spans a couple of months, starting from August 2017, the "RFC" thread.

DebOps was meant to be used as a "base" for your own infrastructure, so good call I guess. :-) With the monorepo, you can now keep your changes in a separate branch and rebase them on top of 'master' when any new changes show up. Or, in the future, you will be able to easily select a version of DebOps you want to stick with and use that.