Cannot install Zope 4 with ZServer following installation documentation

[jugmac00]

In order to reproduce zopefoundation/Zope#665 with a clean Zope installation, I tried to install Zope 4 with ZServer following the documentation at https://zope.readthedocs.io/en/latest/INSTALL.html and https://zope.readthedocs.io/en/2.13/INSTALL-buildout.html#creating-a-zope-instance

Result:

jugmac00@jugmac00-XPS-13-9370:~/Projects/zserver-test$ bin/mkzopeinstance
Please choose a directory in which you'd like to install
Zope "instance home" files such as database files, configuration
files, etc.

Directory: instance
Please choose a username and password for the initial user.
These will be the credentials you use to initially manage
your new Zope instance.

Username: admin
Password: 
Verify password: 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named Zope2

Step by step...

jugmac00@jugmac00-XPS-13-9370:~/Projects$ mkdir zserver-test
jugmac00@jugmac00-XPS-13-9370:~/Projects$ cd zserver-test/
jugmac00@jugmac00-XPS-13-9370:~/Projects/zserver-test$ virtualenv .venv -p /usr/bin/python2.7
jugmac00@jugmac00-XPS-13-9370:~/Projects/zserver-test$ . .venv/bin/activate
(.venv) jugmac00@jugmac00-XPS-13-9370:~/Projects/zserver-test$ pip install zc.buildout

[buildout]
extends =
    https://zopefoundation.github.io/Zope/releases/4.1/versions-prod.cfg
parts =
    zopeinstance

[zopeinstance]
recipe = plone.recipe.zope2instance
eggs =
    Products.TemporaryFolder
    ZServer
user = admin:adminpassword
http-address = 8080
wsgi = off

(.venv) jugmac00@jugmac00-XPS-13-9370:~/Projects/zserver-test$ buildout

jugmac00@jugmac00-XPS-13-9370:~/Projects/zserver-test$ bin/mkzopeinstance
Please choose a directory in which you'd like to install
Zope "instance home" files such as database files, configuration
files, etc.

Directory: instance
Please choose a username and password for the initial user.
These will be the credentials you use to initially manage
your new Zope instance.

Username: admin
Password: 
Verify password: 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named Zope2

[d-maurer]

Jürgen Gmach wrote at 2019-7-2 03:05 -0700:

In order to reproduce zopefoundation/Zope#665 with a clean Zope installation, I tried to install Zope 4 with ZServer following the documenation at https://zope.readthedocs.io/en/latest/INSTALL.html and https://zope.readthedocs.io/en/2.13/INSTALL-buildout.html#creating-a-zope-instance Result: ``` ***@***.***:~/Projects/zserver-test$ bin/mkzopeinstance Please choose a directory in which you'd like to install Zope "instance home" files such as database files, configuration files, etc. Directory: instance Please choose a username and password for the initial user. These will be the credentials you use to initially manage your new Zope instance. Username: admin Password: Verify password: Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named Zope2

`Zope` does contain a `Zope2` package (at least, it did until very recently). I would start by checking that your installation contains a `Zope2` package. If not, try to unstall it. Should your Zope lack the `Zope2` package, then it was very recently removed and other dependent software (e.g. `ZServer`) might not yet have been adapted.

[jugmac00]

@d-maurer You are right - the package "Zope2" is missing following the above way - which is the way described in the documentation.

So instead of installing it manually, it should be declared as a dependency somewhere, right?

[tseaver]

@jugmac00 I think there is a confusion of terms here: "package" might mean "directory one can import in Python" or it might mean "thing installable from PyPY". In this case, Zope2 is an "importable package", contained in the installable Zope distribution.

I just checked, and the Zope2 importable is present in the Zope-4.1.1 distribution on PyPI (both the source distribution and the wheel).

[jugmac00]

@tseaver This Zope2 thing is rather confusing anyway... cf zopefoundation/Zope#616 and zopefoundation/Zope#625

When I do a pip install Zope2 in the virtualenv (which is usually only used for zc.buildout) I encounter no more errors - but this cannot be the solution. And "obviously" Zope2 on pypi is something different from Zope2 within the Zope package.

I am looking forward very much to reach the end of Python 2 when there is only ZopeX on Python3 and nothing else :-)

[tseaver]

@jugmac00 the Zope2 on PyPI is a "meta-distribution", whose only dependenency is the Zope distribution: it exists only for backward compatibility: unless you are upgrading an old environment (virtualenv, buildout) which installed Zope2, there is no point in installing it now.

The name of the Zope2 Python package provided by the Zope distribution has nothing to do with Python 2.x / Python 3.x: it predates Python 3.0 by about 10 years.

[d-maurer]

The "eggs" used during the generation of bin/mkzopeinstance might have been wrong. Look at the top of this script: you likely will find a sys.path[0:0] = [....] where the "..." lists locations from the effective "eggs" definition; check that the egg/wheel Zope is correctly listed. If it is there, look into this egg: you should see a Zope2 package.

[jugmac00]

The exception gets thrown here in p = ...

ZServer/src/ZServer/Zope2/utilities/mkzopeinstance.py

Lines 208 to 221 in d1f3789

	def get_zope2path(python):
	""" Get Zope2 path from selected Python interpreter.
	"""
	zope2file = ''
	p = os.popen('"%s" -c"import Zope2; print Zope2.__file__"' % python)
	try:
	zope2file = p.readline()[:-1]
	finally:
	p.close()
	if not zope2file:
	# fall back to current Python interpreter
	import Zope2
	zope2file = Zope2.__file__
	return os.path.abspath(os.path.dirname(os.path.dirname(zope2file)))

Looking in the path for Zope in bin/mkzopeinstance everything looks normal...

(zzz) jugmac00@jugmac00-XPS-13-9370:~/Projects/zzz$ cd /home/jugmac00/Projects/zzz/eggs/Zope-4.1-py2.7.egg
(zzz) jugmac00@jugmac00-XPS-13-9370:~/Projects/zzz/eggs/Zope-4.1-py2.7.egg$ ls -la
total 52
drwxr-xr-x 12 jugmac00 jugmac00 4096 Jul  4 11:58 .
drwxr-xr-x 89 jugmac00 jugmac00 4096 Jul  4 11:59 ..
drwxr-xr-x  5 jugmac00 jugmac00 4096 Jul  4 11:58 App
drwxr-xr-x  2 jugmac00 jugmac00 4096 Jul  4 11:58 EGG-INFO
drwxr-xr-x  6 jugmac00 jugmac00 4096 Jul  4 11:58 OFS
drwxr-xr-x  6 jugmac00 jugmac00 4096 Jul  4 11:58 Products
drwxr-xr-x  3 jugmac00 jugmac00 4096 Jul  4 11:58 Shared
drwxr-xr-x  5 jugmac00 jugmac00 4096 Jul  4 11:58 Testing
drwxr-xr-x  3 jugmac00 jugmac00 4096 Jul  4 11:58 zmi
drwxr-xr-x  5 jugmac00 jugmac00 4096 Jul  4 11:58 Zope2
-rw-r--r--  1 jugmac00 jugmac00 2753 Jul  4 11:58 Zope-4.1-py3.7-nspkg.pth
drwxr-xr-x  3 jugmac00 jugmac00 4096 Jul  4 11:58 ZPublisher
drwxr-xr-x  3 jugmac00 jugmac00 4096 Jul  4 11:58 ZTUtils
(zzz) jugmac00@jugmac00-XPS-13-9370:~/Projects/zzz/eggs/Zope-4.1-py2.7.egg$ cd Zope2/
(zzz) jugmac00@jugmac00-XPS-13-9370:~/Projects/zzz/eggs/Zope-4.1-py2.7.egg/Zope2$ ls -la
total 28
drwxr-xr-x  5 jugmac00 jugmac00 4096 Jul  4 11:58 .
drwxr-xr-x 12 jugmac00 jugmac00 4096 Jul  4 11:58 ..
drwxr-xr-x  4 jugmac00 jugmac00 4096 Jul  4 11:58 App
-rw-r--r--  1 jugmac00 jugmac00  793 Jul  4 11:58 ClassFactory.py
-rw-r--r--  1 jugmac00 jugmac00 2014 Jul  4 11:58 __init__.py
drwxr-xr-x  4 jugmac00 jugmac00 4096 Jul  4 11:58 Startup
drwxr-xr-x  4 jugmac00 jugmac00 4096 Jul  4 11:58 utilities

Currently I lack both time and especially knowledge to dig deeper into this issue.

At first, I only wanted to create an instance to reproduce the bug with ZServer and threading reported in zopefoundation/Zope#665

For me buildout is still the great unknown.

[icemac]

Hm, when trying to reproduce this issue I get another one:

$ bin/mkzopeinstance
Traceback (most recent call last):
  File "bin/mkzopeinstance", line 90, in <module>
    sys.exit(ZServer.Zope2.utilities.mkzopeinstance.main())
  File ".../eggs/ZServer-4.0.1-py2.7.egg/ZServer/Zope2/utilities/mkzopeinstance.py", line 57, in main
    if check_buildout(script_path):
  File ".../eggs/ZServer-4.0.1-py2.7.egg/ZServer/Zope2/utilities/mkzopeinstance.py", line 204, in check_buildout
    parser.read(buildout_cfg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 305, in read
    self._read(fp, filename)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 546, in _read
    raise e
ConfigParser.ParsingError: File contains parsing errors: .../Zope/buildout.cfg
    [line 158]: '=> make-docs\n'

I am using a checkout of Zope master and ran bin/buildout before.
It seems that ZServer parses buildout.cfg using ConfigParser which breaks on buildout specific syntax.

I am going to create an issue in the ZServer project.

[icemac]

Using a clean buildout.cfg and a debugger shows that the error message comes from

ZServer/src/ZServer/Zope2/utilities/mkzopeinstance.py

Line 212 in 8025f65

p = os.popen('"%s" -c"import Zope2; print Zope2.__file__"' % python)

It should be adapted like

https://github.com/zopefoundation/Zope/blob/e2cfef9132bae378e2654cb1dd4517021916ff2c/src/Zope2/utilities/mkwsgiinstance.py#L218-L227

in Zope's mkwsgiinstance.py

As both issues are ZServer related, I'm transferring this issue there.

[icemac]

The shown error message does not mean that creating an instance had an error, it is only an internal message which should be suppressed.

[icemac]

See zopefoundation/Zope@eed8f0c as the commit introducing the fix in the Zope repository.

[jugmac00]

@icemac I am not sure what this means... is this issue fixed with your linked commit?

[icemac]

@jugmac00 I think that the liked commit has to be ported to ZServer as it has its own copy of the code.