Failed building wheel for amazon-kclpy

[ghost]

I'm trying to install this module but it fails when building wheel

  Running setup.py bdist_wheel for amazon-kclpy ... error
  Complete output from command /Users/khashayar/Documents/Projects/Work/Terrene/TerreneCore/env/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/private/var/folders/g4/gbyhwndn1sgbqgpnx0d9sb780000gn/T/pip-build-xn64fd3j/amazon-kclpy/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /var/folders/g4/gbyhwndn1sgbqgpnx0d9sb780000gn/T/tmpioptd5o2pip-wheel- --python-tag cp35:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib
  creating build/lib/amazon_kclpy
  copying amazon_kclpy/__init__.py -> build/lib/amazon_kclpy
  copying amazon_kclpy/kcl.py -> build/lib/amazon_kclpy
  creating build/lib/samples
  copying samples/__init__.py -> build/lib/samples
  copying samples/amazon_kclpy_helper.py -> build/lib/samples
  copying samples/sample_kclpy_app.py -> build/lib/samples
  copying samples/sample_kinesis_wordputter.py -> build/lib/samples
  creating build/lib/amazon_kclpy/jars
  copying amazon_kclpy/jars/__init__.py -> build/lib/amazon_kclpy/jars
  copying samples/sample.properties -> build/lib/samples
  running build_scripts
  creating build/scripts-3.5
  copying samples/__init__.py -> build/scripts-3.5
  copying and adjusting samples/amazon_kclpy_helper.py -> build/scripts-3.5
  copying and adjusting samples/sample_kclpy_app.py -> build/scripts-3.5
  copying and adjusting samples/sample_kinesis_wordputter.py -> build/scripts-3.5
  changing mode of build/scripts-3.5/__init__.py from 644 to 755
  changing mode of build/scripts-3.5/amazon_kclpy_helper.py from 644 to 755
  changing mode of build/scripts-3.5/sample_kclpy_app.py from 644 to 755
  changing mode of build/scripts-3.5/sample_kinesis_wordputter.py from 644 to 755
  installing to build/bdist.macosx-10.12-x86_64/wheel
  running install
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/amazonaws/amazon-kinesis-client/1.6.4/amazon-kinesis-client-1.6.4.jar
  Saving http://search.maven.org/remotecontent?filepath=com/amazonaws/amazon-kinesis-client/1.6.4/amazon-kinesis-client-1.6.4.jar -> amazon_kclpy/jars/amazon-kinesis-client-1.6.4.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-cloudwatch/1.11.14/aws-java-sdk-cloudwatch-1.11.14.jar
  Saving http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-cloudwatch/1.11.14/aws-java-sdk-cloudwatch-1.11.14.jar -> amazon_kclpy/jars/aws-java-sdk-cloudwatch-1.11.14.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-core/1.11.14/aws-java-sdk-core-1.11.14.jar
  Saving http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-core/1.11.14/aws-java-sdk-core-1.11.14.jar -> amazon_kclpy/jars/aws-java-sdk-core-1.11.14.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-dynamodb/1.11.14/aws-java-sdk-dynamodb-1.11.14.jar
  Saving http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-dynamodb/1.11.14/aws-java-sdk-dynamodb-1.11.14.jar -> amazon_kclpy/jars/aws-java-sdk-dynamodb-1.11.14.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-kinesis/1.11.14/aws-java-sdk-kinesis-1.11.14.jar
  Saving http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-kinesis/1.11.14/aws-java-sdk-kinesis-1.11.14.jar -> amazon_kclpy/jars/aws-java-sdk-kinesis-1.11.14.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-kms/1.11.14/aws-java-sdk-kms-1.11.14.jar
  Saving http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-kms/1.11.14/aws-java-sdk-kms-1.11.14.jar -> amazon_kclpy/jars/aws-java-sdk-kms-1.11.14.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-s3/1.11.14/aws-java-sdk-s3-1.11.14.jar
  Saving http://search.maven.org/remotecontent?filepath=com/amazonaws/aws-java-sdk-s3/1.11.14/aws-java-sdk-s3-1.11.14.jar -> amazon_kclpy/jars/aws-java-sdk-s3-1.11.14.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar
  Saving http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar -> amazon_kclpy/jars/jackson-annotations-2.6.0.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/core/jackson-core/2.6.6/jackson-core-2.6.6.jar
  Saving http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/core/jackson-core/2.6.6/jackson-core-2.6.6.jar -> amazon_kclpy/jars/jackson-core-2.6.6.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/core/jackson-databind/2.6.6/jackson-databind-2.6.6.jar
  Saving http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/core/jackson-databind/2.6.6/jackson-databind-2.6.6.jar -> amazon_kclpy/jars/jackson-databind-2.6.6.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.6/jackson-dataformat-cbor-2.6.6.jar
  Saving http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.6/jackson-dataformat-cbor-2.6.6.jar -> amazon_kclpy/jars/jackson-dataformat-cbor-2.6.6.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/google/guava/guava/18.0/guava-18.0.jar
  Saving http://search.maven.org/remotecontent?filepath=com/google/guava/guava/18.0/guava-18.0.jar -> amazon_kclpy/jars/guava-18.0.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar
  Saving http://search.maven.org/remotecontent?filepath=com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar -> amazon_kclpy/jars/protobuf-java-2.6.1.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=commons-codec/commons-codec/1.9/commons-codec-1.9.jar
  Saving http://search.maven.org/remotecontent?filepath=commons-codec/commons-codec/1.9/commons-codec-1.9.jar -> amazon_kclpy/jars/commons-codec-1.9.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=commons-lang/commons-lang/2.6/commons-lang-2.6.jar
  Saving http://search.maven.org/remotecontent?filepath=commons-lang/commons-lang/2.6/commons-lang-2.6.jar -> amazon_kclpy/jars/commons-lang-2.6.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar
  Saving http://search.maven.org/remotecontent?filepath=commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar -> amazon_kclpy/jars/commons-logging-1.1.3.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=joda-time/joda-time/2.8.1/joda-time-2.8.1.jar
  Saving http://search.maven.org/remotecontent?filepath=joda-time/joda-time/2.8.1/joda-time-2.8.1.jar -> amazon_kclpy/jars/joda-time-2.8.1.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar
  Saving http://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar -> amazon_kclpy/jars/httpclient-4.5.2.jar
  Attempting to retrieve remote jar http://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar
  Saving http://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar -> amazon_kclpy/jars/httpcore-4.4.4.jar
  Checking .pth file support in build/bdist.macosx-10.12-x86_64/wheel/
  /Users/khashayar/Documents/Projects/Work/Terrene/TerreneCore/env/bin/python3.5 -E -c pass
  TEST FAILED: build/bdist.macosx-10.12-x86_64/wheel/ does NOT support .pth files
  error: bad install directory or PYTHONPATH

  You are attempting to install a package to a directory that is not
  on PYTHONPATH and which Python does not read ".pth" files from.  The
  installation directory you specified (via --install-dir, --prefix, or
  the distutils default setting) was:

      build/bdist.macosx-10.12-x86_64/wheel/

  and your PYTHONPATH environment variable currently contains:

      ''

  Here are some of your options for correcting the problem:

  * You can choose a different installation directory, i.e., one that is
    on PYTHONPATH or supports .pth files

  * You can add the installation directory to the PYTHONPATH environment
    variable.  (It must then also be on PYTHONPATH whenever you run
    Python and want to use the package(s) you are installing.)

  * You can set up the installation directory to support ".pth" files by
    using one of the approaches described here:

    https://setuptools.readthedocs.io/en/latest/easy_install.html#custom-installation-locations


  Please make the appropriate changes for your system and try again.

Python version: 3.5.2
OS X: 10.12 (also tried on 10.11, same error)

[whummer]

+1 happens here as well under Python 2.7.11 on OS X 10.11.

[pfifer]

I'm not able to recreate this on my system with either Python 2.7.12, or 3.5.2. I am using python installed from Homebrew in both cases where the results are being installed into a directory owned by me.

Is the command line you're using for install different than python/python3 setup.py install?

[ghost]

I install it with pip which in turn uses the python setup.py install and I'm using the homebrew version of python

@whummer I was able to resolve this by removing a few lines from the setup.py (source) It's not a proper fix but it got rid of the error

[pfifer]

I was trying to install it directly from the source directory, and not pip. Once I tried using the pip installation I did get see the same thing.

To make it more confusing it appears the installation actually succeeds, and the egg is dropped in
/usr/local/lib/python2.7/site-packages/amazon_kclpy-1.3.1-py2.7.egg/
or
/usr/local/lib/python3.5/site-packages/amazon_kclpy-1.3.1-py3.5.egg/

This is also using Homebrew.

The setup.py is working somewhat correctly, but I don't want it to generate the error message. I'm not sure if the setup actually needs to override install, or call do_egg_install directly this way. Switching to just calling install.run(self) appears to work correctly on my laptop.

Can you check whether the egg is showing up in one of the directories above, even after the pip install failure?

[whummer]

@pfifer I can confirm that the egg is showing up correctly in the directory lib/python2.7/site-packages/amazon_kclpy-1.3.1-py2.7.egg/, including all required JAR files, even after the pip install failure. Can we maybe apply the fix suggested by @kpourdeilami ? Thanks

[pfifer]

I'm looking into a complete fix similar to what @kpourdeilami suggested. I'm trying to track down the reason setup.py does the override of install the way it does.

I hope to have a fix complete for the next release.

[pfifer]

This should be fixed in release 1.4.0. Please let us know if you have any problems.