fix(python): dependency versioning in Python

[garnaat]

Fix the generation of the dependency version strings for the install_requires section of setup.py. Fixes #676.

The version string placed in the setup.py file for dependencies is incorrect. This results in incompatible modules being installed. This fix changes the generated version string for dependencies to correct the problem.

Fixes #696

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[eladb]

Just to make sure, ~=1.4.0 will be compatible with 1.6.5, correct?

[garnaat]

Not totally sure I understand the question. If you use a version string of "~=1.4.0" for a dependency, it will be compatible with 1.4.*. If the only version it can find is 1.6.5, it will fail. Is that not correct?

[eladb]

In JavaScript the version spec we use is “^1.3.0” which means to use the latest compatible version. This basically includes all versions >= 1.3.0 and < 2.0.0 since by definition those should be backwards compatible with 1.3.0. We should use the same spec for python I think, no?

[garnaat]

I thought that was the problem we were trying to fix in aws/aws-cdk#3517. If I try to install version 1.0.0 of a module and it has dependencies, all of the dependencies come in as the latest available version (e.g. 1.3.0) rather than also being pinned to 1.0.0.

[eladb]

As long as 1.3.0 is compatible with 1.0.0, I don’t see a problem.

[eladb]

Mmmm... looking at the CDK issue, I am curious how this can happen because basically there is a violation of the requirement for the transitive dependency. If there’s a top level dependency A-1.0.0 and B-1.0.0 and day A has a transitive dependency on C, which takes a dependency on B. If we bring in [email protected] it’s requirement on B would be ^1.3.0 which mean that B is not compatible.

[garnaat]

Well, from the issue above it appears that it is causing a problem. Plus, I guess if I pinned my dependency to a specific version my expectation is that all transitive dependencies would be pinned to the same version. If we don't think that is the right behavior this fix is not needed as there really isn't a problem to fix.

[garnaat]

I don't think this PR should be merged. It would mean that Python would work differently than the other languages. Instead, I will investigate the linked issue and try to reproduce it and figure out why it caused an error.

[eladb]

Makes sense

[RomainMuller]

Closing this PR without merging. We can dig it up if we end up actually needing to go down that path.

[joekiller]

@eladb @RomainMuller this not being merged causes aws/aws-cdk#4957 to cascade into any of my libraries even if I pin them at 1.15.0 because

https://github.com/aws/jsii/blame/fe942b3f358335125f9f0800fc34ac06be93dcff/packages/jsii-pacmak/lib/targets/python.ts#L1096-L1100

is making an incorrect compatible release specification (https://www.python.org/dev/peps/pep-0440/#compatible-release)

aws-cdk.aws-cloudformation~=1.15,>=1.15.0 means >=1.5, ==1.* which then loads 1.16.0.

[RomainMuller]

I think the reasoning is slightly different now however - the code generator generates a dependency statement that is NOT what was declared in the initial module!

[joekiller]

@RomainMuller I can understand that concern

Regarding the python dependency itself keying off depInfo.version and exclusively using the ~=X.Y.Z only to keep minor versions from affecting downstream projects pinned at ~= X.Y.0

[joekiller]

@RomainMuller just to be thorough. There is a explicit difference in the interpretation of versions that makes an incorrect compatibility presumption with regard to the current python implementation where ~=X.Y,>=X.Y.Z is being returned for the python version.

In npmjs tilde ranges the same ~=X.Y,>=X.Y.Z is equivalent to >=X.Y.0 <X.(Y+1).0 or >=X.Y.0 <X.(Y+1).0

In python compatible release ~=X.Y,>=X.Y.Z is the equivalent of >= X.Y, == X.* which is not the same compatibility intention as the npm tilde range.

The change in this PR where the version is set to be exclusively as ~=X.Y.Z is valid and should be accepted to keep the two intended versions compatibility layer between the two package managers.

I wasn't sure if this difference was clear or not so please forgive me posting twice :)

[joekiller]

So at risk of being completely redundant if the code was updated such that the python version uses depInfo.version the string still needs to be updated to be exclusively ~=X.Y.Z.