issues with relative packages #540
Comments
|
The actual problem you're experiencing seems to be that you have a setup.py file somewhere with an explicit pinned dependency on a version of heliograf that doesn't exist in a repository pipenv can query. This is fine, because you have that package installed and you claim it is the right version, but note that even pip doesn't know which version it is because it is installed from VCS. I'm not sure this is a pipenv issue. |
|
try using an absolute path instead, see if that changes anything |
|
@techalchemy I think you misunderstand - this isn't a published package anywhere, it isn't installed directly from VCS, its installed from a relative path to my project. I don't intend to pin any specific version, I just want it installed as an editable package. @kennethreitz that fixed it! I'd love to continue using a relative path in the Pipefile so I can use the same Pipfile with my collaborators. I also noticed that the Pipefile seemed to lowercase the whole filepath but that didn't affect anything. |
|
@AlJohri ah ok I misunderstood indeed, so it's attributing the following version pinning to the editable VCS version? Is interpreted as As pipenv is a dependency manager I'm not sure it's a good idea to use relative paths with it.. the idea is to have a portable Pipfile, to any degree possible, without restricting the ability to still install stuff. I think we can resolve this by enforcing absolute filepaths, I'll look into it in a bit. |
|
I use 'enforcement' loosely, I really mean 'resolution' -- we can resolve the paths when we either lock or do dependency resolutions or both |
|
I think we should allow relative paths here, just not encourage their use. |
|
Maybe even issue a warning that it's likely a bad idea. |
|
I rely on them pretty heavily in my "multi-app with shared library" repositories for storing utility functions, exceptions, maybe some data structures etc.
Why are they a bad idea?
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: Kenneth Reitz <[email protected]>
Sent: Thursday, September 14, 2017 11:23:53 AM
To: kennethreitz/pipenv
Cc: Al Johri; Mention
Subject: Re: [kennethreitz/pipenv] issues with relative packages (#540)
Maybe even issue a warning that it's likely a bad idea.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://github.com/kennethreitz/pipenv/issues/540#issuecomment-329517307>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ACqSzEBc-iqXr5t0PbfDeM2xVNSRWBMTks5siUUJgaJpZM4PW4KW>.
|
|
I also find it hugely useful as an alternative to mucking around with PYTHONPATH or doing weird "import site; site.insert..." hacks and instead running "pip install -e" on a bunch of packages and now voila you can use them in your Jupyter notebooks and scripts that may happen to be in a different folder. |
|
Yeah, you know what you're doing. They're a bad idea for people that don't know what they're doing, basically. |
|
@kennethreitz I'm not suggesting not to allow them, just to convert them when we resolve them |
|
yeah, i don't think a lockfile should ever contain relative paths. that's dangerous territory. |
|
unless they're within the codebase |
|
we only support absolute paths for now |
|
Context: We spent a good chunk of time on this today. It isn't straightforward when 'locking' dependencies, hashing paths, installing, etc. The recommended approach would be to use pip directly if you need a relative path or to use an absolute path. |
|
thanks for looking into this @kennethreitz @techalchemy! is there any chance to just save the dependency to the Pipfile without any hashing or locking etc.? I'd like to ideally tell my users / collaborators to "just run" I can wrap the |
|
this is probably something we can implement although I think we are going to be slowing down release cycles here. @kennethreitz My proposal for a solution would be to just add an extra check to see if something is a file and, if so, resolve its absolute path upon locking. |
|
@kennethreitz should this issue be kept open for now? |
|
Got a new error now: This line was added to my Pipfile: |
|
Using this workaround for now: |
|
I don't suppose you're going to tell us whether that actually is what is on line 41 of your Pipfile ?
In the meantime, can you show us the output of pipenv version and start using --verbose with pipenv install
… On Sep 17, 2017, at 10:47 PM, Al Johri ***@***.***> wrote:
Got a new error now:
$ pipenv install -e ../../lib/python
Installing -e ../../lib/python…
Obtaining file:///Users/johria/Development/heliograf/lib/python
Installing collected packages: heliograf
Found existing installation: heliograf 0.1.0
Uninstalling heliograf-0.1.0:
Successfully uninstalled heliograf-0.1.0
Running setup.py develop for heliograf
Successfully installed heliograf
Adding -e ../../lib/python to Pipfile's [packages]…
Locking [dev-packages] dependencies…
Traceback (most recent call last):
File "/Users/johria/.pyenv/versions/3.6.0/bin/pipenv", line 11, in <module>
sys.exit(cli())
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/cli.py", line 1434, in install
do_lock(system=system)
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/cli.py", line 852, in do_lock
deps = convert_deps_to_pip(project.dev_packages, r=False)
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/project.py", line 252, in dev_packages
for k, v in self.parsed_pipfile.get('dev-packages', {}).items():
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/project.py", line 176, in parsed_pipfile
return contoml.loads(f.read())
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/patched/contoml/__init__.py", line 15, in loads
elements = parse_tokens(tokens)
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/patched/prettytoml/parser/__init__.py", line 17, in parse_tokens
return _parse_token_stream(TokenStream(tokens))
File "/Users/johria/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pipenv/patched/prettytoml/parser/__init__.py", line 32, in _parse_token_stream
raise ParsingError('Failed to parse line {}'.format(pending.head.row))
prettytoml.parser.errors.ParsingError: Failed to parse line 41
This line was added to my Pipfile:
1308583 = {file = "../../lib/python", editable = true}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
@techalchemy sorry if that wasn't clear; the line I posted below the stack trace starting with
Full Pipfile: Line 41 is the line that starts with |
@kennethreitz I tried using an absolute path but I'm still running into issues. I'm using pipenv 7.3.7. |
|
@kennethreitz should this issue be re-opened? the issue exists with absolute paths as well |
|
happy to help if you point me in a direction :) |
|
this is working fine for me in my tests |
|
@techalchemy I changed my naming convention and removed the trailing slash but still ran into the error with an absolute path. I'm using pipenv 7.4.4. does it have to do with the lowercasing of the filepath? resulted in directory structure: setup.py file: |
|
So for whatever reason the path we are passing the resolver for |
|
Nope, I'm lying. It's lowercasing your path when it tries to lock. |
|
This may be fixed by 4638701 |
|
should be fixed now :) |
|
still seems like its being lowercased? I tried to verify I have the latest version from VCS same error: |
|
i actually just fixed the thing, spoke too soon |
|
need to make sure if works with paths too |
|
try with latest release |
|
@kennethreitz didn't seem to work, still lowercasing: |
|
did you use latest? |
|
you need to re-lock |
|
just use relative paths now, they should work fine. |
|
I ran: And it worked! 🎉 Does the Pipfile.lock diff look correct? I see the addition of both Pipfile.lock diff --git a/apps/api/Pipfile.lock b/apps/api/Pipfile.lock
index af52adc..3359be0 100644
--- a/apps/api/Pipfile.lock
+++ b/apps/api/Pipfile.lock
@@ -189,6 +189,10 @@
],
"version": "==0.90.0"
},
+ "ec98b80": {
+ "editable": true,
+ "path": "../../lib/python/heliograf"
+ },
"envdir": {
"hashes": [
"sha256:af5f012e09360cd9af4631fe963872f76a6bf4f08da9c07b3395511a56614bfa",
@@ -196,6 +200,10 @@
],
"version": "==0.7"
},
+ "heliograf": {
+ "hashes": [],
+ "version": "==0.1.0"
+ },
"htmlmin": {
"hashes": [
"sha256:645fdd70615bd120f8b110dfaf93f9b201c88ee7899dc5a4c51ca3ffe004b3a5",Pipfile diff --git a/apps/api/Pipfile b/apps/api/Pipfile
index d5f0e20..6138972 100644
--- a/apps/api/Pipfile
+++ b/apps/api/Pipfile
@@ -38,6 +38,7 @@ tweepy = "*"
uwsgi = "*"
whitenoise = "==4.0b3"
django-tinymce4-lite = {git = "https://github.com/romanvm/django-tinymce4-lite", ref = "develop"}
+ec98b80 = {path = "../../lib/python/heliograf", editable = true}
[dev-packages] |
|
Hm so it has added an entry for both the package name as well as the hash? that is probably bad |
|
oh yeah, we resolve those... |
|
technically correct. |
* Issues - pypa#949, pypa#939, pypa#936 and to a lesser extent pypa#817 pypa#540 and more * Fixed: Local file path installation (resolves in pipfile as relative path) * Pass file:// URI to Requirements library for resolving TODO: * Ignore non-explicit directory paths lacking os.sep or ./ * Add tests
* Issues - pypa#949, pypa#939, pypa#936 and to a lesser extent pypa#817 pypa#540 and more * Fixed: Local file path installation (resolves in pipfile as relative path) * Pass file:// URI to Requirements library for resolving TODO: * Ignore non-explicit directory paths lacking os.sep or ./ * Add tests
leave kennethreitz/pipenv#540 alone till kennethreitz/pipenv#1104 releases
@kennethreitz I'm having trouble installing local packages in relative directories via
-e.This is the initial requirements.txt file:
This is what the converted Pipfile looks like:
heliografin this case is the name in setup.py of the relative package.I believe the issue may be that the Pipfile.lock doesn't have the directory?
While the above is starting from the requirements.txt, this kind of works:
pipenv install -e ../../lib/pythonIt adds another entry into the Pipefile like this:
I can see that the package is installed properly by running
pipenv run pythonand importing the package. In addition, doing apipenv shellfollowed by apip freezeshows:potentially related issues:
versions:
The text was updated successfully, but these errors were encountered: