Include support for using Pkg with SOCKS proxies #10754
Comments
|
Welcome. What we're shortest on is qualified labor. We've had a to-do item to rewrite our package manager using libgit2 instead of shelling out to command-line git, that is going exceptionally slowly for lack of anyone working on it. I'm guessing/assuming that would help here, but I'm not absolutely sure. I believe the majority of the shelling-out to git happens here: https://github.com/JuliaLang/julia/blob/master/base/pkg/git.jl so if you have a local from-source build, you can try modifying the commands that get created there to add the extra proxy wrapper. Hopefully there's a more general solution for this issue but that'll need someone who knows what they're talking about. Sounds like you do, so PR's would be great if you have suggestions. |
As far as I can tell (corrections welcome), proxy config is basically program-specific on Linux, so I'm not quite sure how this would work. One option is to use redsocks or tsocks. Or for git only, set We just shell out to the first "git" found on linux, so you might be able to get away with a helper script such as below (assuming ~/bin/git: |
Yes. Both redsocks and tsocks have issues with leaking packets, allowing leaks through file descriptors passed over UNIX sockets, etc, due to the fact that both (IIRC) only wrap
(This is with Julia packaged from Debian Jessie, however I'm totally willing to stop being lazy and compile Julia from source in order to further debug.) Hence the above lengthy process of fetching the requisite dependencies manually. Using your idea to proxy the first And combined with having already done This is a bit hacky… Perhaps it is possible for some more elegant solution? Also, that segmentation fault is a bit troubling. |
|
Thanks for the patient explanation. The reason I noticed
Looks like someone else has seen this with a libuv-based project. So the segfault may be upstream. If you want to debug the libuv side, we could take a patch for that on our fork (and try to get it upstreamed). Beyond that though, I could imagine that |
|
@lhnorton Re:
Perhaps GH ate the end of your comment? |
|
Sorry about that, I accidentally hit some magic key combo and then edited in-place. Now in the web view it reads: "I could imagine that pipe2 is not a syscall that torsocks wants to support." Or at least, might not be -- due to the concern about compromised syscalls. |
|
Do you have any suggestions for what a I would say that anyone in a situation where compromised syscalls could be a physical or livelihood threat should probably not be installing packages from METADATA automatically right now, and not at all without doing your own code review. |
|
@isislovecruft made me look at this. Since I never used this before, and don't particularly have the time to learn all about it, is this the expected behaviour, starting from a fresh install? There's a bunch of files that appear in the subdirectory, and and my tor traffic momentarily spikes, so I assume it's doing the right thing.... If so, I have a bunch of torsocks patches that make this work. This might be enough: https://github.com/Yawning/torsocks-git-hax/blob/master/bug13294.patch NB:
|
|
That's pretty normal behavior, aside from the torsocks warning, obviously. Julia itself doesn't use |
|
Thanks for the input @Yawning. The fact that our libuv fork doesn't pass its tests is a known bug, JuliaLang/libuv#29 - that would be ideally fixed by upstreaming our pipe API changes so we no longer need to use a fork. |
|
@tkelman No problem. I'm a bit worried that I might have broke additional things since less of the tests pass (beyond the expected udp test failures), so someone should evaluate if tcp networking from within julia itself still works when run through the patched torsocks. But if the usage pattern is "only run it through torsocks when updating packages, this is less of a concern, since the git client that's being fork/execed will work. The torsocks warning that appears is harmless since the libuv code itself falls back to using |
|
Perhaps #17783 may help here. |
|
Fixed by #17783. |
I am on a network behind a very restrictive firewall, and I can only use Tor to reach the internet. I have seen issues like #3001, and my git is globally configured to use
https://instead ofgit://(by the way, that is a severe security issue which actually warrants filing CVE, I strongly suggest fixing #7005).I suggest, for Julia users like myself with restricted connections, that Julia adopt support for respecting system proxy settings, as well as support for both HTTP and SOCKS proxies. My apologies that I am still quite new to Julia, and do not yet have any more concrete suggestions as to where/how to implement this support.
To show you a bit of the difficultly that your new users are facing, the following comprise my attempts to get IJulia installed:
(Towards the end, the "Updating METADATA" task merely hung indefinitely, with no signs of network activity.)
Long story short, I did get IJulia installed and working… but it was painful.
I really love the design of Julia, but I fear that treating new users like this is going to make many who come to your community, excited to try a new language, give up quickly thinking that the language is buggy and incomplete, or still too young for serious work.
The text was updated successfully, but these errors were encountered: