Pkg.update() hangs when updating private github repo #16373

Cody-G · 2016-05-14T20:44:53Z

In the latest 0.5 master, Pkg.update() often hangs indefinitely when updating a private repository. It prompts for my Github username and password but doesn't respond after I enter them. If I press ctrl+c several times I can abort updating that package, and then some (but not all) of my other private repositories are able to update successfully.

The text was updated successfully, but these errors were encountered:

wildart · 2016-05-14T23:13:41Z

I hope #16308 will fix this.

Cody-G · 2016-05-15T00:39:16Z

Great, thanks! I'll follow along and let you know.

Cody-G · 2016-05-16T19:13:56Z

I pulled your credentials branch (#16308) and tested it. I ran Pkg.update several times and credential caching only worked sometimes. Sometimes I only needed to enter credentials once, sometimes multiple times. Unfortunately it still hangs for me sometimes after entering credentials. But a more serious problem is that one call to Pkg.update resulted in this segfault (note that all packages were already up-to-date as of seconds before):

INFO: Updating Gtk master...
INFO: Computing changes...
*** Error in ./julia: double free or corruption (fasttop): 0x00000000027ce400 ***
signal (6): Aborted
while loading no file, in expression starting on line 0
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f11c64b4393)
unknown function (ip: 0x7f11c64c066d)
*** Error in ./julia: corrupted double-linked list: 0x0000000002ac2150 ***

wildart · 2016-05-18T02:17:07Z

Can you reproduce this segfault? I do not see any julia functions in the stack trace.

Cody-G · 2016-05-18T17:54:18Z

I rebased your branch on the latest master and tried again. I have run Pkg.update() about 10 times now and haven't seen another segfault. I do however still get at least one private repo that hangs every time I update. I haven't seen any consistency in which packages hang, it seems to happen randomly. Also the credential caching is still not working (but I'm not sure if you even meant to fix that with this branch).

wildart · 2016-05-18T18:29:57Z

Can you post an exception stack trace when you interrupt hanging repo update? If you are on latest master there should be plenty of output from Pkg.update.

Cody-G · 2016-05-18T22:37:36Z

Looks like when I tried earlier today I was on the wrong Julia branch, oops! Now when I tried the correct branch it was improved, but still not totally fixed. Most of the time Pkg.update() runs fine, but out of 8 tries I got 2 different failures. Here is the first:

INFO: Updating Gtk master...
INFO: Computing changes...
INFO: No packages to install, update or remove                                                                                                                                                                                   
ERROR: Update finished with errors.                                                                                                                                                                                              
=> Package BlockRegistration cannot be updated.                                                                                                                                                                                  
ArgumentError: embedded NULs are not allowed in C strings: "ͪ<W\0\0\0\0ͪ<"                                                                                                                                                         
 [inlined code] from ./base.jl:21                                                                                                                                                                                                
 in unsafe_convert(::Type{Cstring}, ::String) at ./c.jl:87                                                                                                                                                                       
 in credentials_callback(::Ptr{Ptr{Void}}, ::Cstring, ::Cstring, ::UInt32, ::Ptr{Void}) at ./libgit2/callbacks.jl:180                                                                                                            
 [inlined code] from ./c.jl:89                                                                                                                                                                                                   
 in #fetch#44(::Base.LibGit2.FetchOptions, ::String, ::Function, ::Base.LibGit2.GitRemote, ::Array{AbstractString,1}) at ./libgit2/remote.jl:70                                                                                  
 in #fetch#85(::String, ::String, ::Array{AbstractString,1}, ::Nullable{Base.LibGit2.CachedCredentials}, ::Function, ::Base.LibGit2.GitRepo) at ./libgit2.jl:159                                                                 
 in (::Base.Pkg.Entry.##31#34)(::Base.LibGit2.GitRepo) at ./pkg/entry.jl:390                                                                                                                                                     
 in with(::Base.Pkg.Entry.##31#34, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:600                                                                                                                                             
 in update(::String) at ./pkg/entry.jl:382
 [inlined code] from ./promotion.jl:248
 in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#update,Tuple{String}})() at ./pkg/dir.jl:31
 in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#update,Tuple{String}}, ::String) at ./file.jl:48
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{String,N}) at ./pkg/dir.jl:31
 [inlined code] from ./pkg/dir.jl:25
 in update() at ./pkg.jl:201
 in eval(::Module, ::Any) at ./boot.jl:226
 [inlined code] from ./REPL.jl:3
 in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:62
 [inlined code] from ./REPL.jl:92
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46

In the second failure it hanged, but in a different place than before. It's during "Computing changes...."

When I interrupted that I saw this rather uninformative trace:

INFO: Updating METADATA...
Username for 'https://github.com':Cody-G
Password for 'https://[email protected]':
INFO: Updating BoxTrees master...
INFO: Updating BlockRegistration master...
INFO: Updating TileTrees master...
WARNING: Cannot perform fast-forward merge.
INFO: Updating Optim teh/constrained...
INFO: Updating ImagePlayer master...
INFO: Updating BlockRegistrationScheduler master...
INFO: Updating GtkInteract master...
INFO: Updating CellSegmentation master...
INFO: Updating Requires master...
INFO: Updating Gtk master...
INFO: Computing changes...
^C^C^C^C^C^C^C^C^C^C^CWARNING: Force throwing a SIGINT
Segmentation fault (core dumped)

Cody-G · 2016-05-18T22:39:29Z

Just to make sure we're on the same page, here's my Julia version (the 11 commits are all yours):

Version 0.5.0-dev+4167 (2016-05-18 18:54 UTC)
credentials/eff19c9* (fork: 11 commits, 0 days)
x86_64-linux-gnu

wildart · 2016-05-19T05:29:54Z

The SIGINT segfault is more general issue then the package manager. It's a tricky one. I got one sometime ago, #11382, but could not reproduce it.

As for exceptions during Pkg.update, does your password contain Unicode chars? How did you manage to enter string with zeros in it, <W\0\0\0\0ͪ<?

Cody-G · 2016-05-19T15:24:55Z

I have been getting the SIGINT segfault most of the times that I have to interrupt Pkg.update(). Do you think I should re-open that issue?

As for my password, I checked and the problem occurs regardless of whether I have unicode characters or purely ascii characters. The string printed (<W\0\0\0\0ͪ<) has no obvious relationship with my password. Perhaps memory is getting corrupted somewhere and my password is turning to jibberish?

wildart · 2016-05-19T20:44:21Z

I found cause of the segfault, it's strings from getpass call. It looks like I incorrectly used them.

Cody-G · 2016-05-19T21:05:28Z

Great, I'm glad to hear you found it! Let me know when you want me to test it again.

wildart · 2016-05-19T21:22:20Z

@Cody-G Could you check latest update?

tkelman · 2016-05-19T21:36:12Z

What was the fix? Can we add a test that would have hit the problematic code path?

edit: if it was only getpass that needed to change then that would be tough to test on CI unfortunately

Cody-G · 2016-05-19T21:45:29Z

I think we're getting close. The very first time I ran Pkg.update it got stuck and I had to interrupt it. The segfault is gone, and the trace seems useful (see below). I updated about 10 more times and it worked properly every time after that.

INFO: Updating METADATA...
INFO: Updating cache of Plots...
Username for 'https://github.com':Cody-G
Password for 'https://[email protected]':


^C^C^C^C^C^C^C^C^CWARNING: Force throwing a SIGINT
INFO: Updating BlockRegistration master...
INFO: Updating TileTrees master...
WARNING: Cannot perform fast-forward merge.
INFO: Updating Optim teh/constrained...
INFO: Updating ImagePlayer master...
INFO: Updating BlockRegistrationScheduler master...
INFO: Updating GtkInteract master...
INFO: Updating CellSegmentation master...
INFO: Updating Requires master...
INFO: Updating Gtk master...
INFO: Computing changes...
INFO: Cloning cache of RecipesBase from https://github.com/JuliaPlots/RecipesBase.jl.git
INFO: Upgrading FixedSizeArrays: v0.1.1 => v0.1.2
INFO: Upgrading Plots: v0.5.4 => v0.6.2
INFO: Installing RecipesBase v0.0.1
ERROR: Update finished with errors.
=> Package BoxTrees cannot be updated.
InterruptException:
 in macro expansion at ./libgit2/error.jl:96 [inlined]
 in #fetch#44(::Base.LibGit2.FetchOptions, ::String, ::Function, ::Base.LibGit2.GitRemote, ::Array{AbstractString,1}) at ./libgit2/remote.jl:70
 in (::Base.LibGit2.#kw##fetch)(::Array{Any,1}, ::Base.LibGit2.#fetch, ::Base.LibGit2.GitRemote, ::Array{AbstractString,1}) at ./null:0
 in #fetch#85(::String, ::String, ::Array{AbstractString,1}, ::Nullable{Base.LibGit2.CachedCredentials}, ::Function, ::Base.LibGit2.GitRepo) at ./libgit2.jl:159
 in (::Base.LibGit2.#kw##fetch)(::Array{Any,1}, ::Base.LibGit2.#fetch, ::Base.LibGit2.GitRepo) at ./null:0
 in (::Base.Pkg.Entry.##31#34)(::Base.LibGit2.GitRepo) at ./pkg/entry.jl:390
 in with(::Base.Pkg.Entry.##31#34, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:600
 in update(::String) at ./pkg/entry.jl:382
 in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#update,Tuple{String}})() at ./pkg/dir.jl:31
 in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#update,Tuple{String}}, ::String) at ./file.jl:48
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{String,N}) at ./pkg/dir.jl:31
 in update() at ./pkg.jl:201
 in eval(::Module, ::Any) at ./boot.jl:225
 in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:62
 in macro expansion at ./REPL.jl:92 [inlined]
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46
 in update(::String) at ./pkg/entry.jl:422
 in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#update,Tuple{String}})() at ./pkg/dir.jl:31
 in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#update,Tuple{String}}, ::String) at ./file.jl:48
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{String,N}) at ./pkg/dir.jl:31
 in update() at ./pkg.jl:201
 in eval(::Module, ::Any) at ./boot.jl:225
 in macro expansion at ./REPL.jl:92 [inlined]
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46

Cody-G · 2016-05-19T21:58:55Z

I was able to trigger the same error a couple more times by waiting a few minutes between Pkg.update() calls.

wildart · 2016-05-19T23:08:01Z

I see, update doesn't stop after InterruptException.

felipenoris · 2016-07-21T12:16:32Z

I also got this today: in the latest 0.5 master, Pkg.update() hanged. It prompts for my Github username and password but doesn't respond after I enter them. I don't have any private repos, but I had a dirty package pointing to a forked repo that I deleted after merging a PR. After running Pkg.free() the problem was gone.

tkelman · 2016-07-22T07:16:00Z

try again on latest master, #17539 should fix this. if not fixed, leave a comment and we'll reopen.

tkelman added the packages Package management and loading label May 14, 2016

tkelman mentioned this issue May 18, 2016

LibGit2 credentials callback overhaul #16308

Merged

tkelman closed this as completed Jul 22, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pkg.update() hangs when updating private github repo #16373

Pkg.update() hangs when updating private github repo #16373

Cody-G commented May 14, 2016

wildart commented May 14, 2016

Cody-G commented May 15, 2016

Cody-G commented May 16, 2016 •

edited by wildart

Loading

wildart commented May 18, 2016

Cody-G commented May 18, 2016

wildart commented May 18, 2016

Cody-G commented May 18, 2016 •

edited by wildart

Loading

Cody-G commented May 18, 2016

wildart commented May 19, 2016 •

edited

Loading

Cody-G commented May 19, 2016 •

edited

Loading

wildart commented May 19, 2016

Cody-G commented May 19, 2016

wildart commented May 19, 2016

tkelman commented May 19, 2016 •

edited

Loading

Cody-G commented May 19, 2016

Cody-G commented May 19, 2016

wildart commented May 19, 2016

felipenoris commented Jul 21, 2016

tkelman commented Jul 22, 2016

Pkg.update() hangs when updating private github repo #16373

Pkg.update() hangs when updating private github repo #16373

Comments

Cody-G commented May 14, 2016

wildart commented May 14, 2016

Cody-G commented May 15, 2016

Cody-G commented May 16, 2016 • edited by wildart Loading

wildart commented May 18, 2016

Cody-G commented May 18, 2016

wildart commented May 18, 2016

Cody-G commented May 18, 2016 • edited by wildart Loading

Cody-G commented May 18, 2016

wildart commented May 19, 2016 • edited Loading

Cody-G commented May 19, 2016 • edited Loading

wildart commented May 19, 2016

Cody-G commented May 19, 2016

wildart commented May 19, 2016

tkelman commented May 19, 2016 • edited Loading

Cody-G commented May 19, 2016

Cody-G commented May 19, 2016

wildart commented May 19, 2016

felipenoris commented Jul 21, 2016

tkelman commented Jul 22, 2016

Cody-G commented May 16, 2016 •

edited by wildart

Loading

Cody-G commented May 18, 2016 •

edited by wildart

Loading

wildart commented May 19, 2016 •

edited

Loading

Cody-G commented May 19, 2016 •

edited

Loading

tkelman commented May 19, 2016 •

edited

Loading