Live Loader: Issue with dgraph's default addr #4019

MichelDiz · 2019-09-18T02:49:06Z

What version of Dgraph are you using?

1.1.0 (ibrahim:badger-update - but it's not related to that branch)

Have you tried reproducing the issue with the latest release?

Yes

What is the hardware spec (RAM, OS)?

Tested on Windows and Darwin

Steps to reproduce the issue (command/config used to run Dgraph).

Start live load with defaults settings. Or set the addr (flag -a)

Usage:
  dgraph live [flags]

Flags:
  -a, --alpha string   Comma-separated list of Dgraph alpha gRPC server 
addresses (default "127.0.0.1:9080")

This issue also happened in Dgo. As Live Loader uses Dgo, I think Dgo should be checked too.

The error logs

Running transaction with dgraph endpoint: 127.0.0.1:9080
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...

The text was updated successfully, but these errors were encountered:

ashish-goswami · 2019-09-18T15:47:37Z

Caused by #4020.
Closing this.

MichelDiz · 2019-09-18T17:32:07Z

This still happening on master - I've build it again in Darwin and tested.

➜  dgraph git:(master) git rev-parse HEAD
a5dbfc80e029c096c29024d2c0b2497fff265ffd
➜  dgraph git:(master) git describe
v1.1.0-45-ga5dbfc80

git commit => a5dbfc8

➜  master(18-09-19) dgraph live -f 21million.rdf.gz -s release.schema.txt                  
[Decoder]: Using assembly version of decoder
I0918 14:24:07.178355   18010 init.go:98] 

Dgraph version   : 
Dgraph SHA-256   : 22cac961c0e975b29f6f41f1255cf769eba0cca67879d8788dbcea7a929e0898
Commit SHA-1     : 
Commit timestamp : 
Branch           : 
Go version       : go1.12.9

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.



Running transaction with dgraph endpoint: 127.0.0.1:9080
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
^C

MichelDiz · 2019-09-19T16:48:15Z

I'll leave it opened to keep tracking it. And test it again after #4020 is solved.

MichelDiz · 2019-09-20T16:27:46Z

I did more tests around this

Same error with:
v1.1.0
don't work ➜ dgraph-darwin-amd64 ./dgraph live -f rdf.rdf -s schema.schema
works ➜ dgraph-darwin-amd64 ./dgraph live -f rdf.rdf -s schema.schema -a localhost:9080

v1.0.15
don't work ➜ Dgraph v1.0.15 ./dgraph live -r 21million.rdf.gz -s release.schema.txt
works ➜ ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080

v1.0.13
don't work ➜ Dgraph v1.0.13 ./dgraph live -r 21million.rdf.gz -s release.schema.txt
works ➜ ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080

v1.0.6
don't work ➜ v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema
works ➜ v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080

I do not understand why this happens, because 127.0.0.1 is Localhost on any operating system.

All the Logs

➜  dgraph-darwin-amd64 ./dgraph live -f rdf.rdf -s schema.schema          
[Decoder]: Using assembly version of decoder
I0920 12:56:58.569629   49642 init.go:98] 

Dgraph version   : v1.1.0
Dgraph SHA-256   : 98db2956f6dd8b7b9b88e02962d2036845b057fe5fe953190eaafac0a83dfcce
Commit SHA-1     : ef7cdb28
Commit timestamp : 2019-09-04 00:12:51 -0700
Branch           : HEAD
Go version       : go1.12.7

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.



Running transaction with dgraph endpoint: 127.0.0.1:9080
While trying to setup connection: context deadline exceeded. Retrying...
While trying to setup connection: context deadline exceeded. Retrying...
^C

Dgraph v1.0.15 ./dgraph live -r 21million.rdf.gz -s release.schema.txt                  
I0920 13:04:56.625526   49918 init.go:88] 

Dgraph version   : v1.0.15
Commit SHA-1     : ff5ee1e2
Commit timestamp : 2019-05-30 15:46:55 -0700
Branch           : HEAD
Go version       : go1.12.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x133341549
badger 2019/09/20 13:04:56 INFO: All 0 tables opened in 0s

Processing release.schema
Error while processing schema file "release.schema": rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>
badger 2019/09/20 13:04:56 INFO: Got compaction priority: {level:0 score:1.73 dropPrefix:[]}

./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080
I0920 13:05:38.323393   49940 init.go:88] 

Dgraph version   : v1.0.15
Commit SHA-1     : ff5ee1e2
Commit timestamp : 2019-05-30 15:46:55 -0700
Branch           : HEAD
Go version       : go1.12.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x573696955
badger 2019/09/20 13:05:38 INFO: All 0 tables opened in 0s

Processing release.schema.txt
Processed schema file "release.schema.txt"

Processing 21million.rdf.gz
[    2s] Txns: 103 RDFs: 103000 RDFs/sec: 49882 Aborts: 1
[    4s] Txns: 204 RDFs: 204000 RDFs/sec: 50149 Aborts: 2
[    6s] Txns: 284 RDFs: 284000 RDFs/sec: 46803 Aborts: 3

Dgraph v1.0.13 ./dgraph live -r 21million.rdf.gz -s release.schema.txt
I0920 13:08:30.621463   50037 init.go:88] 

Dgraph version   : v1.0.13
Commit SHA-1     : 691b3b35
Commit timestamp : 2019-03-09 19:33:59 -0800
Branch           : HEAD
Go version       : go1.11.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x764103728
badger 2019/09/20 13:08:30 INFO: All 0 tables opened in 0s

Processing release.schema
Error while processing schema file "release.schema": rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>
badger 2019/09/20 13:08:30 WARNING: While forcing compaction on level 0: Unable to fill tables
E0920 13:08:30.626790   50037 xidmap.go:120] Error while getting lease: rpc error: code = Unavailable desc = transport is closing

./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080
I0920 13:08:59.961881   50046 init.go:88] 

Dgraph version   : v1.0.13
Commit SHA-1     : 691b3b35
Commit timestamp : 2019-03-09 19:33:59 -0800
Branch           : HEAD
Go version       : go1.11.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x959542565
badger 2019/09/20 13:08:59 INFO: All 0 tables opened in 0s

Processing release.schema.txt
Processed schema file "release.schema.txt"

Processing 21million.rdf.gz
[    2s] Txns: 56 RDFs: 56000 RDFs/sec: 27866 Aborts: 0
[    4s] Txns: 125 RDFs: 125000 RDFs/sec: 31179 Aborts: 1
[    6s] Txns: 175 RDFs: 175000 RDFs/sec: 29123 Aborts: 2
[    8s] Txns: 216 RDFs: 216000 RDFs/sec: 26967 Aborts: 2
[   10s] Txns: 258 RDFs: 258000 RDFs/sec: 25776 Aborts: 3

➜  v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema                      
2019/09/20 13:14:43 run.go:337: Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x141447596

Processing release.schema
2019/09/20 13:14:43 rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>

➜  v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d localhost:9080    
2019/09/20 13:18:02 run.go:337: Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x524737574

Processing release.schema.txt
2019/09/20 13:18:02 run.go:354: Processed schema file

Processing 21million.rdf.gz
Total Txns done:       72 RDFs per second:   35360 Time Elapsed: 2s, Aborts: 25
Total Txns done:      131 RDFs per second:   32457 Time Elapsed: 4s, Aborts: 42
Total Txns done:      182 RDFs per second:   30152 Time Elapsed: 6s, Aborts: 66
Total Txns done:      237 RDFs per second:   29492 Time Elapsed: 8s, Aborts: 84

➜  v1.0.6 ./dgraph live -r 21million.rdf.gz -s release.schema.txt -d 127.0.0.1:9080 
2019/09/20 13:18:30 run.go:337: Creating temp client directory at /var/folders/x1/bz12z9ms4b7cx46yzptgb2fr0000gn/T/x764043649

Processing release.schema.txt
2019/09/20 13:18:30 rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: <nil>

danielmai · 2019-09-20T16:45:09Z

Seems like a machine config issue, @MichelDiz. I tried the same commands you shared on Mac and they work with the default --alpha flag value 127.0.0.1:9080.

with Dgraph v1.1.0:

dgraph live -f 21million.rdf.gz -s release.schema.txt

MichelDiz · 2019-09-20T17:51:25Z

I’ve created a VM with Windows 10 and tested it there. It works fine.

So it is indeed something else very odd. I can't figure out what is it. So in case, this issue happens to other users, please set the addr as localhost.

polymorphic · 2024-10-09T22:00:28Z

I ran into the same problem running self-hosted v24.0.2 on Sequoia. I solved it by running the live loader within the alpha container rather than from a new container; I had to add an additional CLI parameter specifying the zero node:

root@4b55410e4c5f:/dgraph# dgraph live -f data.rdf.gz -s schema.txt -a localhost:9080 -z zero:5080

I suspect the issue is caused by the way docker networking works on a Mac.

MichelDiz changed the title ~~Live Loader: Issue with dgraph's default addr (tested on Windows and Darwin)~~ Live Loader: Issue with dgraph's default addr Sep 18, 2019

MichelDiz added area/live-loader Issues related to live loading. kind/bug Something is broken. labels Sep 18, 2019

mangalaman93 assigned ashish-goswami Sep 18, 2019

mangalaman93 added priority/P0 Critical issue that requires immediate attention. status/accepted We accept to investigate/work on it. labels Sep 18, 2019

ashish-goswami closed this as completed Sep 18, 2019

MichelDiz reopened this Sep 19, 2019

MichelDiz closed this as completed Sep 20, 2019

MichelDiz mentioned this issue May 6, 2020

dgraph live fails on macOS Mojave #5378

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Live Loader: Issue with dgraph's default addr #4019

Live Loader: Issue with dgraph's default addr #4019

MichelDiz commented Sep 18, 2019 •

edited

Loading

ashish-goswami commented Sep 18, 2019

MichelDiz commented Sep 18, 2019 •

edited

Loading

MichelDiz commented Sep 19, 2019

MichelDiz commented Sep 20, 2019 •

edited

Loading

danielmai commented Sep 20, 2019

MichelDiz commented Sep 20, 2019 •

edited

Loading

polymorphic commented Oct 9, 2024

Live Loader: Issue with dgraph's default addr #4019

Live Loader: Issue with dgraph's default addr #4019

Comments

MichelDiz commented Sep 18, 2019 • edited Loading

What version of Dgraph are you using?

Have you tried reproducing the issue with the latest release?

What is the hardware spec (RAM, OS)?

Steps to reproduce the issue (command/config used to run Dgraph).

ashish-goswami commented Sep 18, 2019

MichelDiz commented Sep 18, 2019 • edited Loading

MichelDiz commented Sep 19, 2019

MichelDiz commented Sep 20, 2019 • edited Loading

All the Logs

danielmai commented Sep 20, 2019

MichelDiz commented Sep 20, 2019 • edited Loading

polymorphic commented Oct 9, 2024

MichelDiz commented Sep 18, 2019 •

edited

Loading

MichelDiz commented Sep 18, 2019 •

edited

Loading

MichelDiz commented Sep 20, 2019 •

edited

Loading

MichelDiz commented Sep 20, 2019 •

edited

Loading