Webseed in https not working

[gatisahu]

HTTP webseed is working fine.
HTTPs webseed sending request to webseed server and receiving the chunk but it is not progressing .

we are using response.addHeader("Accept-Ranges", "bytes");

After debugging it seems
some peice the pieceHashed coming false
https://github.com/anacrolix/torrent/blob/master/torrent.go#L2379
While checking I found comparing sha1 hash it is not matching .
Not sure how https playing a role here

tls config

tlsConfig := &tls.Config{
RootCAs: caCertPool,
InsecureSkipVerify: false, // Verify server's certificate
MinVersion: tls.VersionTLS12,
}

cfg.WebTransport = &http.Transport{
TLSClientConfig: tlsConfig,
}

full status

Torrents: 1

p0.51297892-el8.parcel
0.017206% of 13997539212 bytes (14 GB)
Infohash: ea34737fb4f35c2f5d5161a911f45befa1cb1d3a
Metadata length: 33495
Piece length: 8388608 (512 chunks)
Num Pieces: 1669 (0 completed)
Piece States: 1. 1.P 5. 1.P 1661.
Piece availability frequency: 1: 1669
Reader Pieces:
Enabled trackers:
URL Extra
DHT Announces: 0
(torrent.TorrentStats) {
ConnStats: (torrent.ConnStats) {
BytesWritten: (torrent.Count) 1545,
BytesWrittenData: (torrent.Count) 0,
BytesRead: (torrent.Count) 1545,
BytesReadData: (torrent.Count) 0,
BytesReadUsefulData: (torrent.Count) 0,
BytesReadUsefulIntendedData: (torrent.Count) 0,
ChunksWritten: (torrent.Count) 0,
ChunksRead: (torrent.Count) 2672275,
ChunksReadUseful: (torrent.Count) 2672275,
ChunksReadWasted: (torrent.Count) 0,
MetadataChunksRead: (torrent.Count) 0,
PiecesDirtiedGood: (torrent.Count) 0,
PiecesDirtiedBad: (torrent.Count) 0
},
TotalPeers: (int) 0,
PendingPeers: (int) 0,
ActivePeers: (int) 0,
ConnectedSeeders: (int) 0,
HalfOpenPeers: (int) 0,
PiecesComplete: (int) 0
}
webseeds:

• https://ccycloud-1.gsahu.root.comops.site:7183/cmf/parcel/download/p0.51297892-el8.parcel
  last unhandled error: never
  bep40-prio: fa2f2406
  last msg: never, connected: never, last helpful: 15.94s ago, itime: 38m36.287724526s, etime: 38m28.713064362s
  1669/1669 completed, 4 pieces touched, good chunks: 2672275/2672275:0 reqq: 108+0/(128/128):0/1024, flags: i:WS:, dr: 0.0 KiB/s
  requested pieces: 7(108)

[gatisahu]

This is not specific to https issue. When I am using v1.56.0 I am seeing below error

banning webseed peer for "https://######/parcel/download/some.parcel" for being sole dirtier of piece 6 after failed piece check [ github.com/anacrolix/torrent torrent.go:2458 ]

I have tried to run master I don't see the same issue. Still I see when there is a gap the webseed don't download.
just for testing I have added ws.peer.updateRequests("onSetInfo") in addWebSeed method and it is working .
I think when request order happen it call webseed and if webseed is added after this then webseed don't do anything .

[anacrolix]

See #964 (comment).

[gatisahu]

The code is wait on ws.requesterCond.Wait() .Even if signal to wake up ws.peer.requestState.Requests will be null ,so it will not go inside critical section.
https://github.com/anacrolix/torrent/blob/master/webseed-peer.go#L97

On torrent during addwebseed adding ws.peer.updateRequests("onSetInfo") is creating request and working fine .
This will cause any regression ?

[anacrolix]

This should be fixed in 3b0b61f.

[gatisahu]

Thanks for fixing . Now webseed is working fine in delayed case.Now it is breaking the normal use case where I have tested with 5 nodes and adding webseed after every 5 secs.
Without the code changes it is working fine.

We add add peer every 90 secs.

With the changes from 5 nodes 3 nodes only download from webseed and not exchange message through peer .Two did not start after restart it start again . I will test little bigger cluster and update .

seeing message in log
2024-08-27T08:31:09.516Z debug log/logger.go:157 error receiving handshakes on *torrent.PeerConn 0xc00061ea88 [flags=I,U,v1 id="\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", exts=0000000000000000 (), v=%!q()]: EOF [ github.com/anacrolix/torrent client.go:1066 ]

full status
Peer ID: "-GT0003-rhyL\xdau\xdc\x1ef\xe7'-"
Extension bits: 0000000000100005 (ltep, fast, dht)
Announce key: 66e7272d
Banned IPs: 0
(torrent.ClientStats) {
ConnStats: (torrent.ConnStats) {
BytesWritten: (torrent.Count) 123616,
BytesWrittenData: (torrent.Count) 0,
BytesRead: (torrent.Count) 8405544849,
BytesReadData: (torrent.Count) 8405385216,
BytesReadUsefulData: (torrent.Count) 8405385216,
BytesReadUsefulIntendedData: (torrent.Count) 8405385216,
ChunksWritten: (torrent.Count) 0,
ChunksRead: (torrent.Count) 1002,
ChunksReadUseful: (torrent.Count) 1002,
ChunksReadWasted: (torrent.Count) 0,
MetadataChunksRead: (torrent.Count) 0,
PiecesDirtiedGood: (torrent.Count) 1002,
PiecesDirtiedBad: (torrent.Count) 0
},
ActiveHalfOpenAttempts: (int) 0,
NumPeersUndialableWithoutHolepunch: (int) 0,
NumPeersUndialableWithoutHolepunchDialedAfterHolepunchConnect: (int) 0,
NumPeersDialableOnlyAfterHolepunch: (int) 0,
NumPeersDialedSuccessfullyAfterHolepunchConnect: (int) 0,
NumPeersProbablyOnlyConnectedDueToHolepunch: (int) 0
}

Torrents: 1

CDH-7.3.1-1.cdh7.3.1.p0.56439301-el8.parcel
59.563233% of 14111700715 bytes (14 GB)
Infohash: 58f1a52e8c0315a8a2cce6ea8b755a1fcb1a214c
Metadata length: 33773
Piece length: 8388608 (0.0625 chunks)
Num Pieces: 1683 (1002 completed)
Piece States: 918C 30. 84C 651.
Piece availability frequency: 5: 1683
Reader Pieces:
Enabled trackers:
URL Extra
DHT Announces: 0
(torrent.TorrentStats) {
ConnStats: (torrent.ConnStats) {
BytesWritten: (torrent.Count) 123616,
BytesWrittenData: (torrent.Count) 0,
BytesRead: (torrent.Count) 8405544849,
BytesReadData: (torrent.Count) 8405385216,
BytesReadUsefulData: (torrent.Count) 8405385216,
BytesReadUsefulIntendedData: (torrent.Count) 8405385216,
ChunksWritten: (torrent.Count) 0,
ChunksRead: (torrent.Count) 1002,
ChunksReadUseful: (torrent.Count) 1002,
ChunksReadWasted: (torrent.Count) 0,
MetadataChunksRead: (torrent.Count) 0,
PiecesDirtiedGood: (torrent.Count) 1002,
PiecesDirtiedBad: (torrent.Count) 0
},
TotalPeers: (int) 4,
PendingPeers: (int) 0,
ActivePeers: (int) 4,
ConnectedSeeders: (int) 4,
HalfOpenPeers: (int) 0,
PiecesComplete: (int) 1002
}
webseeds:

• http://ccycloud-1.quasar-gzqwpo.root.comops.site:7180/cmf/parcel/download/CDH-7.3.1-1.cdh7.3.1.p0.56439301-el8.parcel
  last unhandled error: never
  bep40-prio: e97fd7f2
  last msg: never, connected: never, last helpful: 380.82s ago, itime: 18m47.653748946s, etime: 20.623776161s
  1683/1683 completed, 0 pieces touched, good chunks: 1002/1002:0 reqq: 0+0/(1/128):0/1024, flags: i:WS:, dr: 398005.9 KiB/s
  requested pieces:
  4 peer conns:
• 10.140.34.4:53220-10.140.17.130:7191
  peer id: "-GT0003-p\xaa\xea\xf9E\xc4\u01a7\u0633H\xfb"
  extensions: 0000000000100005 (ltep, fast, dht)
  ltep extensions: map[ut_holepunch:2 ut_metadata:1 ut_pex:3]
  pex: 4 conns, 0 unsent events
  bep40-prio: c4c47687
  last msg: 12.65s ago, connected: 1112.69s ago, last helpful: never, itime: 15m26.277562491s, etime: 15m26.277538909s
  1683/1683 completed, 0 pieces touched, good chunks: 0/0:0 reqq: 2+0/(1/1024):0/1024, flags: i:M,e,v1:i, dr: 0.0 KiB/s
  requested pieces: 1270(1) 1280(1)
• 10.140.34.4:41280-10.140.44.3:7191
  peer id: "-GT0003-r\rl\xf6\xf1\x13\xadz\xb5\x95\xaa\x12"
  extensions: 0000000000100005 (ltep, fast, dht)
  ltep extensions: map[ut_holepunch:2 ut_metadata:1 ut_pex:3]
  pex: 4 conns, 0 unsent events
  bep40-prio: 5442fdae
  last msg: 12.65s ago, connected: 1112.69s ago, last helpful: never, itime: 18m32.687891133s, etime: 4m4.358652272s
  1683/1683 completed, 0 pieces touched, good chunks: 0/0:0 reqq: 0+0/(1/1024):0/1024, flags: i:M,e,v1:, dr: 0.0 KiB/s
  requested pieces:
• 10.140.34.4:7191-10.140.53.14:33406
  peer id: "-GT0003-\xb3\xacsv\xd0\x17\x9ep-\xff\v\"
  extensions: 0000000000100005 (ltep, fast, dht)
  ltep extensions: map[ut_holepunch:2 ut_metadata:1 ut_pex:3]
  pex: 4 conns, 0 unsent events
  bep40-prio: 4dd48044
  last msg: 36.32s ago, connected: 364.83s ago, last helpful: never, itime: 6m3.780153238s, etime: 6m3.780137646s
  1683/1683 completed, 0 pieces touched, good chunks: 0/0:0 reqq: 2+0/(1/1024):0/1024, flags: i:I,e,v1:i, dr: 0.0 KiB/s
  requested pieces: 1404(1) 1405(1)
• 10.140.34.4:51946-10.140.50.129:7191
  peer id: "-GT0003--^k\xf5\x0e\xa0N\xaf\xcd7\xc9\xf6"
  extensions: 0000000000100005 (ltep, fast, dht)
  ltep extensions: map[ut_holepunch:2 ut_metadata:1 ut_pex:3]
  pex: 4 conns, 0 unsent events
  bep40-prio: 250d6df3
  last msg: 12.65s ago, connected: 1112.69s ago, last helpful: never, itime: 15m37.096558533s, etime: 15m37.096530863s
  1683/1683 completed, 0 pieces touched, good chunks: 0/0:0 reqq: 2+0/(1/1024):0/1024, flags: i:M,e,v1:i, dr: 0.0 KiB/s
  requested pieces: 938(1) 933(1)

[anacrolix]

It looks like connectivity issues because you're downloading so fast!

[gatisahu]

Webseed I have increased the chunk size 16 kb to 1 mb . You are suspecting it is choking network bandwidth of host downloading from webseed ?

[anacrolix]

Yes

[gatisahu]

Thanks it is working fine .Now default chunk size is 16 kb which is very low. Is there a specification or recommended chunk size for webseed

[anacrolix]

It's recommended to leave the chunk size at the default of 16 KiB and only increase if you are seeing significant overhead or running your own swarm.

[gatisahu]

After decreasing it is working fine .Now I am testing a larger cluster in aws where there is multiple AZ so Our logic is every rac will have a rac leader and download from webseed .So is there way I can only increase the chunk size of webseed only do at least download 1% of file in one http request else the webseed server will rate limit it .If you can provide some code pointer and your thought I can try the implementation and verify in aws .

[anacrolix]

It's not exposed as a config option, but you would want to modify this

torrent/torrent.go

Line 2860 in dcfb61f

const maxRequests = 16

And increase chunk size 😄