How do we adjust the choking algorithm?

[sandeshbhatjr]

I use the bit torrent protocol to distribute data amongst different peers, all of whom I control. Is there any easy way to completely modify/disable the choking algorithm?

[anacrolix]

Yes, I believe there's an aggressive upload option or something, I'll follow up when I'm on a PC

[sandeshbhatjr]

@anacrolix So, I investigated a bit further, but I am not sure I understand the logic here. My mental model was that the writeBuffer is where you store the file data before you upload it. In this case, the writeBuffer is not filling up for some reason.

In the loop below, the writer starts off with elements in the writeBuffer (which is not empty in the beginning). It then swaps it with frontBuffer, and in the subsequent iteration the writeBuffer is empty. It remains empty after the first iteration, so basically the loop just waits until the timer runs out, and repeats with an empty writeBuffer.

https://github.com/anacrolix/torrent/blob/e10ffd133acfb476b1ebd69f9380a4e300ad75b9/peer-conn-msg-writer.go#L67C1-L91C4

[sandeshbhatjr]

On second thought, I think I got the logic wrong above. The upload function just adds to the write buffer via the sendChunk fn, right?

The problem, I think, is the loop over c.peerRequests in the upload fn; c.peerRequests is always empty. So nothing really happens in the upload fn. I will try to investigate why peerRequests is turning out empty. Please do correct me if I am wrong here.

[anacrolix]

Yeah I'm pretty sure the buffer swapping stuff works fine, but I could be wrong. If peerRequests is empty that would definitely explain the issues you are seeing.

[anacrolix]

I can't see anything that cause it. Do you have an upload rate limit set?

[sandeshbhatjr]

@anacrolix Yes, the buffer is fine. It's definitely the peer request being consistently empty. I used WriteStatus() method of the client to get info about the peers and their connections.

The peers connect and stay connected, however they do not send any requests for pieces. They send one unchoke message in the beginning and then no messages are sent after that. Maybe you can point me to the code where the logic for requesting pieces is?

I do not use any upload limit in my config.

Thanks again for the support!

[anacrolix]

Have you set download priorities on the torrent?

[sandeshbhatjr]

I don't think so. My settings are as follows with dataDir, port and remotes passed in as CLI arguments:

cfg := NewDefaultClientConfig()
cfg.NoDHT = true
cfg.DisableTrackers = true
cfg.DisablePEX = true
cfg.DisableAggressiveUpload = false
cfg.DataDir = dataDir
cfg.NoDefaultPortForwarding = true
cfg.PieceHashersPerTorrent = 10
cfg.Extensions = peer_protocol.NewPeerExtensionBytes(peer_protocol.ExtensionBitV2, peer_protocol.ExtensionBitFast)
// upload after all pieces are downloaded
cfg.Seed = true
cfg.AlwaysWantConns = true
cfg.AcceptPeerConnections = true
cfg.EstablishedConnsPerTorrent = 20
// upload completed pieces to while remaining pieces are downloading
cfg.DisableAggressiveUpload = false
cfg.ListenPort = int(port)
client, err := torrent.NewClient(&cfg)

I then just run it in a loop with a fixed peer list I call remotes (which is []netip.AddrPort) as follows:

t, _ := client.AddTorrentFromFile(torrentFilepath)
if err != nil {
	// do something
}
<-t.GotInfo() // blocking until torrent is added
// handle the case where the file is changed midway?
for {
	select {
	case <-ctx.Done():
		return
	default:
		// monitor status 
		if ((t.Stats().HalfOpenPeers == 0) && (t.Stats().PendingPeers == 0)) {
			for _, remoteNode := range remotes {
				// update peers regularly using peer mapper
				_ = t.AddPeers([]torrent.PeerInfo{
					{
						Source: torrent.PeerSourceDirect,
						Addr: remoteNode,
						Trusted: true,
					},
				})
			}
		}
		if t.BytesMissing() > 0 {
			t.DownloadAll()
		}
		time.Sleep(REFRESH_PEER_INTERVALS) // TODO: This is so hacky!
	}
}

[sandeshbhatjr]

@anacrolix Any luck figuring out the issue? Or am I doing something wrong with configurations?

[anacrolix]

The configuration looks fine.

Just to confirm, you are using the builtin upload/choking implementation, you haven't made any modifications? The issue is that peers won't start uploading until they've fully obtained a torrent.

I think this needs a test of say 10 peers, with a single initial seeder, all connected to each other. The goals would be to confirm that they request different pieces from the seeder and upload to one another throughout.

I have some tests in place already that do a seeder/leecher/leecher leecher arrangement like the one you described, there may be some where the leecher has a smaller cache than the entire size of the torrent. That should show similar behaviour to what you describe, but might not match exactly.

[anacrolix]

@sandeshbhatjr do you mind looking at #916? I believe it's related. I have added a test there, it should be similar to the problem you were encountering.

[anacrolix]

This should be fixed by 528c5f6.