What is the "source" field used for?

[josecelano]

I do not know what it is used for and I do not find official documentation about that field.

The field is included in the "info" dictionary, so it's included when calculating the torrent infohash.

Sample torrent:

{
   "announce": "http://localhost",
   "comment": "Comments",
   "created by": "qBittorrent v4.4.1",
   "creation date": 1690882922,
   "info": {
      "length": 172204,
      "name": "mandelbrot_2048x2048.png",
      "piece length": 16384,
      "pieces": "<hex>7D 91 71 0D 9D 4D BA 88 9B 54 20 54 D5 26 72 8D 5A 86 3F E1 21 DF 77 C7 F7 BB 6C 77 96 21 66 25 38 C5 D9 CD AB 8B 08 EF 8C 24 9B B2 F5 C4 CD 2A DF 0B C0 0C F0 AD DF 72 90 E5 B6 41 4C 23 6C 47 9B 8E 9F 46 AA 0C 0D 8E D1 97 FF EE 68 8B 5F 34 A3 87 D7 71 C5 A6 F9 8E 2E A6 31 7C BD F0 F9 E2 23 F9 CC 80 AF 54 00 04 F9 85 69 1C 77 89 C1 76 4E D6 AA BF 61 A6 C2 80 99 AB B6 5F 60 2F 40 A8 25 BE 32 A3 3D 9D 07 0C 79 68 98 D4 9D 63 49 AF 20 58 66 26 6F 98 6B 6D 32 34 CD 7D 08 15 5E 1A D0 00 09 57 AB 30 3B 20 60 C1 DC 12 87 D6 F3 E7 45 4F 70 67 09 36 31 55 F2 20 F6 6C A5 15 6F 2C 89 95 69 16 53 81 7D 31 F1 B6 BD 37 42 CC 11 0B B2 FC 2B 49 A5 85 B6 FC 76 74 44 93</hex>",
      "private": 1,
      "source": "Source"
   },
   "url-list": "http://localhost/mandelbrot_2048x2048.png"
}

Some BitTorrent clients include it (qBitorrent or this online torrent generator https://kimbatt.github.io/torrent-creator/), but others don't.

I have not found any BEP about it yet.

[josecelano]

@Kimbatt has pointed me to the solution here: Kimbatt/torrent-creator#10 (comment).

[josecelano]

I'm going to try to reply to my question, but It might be inexact because I have not found confirmations from high-reputation sources with a good detailed explanation.

The best explanation I've found so far it's this one by @2wicked:

#7965 (comment)

Private trackers that use this often fill in a three letter code specific for their tracker. Users that upload torrents can add it themselves (using dottorrent-gui for example) or the tracker will add it and the uploader needs to download the torrent with added tag from the tracker to start the seeding. Often people use different a locations for uploads than the default download location. So they will have to move files or edit download location and do a force recheck. If it would be implemented in the normal torrent creation process it would be much easier especially for novice users.

If if was not clear this is done to be able to cross-seed the same content between different private trackers. If no difference is made between the torrents the tracker addresses of the 2nd torrent will be added to the first and the seeders of both trackers will be mixed by the client.

Definitions

To remove the ambiguity, I will define a couple of terms:

• Tracker: it's an online service (using UDP or HTTP protocols) that can be used to find seeders for a given torrent. Peers announce themselves to the tracker periodically.
• Torrent Index: website where you can look for torrents. You can search and download the torrent file (metainfo file) or the magnet link.
• Cross-seeding: the same peer (BitTorrent client) could share the same content (torrent file) with several trackers in order to increase the torrent availability.

NOTICE: torrent indexes are often linked to a tracker, and sometimes people refer to both (index-tracker) simply as "tracker"·

More about cross-seeding (by https://www.perplexity.ai/)

Cross-seeding in BitTorrent refers to the practice of using the same downloaded content to seed multiple torrents from different trackers or torrent sites. It allows users to share the same file across different torrent communities, maximizing the availability and distribution of the content.

When cross-seeding, you typically download a torrent file from one tracker and then use that downloaded content to seed the same file on another tracker. This process helps to increase the number of seeders for a particular torrent, improving download speeds and availability for other users.

Cross-seeding can be beneficial in several ways:

1. Redundancy: By seeding the same content on multiple trackers, you ensure that the file remains available even if one tracker goes offline or becomes inaccessible.

2. Increased availability: More seeders mean a higher chance of finding peers and faster download speeds for users across different trackers.

3. Building reputation: By cross-seeding and maintaining a good seeding ratio on multiple trackers, users can build a positive reputation within the torrent community.

It's important to note that cross-seeding should be done within the rules and guidelines of each tracker. Some trackers may have specific rules regarding cross-seeding, so it's essential to familiarize yourself with the policies of each tracker before engaging in cross-seeding activities.
Overall, cross-seeding is a practice that allows users to share the same downloaded content across multiple trackers, increasing availability and improving download speeds for users in different torrent communities.

Source file in the info dictionary

An example torrent in a json format could be like this:

{
   "announce": "http://localhost",
   "comment": "Comments",
   "created by": "qBittorrent v4.4.1",
   "creation date": 1690882922,
   "info": {
      "length": 172204,
      "name": "mandelbrot_2048x2048.png",
      "piece length": 16384,
      "pieces": "<hex>7D 91 71 0D 9D 4D BA 88 9B 54 20 54 D5 26 72 8D 5A 86 3F E1 21 DF 77 C7 F7 BB 6C 77 96 21 66 25 38 C5 D9 CD AB 8B 08 EF 8C 24 9B B2 F5 C4 CD 2A DF 0B C0 0C F0 AD DF 72 90 E5 B6 41 4C 23 6C 47 9B 8E 9F 46 AA 0C 0D 8E D1 97 FF EE 68 8B 5F 34 A3 87 D7 71 C5 A6 F9 8E 2E A6 31 7C BD F0 F9 E2 23 F9 CC 80 AF 54 00 04 F9 85 69 1C 77 89 C1 76 4E D6 AA BF 61 A6 C2 80 99 AB B6 5F 60 2F 40 A8 25 BE 32 A3 3D 9D 07 0C 79 68 98 D4 9D 63 49 AF 20 58 66 26 6F 98 6B 6D 32 34 CD 7D 08 15 5E 1A D0 00 09 57 AB 30 3B 20 60 C1 DC 12 87 D6 F3 E7 45 4F 70 67 09 36 31 55 F2 20 F6 6C A5 15 6F 2C 89 95 69 16 53 81 7D 31 F1 B6 BD 37 42 CC 11 0B B2 FC 2B 49 A5 85 B6 FC 76 74 44 93</hex>",
      "private": 1,
      "source": "Source"
   },
   "url-list": "http://localhost/mandelbrot_2048x2048.png"
}

The info attribute is the info key dictionary. Torrent files are encoded using Bencode format. A dictionary is an object with multiple keys. The info dictionary for the previous torrent is:

{
      "length": 172204,
      "name": "mandelbrot_2048x2048.png",
      "piece length": 16384,
      "pieces": "<hex>7D 91 71 0D 9D 4D BA 88 9B 54 20 54 D5 26 72 8D 5A 86 3F E1 21 DF 77 C7 F7 BB 6C 77 96 21 66 25 38 C5 D9 CD AB 8B 08 EF 8C 24 9B B2 F5 C4 CD 2A DF 0B C0 0C F0 AD DF 72 90 E5 B6 41 4C 23 6C 47 9B 8E 9F 46 AA 0C 0D 8E D1 97 FF EE 68 8B 5F 34 A3 87 D7 71 C5 A6 F9 8E 2E A6 31 7C BD F0 F9 E2 23 F9 CC 80 AF 54 00 04 F9 85 69 1C 77 89 C1 76 4E D6 AA BF 61 A6 C2 80 99 AB B6 5F 60 2F 40 A8 25 BE 32 A3 3D 9D 07 0C 79 68 98 D4 9D 63 49 AF 20 58 66 26 6F 98 6B 6D 32 34 CD 7D 08 15 5E 1A D0 00 09 57 AB 30 3B 20 60 C1 DC 12 87 D6 F3 E7 45 4F 70 67 09 36 31 55 F2 20 F6 6C A5 15 6F 2C 89 95 69 16 53 81 7D 31 F1 B6 BD 37 42 CC 11 0B B2 FC 2B 49 A5 85 B6 FC 76 74 44 93</hex>",
      "private": 1,
      "source": "Source"
   }

Notice that some fields are common for single-file and multi-file torrents, but other files are exclusive. Some are also optional.

In the Torrust Tracker, we are using this struct for the torrent info dictionary:

pub struct TorrentInfo {
    pub name: String,
    pub pieces: Option<ByteBuf>,
    pub piece_length: i64,
    pub md5sum: Option<String>,           // single-file
    pub length: Option<i64>,              // single-file
    pub files: Option<Vec<TorrentFile>>,  // multi-file
    pub private: Option<u8>,
    pub path: Option<Vec<String>>,        // ???
    pub root_hash: Option<String>,        // BiTorrent specification v2 (BEP 52)
    pub source: Option<String>,
}

More info about the info dictionary:

• https://wiki.theory.org/BitTorrentSpecification#Info_Dictionary
• https://www.bittorrent.org/beps/bep_0003.html

The torrent info-hash is the sha1 hash of the info dictionary in encoded format.

NOTICE that encoded format forces the dictionary to be stored alphabetically.

Since the source field is included in the info part of the torrent file, that means when you change the source field value, you get a different torrent infohash.

Why was the source field added?

When you create a torrent, you can append a list of trackers that are tracking that torrent. For example, when you create a new torrent with qBittorrent you see:

It seems some private trackers impose some restrictions. For some reason, some private trackers want you to use only their tracker URL. When you upload a torrent to a index-tracker, the index-tracker can remove all tracker URLs and add only their URL. When you download your uploaded torrent you get a different infohash.

NOTICE: I guess the reason is they do not want to let it known to other trackers that they have the torrent indexed.

Sometimes you want to cross-seed (share the torrent with two trackers, two swarms, two download communities) the torrent while avoiding being banned from the index-trackers.

That means you want to:

• Have the same torrent twice, one with each tracker URL.
• Use the same content file for seeding to avoid file duplication on your disk and duplicate download.

Usually, BitTorrent clients do not allow you to have two torrents with the same content:

People have been solving in different ways:

• Opening more than one BitTorrent client. In this case, you have either duplicate files or you use symbolic links.
• Slightly changing one of the files in the info dictionary to generate a different infohash. FOr example the name of the file being seeded.

That leads to a lot of manual work, or you can use some tools:

• https://github.com/cross-seed/cross-seed
• https://github.com/ccf-2012/seedcross

Final answer

Some private trackers started adding the source field in order to generate a different infohash so that people can easily add duplicate torrent content (different hash, same content).

When you upload the torrent to the index-tracker, the track adds/changes the source value. It's commonly used a three-letter code representing the tracker like:

• GGn
• PTP
• BLU

Clients supporting the source field

As far as I know, clients mainly support it only by adding the field when you create the torrent:

• Need "source flag" functionality BiglySoftware/BiglyBT#1997
• [feature request] Add source flag/tag when creating torrents #7965

But they do not make it easier to cross-seed torrents. For example, in qBittorrent v4.5.4 if you try to add two torrents with different info-hashes but the same content, you will get a message like this:

I suppose this practice only helps private trackers to make sure people are using only their tracker URL in the torrent files they share. If the clients want to avoid downloading the same file again from the new tracker, they have to stop the download process and tell the client software to use the same location as the previous download. For the qBittorrent client, that's called force recheck

Open questions

• Does it make sense to use the source field with a public tracker?

cc @Zorlin @2wicked @Chocobo1 @da2ce7 @Kimbatt

[trim21]

yes, you are right. Source will become info.source field in torrent, it's for pt, and it will contribute to info hash.