Overhaul core Tracker: refactor statistics module

[josecelano]

Parent issue: #1181
Relates to: #753

The recently extracted tracker-core package contains a statistics module:

tree packages/tracker-core/src/statistics/
packages/tracker-core/src/statistics/
├── event
│   ├── handler.rs
│   ├── listener.rs
│   ├── mod.rs
│   └── sender.rs
├── keeper.rs
├── metrics.rs
├── mod.rs
├── repository.rs
├── services.rs
└── setup.rs

2 directories, 10 files

The module was already there (inside the core module) when I extracted the new package. However, now that the code is better organized I've realized that the core tracker doesn't send any stats event and all events are related to the UDP or HTTP trackers. That means the logic belongs to the higher level layer.

The Torrust Tracker application has these layers:

                   Main Torrust Tracker
                              |
              Axum HTTP tracker server (`packages\axum-http-tracker`. This hasn't been extracted yet)
                              |
                 HTTP tracker protocol (`packages\http-protocol`)
                              |
                          Core tracker (`packages\tracker-core`)

• Axum HTTP tracker server layer: the delivery layer. The public layer the application uses to interact with users.
• HTTP tracker protocol: logic that could be re-used even if we move to a different HTTP framework in the future or if we decide to have different implementations like for example an ActixWeb HTTP tracker server. In fact, there is a Torrust's fork using ActixWeb. Having these two independent layers will allow us in the future to easily move to new frameworks. Frameworks change faster and more often than BitTorrent protocols.

The UDP tracker should have the same layers:

                   Main Torrust Tracker
                              |
             Custom UDP tracker server (`packages\udp-tracker`. This hasn't been extracted yet)
                              |
                 UDP tracker protocol  (`packages\udp-protocol`. This hasn't been extracted yet)
                              |
                          Core tracker (`packages\tracker-core`)

However, for the UDP tracker, at the level of UDP tracker protocol we are also using the aquatic UDP protocol.

I'm going to move the module back to the main lib. I also see the opportunity to split it into two separate modules:

• Statistics for UDP tracker
• Statistics for HTTP tracker

I will basically clone the code and move a copy to each server and finally remove the metrics that don't belong to the module. The final goal is let each type of tracker (UDP or HTTP) handle their or stats.

NOTE: I will not change the API endpoint exposing the metrics. That endpoint will remain the same:

https://tracker.torrust-demo.com/api/v1/stats

{
  "torrents": 331388,
  "seeders": 124488,
  "completed": 17472,
  "leechers": 250249,
  "tcp4_connections_handled": 14,
  "tcp4_announces_handled": 14,
  "tcp4_scrapes_handled": 0,
  "tcp6_connections_handled": 0,
  "tcp6_announces_handled": 0,
  "tcp6_scrapes_handled": 0,
  "udp_requests_aborted": 60222,
  "udp_requests_banned": 7219182,
  "udp_banned_ips_total": 14889,
  "udp_avg_connect_processing_time_ns": 187574,
  "udp_avg_announce_processing_time_ns": 5557430,
  "udp_avg_scrape_processing_time_ns": 188447,
  "udp4_requests": 40682829,
  "udp4_connections_handled": 6475337,
  "udp4_announces_handled": 24841768,
  "udp4_scrapes_handled": 590641,
  "udp4_responses": 33462658,
  "udp4_errors_handled": 1541103,
  "udp6_requests": 0,
  "udp6_connections_handled": 0,
  "udp6_announces_handled": 0,
  "udp6_scrapes_handled": 0,
  "udp6_responses": 0,
  "udp6_errors_handled": 0
}

In the future I will like to change it to this:

{
  "torrent_metrics": {
    "torrents": 331388,
    "seeders": 124488,
    "completed": 17472,
    "leechers": 250249,
  }
  "http_tracker_metrics": {
    "tcp4_connections_handled": 14,
    "tcp4_announces_handled": 14,
    "tcp4_scrapes_handled": 0,
    "tcp6_connections_handled": 0,
    "tcp6_announces_handled": 0,
    "tcp6_scrapes_handled": 0,
  }
  "udp_tracker_metrics": {
    "requests_aborted": 60222,
    "requests_banned": 7219182,
    "banned_ips_total": 14889,
    "avg_connect_processing_time_ns": 187574,
    "avg_announce_processing_time_ns": 5557430,
    "avg_scrape_processing_time_ns": 188447,
    "udp4": {
      "requests": 40682829,
      "connections_handled": 6475337,
      "announces_handled": 24841768,
      "scrapes_handled": 590641,
      "responses": 33462658,
      "errors_handled": 1541103,
    }
    "udp6": {
      "requests": 0,
      "connections_handled": 0,
      "announces_handled": 0,
      "scrapes_handled": 0,
      "responses": 0,
      "errors_handled": 0
    }
}

And also rename the metrics to follow Prometheus recommendations.

cc @da2ce7

[josecelano]

Hi @da2ce7, I've finally introduced a new layer in the app architecture because I didn't fine a good place to put the stats:

HTTP tracker:

                   Main Torrust Tracker
                              |
              Axum HTTP tracker server (`packages\axum-http-tracker`. This hasn't been extracted yet)
                              |
                    HTTP tracker core  (`src\packages\http_tracker_core`)
                              |
                 HTTP tracker protocol (`packages\http-protocol`)
                              |
                          Core tracker (`packages\tracker-core`)

• Axum HTTP tracker server: the top layer that exposes the user's service with a framework. Basically Axum stuff. It could be another framework in the future.
• HTTP tracker core: extra logic (like statistics) that might be useful for other implementations regardless what HTTP frameworks you are using.
• HTTP tracker protocol: mandatory logic needed by any HTTP tracker implementation (official specification)
• Core tracker: announce and scrape logic without delivery mechanism.

Same for the UDP tracker.

[josecelano]

I've also created a new temporary dir (src/packages) with modules I will extract intro crates (/packages).

[josecelano]

I've also created sub-issues for extracting the packages in #753