Skip to content
This repository has been archived by the owner on Nov 13, 2018. It is now read-only.

Update Traffic Ops to use Daily Summary graph from InfluxDb/Grafana #812

Merged
merged 6 commits into from
Nov 23, 2015
Merged

Update Traffic Ops to use Daily Summary graph from InfluxDb/Grafana #812

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
376990f
Traffic Ops now uses daily graphs from InfluxDb/Grafana to show daily…
Nov 23, 2015
97b5e0b
Merge branch 'master' into dev
Nov 23, 2015
b2473d6
update docs to reflect new traffic ops params for daily graph
Nov 23, 2015
e76dc16
fix link to docs
Nov 23, 2015
50064e5
fixed malformed table
Nov 23, 2015
a32e220
tweak css to make values easier to read
Nov 23, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/source/admin/traffic_ops_using.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -293,11 +293,11 @@ The fields in the Delivery Service view are:
| Range Request Handling | (experimental) How to treat range requests: |
| | |
| | - 0 Do not cache (ranges requested from files taht are already cached due to a non range request will be a HIT) |
| | - 1 Use the `background_fetch <https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/background_fetch.en.html>`_ plugin. |
| | - 1 Use the `background_fetch <https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/background_fetch.en.html>`_ plugin. |
| | - 2 Use the cache_range_requests plugin. |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Delivery Service DNS TTL | The Time To Live on the DNS record for the Traffic Router A and AAAA records (``tr.<deliveryservice>.<cdn-domain>``) for a HTTP delivery service *or* for the A and |
| | AAAA records of the edge name (``edge.<deliveryservice>.<cdn-domain>``). |
| | AAAA records of the edge name (``edge.<deliveryservice>.<cdn-domain>``). |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Origin Server Base URL | The Origin Server's base URL. This includes the protocol (http or https). Example: ``http://movies.origin.com`` |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Expand All @@ -319,9 +319,9 @@ The fields in the Delivery Service view are:
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Mid Header Rewrite Rules | Header Rewrite rules to apply for this delivery service at the MID tier. See :ref:`rl-header-rewrite`. [1]_ |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Regex Remap Expression | Regex Remap rule to apply to this delivery service at the Edge tier. See `ATS documentation on regex_remap <https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/regex_remap.en.html>`_. [1]_ |
| Regex Remap Expression | Regex Remap rule to apply to this delivery service at the Edge tier. See `ATS documentation on regex_remap <https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/regex_remap.en.html>`_. [1]_ |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Cache URL expression | Cache URL rule to apply to this delivery service. See `ATS documentation on cacheurl <https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/cacheurl.en.html>`_. [1]_ |
| Cache URL expression | Cache URL rule to apply to this delivery service. See `ATS documentation on cacheurl <https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/cacheurl.en.html>`_. [1]_ |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Raw remap text | For HTTP and DNS deliveryservices, this will get added to the end of the remap line on the cache verbatim. For ANY_MAP deliveryservices this is the remap line. [1]_ |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Expand Down
6 changes: 5 additions & 1 deletion docs/source/admin/traffic_stats.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ Configuration

**Configuring Traffic Ops to use Grafana Dashboards**

To configure Traffic Ops to use Grafana Dashboards, you need to enter the following parameters and assign them to the GLOBAL profile. This assumes you followed the above instructions to install and configure InfluxDB and Grafana. You will need to place 'cdn-stats' and 'deliveryservice-stats' with the name of your dashboards.
To configure Traffic Ops to use Grafana Dashboards, you need to enter the following parameters and assign them to the GLOBAL profile. This assumes you followed the above instructions to install and configure InfluxDB and Grafana. You will need to place 'cdn-stats','deliveryservice-stats', and 'daily-summary' with the name of your dashboards.

+---------------------------+------------------------------------------------------------------------------------------------+
| parameter name | parameter value |
Expand All @@ -173,3 +173,7 @@ Configuration
+---------------------------+------------------------------------------------------------------------------------------------+
| visual_status_panel_2 | https://<grafanaHost>/dashboard/solo/db/cdn-stats?panelId=1&fullscreen&from=now-24h&to=now-60s |
+---------------------------+------------------------------------------------------------------------------------------------+
| daily_bw_url | https://<grafanaHost>/dashboard/solo/db/daily-summary?panelId=1&fullscreen&from=now-3y&to=now |
+---------------------------+------------------------------------------------------------------------------------------------+
| daily_served_url | https://<grafanaHost>/dashboard/solo/db/daily-summary?panelId=2&fullscreen&from=now-3y&to=now |
+---------------------------+------------------------------------------------------------------------------------------------+
90 changes: 59 additions & 31 deletions traffic_ops/app/lib/Extensions/TrafficStats/API/CacheStats.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,24 +96,33 @@ sub get_db_name {
return $conf->{cache_stats_db_name};
}

sub current_bandwidth {
sub get_stat {
my $self = shift;
my $cdn = $self->param('cdnName');
my $query = "SELECT sum(value)/6 FROM \"bandwidth\" WHERE time < now() - 60s and time > now() - 120s";
if ($cdn) {
$query = "SELECT sum(value)/6 FROM \"bandwidth\" WHERE time < now() - 60s and time > now() - 120s and cdn = \'$cdn\'";
}
my $response_container = $self->influxdb_query("cache_stats", $query);
my $database = shift;
my $query = shift;

my $response_container = $self->influxdb_query($database, $query);
my $response = $response_container->{'response'};
my $content = $response->{_content};
my $summary_content;
my $bandwidth = "err";

if ( $response->is_success() ) {
$summary_content = decode_json($content);
$bandwidth = $summary_content->{results}[0]{series}[0]{values}[0][1];
$bandwidth = $bandwidth/1000000;
return $summary_content->{results}[0]{series}[0]{values}[0][1];
}

return "err";
}

sub current_bandwidth {
my $self = shift;
my $cdn = $self->param('cdnName');
my $query = "SELECT sum(value)/6 FROM \"bandwidth\" WHERE time < now() - 60s and time > now() - 120s";
if ($cdn) {
$query = "SELECT sum(value)/6 FROM \"bandwidth\" WHERE time < now() - 60s and time > now() - 120s and cdn = \'$cdn\'";
}
return $self->success({"bandwidth" => $bandwidth});
my $bandwidth = $self->get_stat("cache_stats", $query);
return $self->success({"bandwidth" => $bandwidth/1000000});
}

sub current_connections {
Expand All @@ -123,15 +132,7 @@ sub current_connections {
if ($cdn) {
$query = "select sum(value)/6 from \"ats.proxy.process.http.current_client_connections\" where time > now() - 120s and time < now() - 60s and cdn = \'$cdn\'";
}
my $response_container = $self->influxdb_query("cache_stats", $query);
my $response = $response_container->{'response'};
my $content = $response->{_content};
my $summary_content;
my $connections = "err";
if ( $response->is_success() ) {
$summary_content = decode_json($content);
$connections = $summary_content->{results}[0]{series}[0]{values}[0][1];
}
my $connections = $self->get_stat("cache_stats", $query);
return $self->success({"connections" => $connections});
}

Expand All @@ -142,18 +143,45 @@ sub current_capacity {
if ($cdn) {
$query = "select sum(value)/6 from \"maxKbps\" where time > now() - 120s and time < now() - 60s and cdn = \'$cdn\'";
}
my $response_container = $self->influxdb_query("cache_stats", $query);
my $response = $response_container->{'response'};
my $content = $response->{_content};
my $summary_content;
my $capacity = "err";
if ( $response->is_success() ) {
$summary_content = decode_json($content);
$capacity = $summary_content->{results}[0]{series}[0]{values}[0][1];
$capacity = $capacity/1000000; #convert to Gbps
$capacity = $capacity * 0.85; # need a better way to figure out percentage of max besides hard-coding
}
my $capacity = $self->get_stat("cache_stats", $query);
$capacity = $capacity/1000000; #convert to Gbps
$capacity = $capacity * 0.85; # need a better way to figure out percentage of max besides hard-coding
return $self->success({"capacity" => $capacity});
}

sub daily_summary {
my $self = shift;
my $query = "";
my $database = "daily_stats";
my $total_bytesserved = 0;

my $daily_stats;
my @max_gbps;
my @pb_served;
#get cdns
my @cdn_names = $self->db->resultset('Server')->search({ 'type.name' => 'EDGE' }, { prefetch => [ 'cdn', 'type' ], group_by => 'cdn.name' } )->get_column('cdn.name')->all();
foreach my $cdn (@cdn_names) {
my $bytes_served;
my $max;
#get max bw
$max->{"cdn"} = $cdn;
$bytes_served->{"cdn"} = $cdn;
my $max_bw = $self->get_stat($database, "select max(value) from \"daily_maxgbps\" where cdn = \'$cdn\'");
$max->{"highest"} = $max_bw;
#get last bw
my $last_bw = $self->get_stat($database, "select last(value) from \"daily_maxgbps\" where cdn = \'$cdn\'");
$max->{"yesterday"} = $last_bw;
push(@max_gbps, $max);
#get bytesserved
my $bytesserved = $self->get_stat($database, "select sum(value) from \"daily_bytesserved\" where cdn = \'$cdn\'");
$bytes_served->{"bytesServed"} = $bytesserved/1000;
push(@pb_served, $bytes_served);
$total_bytesserved += $bytesserved;
}
push(@pb_served, ({cdn => "total", bytesServed => $total_bytesserved/1000}));
$daily_stats->{"maxGbps"} = \@max_gbps;
$daily_stats->{"petaBytesServed"} = \@pb_served;
return $self->success({%$daily_stats});
}

1;
1 change: 1 addition & 0 deletions traffic_ops/app/lib/TrafficOpsRoutes.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -775,6 +775,7 @@ sub traffic_stats_routes {
$r->get( "internal/api/$version/current_bandwidth" => [ format => [qw(json)] ] )->to( 'CacheStats#current_bandwidth', namespace => $namespace );
$r->get( "internal/api/$version/current_connections" => [ format => [qw(json)] ] )->to( 'CacheStats#current_connections', namespace => $namespace );
$r->get( "internal/api/$version/current_capacity" => [ format => [qw(json)] ] )->to( 'CacheStats#current_capacity', namespace => $namespace );
$r->get( "internal/api/$version/daily_summary" => [ format => [qw(json)] ] )->to( 'CacheStats#daily_summary', namespace => $namespace );
}

sub catch_all {
Expand Down
18 changes: 12 additions & 6 deletions traffic_ops/app/lib/UI/VisualStatus.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,15 +92,21 @@ sub graphs_redis {
sub daily_summary {
my $self = shift;

my @cdn_names = $self->db->resultset('Server')->search({ 'type.name' => 'EDGE' }, { prefetch => [ 'cdn', 'type' ], group_by => 'cdn.name' } )->get_column('cdn.name')->all();
my $pparam =
$self->db->resultset('ProfileParameter')
->search( { -and => [ 'parameter.name' => 'daily_bw_url', 'profile.name' => 'GLOBAL' ] }, { prefetch => [ 'parameter', 'profile' ] } )->single();
my $bw_url = defined($pparam) ? $pparam->parameter->value : undef;
$pparam =
$self->db->resultset('ProfileParameter')
->search( { -and => [ 'parameter.name' => 'daily_served_url', 'profile.name' => 'GLOBAL' ] }, { prefetch => [ 'parameter', 'profile' ] } )->single();
my $served_url = defined($pparam) ? $pparam->parameter->value : undef;

my $tool_instance =
$self->db->resultset('Parameter')->search( { -and => [ name => 'tm.instance_name', config_file => 'global' ] } )->get_column('value')->single();
my @cdn_names = $self->db->resultset('Server')->search({ 'type.name' => 'EDGE' }, { prefetch => [ 'cdn', 'type' ], group_by => 'cdn.name' } )->get_column('cdn.name')->all();

$self->stash(
cdn_names => \@cdn_names,
tool_instance => $tool_instance,
graph_page => 1,
daily_bw_url => $bw_url,
daily_served_url => $served_url,
cdn_names => \@cdn_names
);

&navbarpage($self);
Expand Down
Loading