Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For Code Review: All files #8

Closed
wants to merge 106 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
06753d6
set up project structure
huyan0 Jul 30, 2020
287a4e5
placeholder text for README
huyan0 Jul 30, 2020
c7e2a42
finish Test_validateMetrics
huyan0 Jul 30, 2020
0f6033f
finish Test_validateMetrics
huyan0 Jul 30, 2020
0686608
finish Test_validateMetrics
huyan0 Jul 30, 2020
a896535
remove extra import
huyan0 Jul 30, 2020
6335403
fix format error so that make runs
huyan0 Jul 30, 2020
f76b22a
update unit tests
huyan0 Jul 30, 2020
91dddb9
Testbed for the config and factory constructions:
Jul 30, 2020
66224aa
Testbed for the config and factory constructions, possible README.md …
Jul 30, 2020
2e4b1e1
Testbed oop
Jul 30, 2020
01441e7
add metrictest util code
huyan0 Jul 31, 2020
a9eaaec
tempeorary code
huyan0 Jul 31, 2020
7cf119b
finish testutil and Test_addSample
huyan0 Jul 31, 2020
5625330
add more unit tests
huyan0 Aug 1, 2020
cbd7f6e
add unit tests
huyan0 Aug 2, 2020
a4d7eeb
use assert.Exactly
huyan0 Aug 2, 2020
ae1758c
finish Test_handleHistogramMetric
huyan0 Aug 2, 2020
93df3e9
fix Test_handleScalarMetric to include metric name in label set
huyan0 Aug 2, 2020
346fce0
finish Test_handleSummaryMetric
huyan0 Aug 2, 2020
d215b33
add Test_shutdown
huyan0 Aug 2, 2020
b001c18
add unit test for functions
huyan0 Aug 2, 2020
64aad65
add unit tests for cortex.go
huyan0 Aug 3, 2020
b55576e
associated methods with cortexExporter
huyan0 Aug 3, 2020
ad4f1b1
some more config and factory stuff:
Aug 3, 2020
5806a51
M branch 'unit-tests' of https://github.com/open-o11y/opentelemetry-c…
Aug 3, 2020
d670578
config and some factory, export skeleton
Aug 3, 2020
d125a28
config and some factory, export skeleton
Aug 3, 2020
34b40fe
add nil test cases for cortex.go
huyan0 Aug 3, 2020
32a9db9
implement and test validateMetrics and addSample
huyan0 Aug 3, 2020
c466bd9
implement and test timeSeriesSignature
huyan0 Aug 3, 2020
c7a0dc1
implement and test createLabelSet
huyan0 Aug 3, 2020
b978fd4
finish createMetricsExporter method
huyan0 Aug 3, 2020
3992ea7
work
Aug 3, 2020
01ad87a
factory
Aug 3, 2020
da4dbcc
no more const labels for config
Aug 3, 2020
0312ced
implement and test handleScalarMetric
huyan0 Aug 3, 2020
d868fd9
implement and test handleScalarMetric
huyan0 Aug 3, 2020
cadd3bf
Update factory.go
huyan0 Aug 3, 2020
184e16a
implement and test handleHistogramMetric
huyan0 Aug 3, 2020
d0d700b
Merge branch 'unit-tests' of github.com:open-o11y/opentelemetry-colle…
huyan0 Aug 3, 2020
a133149
implement and test handleHistogramMetric
huyan0 Aug 3, 2020
222ea87
implement and test handleSummaryMetric
huyan0 Aug 3, 2020
c7ce3c6
implement shutdown
huyan0 Aug 3, 2020
a4f09ab
opt headers for config
Aug 3, 2020
100d482
Mergeae branch 'unit-tests' of https://github.com/open-o11y/opentelem…
Aug 3, 2020
ca1295e
undupped create Metrics Exporter
Aug 3, 2020
4bc9a4f
test shutdown
huyan0 Aug 3, 2020
db605f2
Merge branch 'unit-tests' of github.com:open-o11y/opentelemetry-colle…
huyan0 Aug 3, 2020
4a29b98
yes
Aug 3, 2020
c4d8d47
okay, worksMerge branch 'unit-tests' of https://github.com/open-o11y/…
Aug 3, 2020
6073ce3
remove usage of getSample in cortex.go
huyan0 Aug 3, 2020
3c12ab7
Merge branch 'unit-tests' of github.com:open-o11y/opentelemetry-colle…
huyan0 Aug 3, 2020
37db943
implement and test newMetricsExporter
huyan0 Aug 4, 2020
e9ac299
Test_pushMetrics not working
huyan0 Aug 4, 2020
53fbe2b
add comments to and format cortex.go
huyan0 Aug 4, 2020
96807eb
make cortex_test readable
huyan0 Aug 4, 2020
4182996
modify things in factory.go
huyan0 Aug 4, 2020
8ac6b25
update TODOs in cortex_test
huyan0 Aug 4, 2020
9a69f18
updated go.mod
huyan0 Aug 4, 2020
cf8fcaf
add headers field to exporter struct
huyan0 Aug 4, 2020
f223b37
fix config_test
huyan0 Aug 4, 2020
3371a59
add todos
huyan0 Aug 4, 2020
2cb879a
create helper.go and helper_test.go
huyan0 Aug 4, 2020
071b82d
add cortex.go and helper.go with tests
huyan0 Aug 4, 2020
fda582b
fix cortex_test.go
huyan0 Aug 4, 2020
05bcd9d
add comments to tests and address style issues
huyan0 Aug 5, 2020
4042156
add DO NOT REVIEW message in certain files
huyan0 Aug 5, 2020
cc12828
added export and test export and helpers, will comment and annotate t…
Aug 5, 2020
abc6339
added export and test export and helpers, will comment and annotate t…
Aug 5, 2020
af91626
added all the files, should we review them all?
Aug 5, 2020
2e8d244
added all files for prelim review
Aug 5, 2020
f65d4ae
update README
huyan0 Aug 5, 2020
9abf701
add more comments and format
huyan0 Aug 5, 2020
48ab3ec
remove DO NOT REVIEW in facotry.go
huyan0 Aug 5, 2020
6686e24
change Export to export
huyan0 Aug 5, 2020
35f7dbf
change format and add comments
huyan0 Aug 5, 2020
6459a97
fix import statement
huyan0 Aug 5, 2020
a3a7d48
fix import format
huyan0 Aug 5, 2020
79f0c1b
fix import format
huyan0 Aug 5, 2020
2fe4cb3
fix case error in config_test.go
huyan0 Aug 5, 2020
310af0f
fixed test bugs
Aug 5, 2020
29ebc9f
Add link to Cortex specification
huyan0 Aug 5, 2020
03eb0ae
Update config.go
huyan0 Aug 5, 2020
6f3e9b7
fix Test_pushMetrics error
huyan0 Aug 5, 2020
c30c7a3
Update README.md
huyan0 Aug 5, 2020
c029ba7
add extra line at the end
huyan0 Aug 5, 2020
ef86ae4
add extra line
huyan0 Aug 5, 2020
cb0009b
add blank line at the end of the file
huyan0 Aug 5, 2020
b0135df
fix merge conflict
huyan0 Aug 6, 2020
6059ee6
update README.md
huyan0 Aug 6, 2020
4ae8d41
rename testutil.go to testutil_test.go
huyan0 Aug 6, 2020
4867718
address Wesley's comments
huyan0 Aug 6, 2020
b9a56a1
address Eric's comments
huyan0 Aug 6, 2020
e173d35
address EdZou's comments
huyan0 Aug 6, 2020
7db4e71
add histogram test to Test_pushMetrics
huyan0 Aug 6, 2020
cfeefe6
rename lbls to labels
huyan0 Aug 6, 2020
0a72c51
improve code coverage and build success
huyan0 Aug 7, 2020
9ca9300
improve code coverage and build success
huyan0 Aug 7, 2020
7b41ecd
3am reformatting, very minor, more to come tomorrow before the PR hehe
Aug 7, 2020
7e66b32
rename to prometheus remote write exporter
huyan0 Aug 7, 2020
5f7b486
change name to Prometheus remote write
huyan0 Aug 7, 2020
71ded56
update README.md
huyan0 Aug 7, 2020
95938e8
another round of lookovers
Aug 7, 2020
f0ddcae
add comments
Aug 7, 2020
16553e0
yay
Aug 7, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions exporter/prometheusremotewriteexporter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
This Exporter sends metrics data in Prometheus TimeSeries format to any Prometheus Remote Write compatible backend, such as Cortex.

Non-cumulative monotonic, histogram, and summary OTLP metrics are dropped by this exporter.

The following settings are required:

- `endpoint`: protocol:host:port to which the exporter is going to send traces or metrics, using
the HTTP/HTTPS protocol.

- `namespace`: suffix to metric name attached to each metric.

The following settings can be optionally configured:
- `headers`: additional headers attached to each HTTP request. `X-Prometheus-Remote-Write-Version` cannot be set by users
and is attached to each request.
- `insecure` (default = false): whether to enable client transport security for
the exporter's connection.
- `ca_file`: path to the CA cert. For a client this verifies the server certificate. Should
only be used if `insecure` is set to true.
- `cert_file`: path to the TLS cert to use for TLS required connections. Should
only be used if `insecure` is set to true.
- `key_file`: path to the TLS key to use for TLS required connections. Should
only be used if `insecure` is set to true.
- `timeout` (default = 5s): How long to wait until the connection is close.
- `read_buffer_size` (default = 0): ReadBufferSize for HTTP client.
- `write_buffer_size` (default = 512 * 1024): WriteBufferSize for HTTP client.

Example:

```yaml
exporters:
prometheusremotewrite:
endpoint: "http://some.url:9411/api/prom/push"
```
The full list of settings exposed for this exporter are documented [here](./config.go)
with detailed sample configurations [here](./testdata/config.yaml).

_Here is a link to the overall project [design](https://github.com/open-telemetry/opentelemetry-collector/pull/1464)_

File structure:

- `cortex.go`: exporter implementation. Converts and sends OTLP metrics

- `helper.go`: helper functions that cortex.go uses. Performs tasks such as sanitizing label and generating signature string

- `config.go`: configuration struct of the exporter

- `factory.go`: initialization methods for creating default configuration and the exporter

Feature in development: derive Prometheus `job` or `instance` label from Resource, or allow users to configure which Resource attributes needs to be added as metric label_
43 changes: 43 additions & 0 deletions exporter/prometheusremotewriteexporter/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
// Copyright 2020 The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// This package defines the standard and necessary parameters of the exporter config struct.
// The yaml file for the entire collector pipelne must include a section underneath
// `exporters` titled `prometheusremotewrite(/#)`. Example in testdata/config.yaml.

package prometheusremotewriteexporter

import (
"go.opentelemetry.io/collector/config/confighttp"
"go.opentelemetry.io/collector/config/configmodels"
"go.opentelemetry.io/collector/exporter/exporterhelper"
)

// Config defines configuration for Remote Write exporter.
type Config struct {
// squash ensures fields are correctly decoded in embedded struct.
configmodels.ExporterSettings `mapstructure:",squash"`
exporterhelper.TimeoutSettings `mapstructure:",squash"`

exporterhelper.QueueSettings `mapstructure:"sending_queue"`
exporterhelper.RetrySettings `mapstructure:"retry_on_failure"`

// Namespace if set, exports metrics under the provided value.*/
Namespace string `mapstructure:"namespace"`

// Optional headers configuration for authorization and security/extra metadata
Headers map[string]string `mapstructure:"headers"`

HTTPClientSettings confighttp.HTTPClientSettings `mapstructure:"http_setting"`
}
92 changes: 92 additions & 0 deletions exporter/prometheusremotewriteexporter/config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
// Copyright 2020 The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package prometheusremotewriteexporter

import (
"path"
"testing"
"time"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

"go.opentelemetry.io/collector/component/componenttest"
"go.opentelemetry.io/collector/config/confighttp"
"go.opentelemetry.io/collector/config/configmodels"
"go.opentelemetry.io/collector/config/configtest"
"go.opentelemetry.io/collector/config/configtls"
"go.opentelemetry.io/collector/exporter/exporterhelper"
)

// TestLoadConfig checks whether yaml configuration can be loaded correctly
func TestLoadConfig(t *testing.T) {
factories, err := componenttest.ExampleComponents()
assert.NoError(t, err)

factory := NewFactory()
factories.Exporters[typeStr] = factory
cfg, err := configtest.LoadConfigFile(t, path.Join(".", "testdata", "config.yaml"), factories)

require.NoError(t, err)
require.NotNil(t, cfg)

// From the default configurations -- checks if a correct exporter is instantiated
e0 := cfg.Exporters["prometheusremotewrite"]
assert.Equal(t, e0, factory.CreateDefaultConfig())

// checks if the correct Config struct can be instantiated from testdata/config.yaml
e1 := cfg.Exporters["prometheusremotewrite/2"]
assert.Equal(t, e1,
&Config{
ExporterSettings: configmodels.ExporterSettings{
NameVal: "prometheusremotewrite/2",
TypeVal: "prometheusremotewrite",
},
TimeoutSettings: exporterhelper.TimeoutSettings{
Timeout: 10 * time.Second,
},
QueueSettings: exporterhelper.QueueSettings{
Enabled: true,
NumConsumers: 2,
QueueSize: 10,
},
RetrySettings: exporterhelper.RetrySettings{
Enabled: true,
InitialInterval: 10 * time.Second,
MaxInterval: 1 * time.Minute,
MaxElapsedTime: 10 * time.Minute,
},
Namespace: "test-space",

Headers: map[string]string{
"prometheus-remote-write-version": "0.1.0",
"tenant-id": "234"},

HTTPClientSettings: confighttp.HTTPClientSettings{
Endpoint: "localhost:8888",
TLSSetting: configtls.TLSClientSetting{
TLSSetting: configtls.TLSSetting{
CAFile: "/var/lib/mycert.pem", //This is subject to change, but currently I have no idea what else to put here lol
},
Insecure: false,
},
ReadBufferSize: 0,

WriteBufferSize: 512 * 1024,

Timeout: 5 * time.Second,
},
})
}
Loading