[Datadog] Add support for Logs Public config API #312
Conversation
ghost
added
the
| "strings" | ||
| ) | ||
|
|
||
| const ( |
There was a problem hiding this comment.
Can we group remappers and processors please ? It should ease readability
There was a problem hiding this comment.
I was sorting them alphabetically. It should be easier for the people who are not familiar with logs backend related logic? In addition, they are all called processors.
There was a problem hiding this comment.
I don’t think so, it just grouping by suffix basically.
There was a problem hiding this comment.
const (
tfArithmeticProcessor = "arithmetic_processor"
tfAttributeRemapperProcessor = "attribute_remapper"
tfCategoryProcessor = "category_processor"
tfDateRemapperProcessor = "date_remapper"
tfGrokParserProcessor = "grok_parser"
tfMessageRemapperProcessor = "message_remapper"
tfNestedPipelineProcessor = "pipeline"
tfServiceRemapperProcessor = "service_remapper"
tfStatusRemapperProcessor = "status_remapper"
tfTraceIdRemapperProcessor = "trace_id_remapper"
tfUrlParserProcessor = "url_parser"
tfUserAgentParserProcessor = "user_agent_parser"
)
To me the order is alphabetic:
tfArithmeticProcessor
tfAttributeRemapperProcessor
tfCategoryProcessor
tfDateRemapperProcessor
tfGrokParserProcessor
tfMessageRemapperProcessor
tfNestedPipelineProcessor
tfServiceRemapperProcessor
tfStatusRemapperProcessor
tfTraceIdRemapperProcessor
tfUrlParserProcessor
tfUserAgentParserProcessor
| tfUserAgentParserProcessor = "user_agent_parser" | ||
| ) | ||
|
|
||
| var tfProcessorTypes = map[string]string{ |
| tfUserAgentParserProcessor: datadog.UserAgentParserType, | ||
| } | ||
|
|
||
| var ddProcessorTypes = map[string]string{ |
| page_title: "Datadog: datadog_logs_pipeline" | ||
| sidebar_current: "docs-datadog-resource-logs-pipeline" | ||
| description: |- | ||
| Provides a Datadog logs pipeline resource. This can be used to create and manage logs pipelines. |
There was a problem hiding this comment.
Nit: Logs pipelines & processors
There was a problem hiding this comment.
Why do we need to explicitly state processors? Shouldn't it be a property of pipeline? From the current documentation for public config API, we don't have explicit endpoints to access processors.
|
|
||
| # datadog_logs_pipeline | ||
|
|
||
| Provides a Datadog [Logs Pipeline API](https://docs.datadoghq.com/api/?lang=python#logs-pipelines) resource. This can be used to create and manage Datadog logs pipelines. |
|
Hi @ruthnaebeck, I added a recommendation at the end of website/docs/r/logs_pipeline.html.markdown could you check again please? |
|
|
||
| const ( | ||
| tfArithmeticProcessor = "arithmetic_processor" | ||
| tfAttributeRemapperProcessor = "attribute_remapper" |
There was a problem hiding this comment.
Why the processor suffix for a remapper? It is a TF convention?
There was a problem hiding this comment.
My understanding is processor is a general name which describes the processor properties of pipeline. Either a remapper or something else, they are all processors. I want to make it clear that they are all processors in TF.
There was a problem hiding this comment.
Allright folks, I think all the questions/comments have been addressed or replied to and I see no more issues preventing me from merging this PR.
Big thanks to everyone who added their reviews and huge thanks to @tt810 for implementing all of this functionality!
Merging 🎉
Looks good :-) |
Supporting logs Public config API pipeline and pipeline order. for more information please read this