Skip to content

Commit

Permalink
add sample
Browse files Browse the repository at this point in the history
  • Loading branch information
linrunqi08 committed Feb 29, 2024
1 parent 1129f0a commit 58c3e69
Show file tree
Hide file tree
Showing 2 changed files with 98 additions and 0 deletions.
59 changes: 59 additions & 0 deletions docs/cn/plugins/processor/native/processor-spl.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Json解析原生处理插件

## 简介

`processor_parse_json_native`插件解析事件中`Json`格式字段内容并提取新字段。

## 版本

[Stable](../stability-level.md)

## 配置参数

| **参数** | **类型** | **是否必填** | **默认值** | **说明** |
| --- | --- | --- | --- | --- |
| Type | string || / | 插件类型。固定为processor\_spl。 |
| Script | string || / | spl语句 |
| TimeoutMilliSeconds | int || 1000 | 执行脚本的超时时间,单位为毫秒 |
| MaxMemoryBytes | int || 50*1024*1024 | spl执行最大内存配置,单位为字节 |

## 样例

采集文件`/home/test-log/test.log`,解析日志内容并提取字段,并将结果输出到stdout。

- 输入

```
[2024-01-05T12:07:00.123456] {"message": "this is a msg", "level": "INFO", "garbage": "xxx"} java.lang.Exception: exception
```

- 采集配置

```yaml
enable: true
inputs:
- Type: input_file
FilePaths:
- /home/test-log/test.log
Multiline:
StartPattern: \[\d+.*
processors:
- Type: processor_spl
Script: '* | parse-regexp content, ''\[([^]]+)]\s+([^}]+})\s+([\S\s]*)'' as time,json,stack | parse-json json | project-away garbage,json,content'
flushers:
- Type: flusher_stdout
OnlyStdout: true
```
- 输出
```json
{
"__tag__:__path__": "/home/test-log/test.log",
"message": "this is a msg",
"level": "INFO",
"time": "2024-01-05T12:07:00.123456",
"stack": "java.lang.Exception: exception",
"__time__": "1657161028"
}
```
39 changes: 39 additions & 0 deletions example_config/data_pipelines/file-spl.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Copyright 2022 iLogtail Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# To collect json format data from file.
# Before Process:
# [2024-01-05T12:07:00.123456] {"message": "this is a msg", "level": "INFO", "garbage": "xxx"} java.lang.Exception: exception
# After Process:
# {"message": "this is a msg", "level": "INFO", "time": "2024-01-05T12:07:00.123456", "stack":"java.lang.Exception: exception"}
# You may change SourceKey and other parameters.
# For a full list of parameters, visit https://ilogtail.gitbook.io/ilogtail-docs/plugins/processor/spl

enable: true
inputs:
- Type: input_file
FilePaths:
- /home/test-log/test.log
Multiline:
StartPattern: \[\d+.*
processors:
- Type: processor_spl
Script: '* | parse-regexp content, ''\[([^]]+)]\s+([^}]+})\s+([\S\s]*)'' as time,json,stack | parse-json json | project-away garbage,json,content'
flushers:
- Type: flusher_sls
Endpoint: cn-xxx.log.aliyuncs.com
ProjectName: test_project
LogstoreName: test_logstore
- Type: flusher_stdout
OnlyStdout: true

0 comments on commit 58c3e69

Please sign in to comment.