yaml/v2: use stack based parser.

[pwhelan]

Summary

This PR includes a refactoring of the YAML parser to use a stack based parser instead of using a single state that is continuously modified.

This simplifies the code as well as allowing the reuse of code where appropriate, ie: properties for customs, inputs, filters, outputs, processors.

I also introduced debug output for the parser which requires a modification to fluent-bit to allow setting verbose mode when parsing command line arguments. A PR to do so is forthcoming.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• Run local packaging test showing all targets (including any new ones) build.
• Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• Documentation required for this feature

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

---

• To see the specific tasks where the Asana app for GitHub is being used, see below:
  • https://app.asana.com/0/0/1205171678259551

[pwhelan]

This is a test run using the an input with a processor through a filter and then to two different outputs:

---
pipeline:
  inputs:
    - name: dummy
      dummy: '{"boo": "far"}'
      processors:
        logs:
          - name: record_modifier
            record: processed true
  filters:
    - name: record_modifier
      match: "*"
      record:
        - powered_by calyptia
        - hostname localhost
  outputs:
    - name: stdout
      format: json_lines
      match: "*"
    - name: exit
      match: "*"
      flush_count: 10

bash$ valgrind --leak-check=full ./bin/fluent-bit -c input-simple.yaml
==492741== Memcheck, a memory error detector
==492741== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==492741== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==492741== Command: ./build/bin/fluent-bit -c build/input-simple.yaml
==492741== 
Fluent Bit v2.1.7
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/07/06 14:14:36] [ info] [fluent bit] version=2.1.7, commit=fb7d4c8c9b, pid=492741
[2023/07/06 14:14:36] [ info] [storage] ver=1.2.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/07/06 14:14:36] [ info] [cmetrics] version=0.6.3
[2023/07/06 14:14:37] [ info] [output:stdout:stdout.0] worker #0 started
[2023/07/06 14:14:36] [ info] [ctraces ] version=0.3.1
[2023/07/06 14:14:37] [ info] [input:dummy:dummy.0] initializing
[2023/07/06 14:14:37] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2023/07/06 14:14:37] [ info] [sp] stream processor started
{"date":1688667277.249725,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667278.273059,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667279.236527,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667280.236557,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667281.253871,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667282.23624,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667283.236297,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667284.236255,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667285.236251,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667286.2378,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
{"date":1688667287.236259,"boo":"far","processed":"true","powered_by":"calyptia","hostname":"localhost"}
[2023/07/06 14:14:47] [ warn] [engine] service will shutdown in max 5 seconds
[2023/07/06 14:14:47] [ info] [input] pausing dummy.0
[2023/07/06 14:14:48] [ info] [engine] service has stopped (0 pending tasks)
[2023/07/06 14:14:48] [ info] [input] pausing dummy.0
[2023/07/06 14:14:48] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/07/06 14:14:48] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==492741== 
==492741== HEAP SUMMARY:
==492741==     in use at exit: 0 bytes in 0 blocks
==492741==   total heap usage: 2,974 allocs, 2,974 frees, 5,978,725 bytes allocated
==492741== 
==492741== All heap blocks were freed -- no leaks are possible
==492741== 
==492741== For lists of detected and suppressed errors, rerun with: -s
==492741== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

[pwhelan]

There is a pending compile bug on windows:

D:\a\fluent-bit\fluent-bit\src\config_format\flb_cf_yaml.c(471): error C2065: 'state': undeclared identifier
D:\a\fluent-bit\fluent-bit\src\config_format\flb_cf_yaml.c(471): error C2223: left of '->file' must point to struct/union
D:\a\fluent-bit\fluent-bit\src\config_format\flb_cf_yaml.c(471): error C2198: 'read_config': too few arguments for call
D:\a\fluent-bit\fluent-bit\src\config_format\flb_cf_yaml.c(610): warning C4020: 'read_glob': too many actual parameters
D:\a\fluent-bit\fluent-bit\src\config_format\flb_cf_yaml.c(1416): warning C4047: '=': 'flb_sds_t' differs in levels of indirection from 'int'
NMAKE : fatal error U1077: '"C:\Program Files\CMake\bin\cmake.exe" -E cmake_cl_compile_depends --dep-file=CMakeFiles\fluent-bit-static.dir\config_format\flb_cf_yaml.c.obj.d --working-dir=D:\a\fluent-bit\fluent-bit\build\src --filter-prefix="Note: including file: " -- C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1435~1.322\bin\HostX86\x86\cl.exe @C:\Users\RUNNER~1\AppData\Local\Temp\nm1711.tmp' : return code '0x2'

[cosmo0920]

Most of the PR looks good to me.
I added a nitpick comment.

[niedbalski]

LGTM