Support list format for parsers_file in YAML service section

[lecaros]

Is your feature request related to a problem? Please describe.

The YAML configuration format does not support list/array syntax for the parsers_file property in the service section, while the classic conf format allows multiple parsers_file directives. This creates an inconsistency between configuration formats and forces users to use workarounds when migrating from classic conf to YAML.

In classic conf format, multiple parser files can be loaded like this:

[SERVICE]
    parsers_file parser1.conf
    parsers_file parser2.conf
    parsers_file parser3.conf

However, the equivalent YAML syntax fails:

service:
  parsers_file:
    - parser1.yaml
    - parser2.yaml
    - parser3.yaml

Error:

[error] [config] YAML error found in file "config.yaml", line 3, column 4: unexpected event 'sequence-start-event' (7) in state 'section-value' (5).

Technical Details:

The YAML parser explicitly rejects sequences in the service section (except for env):

• Source: src/config_format/flb_cf_yaml.c:1856-1861
• When a YAML_SEQUENCE_START_EVENT is encountered in the service section, it throws an error unless state->section == SECTION_ENV

Describe the solution you'd like

Support list/array syntax for parsers_file in the YAML service section to match the behavior of classic conf format:

service:
  flush: 1
  log_level: info
  parsers_file:
    - /path/to/parser1.yaml
    - /path/to/parser2.yaml
    - /path/to/parser3.yaml

pipeline:
  inputs:
    - name: tail
      path: /var/log/app.log
  outputs:
    - name: stdout
      match: '*'

This would allow multiple parser files to be loaded in the same way as the classic conf format, maintaining feature parity between configuration formats.

Describe alternatives you've considered

There are currently three workarounds:

1. Use the includes directive (loads entire YAML files):

service:
  flush: 1
  log_level: info

includes:
  - /path/to/parser1.yaml
  - /path/to/parser2.yaml

pipeline:
  inputs:
    - name: tail
      path: /var/log/app.log
  outputs:
    - name: stdout
      match: '*'

2. Define parsers inline in the main YAML file:

service:
  flush: 1
  log_level: info

parsers:
  - name: apache
    format: regex
    regex: "^(?<host>[^ ]*) ..."
    time_key: time
    time_format: "%d/%b/%Y:%H:%M:%S %z"

  - name: nginx
    format: regex
    regex: "^(?<remote>[^ ]*) ..."
    time_key: time
    time_format: "%d/%b/%Y:%H:%M:%S %z"

pipeline:
  inputs:
    - name: tail
      path: /var/log/app.log
  outputs:
    - name: stdout
      match: '*'

3. Use duplicate keys (breaks YAML conventions):

service:
  parsers_file: parser1.yaml
  parsers_file: parser2.yaml  # Last value overwrites

While workarounds exist, they either:

• Mix parser definitions with includes (workaround 1)
• Require consolidating all parsers into one file or the main config (workaround 2)
• Break YAML conventions or lose data (workaround 3)

Additional context

This feature would improve:

• Migration path: Easier transition from classic conf to YAML format
• Configuration consistency: Same capabilities across both configuration formats
• User experience: More intuitive YAML syntax that follows common YAML patterns
• Modularity: Ability to organize parsers across multiple files without using includes for entire configurations

The classic conf format handles multiple parsers_file entries by calling flb_parser_conf_file() for each occurrence (see src/flb_config.c:770-776). The YAML parser could implement similar behavior for sequence values.

[edsiper]

@lecaros @cosmo0920

Looking at the current options and how we extended parsers configuration in Yaml vs classic mode, I don't see the value of extending the old mechanism (parsers_file) since parsers can easily live in a separate Yaml files and be included:

includes:

• my-parsers.yaml

where my-parser.yaml:

parsers:
- name: docker
format: json

the effort was to register them outside of service section, so extending service section goes into the opposite direction.

btw, PRs mentioned #1, #2 and #3 are not related with the context of this issue

[lecaros]

my bad with formatting IRT the PRs, it was intended to number the workarounds, not any PR 😬
IMO, we should either remove the parsers_file from yaml format (and update the docs), or (get closer to) comply with YAML format definitions. It's not a extension of the feature, it's giving the users a consistent experience while configuring using YAML format.

[edsiper]

I would put the effort in documentation; we have to consider that many users still use classic mode :/ , and docs/guides which reflect best practices in Yaml and the new formats for these scenarios will help