elasticsearch sink ignores SECRET[...]

[sim0nx]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Since 0.57.0 ENV var substitution should no longer be used thus the idea was to switch to some secret backend (exec or file).
The exec backend should call a simple python script that outputs the requested secret (using v1 proto) as per your blog post from 2022.

The problem is that although vector correctly loads the config without error or warning, it ignores the secret backend config and never calls the script. Specifying an invalid filename, invalid secret key, doesn't bother it... no error output.

The username / password combo used for ES is just the plain literal:

• SECRET[mysecrets.elasticsearch_user]
• SECRET[mysecrets.elasticsearch_password]

I highly suspect that for whatever reason auth.user and auth.password of the ES sink does not support parsing for SECRET[...].

ENV var substitution works fine, but as this is deprecated and to be removed in a future version, I want to avoid that usage.

Any help would be appreciated to solving this.

Configuration

[secret.mysecrets]
type = "exec"
command = ["/etc/vector/vector_get_secret.py"]


[sinks.sink_es]
type = "elasticsearch"
inputs = ["json_remap"]

endpoints = ["https://.................."]
api_version = "v8"

mode = "bulk"
bulk.action = "index"
bulk.index = "..."
compression = "none"
encoding.timestamp_format = "rfc3339"
healthcheck.enabled = true

id_key = "_id"

auth.strategy = "basic"
auth.user = "SECRET[mysecrets.elasticsearch_user]"
auth.password = "SECRET[mysecrets.elasticsearch_password]"

Version

0.57.0

Debug Output

Example Data

No response

Additional Context

No response

References

No response