Skip to content

format: add support for ndjson_batch#155

Open
yaauie wants to merge 1 commit intologstash-plugins:mainfrom
yaauie:format-ndjson_batch
Open

format: add support for ndjson_batch#155
yaauie wants to merge 1 commit intologstash-plugins:mainfrom
yaauie:format-ndjson_batch

Conversation

@yaauie
Copy link
Contributor

@yaauie yaauie commented Jan 29, 2026

Adds support for format => ndjson_batch, which is batch-oriented and sends newline-delimited sequence of events

@yaauie yaauie requested a review from jsvd January 29, 2026 05:23
@@ -1,3 +1,6 @@
## 6.1.0
- Added support for batched newline-delimited json with `format => ndjson_batch`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Added support for batched newline-delimited json with `format => ndjson_batch`
- Added support for batched newline-delimited json with `format => ndjson_batch` [#155](https://github.com/logstash-plugins/logstash-output-http/pull/155)

If form, then the body will be the mapping (or whole event) converted
into a query parameter string, e.g. `foo=bar&baz=fizz...`
When set to a batch-oriented `format`, each batch of events received by this output will be transmitted in a single http request.
This is particularly useful for high throughput scenarios such as sending data between Logstash instances.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is particularly useful for high throughput scenarios such as sending data between Logstash instances.
This is recommended for most workloads, as batching avoids severely limiting throughput due to the request's round-trip time. For example, an RTT of 5ms will limit throughput to 200 events per second, per pipeline worker.

This is particularly useful for high throughput scenarios such as sending data between Logstash instances.

If message, then the body will be the result of formatting the event according to message
Non-batch formats send one http request per event, which can have significantly lower throughput.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Non-batch formats send one http request per event, which can have significantly lower throughput.
Non-batch formats send one HTTP request per event, which will have significantly lower throughput.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants