Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/getting-started/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ For further details, please refer to [KIP-1120](https://cwiki.apache.org/conflue
* The metrics `org.apache.kafka.server:type=AssignmentsManager.QueuedReplicaToDirAssignments`, `org.apache.kafka.storage.internals.log:type=RemoteStorageThreadPool.RemoteLogReaderTaskQueueSize`, and `org.apache.kafka.storage.internals.log:type=RemoteStorageThreadPool.RemoteLogReaderAvgIdlePercent` have been deprecated and will be removed in Kafka 5.0. As replacements, the following metrics have been introduced, which report the same information: `kafka.server:type=AssignmentsManager.QueuedReplicaToDirAssignments`, `kafka.log.remote:type=RemoteStorageThreadPool.RemoteLogReaderTaskQueueSize`, and `kafka.log.remote:type=RemoteStorageThreadPool.RemoteLogReaderAvgIdlePercent`. For further details, please refer to [KIP-1100](https://cwiki.apache.org/confluence/x/6oqMEw).
* A new metric `AvgIdleRatio` has been added to the `ControllerEventManager` and `MetadataLoader` groups. These metrics measure the average idle ratio of their respective event queue threads, providing visibility into how much time each component spends waiting for events versus processing them. The metric value ranges from 0.0 (always busy) to 1.0 (always idle).
* Deprecated `org.apache.kafka.streams.KafkaStreams$CloseOptions` and its related methods, such as `KafkaStreams#close(org.apache.kafka.streams.KafkaStreams$CloseOptions)`. As a replacement, please use `org.apache.kafka.streams.CloseOptions` and `KafkaStreams#close(org.apache.kafka.streams.CloseOptions)`. For further details, please refer to [KIP-1153](https://cwiki.apache.org/confluence/x/QAq9F).
* A new implementation of `ConnectorClientConfigOverridePolicy`, `AllowlistConnectorClientConfigOverridePolicy`, has been added. This enables specifying the configurations that connectors can override via `connector.client.config.override.allowlist`. From Kafka 5.0.0, this will be the default [connector.client.config.override.policy](documentation/#connectconfigs_connector.client.config.override.policy) policy. The `PrincipalConnectorClientConfigOverridePolicy` policy is now deprecated and will be removed in Kafka 5.0.0. For further details, please refer to [KIP-1188](https://cwiki.apache.org/confluence/x/2IkvFg).
* A new implementation of `ConnectorClientConfigOverridePolicy`, `AllowlistConnectorClientConfigOverridePolicy`, has been added. This enables specifying the configurations that connectors can override via `connector.client.config.override.allowlist`. From Kafka 5.0.0, this will be the default [connector.client.config.override.policy](https://kafka.apache.org/documentation/#connectconfigs_connector.client.config.override.policy) policy. The `PrincipalConnectorClientConfigOverridePolicy` policy is now deprecated and will be removed in Kafka 5.0.0. For further details, please refer to [KIP-1188](https://cwiki.apache.org/confluence/x/2IkvFg).
* It is now possible to specify the start time for a Kafka Streams punctuation, instead of relying on the non-deterministic time when you register it. For further details, please refer to [KIP-1146](https://cwiki.apache.org/confluence/x/9QqWF).
* Added an optional `--node-id` flag to the `FeatureCommand` command. It specifies the node to describe. If not provided, an arbitrary node is used.

Expand Down
2 changes: 1 addition & 1 deletion docs/kafka-connect/user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ type: docs
-->


The [quickstart](../getting-started/quickstart) provides a brief example of how to run a standalone version of Kafka Connect. This section describes how to configure, run, and manage Kafka Connect in more detail.
The [quickstart](../../getting-started/quickstart) provides a brief example of how to run a standalone version of Kafka Connect. This section describes how to configure, run, and manage Kafka Connect in more detail.

## Running Kafka Connect

Expand Down
12 changes: 6 additions & 6 deletions docs/security/security-model-connect.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ type: docs
-->


This page extends the [Apache Kafka security model](security-model) to Kafka Connect. A worker authenticates to the Kafka cluster over a configured `SASL_SSL`/`SSL` listener exactly like any other client, so everything the core model says about authentication, authorization, and transport encryption to the brokers applies unchanged. What follows covers only what Connect adds on top — chiefly its own control plane, the REST API, and the fact that it runs user-supplied code.
This page extends the [Apache Kafka security model](../security-model) to Kafka Connect. A worker authenticates to the Kafka cluster over a configured `SASL_SSL`/`SSL` listener exactly like any other client, so everything the core model says about authentication, authorization, and transport encryption to the brokers applies unchanged. What follows covers only what Connect adds on top — chiefly its own control plane, the REST API, and the fact that it runs user-supplied code.

## Things You Need To Know

- **Connect inherits the broker's client security model.** Authentication to the brokers, broker-side authorization, and transport encryption are exactly as described in the [core security model](security-model). This page only describes what Connect layers on top.
- **Connect inherits the broker's client security model.** Authentication to the brokers, broker-side authorization, and transport encryption are exactly as described in the [core security model](../security-model). This page only describes what Connect layers on top.
- **The REST API is unauthenticated by default.** Out of the box, anyone who can reach the REST port can create, reconfigure, stop, or delete any connector. Because connectors and plugins run arbitrary code, REST access lets a caller run anything the worker's installed plugins allow.
- **Connect plugins run arbitrary code.** Connectors, converters, transformations, predicates, and REST extensions loaded from `plugin.path` execute in the worker JVM with its privileges. Install only plugins you trust.
- **The REST API is a shared control plane with no per-connector isolation.** There is no notion of connector ownership: any caller allowed onto the API can act on every connector and read its configuration.
Expand All @@ -54,7 +54,7 @@ Connect enables no authentication on the REST API by default. There are two comm
- **Reverse proxy.** Terminate authentication (mTLS, OIDC, basic auth, etc.) in a proxy in front of the workers and allow only the proxy to reach the REST port.
- **REST extension.** Register an authentication extension via `rest.extension.classes`. The built-in `BasicAuthSecurityRestExtension` performs JAAS-based HTTP basic authentication against a configured `LoginModule`. The reference `PropertyFileLoginModule` is **not** intended for production, as it stores credentials in cleartext; production deployments should configure a `LoginModule` that authenticates against a real credential store.

REST authentication only establishes *who is calling*; it does not, on its own, authorize that caller on a per-connector basis (see Authorization below). Separately, the worker's authentication *to the Kafka brokers* uses the standard `ssl.*`/`sasl.*` client configs described in the [core model](security-model).
REST authentication only establishes *who is calling*; it does not, on its own, authorize that caller on a per-connector basis (see Authorization below). Separately, the worker's authentication *to the Kafka brokers* uses the standard `ssl.*`/`sasl.*` client configs described in the [core model](../security-model).

## Authorization

Expand All @@ -70,12 +70,12 @@ The practical consequences are that you cannot grant Connect REST API access to

Two independent channels need TLS:

- **Worker-to-Kafka.** Configured with the standard `ssl.*` client properties, exactly as in the [core model](security-model).
- **Worker-to-Kafka.** Configured with the standard `ssl.*` client properties, exactly as in the [core model](../security-model).
- **REST API (and inter-worker).** Enable an `https` listener. By default the REST server reuses the worker's `ssl.*` settings; to configure the REST endpoint independently of the Kafka client, use the `listeners.https.*` prefixed properties (when the prefix is used, the unprefixed `ssl.*` settings are ignored for the REST server). The same settings secure inter-worker forwarding in distributed mode.

## Secrets in Configuration

Connector configurations frequently contain credentials for external systems. As in the [core model](security-model), reference them indirectly through a `ConfigProvider` rather than inlining them, and set `allowed.paths` on the file-based providers to constrain which directories they can read. Two Connect-specific caveats:
Connector configurations frequently contain credentials for external systems. As in the [core model](../security-model), reference them indirectly through a `ConfigProvider` rather than inlining them, and set `allowed.paths` on the file-based providers to constrain which directories they can read. Two Connect-specific caveats:

- **Config providers are resolved through the shared REST API.** A caller who can guess or enumerate a provider alias can resolve its full value, so a provider is only as isolated as the REST API in front of it.
- **What goes in the config topic depends on how secrets are supplied.** A secret placed directly in a connector configuration is written to `config.storage.topic` as an ordinary Kafka record, and is therefore only as protected as that topic's ACLs and the brokers' at-rest story. A `ConfigProvider` reference keeps the secret itself out of the topic — only the template string (`${alias:fields}`) is stored, not the resolved value. Note, however, that this does not hide the secret from REST API callers: anyone who knows a valid template string can usually retrieve the resolved value through the REST API.
Expand All @@ -86,7 +86,7 @@ Connect loads connectors, converters, single-message transforms, predicates, `Co

## Known Non-Findings

In line with the [core model's classification](security-model), the following follow from Connect's design and are not, on their own, security vulnerabilities:
In line with the [core model's classification](../security-model), the following follow from Connect's design and are not, on their own, security vulnerabilities:

- **File-based connectors granting disk access.** Adding the file connectors to a worker effectively grants read/write access to the worker's local disk. This is the connector's intended function, not a flaw.
- **Local-disk-only weaknesses in the file-based config providers.** Issues that require an attacker to already have local disk access on a Connect worker — for example the ability to create arbitrary files or symlinks — are not security issues, because such access is outside Connect's trust boundary. Path-validation robustness in these providers may still be hardened independently.
Expand Down
4 changes: 2 additions & 2 deletions docs/security/security-model-streams.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ type: docs
-->


This page extends the [Apache Kafka security model](security-model) to Kafka Streams. From the cluster's perspective a Streams application is an ordinary Kafka client and inherits the broker's authentication, authorization, and transport-encryption model through the same `ssl.*` and `sasl.*` client configs.
This page extends the [Apache Kafka security model](../security-model) to Kafka Streams. From the cluster's perspective a Streams application is an ordinary Kafka client and inherits the broker's authentication, authorization, and transport-encryption model through the same `ssl.*` and `sasl.*` client configs.

## Things You Need To Know

Expand All @@ -36,7 +36,7 @@ This page extends the [Apache Kafka security model](security-model) to Kafka Str

## Known Non-Findings

In line with the [core model's classification](security-model), the following are not, on their own, security vulnerabilities:
In line with the [core model's classification](../security-model), the following are not, on their own, security vulnerabilities:

- **Application-level issues in a Streams topology.** Streams runs inside the user's application, so bugs in user-supplied processors, state handling, or topology code fall within the application's trust boundary, not the broker's.
- **Local state being readable on the host.** State stores and changelogs on disk are protected by the host filesystem, not by Kafka; reading them requires local access that is outside the model.
6 changes: 3 additions & 3 deletions docs/security/security-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,8 +128,8 @@ Broker, client, and Connect properties files contain keystore passwords, SASL cr

The components built on top of the Kafka clients have their own security models, covered on separate pages:

- [Kafka Connect](security-model-connect)
- [Kafka Streams](security-model-streams)
- [Kafka Connect](../security-model-connect)
- [Kafka Streams](../security-model-streams)

## Development and Test Tooling

Expand Down Expand Up @@ -162,7 +162,7 @@ The following follow directly from the model above and are not, on their own, se
- **Unauthenticated or unencrypted access to a default cluster.** Security is off by default; an open `PLAINTEXT` listener with no authorizer is a deployment choice, not a defect.
- **A trusted principal performing an authorized operation.** Admin and inter-broker actions by a principal that holds the relevant ACLs — or by a `super.users` entry — are expected behaviour.
- **Findings in development and test tooling.** Issues in `tools`, `bin`, Trogdor, `tests`, and similar are out of scope (see Development and Test Tooling above).
- **Kafka Streams application-level issues.** Streams runs inside the user's application, so its security boundary is the application's, not the broker's (see the [Kafka Streams security model](security-model-streams)).
- **Kafka Streams application-level issues.** Streams runs inside the user's application, so its security boundary is the application's, not the broker's (see the [Kafka Streams security model](../security-model-streams)).

## Reporting Security Issues

Expand Down
2 changes: 1 addition & 1 deletion docs/streams/developer-guide/app-reset-tool.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ type: docs

You can reset an application and force it to reprocess its data from scratch by using the application reset tool. This can be useful for development and testing, or when fixing bugs.

The application reset tool handles the Kafka Streams [user topics](manage-topics.html#streams-developer-guide-topics-user) (input, and output) and [internal topics](manage-topics.html#streams-developer-guide-topics-internal) differently when resetting the application.
The application reset tool handles the Kafka Streams [user topics](../manage-topics#streams-developer-guide-topics-user) (input, and output) and [internal topics](../manage-topics#streams-developer-guide-topics-internal) differently when resetting the application.

Here's what the application reset tool does for each topic type:

Expand Down
Loading
Loading