docs: filter & comprehensive destination documentation

docs/pages/features/destinations.mdx → docs/pages/destinations.mdx

@@ -2,45 +2,47 @@
 title: Destinations
 ---
 
-Outpost supports multiple event destination types. Each tenant can have multiple destinations, up to a maximum set by the `MAX_DESTINATIONS_PER_TENANT` environment variable (defaulting to `20`). Destinations can be registered either through the tenant-facing portal or with the API. The current supported destination types are:
-
-- Webhooks
-- Hookdeck Event Gateway
-- AWS Kinesis
-- AWS SQS
-- AWS S3
-- Azure Service Bus
-- GCP Pub/Sub
-- RabbitMQ (AMQP)
-
-Plans for additional event destination types include:
-
-- Amazon EventBridge
-- Kafka
+Outpost supports multiple event destination types. Each tenant can have multiple destinations, up to a maximum set by the `MAX_DESTINATIONS_PER_TENANT` environment variable (defaulting to `20`).
 
 > We recommend setting the `MAX_DESTINATIONS_PER_TENANT` value as low as is appropriate for your use case to prevent abuse and performance degradation. Updating the value to a lower value later will not delete existing destinations.
 
-See the [roadmap](/docs/references/roadmap) for more information on planned destination types. To be eligible as a destination type, it must be asynchronous in nature and not run any business logic.
+## Supported Destinations
+
+| Destination | Description |
+| ----------- | ----------- |
+| [Webhook](/docs/destinations/webhook) | Send events via HTTP POST to a URL |
+| [Hookdeck](/docs/destinations/hookdeck) | Route events through Hookdeck Event Gateway |
+| [AWS Kinesis](/docs/destinations/aws-kinesis) | Stream events to Amazon Kinesis |
+| [AWS SQS](/docs/destinations/aws-sqs) | Send events to an Amazon SQS queue |
+| [AWS S3](/docs/destinations/aws-s3) | Store events in an Amazon S3 bucket |
+| [Azure Service Bus](/docs/destinations/azure-service-bus) | Send events to Azure Service Bus |
+| [GCP Pub/Sub](/docs/destinations/gcp-pubsub) | Publish events to Google Cloud Pub/Sub |
+| [RabbitMQ](/docs/destinations/rabbitmq) | Send events to a RabbitMQ exchange |
 
-## Creating a destination
+See the [roadmap](/docs/references/roadmap) for planned destination types. To be eligible as a destination type, it must be asynchronous in nature and not run any business logic.
 
-Destinations can be registered either through the tenant-facing portal or with the API. Since each destination type uses its own credentials and configuration, the required fields are different. Refer to the [Create Destination API](/docs/api/destinations#create-destination) for the required `config` and `credentials` fields for each destination type.
+## Creating a Destination
 
-For example, creating a "Webhook" destination requires the `config.url` field.
+Destinations can be registered through the tenant portal or via the API. Each destination type has its own configuration and credentials. Refer to the [Create Destination API](/docs/api/destinations#create-destination) for the required `config` and `credentials` fields for each destination type.
 
 ```sh
-curl --location 'localhost:3333/api/v1/<TENANT_ID>/destinations' \
+curl --location 'https://<OUTPOST_API_URL>/api/v1/<TENANT_ID>/destinations' \
 --header 'Content-Type: application/json' \
 --header 'Authorization: Bearer <API_KEY>' \
 --data '{
- "type": "<TYPE>",
- "topics": ["<TOPIC>"],
- "config": {
-    "url": "https://example.test/webhooks"
- }
+  "type": "<TYPE>",
+  "topics": ["<TOPIC>"],
+  "config": { ... },
+  "credentials": { ... }
 }'
 ```
 
+## Destination Filtering
+
+Destinations can be configured with filters to selectively receive only events matching specific criteria. This allows tenants to create fine-grained routing rules based on event properties.
+
+See the [Filters](/docs/features/filter) documentation for the complete filter syntax and examples.
+
 ## Getting Destination Types & Schemas
 
 When using the API, you may want to build your own UI to capture user input on the destination configuration. Since each destination requires a specific configuration, the `GET /destination-types` endpoint provides a JSON schema for standardized input fields for each destination type.
@@ -84,9 +86,9 @@ Some destinations will require instructions to configure. For instance, with Pub
 
 Some destinations may have OAuth flow or other managed setup flow that can be triggered with a link. If a `remote_setup_url` is set, then the user should be prompted to follow the link to configure the destination.
 
-See the [building your own UI guide](../../guides/building-your-own-ui.mdx) for recommended UI patterns and wireframes for implementation in your own app.
+See the [building your own UI guide](/docs/guides/building-your-own-ui) for recommended UI patterns and wireframes for implementation in your own app.
 
-## Customizing destination type definitions & instructions
+## Customizing Destination Type Definitions & Instructions
 
 The destination type definitions (label, description, icon, etc) and instructions can be customized by setting the `DESTINATIONS_METADATA_PATH` environment variable to a path on disk containing the destination type definitions and instructions. Outpost will load both the default destination type definitions and any custom destination type definitions and merge them.
 

docs/pages/destinations/aws-kinesis.mdx

@@ -0,0 +1,124 @@
+---
+title: AWS Kinesis
+---
+
+Stream events to an Amazon Kinesis Data Stream.
+
+## Configuration
+
+### Config
+
+| Field | Type | Required | Description |
+| ----- | ---- | -------- | ----------- |
+| `config.stream_name` | string | Yes | The Kinesis stream name |
+| `config.region` | string | Yes | AWS region (e.g., `us-east-1`) |
+| `config.endpoint` | string | No | Custom endpoint URL (for LocalStack, etc.) |
+| `config.partition_key_template` | string | No | JMESPath expression for partition key |
+
+### Credentials
+
+| Field | Type | Required | Description |
+| ----- | ---- | -------- | ----------- |
+| `credentials.key` | string | Yes | AWS Access Key ID |
+| `credentials.secret` | string | Yes | AWS Secret Access Key |
+| `credentials.session` | string | No | AWS Session Token (for temporary credentials) |
+
+### Example
+
+```sh
+curl --location 'https://<OUTPOST_API_URL>/api/v1/<TENANT_ID>/destinations' \
+--header 'Content-Type: application/json' \
+--header 'Authorization: Bearer <API_KEY>' \
+--data '{
+  "type": "aws_kinesis",
+  "topics": ["orders"],
+  "config": {
+    "stream_name": "my-stream",
+    "region": "us-east-1"
+  },
+  "credentials": {
+    "key": "<AWS_ACCESS_KEY_ID>",
+    "secret": "<AWS_SECRET_ACCESS_KEY>"
+  }
+}'
+```
+
+## Partition Key
+
+By default, the event ID is used as the partition key. You can customize this using a JMESPath expression in `partition_key_template`:
+
+```json
+{
+  "config": {
+    "stream_name": "my-stream",
+    "region": "us-east-1",
+    "partition_key_template": "data.customer_id"
+  }
+}
+```
+
+## Record Format
+
+If you publish an event:
+
+```json
+{
+  "topic": "orders",
+  "data": {
+    "order_id": "123",
+    "status": "created"
+  },
+  "metadata": {
+    "source": "checkout-service"
+  }
+}
+```
+
+By default, the Kinesis record includes both metadata and the event's `data` field. The `metadata` object contains system metadata (`event-id`, `topic`, `timestamp`) plus any event metadata from the published event:
+
+```json
+{
+  "metadata": {
+    "event-id": "evt_123",
+    "topic": "orders",
+    "timestamp": "1704067200",
+    "source": "checkout-service"
+  },
+  "data": {
+    "order_id": "123",
+    "status": "created"
+  }
+}
+```
+
+When `DESTINATIONS_AWS_KINESIS_METADATA_IN_PAYLOAD` is set to `false`, only the event's `data` field is sent:
+
+```json
+{
+  "order_id": "123",
+  "status": "created"
+}
+```
+
+## IAM Permissions
+
+The IAM user or role needs the following permission:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": "kinesis:PutRecord",
+      "Resource": "arn:aws:kinesis:*:*:stream/my-stream"
+    }
+  ]
+}
+```
+
+## Operator Configuration
+
+| Environment Variable | Default | Description |
+| -------------------- | ------- | ----------- |
+| `DESTINATIONS_AWS_KINESIS_METADATA_IN_PAYLOAD` | `true` | Include Outpost metadata in the Kinesis record payload |

docs/pages/destinations/aws-s3.mdx

@@ -0,0 +1,126 @@
+---
+title: AWS S3
+---
+
+Store events in an Amazon S3 bucket.
+
+## Configuration
+
+### Config
+
+| Field | Type | Required | Description |
+| ----- | ---- | -------- | ----------- |
+| `config.bucket` | string | Yes | The S3 bucket name |
+| `config.region` | string | Yes | AWS region (e.g., `us-east-1`) |
+| `config.key_template` | string | No | JMESPath expression for S3 object key |
+| `config.storage_class` | string | No | S3 storage class (e.g., `STANDARD`, `GLACIER`) |
+| `config.endpoint` | string | No | Custom endpoint URL (for LocalStack, etc.) |
+
+### Credentials
+
+| Field | Type | Required | Description |
+| ----- | ---- | -------- | ----------- |
+| `credentials.key` | string | Yes | AWS Access Key ID |
+| `credentials.secret` | string | Yes | AWS Secret Access Key |
+| `credentials.session` | string | No | AWS Session Token (for temporary credentials) |
+
+### Example
+
+```sh
+curl --location 'https://<OUTPOST_API_URL>/api/v1/<TENANT_ID>/destinations' \
+--header 'Content-Type: application/json' \
+--header 'Authorization: Bearer <API_KEY>' \
+--data '{
+  "type": "aws_s3",
+  "topics": ["orders"],
+  "config": {
+    "bucket": "my-events-bucket",
+    "region": "us-east-1"
+  },
+  "credentials": {
+    "key": "<AWS_ACCESS_KEY_ID>",
+    "secret": "<AWS_SECRET_ACCESS_KEY>"
+  }
+}'
+```
+
+## Object Key
+
+By default, objects are stored with keys in the format: `{timestamp}_{event-id}.json`
+
+You can customize the key using a JMESPath expression in `key_template`:
+
+```json
+{
+  "config": {
+    "bucket": "my-events-bucket",
+    "region": "us-east-1",
+    "key_template": "join('/', [data.customer_id, metadata.\"event-id\", '.json'])"
+  }
+}
+```
+
+## Object Format
+
+Events are stored as JSON files containing the event's `data` field. For example, if you publish an event:
+
+```json
+{
+  "topic": "orders",
+  "data": {
+    "order_id": "123",
+    "status": "created"
+  },
+  "metadata": {
+    "source": "checkout-service"
+  }
+}
+```
+
+The S3 object body will be:
+
+```json
+{
+  "order_id": "123",
+  "status": "created"
+}
+```
+
+Event metadata is stored in the S3 object's metadata (not in the body). This includes system metadata and any event metadata from the published event:
+
+| Metadata Key | Source | Description |
+| ------------ | ------ | ----------- |
+| `event-id` | System | The unique event ID |
+| `topic` | System | The event topic |
+| `timestamp` | System | Event timestamp (Unix) |
+| `*` | Event | Any additional metadata from the published event's `metadata` field |
+
+## IAM Permissions
+
+The IAM user or role needs the following permission:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": "s3:PutObject",
+      "Resource": "arn:aws:s3:::my-events-bucket/*"
+    }
+  ]
+}
+```
+
+## Storage Classes
+
+Supported storage classes:
+
+- `STANDARD` (default)
+- `REDUCED_REDUNDANCY`
+- `STANDARD_IA`
+- `ONEZONE_IA`
+- `INTELLIGENT_TIERING`
+- `GLACIER`
+- `DEEP_ARCHIVE`
+- `GLACIER_IR`