WorkQueue

WorkQueue is a background job processing service designed to execute long-running tasks asynchronously. It is especially useful when requests must be handled from fast contexts (such as PHP or other web runtimes) where waiting for a response is not feasible.

WorkQueue currently focuses on HTTP-based jobs and provides a simple Web API for job submission.

---

Features

• Execute HTTP requests asynchronously
• Configure source IP addresses for outgoing requests
• Graceful restart with zero job loss after binary updates
• Web API for job submission
• Prometheus-compatible metrics endpoint

---

Web API

POST /post/http — Submit an HTTP job

Submits a new HTTP request to be executed in the background.

The endpoint accepts raw JSON in the request body with the following structure:

{
  "url": "https://google.com",
  "method": "GET",
  "body": "base64-encoded raw body",
  "parameters": {
    "foo": "bar"
  },
  "headers": {
    "X-Auth-Email": "example@example.com",
    "Cookie": "foo=bar"
  }
}

Notes:

• method defaults to GET if not provided
• body must be Base64-encoded
• For GET and HEAD requests, parameters are appended to the URL
• For other methods, parameters are sent as request arguments
• All fields are optional except url

Multiple requests may be submitted at once by sending an array of objects.

---

Cloned Requests

WorkQueue also supports request cloning, allowing the same payload to be sent to multiple destinations.

{
  "method": "POST",
  "body": "eyJoZWxsbyI6IndvcmxkIn0=",
  "headers": {
    "X-Sender": "me"
  },
  "clones": [
    {
      "url": "https://first-secret-domain.com",
      "headers": {
        "X-Secret": "puppies"
      }
    },
    {
      "url": "https://second-secret-domain.com",
      "headers": {
        "X-Secret": "kittens"
      }
    }
  ]
}

This request results in two HTTP calls, one to each clone URL, using the same body and base headers defined in the parent request.

---

GET /metrics — Metrics Endpoint

Exposes Prometheus-compatible metrics for monitoring worker activity and queue state.

---

Configuration

WorkQueue is configured via command-line flags:

• -listen Addresses to bind the Web API. Multiple addresses can be specified, separated by commas.

• -pidfile Path to the PID file.

• -pool-size Number of worker goroutines. Default: 50.

• -pool-queue-size Maximum number of jobs allowed in the queue. Default: 10000.

• -ip-routes Source IP routing rules for outgoing HTTP requests. Example: 172.16.0.0/12 -> 172.16.1.1, 0.0.0.0/0 -> auto

---