Skip to content

bug: InferenceQueue is not priority-aware despite being documented as such #49

@jpleva91

Description

@jpleva91

Bug: InferenceQueue ignores Priority

File: internal/scheduler/queue.go

The package doc and struct comment both describe the queue as "priority-aware", and four Priority constants are defined (PriorityWorker, PriorityPlanner, PriorityCorrector, PriorityEvaluator). However, Submit() never uses the priority — it simply blocks on a semaphore channel:

func (q *InferenceQueue) Submit(ctx context.Context, req InferenceRequest) error {
    // ...
    select {
    case q.slots <- struct{}{}:   // FIFO channel — priority ignored
    case <-ctx.Done():
        return ctx.Err()
    }
    // ...
}

The InferenceRequest.Priority field is never read. All requests are served FIFO regardless of their declared priority. This means PriorityEvaluator = 3 gets no advantage over PriorityWorker = 1.

Impact

  • Evaluator and Corrector agents (higher priority) are not given inference slots ahead of Worker agents. Under load this can cause governance-critical evaluation to queue behind routine worker calls.
  • Misleading documentation increases the chance of future callers setting priorities and expecting them to be honoured.

Suggested fix

Replace the semaphore channel with a proper heap-based priority queue (e.g. container/heap) that dequeues by descending priority. Or, if strict priority isn't needed yet, remove the Priority field from InferenceRequest and update the docs to say the queue is FIFO with a concurrency cap.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium prioritybugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions