ollama-agent-queue — Test Cases

Functional

  1. Enqueue valid request
    • Input: full required JSON payload.
    • Expect: pending increments, request appears in queue.json.
  2. Reject missing required fields
    • Input missing agent_id.
    • Expect: non-zero exit + explicit missing field error.
  3. Priority ordering
    • Enqueue 3 requests: normal, urgent, high.
    • Process repeatedly.
    • Expect order: urgent → high → normal.
  4. Duplicate agent_id protection
    • Enqueue same agent_id twice.
    • Expect second enqueue rejected.
  5. Pause/resume behavior
    • Pause queue, enqueue request, run process-once.
    • Expect queue_paused and no processing.
    • Resume then process; expect normal execution.

Reliability

  1. Model timeout handling
    • Process with low timeout on long prompt.
    • Expect callback file with status=timeout and timeout error.
  2. Ollama failure propagation
    • Use invalid model string.
    • Expect status=error and error detail captured.
  3. Crash-safe queue file write
    • Verify queue.json.tmp swaps atomically to queue.json.
  4. Lock-file discipline
    • Create queue.lock manually and run process-once.
    • Expect no new processing while lock exists.
  5. Offline Ollama handling
    • Stop Ollama and run worker/process-once.
    • Expect 3 retries with backoff, queue status paused_ollama_offline, pending items failed with error outputs.
  6. Resource-overload retry behavior
    • Simulate 503 / out of memory response.
    • Expect one 30s backoff retry, then fail/continue if still broken.
  7. Consecutive resource-failure pause
    • Force 3 consecutive resource errors across items.
    • Expect queue status becomes paused and direct alert emitted.
  8. Stale-lock recovery path
    • Create lock older than 10 minutes with current_agent set and no Ollama process active.
    • Expect stale lock cleared, current agent timed out to callback, queue resumes.

Output contract

  1. Result schema completeness
    • Expect fields: agent_id, calling_skill, model, status, result/error, duration_seconds, completed_at.
  2. Callback path honored
    • Set custom callback path.
    • Expect result JSON at that exact path.

Trigger-eval phrases (skill routing)

Should trigger

  1. “Show me the agent queue status”
  2. “How many agents are queued right now”
  3. “Pause the agent queue”
  4. “Something is stuck in the queue — skip current agent”
  5. “Clear the agent queue, we’re starting fresh”
  6. Council of Advisors registers 6 agents simultaneously (automatic)
  7. Memory-consolidator registers 1 agent (automatic)

Should NOT trigger

  1. “Draft an acknowledgment for the donation that came in”
  2. “What’s in my approval queue?” (queue-optimizer domain)
  3. “Run the morning brief”
  4. “Check system health”
  5. “Post to social media”