Experiment Queue Template

Status

Planned — create a reusable experiment-queue template for ONE and Manitec research work.

Required Template Fields

• Experiment title and research question
• Hypothesis and decision it informs
• Owner and status
• System/version/provider under test
• Scope, fixtures, and inputs
• Constraints and security/data-handling boundaries
• Success criteria and failure criteria
• Required output artifacts and trace locations
• Evaluation method, human scoring rubric, and baseline
• Quality, task-success, latency, token/cost, and reliability measurements
• Findings, decision, follow-up, and evidence links

Definition of Done

The template is usable for small tool probes and larger agent-system benchmarks, produces comparable records across runs, and makes adoption/defer/reject decisions auditable.