Field note: Operational questions often arrive as “Does the program work?” or “Why are people leaving?” Those questions matter, but they are too broad to guide proportionate data collection. Research becomes useful when the intended decision and acceptable uncertainty are explicit.
Operational question
Who will use the answer, what will they decide, and which narrower question can the available time, access, and evidence address credibly?
A workable method
- Name the decision and user. State the next decision, its date, the alternatives under consideration, and who owns it. Evidence without a decision pathway is likely to arrive unused.
- Describe the intervention and context. Clarify what is changing, for whom, where, and under which conditions. Include the current practice or plausible comparison.
- Select the primary question type. Distinguish implementation, experience, outcome, impact, mechanism, and value questions. Each calls for different evidence and claims.
- Set scope and limits. Define population, period, primary outcome, subgroup needs, data access, ethical constraints, and what the study will not establish.
What this looks like in practice
“Does simulator training work?” becomes: among newly qualified deck officers in this program, how does completion of three coached collision-avoidance scenarios affect performance on a delayed transfer scenario compared with the current curriculum, and which implementation factors explain variation?
Evidence to collect
Choose a small set of measures before implementation. Record the baseline, the source of each measure, the review cadence, and who is authorized to act on the result.
- alignment between question and decision
- feasibility of collecting credible evidence
- clarity of intended and excluded claims
Field checklist
- Write the decision, accountable owner, and decision date.
- Describe the current workflow and the conditions that shape performance.
- Confirm the source hierarchy, permissions, and local requirements.
- Test the method under representative—not merely convenient—conditions.
- Review both intended outcomes and burden on the people doing the work.
- Record a change, escalation, and stop rule before results arrive.
Watch-out
Do not let available data silently define the question. Start with the decision, then decide whether existing data can answer it, partially answer it, or only describe a convenient proxy.