A collaboration between Lewis McLain & AI
Why Good Analysis Begins Long Before Data — and Why Asking Better Questions Is a Skill That Must Be Practiced
I. The Invisible Starting Line
Every serious analysis begins with a question.
Almost every serious failure begins with the wrong one.
This is uncomfortable because it means that many errors are not technical. They are not caused by bad data, weak models, insufficient funding, or lack of expertise. They occur before any of that—at the moment a question is framed, accepted, and allowed to go unchallenged.
Questions are often inherited rather than chosen. They arrive embedded in headlines, legislation, grant applications, consulting scopes, software templates, or political urgency. By the time anyone pauses to ask whether the question itself is sound, the machinery is already moving.
Once that happens, better data does not fix the problem.
It accelerates it.
Precision is not clarity. A precisely answered wrong question produces results that feel authoritative while being fundamentally misleading. This is why analysis so often fails quietly and confidently.
II. The Four Types of Questions (And Why Only One Sustains Analysis)
Not all questions do the same kind of work. Most confusion in public debate and institutional decision-making comes from treating very different questions as if they were interchangeable.
1. Descriptive Questions
What is happening?
These establish facts, counts, and trends. They are necessary, but inert. Description alone does not explain change, causation, or constraint. Mistaking description for understanding is one of the most common analytical errors.
2. Attributional Questions
Who is responsible?
These arrive early and loudly. They satisfy emotional and political needs, but they tend to collapse complex systems into villains and heroes. Attribution feels like insight, but it usually precedes understanding.
3. Prescriptive Questions
What should we do?
These feel decisive and productive. They are also dangerous when asked prematurely. Prescriptions lock systems into action paths that may be impossible to reverse, even if the diagnosis was wrong.
4. Analytical Questions
What changed, relative to what, over what time horizon, and under which constraints?
These are the least intuitive and least rewarded questions, yet they are the only ones that scale. They slow the conversation down, resist moral shortcuts, and force structure onto complexity.
Most debates skip directly from description to prescription. Analysis happens, if at all, in the margins.
III. Time Horizons: The Quiet Distorter
Every question implies a time frame, whether stated or not. When it goes unstated, it is almost always too short.
Systems behave differently over one year than over five, and differently again over a generation. Short horizons hide maturation effects, suppress lagged consequences, and reward surface solutions. Long horizons expose tradeoffs, reveal inevitabilities, and demand humility.
When someone asks, “Why is this happening now?” without clarifying whether “now” means this quarter, this decade, or this lifecycle stage, the answer will be confident and wrong.
A reliable analytical rule is simple:
If the time horizon is unstated, it is probably distorting the conclusion.
IV. Baselines: The Question Nobody Wants to Ask
“Compared to what?” is the most expensive sentence in analysis.
Baselines are almost always chosen quietly and defended rarely. Yet they determine whether something appears as growth or stagnation, crisis or normal variation, success or failure.
Common baseline errors include:
- Comparing growing systems to static ones
- Comparing interventions to “doing nothing,” which never exists
- Comparing today to yesterday instead of to trend or lifecycle stage
Without a baseline, change has no meaning. Without an agreed-upon baseline, debate becomes endless recalibration rather than understanding.
The refusal—or failure—to ask baseline questions is not a technical oversight. It is often a psychological one. Baselines make certain narratives harder to maintain.
V. The Substitution Problem
Systems do not eliminate pressure. They redirect it.
Every policy, reform, or intervention substitutes one cost, risk, or burden for another. The analytical failure is not unintended consequences; it is unacknowledged substitution.
When analysis celebrates a solution without tracing where pressure moved, it is incomplete by definition. The question “What problem did we solve?” must be followed immediately by “Where did the pressure go?”
Ignoring substitution allows success to be declared in one domain while strain accumulates invisibly in another.
VI. Metrics Are Mirrors, Not Truth
Metrics are indispensable—and dangerous.
They capture what is easy to measure, not necessarily what matters most. They reward visibility, not durability. They improve responsiveness but often degrade resilience.
Measurement should provoke questions, not end them. When metrics become substitutes for judgment, they stop illuminating reality and begin reflecting institutional incentives back at themselves.
What improves on paper may be decaying in practice. The analyst’s task is not to reject metrics, but to interrogate them relentlessly.
VII. The Discipline of the Second Question
Most people ask one good question. Then they stop.
The first question usually reveals curiosity. The second reveals discipline.
- First question: What happened?
- Second question: Relative to what expectation?
- Third question: Why now and not earlier?
- Fourth question: At whose expense did this improve?
- Fifth question: What constraint was binding?
Most analytical errors occur between questions one and two. The pause required to ask the second question feels unproductive, even obstructive. In reality, it is where understanding begins.
VIII. Asking Good Questions Is a Skill — and It Must Be Practiced
The ability to ask good questions is not innate. It is trained.
It requires resisting the urge to sound smart quickly. It requires tolerating ambiguity longer than is comfortable. It requires being willing to appear slow, cautious, or even naïve in environments that reward speed and certainty.
Like any discipline, it improves through repetition:
- Reviewing past analyses and identifying where the wrong question was asked
- Practicing reframing problems in multiple ways before selecting one
- Studying failures not for answers, but for misframed questions
- Learning to sit with incomplete understanding without rushing to closure
Good questioners are not passive. They are rigorous. They know that the hardest work happens before the first chart, model, or recommendation.
IX. What Your Questions Reveal About You
Questions are diagnostic. They reveal far more about the questioner than about the subject being questioned.
They reveal:
- Whether someone is seeking understanding or validation
- Whether they tolerate uncertainty or rush to control
- Whether they think in systems or in narratives
- Whether they are curious about limits or allergic to them
A person who habitually asks attributional questions before analytical ones is revealing impatience with complexity. A person who never asks baseline or time-horizon questions is revealing comfort with surface explanations.
In this sense, questions are a form of moral autobiography. Over time, they expose whether a person is oriented toward truth, persuasion, blame, or reassurance.
X. Analysis as Responsibility
Analysis is not neutral. It shapes how resources are allocated, how authority is exercised, and how force—legal, financial, or moral—is applied.
Bad questions do not merely mislead; they coerce. They narrow the range of permissible answers and foreclose alternatives before they are considered.
The responsibility of the analyst is not certainty. It is honesty about limits, tradeoffs, and unknowns. Asking better questions is not intellectual vanity; it is an ethical act.
Conclusion
The most dangerous answers are not the wrong ones.
They are the ones that emerge from unexamined questions.
Before asking what the data says, before debating solutions, before declaring success or failure, the analyst owes one discipline above all others:
Stop.
Name the question.
Interrogate it.
And be willing to change it.
That pause—unrewarded, uncomfortable, and often invisible—is where real thinking begins.
Let's Talk About It, Lewis! 