Research Toolkit  /  Developing Ideas

Developing Ideas

A contribution is a belief you move. Before you can move one, you have to establish that there is something to explain, name precisely what is not yet known, and articulate the rival accounts your evidence will have to rule out.

“Interesting” means denying a prior. Affirm everything and you have written a summary, not a contribution.

Setting up
Merton’s Three Fragments

Three moves, in order — not a gap hunt

Searching the literature for an unfilled cell produces questions nobody asked. These three moves produce a phenomenon worth explaining, and each depends on the one before it.

01

Establish the phenomenon

Before explaining why something happens, show that it happens — real, prevalent, measurable. Explain an effect nobody has demonstrated and you are explaining an artifact.

02

Specify ignorance

Not “little is known,” which is true of almost everything and interesting about nothing. Name the exact unknown whose resolution would let more knowledge follow.

03

Justify the setting

A context earns its place by offering leverage on your constructs — what you want to compare actually varies there, observably. Not because the data are on your hard drive.

“We should study mentoring more” versus “we do not know whether the mentoring relationship causes the promotions or merely marks the people who would have been promoted anyway.” Only the second is answerable.

The rule
The Three-Story Approach

Three accounts before you may claim evidence for one

Before claiming evidence for a mechanism, articulate at least three theoretically motivated, mechanism-specific accounts of the phenomenon. Not three variables — three stories about how the world produced the fact.

WHY THREE

One cannot be confirmed

With a single story there is nothing for the evidence to rule out. Two collapses into a straw man flanking the answer you intended from the start.

THE REASON

The world is equifinal

Many paths reach the same outcome. The same promotion gap fits mentoring that works, abler people sorted into mentoring, and high-potentials handed mentors as a reward already earned.

DISCIPLINE 01

Equal footing

The stories stand as peers. A rival introduced only to be dismissed was never alive, and the reader can tell.

DISCIPLINE 02

Give each rival its best case

State every account in the form its most convinced supporter would accept, not in a weakened form you can knock down. A rival you never took seriously cannot be ruled out convincingly, and the reader can tell. Treat the colleague who asks “couldn’t it just be…?” as someone filling in your map, not attacking you.

Getting the mechanism wrong means prescribing the wrong fix. Assigning more mentors does nothing for a gap produced by sorting; training mentors cannot touch a gap produced by who gets chosen in the first place.

Framework 01
Coleman’s Boat

Connecting the levels

The macro-to-macro arrow is the correlation everyone can already see. The explanation is the trip down to the actor’s situation, across to the action, and back up to the outcome.

Macro condition Market, policy, structure Macro outcome The disparity observed Actor’s situation Beliefs · desires · opportunities Actor’s action The choice made THE CORRELATION — NOT AN EXPLANATION SITUATIONAL ACTION-FORMATION TRANSFORMATIONAL
Coleman’s boat. The top arrow is the correlation everyone can see. The mechanism is the trip down, across, and back up.
Framework 02
Elster’s Diagram

Choosing a context where the stories diverge

Each rival account implies things you would see if it were running, and things you would not. Lay them side by side and the design brief writes itself: find the setting where the fingerprints differ, and collect evidence where they differ most.

The argument as an Elster diagram A phenomenon at the top, the rival mechanisms that could have produced it in the middle, and beneath each one the implications that would follow if it were the process actually running. This is where you decide what evidence would change your mind, before you collect any. START HERE · THE FOCAL PHENOMENON State the fact that needs explaining: real, prevalent, measurable. Not a hypothesis, and not a gap in a literature. Something a reader cannot wave away, established before you explain it. If the phenomenon can be dismissed, everything below is explaining an artifact. each arrow reads: this process, if running, would produce that phenomenon ALTERNATIVE M2 · A rival account A competing process that would generate the same phenomenon. State it as its strongest proponent would. A rival introduced only to be knocked down was never alive, and the reader can tell. PREFERRED M1 · Your mechanism Name an actor and a process, not a variable. It needs a subject and a verb: who does what, under what conditions, with what consequence. Give it an informative label such as learning, matching, or valuation. ALTERNATIVE M3 · A rival account A third live process. Three is the minimum: one cannot be confirmed because nothing is ruled out, and two collapses into a straw man flanking the answer you intended from the start. THEN ASK · WHAT WOULD FOLLOW IF THIS ONE WERE TRUE? Each mechanism implies things you would see, and things you would not. Those implications, not the phenomenon, are what separate the three. NOT OBSERVED · TELLS AGAINST The strongest element on the diagram. The rival’s own logic requires this implication and the data do not show it, so the rival is ruled out on its own terms rather than by assertion. It only works if you state it before you look. OBSERVED · SUPPORTS An implication that follows from your mechanism and from no other on the diagram. This is what to design for: a signature, not a shared prediction. Auxiliary observations that fit only your account belong here too. OBSERVED · DOES NOT DISCRIMINATE True, and useless for adjudication. Every mechanism predicts it, so seeing it confirms nothing. Mark these explicitly. The most common inferential error is treating a shared implication as support for the preferred account. WHAT THE DIAGRAM IS FOR Two jobs, both before data collection. It turns a hunch into a decision procedure: you commit in advance to which observations would favour your account and which would sink it, so the finding cannot be assembled after the fact. And it chooses your setting, because the right context is the one where the implications visibly pull apart, not the one whose data you happen to hold. Read the not-observed boxes aloud and you have your threats list. READING THE ELEMENTS Maize — the preferred chain Blue — an alternative chain Dashed — an implication of the box above Modified from Elster, J. (2007). Explaining Social Behavior, Fig. 1.2.
The diagram is a commitment device. Fill it in before collecting data, and the finding cannot be assembled after the fact.

The diagram’s rivals and the design’s threats are one list seen from two sides. Omitted-variable bias is the selection story in methodological dress; reverse causality is the reward story.

← ToolkitFraming Arguments →