The Research Question: What Matters When Nothing Works

A study is a machine for producing an answer. A research question is a claim about what is worth knowing. They are different objects, and nearly all research training in medicine is training in the first one.

The machine gets the scrutiny. Randomization, allocation concealment, power calculations, the pre-registered analysis plan, the handling of missing data, the choice of outcome measure. A protocol goes to a review committee and comes back marked up. A statistician is consulted. Reviewers at a journal spend most of their attention on whether the machine was built correctly.

The question is not examined this way. It arrives from upstream — a mentor's existing program, a funding announcement, the limitations paragraph of the previous paper — and it is treated as a given. The most consequential decision in the entire enterprise is the one nobody supervises.

The question is a separate intellectual object with its own standards. Nursing's research tradition has an explicit apparatus for producing questions. Medicine's largely does not. The difference becomes visible the moment you practice somewhere the proven treatment cannot be delivered.

What PICO Cannot Hold

In 1995, W. Scott Richardson, Mark Wilson, Jim Nishikawa, and Robert Hayward published a short editorial in ACP Journal Club titled "The Well-Built Clinical Question: A Key to Evidence-Based Decisions." It supplied the format that has organized clinical inquiry ever since: patient or population, intervention, comparison, outcome. PICO.

PICO is a good instrument. It forces a vague clinical worry into a form that can be searched and answered, and it has probably done more for bedside reasoning than any other teaching device of the last forty years. It also has four slots, and three of them presuppose that the world contains an intervention, an alternative to it, and a measurable outcome that follows from choosing between them.

Remove those presuppositions.

A composite, invented for this essay and drawn from no patient: a woman in her fifties seen at a rural clinic, hemoglobin A1c above twelve, home four hours away over roads that close, no reliable refrigeration for insulin through the summer, and a daughter who manages the household money and does not accept that the diagnosis is real. The trial evidence about what to do with an A1c above twelve is extensive, high in quality, and inert. The comparison arm does not exist here. The intervention cannot be delivered. PICO has nowhere to put the daughter.

The reflex is to call this an implementation problem rather than a research problem. The science is settled, the delivery is broken, someone in operations should fix it. That reflex is a way of declining to ask a question. It assumes the knowledge is complete and only the plumbing has failed. In the room it is the reverse. The efficacy knowledge is complete, and the knowledge that would change what happens on Monday does not exist, has not been produced, and is not on anyone's grant.

The questions that matter in that room run in three directions the trial apparatus cannot follow.

One runs backward from the encounter. Who decides in this household. What does the family believe is happening inside the body. What did the last clinician say that made the patient stop coming. These are questions about the conditions under which any intervention could be accepted at all, and they are answerable by methods that already exist.

One runs sideways, into the constraint itself. What does rationing do to the person being rationed, and to the clinician doing the rationing. Nursing has been asking a version of this since Andrew Jameton named moral distress in Nursing Practice: The Ethical Issues (Prentice-Hall, 1984) — the condition of knowing the right action and being institutionally prevented from taking it. There is a substantial nursing literature on it now. There is very little asking the same question of the physicians in the same building.

One runs forward, past the endpoint the trial measured. What happened to the household after the diagnosis. What was sold. Who stopped working. The trial's outcome is a laboratory value. The family's outcome is whether the daughter kept her job.

Where Questions Come From

Issue 012 of this publication followed the standpoint tradition, Sandra Harding with Donna Haraway and Patricia Hill Collins, past Risjord and into its methodological consequences. Issue 026 took one of those consequences into research design and set out what Harding means by strong objectivity. Conventional objectivity works by stripping out the individual investigator's position, which leaves untouched the assumptions an entire field holds in common and therefore cannot see. The correction is to start inquiry from the lives of the people a system serves worst, not because their view is purer, but because that is the standing place from which the shared assumptions become visible.

In that essay the argument was about who holds authority over a study. The addition here is narrower. Strong objectivity is a rule for generating questions before it is a rule for evaluating answers. It tells you where to stand while you are deciding what to ask.

The corollary to Risjord's account that has stayed underdeveloped is that the nursing position also generates questions that are not derivative of medicine's, and that the question is where a standpoint does its earliest and most consequential work. By the time a study is designed, the standpoint has already made its decisive contribution and left the room.

A question also fixes the unit of analysis long before anyone argues about statistics, and the unit is itself a claim about where the relevant reality lives. The individual patient is the default unit in trial-derived questions because the individual patient is the thing that can be randomized. The household, the ward, the clinic schedule, and the road are not randomizable, so they are not units, so they are not studied, so nothing accumulates about them. Nothing in that chain is a finding about the world. It is a property of the instrument that decides what gets asked.

The clearest demonstration in the history of health research is Barney Glaser and Anselm Strauss's Awareness of Dying (Aldine, 1965), the hospital fieldwork behind The Discovery of Grounded Theory (Aldine, 1967). Glaser and Strauss went onto wards and asked what was actually happening around dying patients: who knew, who was pretending not to know, and how the staff sustained the pretense. Nobody was asking that. The medical research apparatus of 1965 could not formulate it, because there was no intervention in it.

Jeanne Quint Benoliel, a nurse who worked alongside them at the University of California, San Francisco, carried the inquiry into nursing with The Nurse and the Dying Patient (Macmillan, 1967), and asked a further question: what happens to the nurse assigned to a dying patient and given no preparation for it. That question opened a research program on death and dying in nursing that had not existed before it.

Grounded theory's defining feature is that the question is permitted to change as the data arrives. Pre-registration's defining feature is that it is not. These are not competing virtues. They are instruments for two different stages, and a research culture holding only the second one has quietly decided that the question is settled before the work begins.

An apparatus for producing questions has two procedural requirements. The researcher has to be in the setting before the question is fixed, and the question has to be allowed to change once they are there. Neither is exotic and both are teachable. What keeps them rare in medical training is not their difficulty. It is that a question still in motion reads, to a review committee, as a proposal that is not ready.

The same point sits inside Margarete Sandelowski's defense of plain description, which Issue 030 of this publication took up as one of the designs that fits a constrained primary care setting. She was arguing for the legitimacy of a study whose entire purpose is an accurate account of an event as the people inside it understood it. That is not a lesser study. Issue 030 put the design to work on acceptability, the question of whether an intervention will be taken up at all. Its use here is earlier than that. It is frequently the only thing that can tell you what the question is.

The trial apparatus is not the problem. It is one of the most reliable instruments anyone has built for a particular job: deciding between two available options when the difference between them is too small for intuition to see. Nursing science runs those trials too, and has for decades. The contrast worth drawing is not between disciplines but between instruments. The qualitative traditions have their own failure mode, which is the beautifully described finding that everyone already knew. The two traditions are equipped for different stages of inquiry. The stage at which the question is chosen is the one medicine has left unequipped.

The two pipelines defend different things. Biomedical training defends the design and assumes the question. Nursing science makes the question survive an argument before anyone will discuss the design. That is a difference in what counts as the hard part of the work.

Research Done to Publish, Research Done to Learn

Iain Chalmers and Paul Glasziou put the problem in numbers. In "Avoidable Waste in the Production and Reporting of Research Evidence" (The Lancet, 2009), they estimated that roughly eighty-five percent of biomedical research investment is wasted, and they placed the first and largest source of that waste at the choice of question: research that does not address questions relevant to the people who would use the answers. The Lancet series that followed in 2014 opened with priority setting for that reason.

The James Lind Alliance in the United Kingdom was built as a direct response. Its Priority Setting Partnerships put patients, carers, and clinicians in a room to name the unanswered questions that matter to them, and then hand that list to researchers. The lists that come out do not resemble the lists that come out of investigator meetings. John Ioannidis made the point from the other side in "Why Most Clinical Research Is Not Useful" (PLoS Medicine, 2016): a study can be methodologically impeccable and still useless, and the usual reason is the question.

There is a test for the difference between research done to publish and research done to learn, and it has nothing to do with rigor. Research done to publish begins with a question whose answer can already be predicted within a narrow band. The predictability is what makes the work fundable and the paper acceptable. Research done to learn begins with a question whose answer cannot be predicted, which is also why it is hard to fund.

There is a second test, blunter. A question is real if either answer would change what you do on Monday. If both possible answers leave the clinic running exactly as it runs now, what you have is a study.

Paul Farmer built a career on inverting one bad question. The standard question about tuberculosis treatment failure in rural Haiti was why patients were non-compliant. That question locates the defect in the patient and produces interventions aimed at educating the patient. Issue 009 of this publication argued that a clinical plan built without the determinants of adherence, and then read as the patient's failure when adherence fails, has put the defect in the wrong place. The research version of that error sits one step upstream of the plan, in the question that produced it.

In Infections and Inequalities: The Modern Plagues (University of California Press, 1999), Farmer described what happened when the question was replaced: what conditions would have to hold for a very poor patient to complete a year of therapy. The answer involved food, transport money, and a health worker who came to the house. The difference in treatment completion was not marginal. The pharmacology had not changed. The question had.

That is the whole distinction. The study was within reach of anyone practicing there. The question was not, because asking it required standing somewhere other than the center.

What keeps a clinician working in a thin system is usually described as service, and the description is incomplete rather than wrong. The more honest account is that the questions are better. In a system with everything, most of the fundamental questions have been asked and what remains is increment. Where the system is thin, the fundamental questions are still open, and they are open in front of you, in a room, at a particular hour, attached to a person whose name you know.

Issue 009 also argued that limited-resource medicine teaches reasoning under diagnostic constraint. The research analogue is reasoning under evidentiary constraint: forming a question when the answer you were trained to want does not exist and could not be used if it did. That is not a lesser form of inquiry. It is the form that produces new questions instead of new answers to inherited ones.

A field that only ever produces answers to questions it inherited is not learning. It is maintaining.

About the author

Peter Schindler, MD, PhD is an Assistant Professor of Medicine and Associate Program Director of the Community Health Center Family Medicine Residency Program at the University of Nebraska Medical Center. He practices at Winnebago Comprehensive Health System, OneWorld Community Health Center, and Nebraska Medicine. He completed a Primary Care Research Fellowship at McGill University and holds a BSN from the University of Wisconsin-Oshkosh, an MS and PhD in nursing from Emory University's Laney Graduate School, a Diploma in Tropical Medicine from the Liverpool School of Tropical Medicine, and an MD from the Medical College of Wisconsin-Green Bay. He is board certified by the American Board of Family Medicine. The Interprofessional publishes new essays every week at the intersection of medicine, nursing, and the clinical knowledge that lives between them.

Disclaimer. The views in The Interprofessional are Peter Schindler's own and do not represent the official positions of the University of Nebraska Medical Center, Winnebago Comprehensive Health System, OneWorld Community Health Center, Nebraska Medicine, or any other affiliated institution.