CHAPTER 7: Interpreting and Applying Evidence in Critically Ill Patients

KEY POINTS

• Interpreting evidence requires a basic understanding of how evidence is generated.

• There is not one ideal research design. Diverse research methods are needed to answer a broad range of questions confronting clinicians. However, an understanding of the potential weaknesses of each method is needed for clinicians to determine if the results are strong enough to change practice.

• Observational studies are efficient and can provide data when randomized trials are unavailable or infeasible, but determining causality and estimating treatment effects (causal inference) require complicated methods to identify and account for biases that may, in some cases, be insurmountable.

• Randomized trials are protected against many of the biases of observational research, but they are expensive and time-consuming and may provide uninformative results if underpowered (insufficiently large), poorly designed, or inappropriately analyzed. Assessing and applying clinical trial results require an understanding of the methods used to design and conduct clinical trials.

• Novel trial designs are improving trial efficiency (ie, fewer patients and faster results), and novel methods of analysis are improving the robustness of their interpretation (ie, moving away from interpretations of statistical significance as “positive or negative,” to probabilistic assessments of trial results and more robust subgroup assessments), but these changes are accompanied by an increase in complexity that may be challenging for many clinicians to understand.

INTRODUCTION

Clinicians are confronted daily by dozens of treatment decisions regarding which therapy is best for their patients. Clinical research is intended to assist clinicians in making the best choices for their patients, but medical knowledge is in a constant state of evolution. Treatment guidelines are intended to help clinicians interpret and apply evidence, but they lag years or decades behind the publication of new data. As an additional challenge, recent methodologic advances to improve the efficiency of clinical research and increase the robustness of its analyses have increased the complexity of clinical research far beyond what is traditionally taught in medical school or even master’s level coursework. The goal of this chapter, therefore, is to provide clinicians with a basic understanding of how modern clinical research is conducted and analyzed and to assist clinicians in differentiating high-quality data that should change their practice from low-quality data that is, at best, hypothesis-generating. To accomplish this goal, we will briefly discuss how clinical studies answer questions, the types of studies that are suited for answering different types of questions (with their advantages and limitations), and how results should be interpreted and applied.

HOW DO CLINICAL STUDIES ANSWER QUESTIONS?

Identifying the research question is the most important step when assessing a manuscript regardless of its design. Difficulties in understanding the research question are a major red flag for any clinical study. Good research questions are clear: What is the effect of treatment X versus Y on endpoint Z? Is the incidence of disease W decreasing over time? For each research question, ...