When a real-world evidence study gets scoped, the conversation usually begins with structured data: ICD-10 diagnosis codes, CPT procedure codes, NDC medication codes, LOINC lab values, demographics. These fields are queryable. You can pull a list of patients with I11.0 (hypertensive heart disease with heart failure) in under a minute. That part of RWE construction is largely solved.
The part that breaks teams is everything else. Exposure history documented in a referral letter from a referring physician. Contraindicated conditions mentioned in a surgical consult note but never coded. Adverse events described narratively in a follow-up visit note and not attached to any billing event. Drug dose adjustments written in provider shorthand ("halved losartan due to hypotension"). The structured fields give you a skeleton. The notes contain the actual clinical story, and for most research questions, you need the story.
The anatomy of a typical RWE bottleneck
We have seen this pattern consistently across teams doing retrospective cohort studies. The structured data pull takes a few days. Cohort definition is drafted against structured fields. Then the research question reveals that one or more of the critical variables, typically an exposure definition, an outcome, or an exclusion criterion, cannot be reliably ascertained from structured fields alone. The study requires note review.
At that point, the project timeline approximately doubles. A study designed for a four-month completion window stretches to seven or eight months when note review is added. The bottleneck is not the existence of the notes. The bottleneck is that reading them is currently human labor, allocated from a pool of analysts who are already stretched across multiple studies.
What manual chart review actually costs per study
Published estimates for manual note abstraction time range from 15 to 45 minutes per record, depending on note volume and complexity. For a 500-patient cohort requiring note review for a primary outcome definition, that is somewhere between 125 and 375 analyst hours. At fully-loaded analyst cost, this translates to a cost item that is often the single largest budget line in a retrospective study, exceeding data acquisition costs and statistical analysis combined.
Those are the direct hours. The indirect cost is harder to measure. When note review is blocking analysis, the statistical analysis team waits. When two studies are in the queue for the same analyst pool, one of them slips. When a study slips past a regulatory submission window, the financial consequence can be substantial. The 60 to 80 percent of RWE study time that research operations teams report spending on data extraction rather than analysis is not a methodological choice. It is a capacity constraint masquerading as a methodology.
Structured data alone: what you can and cannot do
We want to be precise about what structured data actually gives you, because the picture is more complex than "notes are better than codes." ICD codes are useful for many research questions. Prevalence estimates at a population level, covariate adjustment in regression models, administrative database studies where notes are not available: all of these rely appropriately on structured fields. The code-only approach is not wrong by definition. It is wrong for specific research questions where the phenomenon of interest requires clinical judgment to identify, and that judgment is documented in notes.
The conditions where note-level verification adds the most value include: primary outcome ascertainment for composite endpoints (where component adjudication matters), exposure characterization that requires dose, duration, and indication (which billing codes do not capture), exclusion criteria based on clinical history rather than active diagnoses, and adverse event attribution where causality assessment is required. For these questions, relying on structured fields introduces systematic bias that cannot be corrected in analysis.
The scale problem is structural, not a personnel issue
It is worth addressing a common framing mistake: teams sometimes describe their note review problem as a hiring problem. "If we had two more abstractors, we would be fine." This may be true for a single study at a given volume. It is not a stable solution as evidence generation programs scale. Each additional cohort study adds note volume that is roughly proportional to patient count. The relationship between program size and required abstraction labor is linear if you do not change the method.
The more durable response to the volume problem is to change the ratio of analyst time to records processed. That means changing the unit of analyst labor from "read a note and extract values" to "review flagged candidates and make judgment calls on ambiguous cases." The former is a rate-limited cognitive task. The latter is a judgment-focused review that benefits from analyst expertise applied to the cases where it actually matters rather than applied uniformly to every record including those that would take any competent analyst thirty seconds to process.
What the pathway forward requires
The reason this problem persists despite years of conversation in the RWE community is not lack of awareness. It is that deploying NLP for note extraction requires domain-specific infrastructure that general tools do not provide. Clinical text has properties that distinguish it from general-purpose text: heavy use of abbreviations and shorthand (SOB for shortness of breath, HTN for hypertension, bilateral LE for bilateral lower extremities), frequent negation structures ("denies chest pain"), section-specific conventions (an Assessment and Plan section carries different semantic weight than a History of Present Illness section for the same diagnosis mention), and template contamination from copy-forward documentation practices.
A pipeline that processes clinical notes reliably enough to be used in research evidence generation needs to be calibrated against clinical text specifically, needs to handle negation and uncertainty markers, and needs to return source citations that allow human reviewers to verify and document their ascertainment decisions. Without citations, automated extraction is unauditable and therefore unusable for regulatory-grade RWE.
The teams that have moved past the bottleneck are not the ones with the most abstractors. They are the ones that restructured their process so that automation handles the candidate identification step and analysts focus on the adjudication step. That restructuring requires a credible extraction tool, a workflow for handling flagged cases, and a defined confidence threshold for auto-acceptance versus analyst review. None of it is trivial to set up, but the setup cost is a one-time investment against a recurring labor problem that compounds with every new study added to the program.
