Cancer Blood Tests Part 2: The clinical trial

Cancer Blood Tests Part 2: The clinical trial

How do you decide whether a clinical trial “worked”? In Part 2 of our Galleri series, we examine the landmark randomized trial of a blood test designed to detect more than 50 cancers. We explore why different outcome measures led to dramatically different headlines, discuss primary versus secondary outcomes, pre-registration, hierarchical testing, and post hoc analyses, and explain why mortality remains the outcome everyone is waiting for. Along the way, we uncover a statistical mystery involving dozens of missing cancers and discover how a little arithmetic can sometimes reveal more than a press release. Statistical topics * cancer screening * exploratory analyses * hierarchical testing * missing data * multiple testing * outcome measures * post hoc analyses * pre-registration * primary and secondary outcomes * randomized clinical trials * screening tests Methodologic Morals * “When the simple numbers don't add up, pay attention. The arithmetic may be trying to tell you something.” * “The first question should not be, did it work? It should be, what counts as success?” References * Giridhar KV, et al. Safety and performance results from PATHFINDER 2, a registrational study of a multi-cancer early detection test in an intended-use population [https://grail.com/wp-content/uploads/2026/05/Giridhar.ASCO-2026.PF2-Primary.ORAL_FINAL-For-GRAIL-Website.pdf]. Presented at the 2026 American Society of Clinical Oncology (ASCO) Annual Meeting. May 2026. * Hubbell E, Clarke CA, Aravanis AM, Berg CD. Modeled Reductions in Late-stage Cancer with a Multi-Cancer Early Detection Test. [https://pubmed.ncbi.nlm.nih.gov/33328254/]Cancer Epidemiol Biomarkers Prev. 2021;30(3):460-468. doi:10.1158/1055-9965.EPI-20-1134 * Neal RD, Johnson P, Clarke CA, et al. Cell-Free DNA-Based Multi-Cancer Early Detection Test in an Asymptomatic Screening Population (NHS-Galleri): Design of a Pragmatic, Prospective Randomised Controlled Trial. [https://pmc.ncbi.nlm.nih.gov/articles/PMC9564213/] Cancers (Basel). 2022;14(19):4818. Published 2022 Oct 1. doi:10.3390/cancers14194818 * ASCO slides: https://grail.com/wp-content/uploads/2026/05/Swanton_ASCO-2026_NHS-Galleri_FINAL-Slides-05.26.2026.pdf [https://grail.com/wp-content/uploads/2026/05/Swanton_ASCO-2026_NHS-Galleri_FINAL-Slides-05.26.2026.pdf] * UK registry protocol:  https://www.isrctn.com/ISRCTN91431511 [https://www.isrctn.com/ISRCTN91431511]  * Clinicaltrials.gov [http://clinicaltrials.gov] protocol: https://clinicaltrials.gov/study/NCT05611632 [https://clinicaltrials.gov/study/NCT05611632]  Common biases in cancer screening studies Cancer screening studies are subject to several well-known biases that can make a screening test appear more effective than it actually is. Three of the most important are: Lead-time bias: Screening advances the time of diagnosis, making survival from diagnosis appear longer even if the patient's lifespan is unchanged. For example, if a screening test detects a Stage II cancer at age 60 that otherwise would have been diagnosed because of symptoms at age 62, but the patient dies at age 68 regardless, survival from diagnosis appears to increase from 6 years to 8 years even though the patient did not live any longer.  Length bias: Screening preferentially detects slower-growing, less aggressive cancers because they remain detectable for longer than fast-growing cancers. For example, a slow-growing cancer that remains in Stage I for 5 years is much more likely to be found by screening than an aggressive cancer that progresses to symptoms within months. This can make screened patients appear to have better survival simply because screening preferentially found the less aggressive cancers.  Overdiagnosis: Screening detects cancers that would never have caused symptoms or death during a person's lifetime, leading to unnecessary diagnosis and treatment. For example, a screening test may detect a very slow-growing prostate or thyroid cancer in an older adult that would never have become clinically important if it had remained undiscovered.  Kristin and Regina’s online courses:  Demystifying Data: A Modern Approach to Statistical Understanding [https://online.stanford.edu/courses/som-xche0033-demystifying-data-modern-approach-statistical-understanding]   Clinical Trials: Design, Strategy, and Analysis [https://online.stanford.edu/courses/som-xche0030-clinical-trials-design-strategy-and-analysis]  Medical Statistics Certificate Program [https://online.stanford.edu/programs/medical-statistics-program]   Writing in the Sciences [https://www.coursera.org/learn/sciwrite]  Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Programs that we teach in: Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Find us on: Kristin -  LinkedIn [https://www.linkedin.com/in/kristin-sainani-642b5914/] & Twitter/X [https://x.com/KristinSainani] Regina - LinkedIn [https://www.linkedin.com/in/reginanuzzo/] & https://www.reginanuzzo.com/ReginaNuzzo.com [http://reginanuzzo.com] * (00:00) - Intro * (03:39) - The Claim: Not Ready for Primetime * (03:58) - Trial Design: 142,000 Participants * (07:50) - The Primary Outcome Problem * (20:29) - The Primary Endpoint: Complete Miss * (22:14) - Three Arguments for the Defense * (28:29) - - Statistical Sleuthing: Missing Cancers * (41:14) - - The Stage Shift Argument * (50:30) - - Rating the Claim

Cancer Blood Tests: Are they ready for primetime? Part 1

Can a single tube of blood really detect dozens of cancers before symptoms appear? We dive into the science behind Galleri, a blood test that claims to detect more than 50 types of cancer from a simple blood draw. Recent headlines about the test ranged from “breakthrough” to “bust” after the release of results from a massive randomized clinical trial. In this Part 1 episode, we explore cell-free DNA, DNA methylation, machine learning, sensitivity, specificity, and positive predictive value. Along the way, we revisit the prenatal screening revolution, ask why detecting cancer earlier doesn’t always help patients, and learn how escaped DNA convicts end up swimming in a giant molecular pool party. And for the first time ever, Normal Curves ends on a cliffhanger: we’ll save the controversial results of that landmark trial for Part 2. Statistical topics * cancer screening * case-control studies * counterfactuals * machine learning * negative predictive value * overdiagnosis * positive predictive value * randomized clinical trials * screening tests * sensitivity and specificity * validation References * Bianchi DW, Chudova D, Sehnert AJ, et al. Noninvasive prenatal testing and incidental detection of occult maternal malignancies [https://jamanetwork.com/journals/jama/fullarticle/2389341]. JAMA. 2015; 314:162-9.  * Liu MC, Oxnard GR, Klein EA, et al. Sensitive and specific multi-cancer detection and localization using methylation signatures in cell-free DNA [https://pmc.ncbi.nlm.nih.gov/articles/PMC8274402/]. Ann Oncol. 2020. 31:745-59.  * Schrag D, Beer T, McDonnell C et al. Blood-based tests for multicancer early detection (PATHFINDER): a prospective cohort study [https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(23)01700-2/abstract]. The Lancet. 402: 1251-60. * Giridhar KV, et al. Safety and performance results from PATHFINDER 2, a registrational study of a multi-cancer early detection test in an intended-use population [https://grail.com/wp-content/uploads/2026/05/Giridhar.ASCO-2026.PF2-Primary.ORAL_FINAL-For-GRAIL-Website.pdf]. Presented at the 2026 American Society of Clinical Oncology (ASCO) Annual Meeting. May 2026. Statistic discussed in the episode PATHFINDER 2 investigators reported that adding Galleri to routine screening increased the number of screen-detected cancers by 6.5-fold. This figure compares 31 cancers detected through USPSTF-recommended screening (for breast, cervical, lung, and colon) with 204 cancers detected when Galleri was added, counting the same 31 conventional-screening cancers in both totals. Thus, describing the increase as 6.5-fold is misleading, since the combination of Galleri plus conventional screening is, by definition, guaranteed to detect at least as many cancers as conventional screening alone. Moreover, everyone in the study received Galleri, whereas conventional screening depended on which tests participants happened to be due for and completed during the study period. The comparison therefore does not involve two equally applied screening strategies. Kristin and Regina’s online courses:  Demystifying Data: A Modern Approach to Statistical Understanding [https://online.stanford.edu/courses/som-xche0033-demystifying-data-modern-approach-statistical-understanding]   Clinical Trials: Design, Strategy, and Analysis [https://online.stanford.edu/courses/som-xche0030-clinical-trials-design-strategy-and-analysis]  Medical Statistics Certificate Program [https://online.stanford.edu/programs/medical-statistics-program]   Writing in the Sciences [https://www.coursera.org/learn/sciwrite]  Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Programs that we teach in: Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Find us on: Kristin -  LinkedIn [https://www.linkedin.com/in/kristin-sainani-642b5914/] & Twitter/X [https://x.com/KristinSainani] Regina - LinkedIn [https://www.linkedin.com/in/reginanuzzo/] & https://www.reginanuzzo.com/ReginaNuzzo.com [http://reginanuzzo.com] * (00:00) - - Introduction * (00:44) - - The Holy Grail of Cancer Testing * (04:31) - - Headlines: Same Data, Opposite Stories * (07:38) - - How Cell-Free DNA Works * (13:54) - - DNA Methylation: GRAIL's Fingerprint * (15:19) - - The Origin Story * (22:18) - - The Pathfinder Studies * (35:01) - - The Paradox: Why Earlier Detection Doesn't Always Help * (40:32) - - The Cliffhanger

Odds Ratios: Do most people get them wrong?

Odds ratios show up everywhere in medical research—but do readers, journalists, and even researchers always know what they mean? In this episode, we tackle one of the most common statistical misunderstandings in science: treating odds ratios like risk ratios. Along the way, we explore puppy photos, fish photos, first-date hookups, sugary drinks, cardiac care, and a listener challenge that started with an informal study of five medical residents and a box of chocolate truffles. We explain why logistic regression produces odds ratios, when odds ratios can wildly exaggerate effects, and why some famous headlines turned out to be much less dramatic than they sounded. Statistical topics * binary outcomes * case-control studies * logistic regression * odds ratios * risk ratios * odds vs risk  Methodological morals * “Just because logistic regression gives you an odds ratio does not mean you have to report it.” * “A lot of bad science communication starts long before the journalist even enters the story.” References * Bleich SN, Herring BJ, Flagg DD, et al. Reduction in purchases of sugar-sweetened beverages among low-income Black adolescents after exposure to caloric information [https://pmc.ncbi.nlm.nih.gov/articles/PMC3483987/]. Am J Public Health. 2012;102:329–35. * Sainani KL. How Statistics Can Mislead [https://pmc.ncbi.nlm.nih.gov/articles/PMC3464840/]. Am J Public Health. 2012. 2012;102:e3–e4. * Bleich SN, Herring BJ, Flagg DD, et al. Bleich et al. respond [https://pmc.ncbi.nlm.nih.gov/articles/PMC3464845/]. Am J Public Health. 2012;102:e4.   * Press video: https://www.youtube.com/watch?v=IFyrqbf1XWs [https://www.youtube.com/watch?v=IFyrqbf1XWs]  * Sainani KL, Schmajuk G, Liu V. A Caution on Interpreting Odds Ratios [https://pmc.ncbi.nlm.nih.gov/articles/PMC2717202/]. Sleep. 2009;32:976. * Schulman KA, Berlin JA, Harless W, et al. The Effect of Race and Sex on Physicians' Recommendations for Cardiac Catheterization [https://pubmed.ncbi.nlm.nih.gov/10029647/]. NEJM. 1999;340:618–26. * Schwartz LM, Woloshin S, Welch HG. Misunderstandings about the Effects of Race and Sex on Physicians' Referrals for Cardiac Catheterization [https://pubmed.ncbi.nlm.nih.gov/10413743/]. NEJM. 1999;341:279–83. * Associated Press. Study Finds Bias in Doctors' Care of Women and Blacks [https://www.nytimes.com/1999/02/25/us/national-news-briefs-doctor-bias-may-affect-heart-care-study-finds.html]. The New York Times. February 25, 1999. * Knol MJ, Duijnhoven RG, Grobbee DE, et al. Potential Misinterpretation of Treatment Effects Due to Use of Odds Ratios and Logistic Regression in Randomized Controlled Trials [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0021248]. PLoS ONE. 2011;6:e21248.   More information on logistic regression and odds ratios: * Sainani KL. Logistic Regression [https://onlinelibrary.wiley.com/doi/full/10.1016/j.pmrj.2014.10.006]. PM&R. 2014;6:1157–62. * Sainani KL. Understanding Odds Ratios [https://pubmed.ncbi.nlm.nih.gov/21402371/]. PM&R. 2011;3:263–67. * Nuzzo RL. Communicating measures of relative risk in plain English. [https://pubmed.ncbi.nlm.nih.gov/35038235/] PM&R. 2022;14:283-287. When outcomes are common, odds ratios can exaggerate effect sizes. Alternatives include: * Presenting raw percentages (absolute risks) * Presenting adjusted percentages from logistic regression (these may be calculated by plugging in means for the covariates) * Converting odds ratios to risk ratios * Reporting risk ratios directly when appropriate Converting Odds Ratios to Risk Ratios: * Zhang J, Yu KF. What's the Relative Risk? A Method of Correcting the Odds Ratio in Cohort Studies of Common Outcomes [https://jamanetwork.com/journals/jama/fullarticle/vol/280/pg/1690]. JAMA. 1998;280:1690–91. * ClinCalc. Odds Ratio to Relative Risk Calculator. *  https://clincalc.com/stats/convertor.aspx [https://clincalc.com/stats/convertor.aspx] * RR = OR / [(1 − P0) + (P0 × OR)] Example: OR=0.51, baseline risk=93.3% RR = 0.51 / [(1 − 0.933) + (0.933 × 0.51)] = 0.51 / (0.067 + 0.476) = 0.51 / 0.543 = 0.94 Thus, an odds ratio of 0.51 corresponds to a risk ratio of 0.94 when the baseline risk is 93.3%. The corresponding unadjusted risk ratio is 86%/93.3%=0.92 Correction: In the episode, we stated that the adjusted risk ratio was 0.92. In fact, it is 0.94, as shown above. 0.92 is the unadjusted risk ratio.  Kristin and Regina’s online courses:  Demystifying Data: A Modern Approach to Statistical Understanding [https://online.stanford.edu/courses/som-xche0033-demystifying-data-modern-approach-statistical-understanding]   Clinical Trials: Design, Strategy, and Analysis [https://online.stanford.edu/courses/som-xche0030-clinical-trials-design-strategy-and-analysis]  Medical Statistics Certificate Program [https://online.stanford.edu/programs/medical-statistics-program]   Writing in the Sciences [https://www.coursera.org/learn/sciwrite]  Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Programs that we teach in: Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Find us on: Kristin -  LinkedIn [https://www.linkedin.com/in/kristin-sainani-642b5914/] & Twitter/X [https://x.com/KristinSainani] Regina - LinkedIn [https://www.linkedin.com/in/reginanuzzo/] & https://www.reginanuzzo.com/ReginaNuzzo.com [http://reginanuzzo.com] * (00:00) - Introduction * (02:54) - What Are Odds Ratios? * (04:02) - Puppy Photos and First Dates * (06:09) - Risk Ratio Explained * (08:10) - Calculating Odds Ratios * (11:09) - Fish Photos and Reversed Numbers * (16:01) - Real-Life Example: Sugary Beverages * (22:08) - How Logistic Regression Works * (31:53) - The Video: Researchers Made the Mistake Themselves * (36:30) - The Cardiac Catheterization Study * (39:24) - The New York Times Printed a Correction * (46:10) - Using OR and RR Interchangeably for Case Control * (47:00) - Reye Syndrome and Aspirin * (49:37) - Rating the Claim and Methodological Morals

Coffee and the Heart: Is caffeine a trigger for AFib?

Does coffee trigger atrial fibrillation — or have doctors been warning people away from caffeine without strong evidence? We dig into two recent randomized clinical trials testing whether caffeinated coffee causes dangerous heart rhythm problems. Along the way, we talk about AFib, survival analysis, intention-to-treat versus as-treated analyses, and one surprisingly elaborate effort to catch clinical trial cheaters with receipts and geolocation tracking. We also explore how a pope may have fueled a European coffee resurgence, why plants make caffeine, and how a game show competition explains hazard ratios. Statistical topics * adherence and compliance * as-treated analysis * confidence intervals * Cox proportional hazards regression * hazard ratios * intention-to-treat analysis * micro-randomization * multiple testing * PICOT * pre-registration * primary vs secondary outcomes * randomized clinical trials * sensitivity analyses * SMART framework * survival analysis Methodological morals * “Never trust conventional wisdom until you see the randomized controlled trial.” * “Trust your participants, but design the study so that they can be honest about their dishonesty.” References * Harrington D, D'Agostino RB Sr, Gatsonis C, et al. New Guidelines for Statistical Reporting in the Journal [https://pubmed.ncbi.nlm.nih.gov/31314974/]. N Engl J Med. 2019;381(3):285-286. doi:10.1056/NEJMe1906559 * Marcus GM, Rosenthal DG, Nah G, et al. Acute Effects of Coffee Consumption on Health among Ambulatory Adults. [https://pubmed.ncbi.nlm.nih.gov/36947466/] N Engl J Med. 2023;388(12):1092-1100. doi:10.1056/NEJMoa2204737 * Wong CX, Cheung CC, Montenegro G, et al. Caffeinated Coffee Consumption or Abstinence to Reduce Atrial Fibrillation: The DECAF Randomized Clinical Trial. [https://pubmed.ncbi.nlm.nih.gov/41206802/]JAMA. 2026;335(4):317-325. doi:10.1001/jama.2025.21056 * @MarcKatzMD’s short video The Pitt- atrial fibrillation cardioversion scene  [https://youtube.com/shorts/MfkO5lpxPf8?si=iZIUvB-M0jSCoZMX] Kristin and Regina’s online courses: Demystifying Data: A Modern Approach to Statistical Understanding [https://online.stanford.edu/courses/som-xche0033-demystifying-data-modern-approach-statistical-understanding]   Clinical Trials: Design, Strategy, and Analysis [https://online.stanford.edu/courses/som-xche0030-clinical-trials-design-strategy-and-analysis]  Medical Statistics Certificate Program [https://online.stanford.edu/programs/medical-statistics-program]   Writing in the Sciences [https://www.coursera.org/learn/sciwrite]  Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Programs that we teach in: Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Find us on: Kristin -  LinkedIn [https://www.linkedin.com/in/kristin-sainani-642b5914/] & Twitter/X [https://x.com/KristinSainani] Regina - LinkedIn [https://www.linkedin.com/in/reginanuzzo/] & https://www.reginanuzzo.com/ReginaNuzzo.com [http://reginanuzzo.com] * (00:00) - - Introduction * (02:15) - - What is AFib? * (04:36) - - Frisky Goats and Satan's Bitter Invention * (10:44) - - How Caffeine Works * (14:43) - - The CRAVE Trial * (15:53) - - PICOT: Evaluating the Study Design * (23:24) - - CRAVE Results * (30:07) - - Catching the Coffee Cheaters * (37:01) - - The DECAF Trial * (41:30) - - Time-to-Event Outcomes * (43:21) - - Hazard Ratios: Balance Beams Over Shark Tanks * (47:06) - - DECAF Results: Team Coffee Wins * (50:38) - - Why Would Coffee Be Protective? * (53:57) - - Rating the Claim

Sleep and Exercise: Does working out on too little sleep speed up aging?

Can exercise actually be bad for you if you don’t get enough sleep? A widely shared claim says yes—that working out while sleep deprived may speed up aging. In this episode, we put that claim under the microscope. We examine the study behind it, unpack how sleep and aging were measured, and explore key statistical ideas like interaction effects and flexible models that can “dance” to the data. With the help of a $400,000 handbag and a man with seven boats, we also break down what it really takes to show that one variable changes the effect of another. What we find: some clear study bloopers, inconsistent modeling results, and interpretations that are flat-out wrong.  Statistical topics * Measurement error  * Model specification * Piecewise linear regression * Regression models * Residual confounding * Splines * Statistical interactions * Survey design Methodological morals * “Before you believe something shocking, ask what had to go wrong to make it true.” * “If slight modeling changes flip the story, there wasn't much story to begin with.” * “Unethical Life Pro Tip: If you do not want your analysis critiqued, then just make it impossible to understand.” Kristin’s Biological Age Calculator [https://www.normalcurves.com/biological-aging-calculator/] References * Original Viral Tweet: Ng D. "People who slept under 6 hours and exercised actually aged faster." [https://x.com/DrDominicNg/status/2030916265859338399] X. March 9, 2026. * Holmer B. Does exercise “age you faster” if you don’t sleep enough? [https://bradyholmer.medium.com/does-exercise-age-you-faster-if-you-dont-sleep-enough-8bb0806ea6c4] Medium. March 16, 2026. * You Y. Chen Y. Liu R., et al. Inverted U-shaped relationship between sleep duration and phenotypic age in US adults: a population-based study [https://www.nature.com/articles/s41598-024-56316-7]. Sci Rep. 2024;14:6247.  * Levine ME, Lu AT, Quach A, et al. An epigenetic biomarker of aging for lifespan and healthspan [https://pmc.ncbi.nlm.nih.gov/articles/PMC5940111/]. Aging. 2018;10:573-591.  Kristin and Regina’s online courses:  Demystifying Data: A Modern Approach to Statistical Understanding [https://online.stanford.edu/courses/som-xche0033-demystifying-data-modern-approach-statistical-understanding]   Clinical Trials: Design, Strategy, and Analysis [https://online.stanford.edu/courses/som-xche0030-clinical-trials-design-strategy-and-analysis]  Medical Statistics Certificate Program [https://online.stanford.edu/programs/medical-statistics-program]   Writing in the Sciences [https://www.coursera.org/learn/sciwrite]  Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Programs that we teach in: Epidemiology and Clinical Research Graduate Certificate Program [https://online.stanford.edu/programs/epidemiology-and-clinical-research-graduate-certificate]  Find us on: Kristin -  LinkedIn [https://www.linkedin.com/in/kristin-sainani-642b5914/] & Twitter/X [https://x.com/KristinSainani] Regina - LinkedIn [https://www.linkedin.com/in/reginanuzzo/] & https://www.reginanuzzo.com/ReginaNuzzo.com [http://reginanuzzo.com] * (00:00) - Introduction * (04:05) - What is NHANES? * (06:38) - The Sleep Duration Results * (12:50) - The 2015 Sleep Mystery * (17:10) - Measuring Biological Aging * (21:35) - The Penalized Cox Regression * (28:16) - Sleep and Aging Results * (30:03) - Cubic Splines and Dancing * (36:49) - Adding Exercise to the Mix * (40:57) - Boats, Handbags, and Interaction Effects * (48:20) - The Cubic Spline Exercise Analysis * (51:21) - The Opposite Result * (55:54) - Academic Writing Gone Wrong * (58:27) - The Writing Makeover * (01:01:12) - Rating the Claim with Gatorinis