Measuring the Impact of Embedded Phishing Training on Ransomware-Lure Click Behavior: A Field Study Across Three Organizations

Abstract

Embedded anti-phishing training, redirecting employees who fail simulated phishing campaigns to immediate instructional content, is among the most widely deployed behavioral defenses against ransomware-delivering social engineering attacks. Despite this widespread adoption, recent large-scale randomized evaluations have produced evidence of only marginal absolute reductions in failure rates, raising fundamental questions about whether the current model of simulated phishing plus embedded training is an effective and efficient use of organizational security resources. A critical gap in the existing literature is the absence of lure-category-specific analyses: virtually all prior evaluations have used heterogeneous phishing campaign portfolios without examining whether training effectiveness varies systematically across the specific delivery techniques—fake software updates, spoofed invoices, and urgency-framed credential resets—most commonly associated with ransomware campaigns. This study aimed to: (1) determine whether embedded 90-second micro-training produces a statistically significant reduction in click-through rates for ransomware-style phishing lures compared with instructor-led training only and a no-training control in a randomized field experiment; (2) examine whether training effectiveness differs across three ransomware-relevant lure categories differing in their reliance on visual artifact inspection versus urgency-driven cognitive override; and (3) assess whether employee engagement with embedded training content as measured by time-on-module moderates the protective effect at the individual level. A six-month randomized field experiment was conducted across three mid-sized organizations (combined n = 1,284 employees). Employees were randomly assigned within department strata to embedded micro-training (n = 428), instructor-led annual training only (n = 426), or a control group receiving standard onboarding only (n = 430). Six monthly simulated campaign waves sent one email from each of three ransomware-relevant lure categories—fake software update notifications, spoofed invoices, and urgent credential-reset requests—to all participants. Click-through rates were compared within conditions using McNemar's test and between conditions using chi-square tests. Individual-level engagement (time-on-module) was assessed as a moderator among the embedded training group. The embedded micro-training group showed a significant overall click-rate reduction from 18.4% to 13.1% (−5.3 pp, p = 0.002), significantly larger than the instructor-led (−1.5 pp, p = 0.214) or control (−1.7 pp, p = 0.389) reductions. Effectiveness was highly lure-dependent: the fake software update category showed a 12.4 pp reduction (p < 0.001); the spoofed invoice category showed a 4.4 pp reduction (p = 0.038); the urgent credential-reset category showed only a 1.8 pp non-significant reduction (p = 0.412). Participants engaging with training content above the median time (>52 seconds) showed significantly larger individual-level reductions in subsequent click likelihood (OR = 0.52, p = 0.011) than below-median engagers (OR = 0.81, p = 0.204). Embedded micro-training produces a measurable but strongly lure-dependent reduction in ransomware-relevant phishing susceptibility. Urgency-based credential-reset lures, among the most common ransomware initial-access vectors, are largely resistant to this intervention, indicating that technical controls including multi-factor authentication and DNS-level phishing filtering are necessary complements regardless of training investment. Engagement with training content rather than mere completion is a key moderator of effectiveness, suggesting that addressing the engagement gap may be as important as content quality in improving training outcomes