Slaughtering Academic Sacred Cows, By Professor X

The Substack post by Adam Mastroianni, titled "I swear the UFO is coming any minute" (published February 17, 2026, on Experimental History), is a lively, quarterly roundup of "unfortunate events in science" — mostly in psychology and behavioural sciences — that poke holes in some of the field's most cherished, textbook-worthy findings. Mastroianni uses the title as a humorous analogy: just as UFO enthusiasts keep insisting "it's coming any minute" despite decades of fizzled sightings and failed prophecies, certain classic studies persist in curricula and popular discourse long after replications flop, archives reveal shenanigans, or reanalyses show the effects were overhyped or non-existent.

The piece isn't a dry academic takedown; it's conversational, witty, and self-deprecating (he even turns the spotlight on his own 2022 paper on public opinion estimation, which faced a critical reanalysis in 2025 claiming people are better at gauging attitude shifts than he originally argued). Mastroianni compiles recent revelations that chip away at foundational claims, arguing that many endure because of small samples, vague theories that resist falsification, lax historical documentation, and the inertia of teaching "classics" in intro courses.

Key Examples from the Post

Here are the standout critiques Mastroianni highlights:

When Prophecy Fails (Leon Festinger et al., 1956): The iconic cognitive dissonance study of a doomsday cult that predicted the world's end via a UFO flood. When the apocalypse didn't arrive, believers supposedly doubled down on their faith. A 2025 archival dive revealed massive contamination: up to half the group were undercover researchers, one even rose to lead the cult and fed quotes for the book. Post-failure, some members quietly distanced themselves or recanted — undermining the clean "dissonance reduction" narrative. Mastroianni ties this to broader dissonance troubles: impossible numbers in early lab studies, a basic effect failing replication in 2023 (with tiny samples), and Elliot Aronson's quip that the theory is "hard to disprove anything" due to its flexibility.

Elizabeth Loftus's 1974 car crash memory study: Participants estimated higher speeds when hearing "smashed" vs. "hit," suggesting language warps memory. Cited thousands of times and central to Loftus's career debunking "repressed" memories, a 2025 re-run with ~10x the sample size found no effect. Mastroianni concedes memory is reconstructive but argues it's not that fragile — one word doesn't reliably overwrite recall, or traffic stops would be comedy gold ("Officer, we just ambled by at 50 k/h").

Oliver Sacks's case studies (e.g., The Man Who Mistook His Wife for a Hat, 1985): Neurological anecdotes that read like gripping tales. A 2025 New Yorker piece quotes Sacks himself calling them part "fairy tales... half-science, half-fable." Parallels to debunked social psych classics like the Stanford Prison Experiment (archival notes show coaching and staging) and Rosenhan's pseudopatient study (similar malfeasance uncovered).

Choice overload / "too many options" paradox: A 1995 study claimed physicians overwhelmed by choices defaulted to no treatment. A 2025 replication found the opposite — more options led to more recommendations. Mastroianni's takeaway: effects are fickle and context-bound; the field often overgeneralises from narrow setups.

These aren't isolated gotchas; they illustrate a pattern where early, sexy findings with small n's get enshrined before rigorous scrutiny arrives. Mastroianni notes pre-internet science often lacked the paper trail we expect today — leading to ironic discoveries of researchers "meticulously documenting their research malfeasance."

Broader Implications and Other Notable Cases That Come to Mind

While Mastroianni focuses on psychology (closely overlapping with social sciences), the replication crisis ripples into sociology textbooks and teaching. Sociology has been slower to confront it than psychology or economics — partly due to methodological fragmentation (qualitative vs. quantitative divides), less emphasis on large-scale experimental replication, and journals rarely mandating data/code sharing or preregistration. Yet similar issues surface:

Classic experiments often taught as gospel (e.g., Milgram's obedience, Asch conformity) face scrutiny over ethics, demand characteristics, or limited generalisability, though they hold up better than priming effects.

Social priming failures (e.g., Bargh's elderly-walking study priming slower movement) sparked early crisis waves in the 2010s — direct replications flopped, yet the idea lingers in pop psych and some intro soc texts.

Textbook inertia: Analyses (e.g., from Ulrich Schimmack's Replication Index) show many social psychology textbooks still feature high-profile failures without caveats, presenting them as settled science. Sociology texts similarly lag in updating for reproducibility concerns.

The replication crisis overall (documented in projects like the 2015 Open Science Collaboration, where ~36% of psych studies replicated) reveals systemic problems: publication bias toward positive results, underpowered studies, flexible analyses (p-hacking), and incentives favouring novelty over robustness. Non-replicable papers often get cited more (a 2021 UCSD study found them cited 153 times more than replicable ones), rewarding splashy claims.

Mastroianni's optimistic close: scrutiny is working— science self-corrects, albeit slowly. These "UFO" moments force humility, better methods (larger samples, preregistration), and healthier scepticism. For me, the data shows exactly the opposite; that fraud and fragility run riot through the human sciences, and STEM is not far behind:

https://www.youtube.com/@SabineHossenfelder/videos

https://www.experimental-history.com/p/i-swear-the-ufo-is-coming-any-minute