The Dual-Use Dilemma: When Medical Breakthroughs Become Biosecurity Risks

The intersection of artificial intelligence and biology is currently operating as a double-edged sword. On one side, we have the promise of bespoke proteins that can kill superbugs and revolutionize drug discovery. On the other, we face the chilling possibility of AI-designed toxins and pathogens that could evade existing detection systems.

Consider the case of the cone snail. These marine molluscs produce conotoxins—proteins that can block ion channels in the nervous system. While some of these are used to create approved treatments for chronic pain, others are lethal. Recently, scientists at Chongqing University, including computational chemist Weiwei Xue, developed an AI tool to design these conotoxins for therapeutic use. While the goal was drug discovery, the project raised immediate alarms within the US government, highlighting a growing fear: the same tool used to heal could, in the wrong hands, be used to harm.

The Rise of the ‘Garage Lab’ and Digital Uplift

For decades, the barrier to creating a biological weapon was expertise. You needed a PhD, specialized equipment, and years of lab experience. AI is systematically dismantling those barriers. This represents what experts call “digital uplift”—the ability of AI to provide actionable, step-by-step guidance to non-experts.

James Black, an AI biosecurity researcher at Johns Hopkins University, identifies two primary tiers of risk. First, You’ll see individuals in “garage labs” who might use chatbots to learn how to produce existing threats like anthrax. Second, there are sophisticated state actors or well-resourced groups who could combine general chatbots with specialist biological software to design entirely new, synthetic bioweapons.

While some, like David Baker of the University of Washington, argue that the global benefits of protein design far outweigh the dangers, the ability of AI to “troubleshoot” lab work means the window for intervention is closing.

The Cat-and-Mouse Game of DNA Screening

The last line of defense against synthetic bioweapons is the synthesis process. When a researcher wants a specific protein, they order the genetic sequence from a company that synthesizes DNA. Many of these firms belong to the International Gene Synthesis Consortium, which screens orders for known toxins or pathogenic sequences.

However, AI is proving adept at “cloaking” these sequences. A study led by Microsoft’s chief scientific officer Eric Horvitz and Bruce Wittmann revealed a critical vulnerability. Their team used open-source protein-design tools to create “synthetic homologues”—molecules that maintain their dangerous function but have genetic sequences different enough to slip past screening software.

The results were sobering: roughly one-quarter of the most dangerous designs initially evaded detection across four participating companies. While software updates eventually reduced this failure rate to about 3%, further research suggests that breaking sequences into fragments of just 25 nucleotides can make them even harder to detect.

Can AI ‘Guard Rails’ Actually Stop a Bad Actor?

To prevent misuse, AI developers have implemented “guard rails.” For example, the genomic language model Evo 2 was trained by excluding viruses that infect humans and other animals, making it poor at designing human-infecting viral sequences.

But these walls are porous. Researchers led by Stanford bioengineer Le Cong demonstrated that a general-purpose AI agent could trick Evo 2 into generating new versions of HIV-1 and SARS-CoV-2 proteins. “fine-tuning” the model with publicly available genome data can restore the very capabilities the developers tried to remove.

This creates a philosophical divide in the scientific community. Brian Hie of Stanford argues that model openness actually contributes to safety by allowing researchers to study and patch vulnerabilities. Conversely, others argue that the availability of these tools makes the “ship has sailed” scenario a reality, where the focus must shift from prevention to detection and counter-attacks.

The Reality Check: Why We Aren’t in a Movie Plot (Yet)

Despite the alarms, a report by the US National Academies of Sciences, Engineering, and Medicine (NASEM) provides a necessary reality check. Designing a pathogen on a computer is vastly different from making one work in the real world.

• The Data Gap: There is a severe lack of high-quality data connecting genetic sequences to actual virulence or transmissibility. AI cannot reliably predict what makes a virus “deadlier” if the data doesn’t exist.
• The Lab Hurdle: Producing pathogens and testing their characteristics remains a physical, messy, and difficult process that AI has not yet simplified.
• The Natural Alternative: As Brian Hie and David Baker note, the natural world already brims with threats. Traditional techniques for introducing random mutations can often achieve harmful results without the need for complex AI design.

Biosecurity FAQ

What do you think? Should biological AI tools be strictly regulated and closed-source, or does openness provide the best defense? Let us know in the comments below or subscribe to our newsletter for more deep dives into the future of biotech.

Explore more on the ethics of synthetic biology and the future of AI-driven medicine.