This project came out of the RIPS (Research in Industrial Projects for Students) summer program at IPAM, one of the NSF's Mathematical Sciences Institutes. I was lucky to be part of the 36-student 2024 cohort, and luckier still in my mentors: Fang Sun (UCLA CS PhD), Luca Ponzoni and Pat Walters (Relay Therapeutics), Susana Serna (RIPS Director), and Dima Shlyaktenko (IPAM Director). Special thanks to my teammates Ellen Li, David Baron, and Walter Virany. There's more about the summer itself on my RIPS page.
Our sponsor was Relay Therapeutics, a precision medicine company in Cambridge, MA that pioneered using heavy computational machinery to accelerate drug discovery. Besides exposing how much my chemistry had atrophied since high school, this project taught me how to dig into the metaphorical guts of industry-standard software: in a few weeks I went from describing molecules as “more or less wiggly” to modeling protein-ligand interactions and imagining new ways to optimize hit-to-lead mutations.
As a math and CS student, I'd often felt that the canonical “interesting problems” were interesting chiefly because of some quirk or symmetry in the underlying mathematical skeleton, with applications tacked on as an afterthought. Fiddling with molecule renderings and manually parsing embeddings for hours gave me a real appreciation for the other direction: beautiful mathematics is often born to fit the topographical contours of an important application, not the other way around.
Abstract
We characterize the differences in performance and behavior across four state-of-the-art generative models for drug discovery: REINVENT4, CReM, SAFE-GPT, and COATI. To do so we build a random forest pipeline for classifying molecules and producing meaningful low-dimensional visualizations, a comparative analysis of the models in a hit-to-lead optimization setting, and a large-scale case study of the distributions of molecules each model generates.
In particular, we develop a new sensitivity metric that captures the variance in those distributions, which helps highlight the limitations of existing models and inform future model development.