This was my first major empirical project, and I had a great time living and learning in Pittsburgh. Thanks to Prof. Graham Neubig and Apurva Gandhi (PhD student at LTI) for their mentorship, and to everyone in the lab and the broader CMU community for making it such a rich summer.
Coming from a mostly theoretical background, this was eye-opening in terms of which intuitions transferred and — more interestingly — which things I'd thought hard about in theory turned out not to matter much in practice. I'd done a lot of work on sample complexity bounds, for instance, and here that simply wasn't the binding constraint: you took as much data as you could get. A useful reminder that the right level of theoretical rigor depends on the problem in front of you.
Abstract
We study whether a learned router over a pool of language models with different cost-performance tradeoffs can cut the cost of software engineering agents while preserving their performance. We frame this as learning a value function over partial SWE trajectories, with the choice of model as the action, using a simple RL framework (advantage-weighted regression).
One interesting observation was a phenomenon of implicit collaboration: the individual models had no idea they were part of a team, but the router mediated coordination between them, and specialized models often collectively outperformed any single model. That raises questions about when multi-agent setups beat individuals, and how theory-guided models of collaboration might help design them.