SERG Seminar

In this edition of our weekly SERG seminar, we will hear from:

Michael Chan on “Multi-Model Routing for Energy-Efficient LLM Code Generation”

Abstract:
The use of Large Language Models for software development is becoming increasingly more common as model capabilities continue to improve. However, inference with such large models results in significant energy costs independent of the complexity of the solution. Many code generation requests vary vastly in complexity, making the uniform use of a single large model inefficient for simpler tasks.

This thesis explores the viability of multi-model routing systems to improve the efficiency of AI systems for code generation tasks. We adapt existing multi-model frameworks and extend prior methodologies to incorporate hardware-level energy measurements. The proposed framework enables systematic comparison of various routing strategies and to quantify their energy-performance trade-offs.

Experimental results on the MBPP and HumanEval benchmarks indicate that model routing can reduce energy consumption for model inference while only incurring a minor reduction in code generation accuracy over using a singular large model. The results show a favorable energy–performance trade-off and demonstrate the potential of model routing as an approach to improve the energy efficiency of AI-assisted coding.

If you are interested to give a talk or host a discussion session in one of our next meetings, please contact Carolin Brandt via Mattermost or email.