Ben Cipollini

Not all questions need GPT-4! We introduce RouteLLM – a routing framework based on human preference data that directs simple queries to a cheaper model. With data augmentation techniques, RouteLLM achieves cost reductions of over 85% on MT Bench and 45% on MMLU while maintaining 95% GPT-4 performance. Comparing against commercial offerings (Martian and Unify AI) on MT Bench and achieve the same performance while being over 40% cheaper. Our model, datasets, and code for serving and evaluating LLM routers are all open-sourced. We are excited to see what the community will build on top! 1/6 (blog post & links👇)