AT&T cuts AI coding costs 56% with minimal performance decline

AT&T found a way to slash its AI coding costs by more than half, and the trick is almost disappointingly simple: stop using the expensive model when a cheaper one works just as well. The telecom giant implemented model routing technology through LiteLLM that redirects routine employee queries, particularly coding-related ones, toward lower-cost open-source models. The result was a 56% reduction in AI coding costs with only a 2% decline in performance quality. For a company processing roughly 45 billion tokens daily through its internal “Ask AT&T” platform, those savings add up fast. The routing playbook The concept behind AT&T’s approach is what the industry calls model routing, essentially a traffic cop for AI queries. Simple questions get sent to lightweight, inexpensive models. Complex tasks still go to premium options from OpenAI and Anthropic. AT&T VP Mark Austin noted that open-source models are narrowing the performance gap with their proprietary counterparts, with a difference of only 6-10 months in capabilities. Currently, open models handle about 40% of employee AI queries at AT&T. The company is targeting 60-70% in the near term. The models doing the ...