Decision Index puts new decision models to the test on speed and cost

Artificial Analysis, the independent AI benchmarking firm best known for its Intelligence Index, now has a scorecard for a newer kind of AI system. The Artificial Analysis Decision Index rates decision models on three things: accuracy, speed and cost. The early verdict is a familiar trade-off. Decision models are faster and cheaper, but they score worse on knowledge and reasoning tasks. What the Decision Index measures It tracks three dimensions at once. The first is accuracy, meaning how often a model gets the answer right. The second is speed, meaning how quickly the answer arrives. The third is cost, meaning how much each answer costs to produce. Viewed together, the results show a clear split. Decision models do well on speed and cost. They fall behind on tasks that need broad world knowledge or multi-step reasoning. So what is a decision model, anyway Decision models are a new category of AI system. They have also been called System One models or typed decision engines. A traditional large language model can write an essay, debug code or draft a breakup text. A decision model does something much narrower. It picks from a set of options defined in advance, such as yes or no, or...