NVIDIA launches model routing infrastructure to direct requests to lowest-cost capable inference model

According to CIO, NVIDIA is entering the model routing market with infrastructure that examines prompts and routes them to the most cost-appropriate model based on performance-price tradeoffs. Model routing has emerged as an enterprise technique to control inference costs as organizations deploy multiple models across different pricing tiers.