Your business is going to rely on hundreds of AI models. Here's why

In today's tech landscape, just as companies often juggle multiple cloud services and databases for different needs, there's a growing trend towards using multiple AI models for various purposes. A recent survey involving over 1,000 IT decision-makers reveals that the most advanced AI adopters are currently managing hundreds of models at the same time. We've entered what's being called the "multi-model AI" era. According to the survey by S&P Global Market Intelligence, sponsored by Vultr, the average number of AI models in use stands at 158, with expectations that this number will climb to 176 within the next year. The survey also found that the most advanced AI users are currently operating with an average of 175 models, with projections to increase this number by 14% to 200 models over the coming year. Those at the second-highest level of AI maturity anticipate an 18% year-over-year growth in model numbers. Additionally, two-thirds of the surveyed managers (66%) are either developing their own models or utilizing open-source options. There are solid reasons for using multiple models across different applications. For instance, a study from MIT highlights a system that employs three models trained on language, vision, and action data to assist robots in planning and executing tasks in household, construction, and manufacturing settings. "Each foundation model captures a different aspect of the decision-making process and collaborates when it's time to make decisions," the MIT researchers explained. This trend is leading to what's known as an "ensemble" approach to AI, where multiple models work together to produce outputs, as Erica Dingman described in a MovableInk post. "The difference between using a single model and an ensemble model is like comparing a solo violin to a full orchestra," she noted. "While each instrument has its value, together they create something truly magical." Moreover, using diverse datasets and continuously updating and training an ensemble of models can help mitigate or eliminate bias in AI outputs. The widespread adoption and diversity of systems supporting AI models are driving this proliferation. AI is increasingly being deployed at the edge, according to the S&P and Vultr survey. "Distributed AI architectures, with the edge playing a crucial role in applications spanning an organization's infrastructure, are likely to become the standard," the survey authors stated. A significant majority (85%) of IT decision-makers believe this shift is likely or extremely likely in their environments, with 32% considering it "extremely likely." The survey identified organizations leading in AI adoption, labeling them as having "transformational AI practices." Half of these leaders are performing "significantly better" than their industry peers at operational levels. Nearly all of these leaders reported improvements in their 2022 to 2023 year-over-year performance across various metrics, including customer satisfaction (90%), revenue (91%), cost reduction/margin expansion (88%), risk management (87%), marketing (89%), and market share (89%). Across all surveyed organizations, AI spending is expected to surpass general IT spending. Nearly nine in ten enterprises (88%) plan to increase AI spending in 2025, with 49% anticipating moderate to significant increases. However, the rapid growth in AI usage brings challenges, particularly concerning existing IT infrastructures. "When it comes to high-demand AI activities like real-time inferencing, respondents are concerned that their current infrastructure may not be sufficient," the survey authors noted. The top three concerns include insufficient CPU or GPU resources (65%), data locality issues (53%), and storage performance issues (50%). The qualitative data from the survey echoed these concerns, with interviewees mentioning scheduling delays for higher-capacity GPU instances in public clouds and potential impacts on data availability. Additionally, there's a growing worry about the cost of infrastructure. "Cost often becomes a more pressing concern once projects are running in production. Historically, organizations have struggled to accurately forecast these costs," the authors concluded.

Related article

Comments (46)