Nvidia Unveils NeMo Software Tools for Enterprises to Create Custom AI Agents

Nvidia, the renowned chip giant, announced on Wednesday the general availability of a new suite of tools designed to help enterprises develop "agentic" artificial intelligence. These tools, dubbed NeMo microservices, are part of Nvidia's broader AI Enterprise software portfolio. They're crafted to customize and optimize AI agents for a wide array of tasks, from enhancing call center operations to boosting software development efficiency.

During a media briefing, Joey Conway, Nvidia's head of generative AI for enterprise, described the NeMo software as a means to deploy AI agents as "digital employees." He highlighted the potential impact across industries, saying, "Our view of where we see things going is that there are over a billion knowledge workers across many industries, geographies, and locations. And our view is that digital employees, or AI agents, will be able to help enterprises get more work done in these various domains and scenarios."

Productivity Gains

Conway shared insights on the tangible productivity benefits these AI agents have already brought to the table. For instance, Amdocs, a software provider for telecommunications companies, has harnessed NeMo microservices to develop billing, sales, and network agents. Their billing agent, tasked with handling customer billing inquiries, achieved a significant 50% increase in "first-call resolution," showcasing a direct impact on customer service efficiency.

The concept of AI agents functioning as digital employees isn't new. It's a narrative that's been gaining traction over the past year, with AI being seen as capable of managing corporate processes just like human employees. Nvidia has been refining its NeMo software for over five years, aiming to accelerate the development of AI models for businesses. In 2022, they expanded their offerings to include on-demand cloud-based pre-built AI models, with the microservices being introduced last October.

New Microservice Components

NeMo's toolkit includes several microservices, two of which, Curator and Retriever, were already available. Curator helps developers construct "pipelines" for cleaning and refining data sets used to train or fine-tune AI models. Retriever, on the other hand, extracts relevant elements from data sources for model use, such as text, graphics, and chart elements.

Complementing these are three new components: Customizer, Evaluator, and Guardrails. The Customizer takes the output from Curator and applies post-training techniques to enhance the model's capabilities. The Evaluator acts as an automated benchmark, testing the model post-Customizer to assess any improvements or new skills acquired. Guardrails ensures compliance and safety by operating at runtime to safeguard enterprise operations.

Updating and Gaining New Abilities

The philosophy behind NeMo is to cycle models through these microservices repeatedly, enabling continuous updates and skill acquisition, a process Nvidia calls a "flywheel." The NeMo microservices integrate with Nvidia's deployment infrastructure software, NIM (Nvidia Inference Microservices), which encapsulates AI models in application containers managed by systems like Kubernetes and accessed through APIs.

Conway emphasized that NeMo simplifies the traditionally complex tasks of training, post-training, evaluating, and revising AI models. He noted, "The focus for NeMo microservices is being able to build these microservices so that the rest of the ecosystem can get started much faster. From our experience, we've seen that these can be quite complicated." He further explained that the NeMo Evaluator consolidates and updates various open-source libraries, making them more reliable and easier to use through simple API calls.