Open Deep Search arrives to challenge Perplexity and ChatGPT Search

If you're in the tech world, you've likely heard about the buzz surrounding Open Deep Search (ODS), the new open-source framework from the Sentient Foundation. ODS is making waves by offering a robust alternative to proprietary AI search engines like Perplexity and ChatGPT Search, and it's all about empowering large language models (LLMs) with advanced reasoning agents that leverage web search and other tools to answer your queries effectively.

The AI Search Landscape

Today's AI search tools, such as Perplexity and ChatGPT Search, are impressive, combining the knowledge and reasoning of LLMs with real-time web search to deliver up-to-date answers. However, these systems are often locked behind proprietary doors, which can make customization and specialized applications a challenge.

Himanshu Tyagi, co-founder of Sentient, shared with VentureBeat, "Most innovation in AI search has been happening behind closed doors. Open-source efforts have historically lagged in usability and performance. ODS aims to bridge this gap, proving that open systems can not only compete but potentially surpass closed counterparts in quality, speed, and flexibility."

Open Deep Search (ODS) Architecture

ODS isn't just another tool; it's a plug-and-play system that you can integrate with both open-source models like DeepSeek-R1 and closed models like GPT-4o and Claude. At its core, ODS has two main components:

Open Search Tool

This component takes your query and scours the web for relevant information to feed back to the LLM as context. It's smart, too—it rephrases your query in various ways to ensure a broad and diverse search. After fetching results from a search engine, it extracts snippets and linked pages, then uses chunking and re-ranking to zero in on the most relevant content. It's particularly adept at handling specific sources like Wikipedia, ArXiv, and PubMed, and can be prompted to prioritize reliable sources when faced with conflicting information.

Open Reasoning Agent

This agent takes your query and, using the base LLM and various tools (including the Open Search Tool), crafts a final answer. Sentient offers two distinct agent architectures within ODS:

ODS-v1

This version uses a ReAct agent framework combined with Chain-of-Thought (CoT) reasoning. ReAct agents alternate between reasoning steps ("thoughts") and actions (like using the search tool) and observations (the results of those actions). ODS-v1 iterates through this process to reach an answer. If the ReAct agent hits a snag, it falls back on CoT Self-Consistency, sampling multiple CoT responses and choosing the most common answer.

ODS-v2

This version employs Chain-of-Code (CoC) and a CodeAct agent, built with the Hugging Face SmolAgents library. CoC harnesses the LLM's ability to generate and execute code snippets to solve problems, while CodeAct uses code generation for planning actions. ODS-v2 can manage multiple tools and agents, making it ideal for tackling complex tasks that might need sophisticated planning and multiple search iterations.

ODS architecture Credit: arXiv

Tyagi explained, "While tools like ChatGPT or Grok offer 'deep research' via conversational agents, ODS operates at a different level—more like the infrastructure behind Perplexity AI—providing the underlying architecture that powers intelligent retrieval, not just summaries."

Performance and Practical Results

Sentient put ODS to the test by pairing it with the open-source DeepSeek-R1 model and pitting it against closed-source competitors like Perplexity AI and OpenAI's GPT-4o Search Preview, as well as standalone LLMs like GPT-4o and Llama-3.1-70B. They used the FRAMES and SimpleQA benchmarks, adapted to assess the accuracy of search-enabled AI systems.

The results? Impressive. Both ODS-v1 and ODS-v2, when teamed up with DeepSeek-R1, outshone Perplexity's flagship products. ODS-v2 with DeepSeek-R1 even surpassed the GPT-4o Search Preview on the complex FRAMES benchmark and came close on SimpleQA.

One fascinating insight was the framework's efficiency. The reasoning agents in both ODS versions learned to use the search tool wisely, often deciding whether another search was needed based on the quality of the initial results. For example, ODS-v2 used fewer web searches on the simpler SimpleQA tasks compared to the more complex, multi-hop queries in FRAMES, optimizing resource use.

Implications for the Enterprise

For businesses looking to harness powerful AI reasoning capabilities with real-time information, ODS is a game-changer. It offers a transparent, customizable, and high-performing alternative to proprietary AI search systems. The ability to plug in preferred open-source LLMs and tools means organizations can tailor their AI stack and avoid being locked into a single vendor.

"ODS was designed with modularity in mind," Tyagi noted. "It dynamically selects which tools to use based on descriptions provided in the prompt. This means it can interact with unfamiliar tools fluently—as long as they're well-described—without needing prior exposure."

However, he cautioned that ODS performance can suffer if the toolset becomes too cluttered, "so careful design is crucial."

Sentient has made the ODS code available on GitHub, inviting the community to explore and contribute.

"Initially, the strength of Perplexity and ChatGPT was their advanced technology, but with ODS, we've leveled this technological playing field," Tyagi concluded. "We now aim to surpass their capabilities through our 'open inputs and open outputs' strategy, enabling users to seamlessly integrate custom agents into Sentient Chat."