HomeMoonshot AI and Tsinghua University Propose PrfaaS Architecture for Data Centers
As large language models (LLMs) demand more computational power during reasoning, conventional service architectures encounter bottlenecks. Moonshot AI, in collaboration with a research team from Tsinghua University, has introduced a novel architecture called Pre-filling as a Service (PrfaaS), aimed at overcoming the constraints of data centers and computing resources in LLM services.
The reasoning process of large language models is generally split into two phases: pre-filling and decoding. Pre-filling is computationally intensive, where the model processes input and creates a key-value cache (KVCache). Decoding is memory bandwidth-intensive, generating outputs sequentially. Traditional setups require both stages to occur within the same data center, leading to constraints in computation and bandwidth.
PrfaaS enables efficient cross-data-center service by offloading pre-filling tasks to dedicated high-compute clusters, using a standard Ethernet network to transfer the generated KVCache to local decoding clusters. Research indicates that this architecture boosts processing performance, achieving a 54% increase in service throughput over traditional models. Real-world case studies also show lower latency and higher efficiency.
The PrfaaS architecture decouples the three main subsystems—computing, networking, and storage—managing each independently. A precise routing mechanism ensures efficient transmission of long requests, preventing congestion from uneven resource allocation seen in traditional approaches. Additionally, a dual-timescale scheduling mechanism adapts to varying traffic patterns, further optimizing resource usage.
As the demand for cross-data-center reasoning grows and new hardware continues to emerge, PrfaaS offers a promising solution for future AI applications.
Related article
Nvidia H200 China deal survives Trump-Xi summit, but not as expected
President Trump traveled to Beijing with Jensen Huang joining the delegation at the eleventh hour, departing two days later with vague remarks about potential chip export developments. No deal materialized. Since Trump first authorized sales in Decem
How to fix Core Web Vitals for SEO ranking?
Creating visually appealing assets is crucial in game development. Pixel art, with its charming retro aesthetic, remains a popular choice. With the advent of AI tools, generating pixel art has become more accessible than ever. This guide explores how
TechCrunch Disrupt 2026 breakout session agenda unveiled
TechCrunch Disrupt 2026 arrives at Moscone West in San Francisco, October 13–15, uniting 10,000+ founders, investors, operators, and tech leaders for three days of insight, networking, and forward-looking dialogue. Register now to save up to $100 on
Related Special Topic Recommendations
Design & Art
Top AI Illustration Tools: Create Editorial Scenes and Cover Art Faster
2026 Latest Top-rated AI Illustration Tools: Our curated list features the best powerfully designed tools for creating stunning editorial scenes and eye-catching cover art quickly. Each option has gone through rigorous real-world tests, offering a free vs paid comparison to help you find the perfect match for your workflow. Try these must-try solutions today to unlock your AI edge and boost your creative productivity. Explore now at XIX.AI.
10 tools
xix.ai
Education and Learning
Top AI Tutors for Exam Prep: Practice Math, Science, and English
2026 Latest Top-Rated Best AI Tutors for Exam Prep are here on XIX.AI! Our curated list features powerful game-changing tools that offer free vs paid comparison, real-world tests, and weekly updated rankings. They help boost your writing efficiency, simplify complex topic learning, and unlock your academic potential. Explore now to Discover your perfect exam prep tool and Start creating today to Unlock your AI edge!
10 tools
xix.ai
Finance
AI Budget Planning Tools for Personal Money Management
2026 Latest Best Top-Rated AI Budget Planning Tools for Personal Money Management are here on XIX.AI! This curated collection features powerful, game-changing solutions that help you conduct real-world tests, understand spending habits, and create personalized plans to boost productivity. Enjoy a free vs paid comparison along with detailed rankings. Must-try tools designed to simplify money management and unlock your AI edge. Explore now!
11 tools
xix.ai
Health & Wellness
AI Sleep Tracking Assistants: Improve Recovery with Better Bedtime Habits
2026 Latest Best Top-Rated AI Sleep Tracking Assistants curated for ultimate recovery. This fully updated weekly guide features powerful tools that deliver real-world tests, free vs paid comparison, and detailed rankings to help you find the must-try solution that boosts sleep quality and supports better bedtime habits. Explore now at XIX.AI to Unlock your AI edge.
10 tools
xix.ai
Data Analysis
AI SQL Copilots for Product and Revenue Analytics
2026 Latest Best Top-Rated AI SQL Copilots for Product and Revenue Analytics! XIX.AI has curated a powerful, game-changing collection of must-try tools that undergo real-world tests through weekly updated rankings. These tools help users boost productivity significantly by accelerating data analysis, streamlining complex queries, and generating accurate insights to drive smarter business decisions. Explore now to Discover your perfect tool for unlocking your AI edge!
14 tools
xix.ai
Education and Learning
Best AI Homework Helpers That Teach: Learn Concepts, Not Just Answers
2026 Latest Best Top-Rated AI Homework Helpers That Teach Core Concepts Not Just Answers Powerful Tools for Students Must-Try Curated List Weekly Updated Free vs Paid Comparison Real-World Tests Accurate Rankings to Help You Unlock Your Full Learning Potential Explore now to Discover your perfect tool for smarter studying.
10 tools
xix.ai
Comments (10)
0/500
![JuanJackson]()
Интересная архитектура PrfaS от Moonshot и Цинхуа для решения проблем с вычислительной мощностью LLM. Надеюсь, это поможет оптимизировать центры обработки данных! 🚀
![EricNelson]()
A arquitetura PrfaS proposta por Moonshot e Tsinghua é um passo interessante para resolver gargalos de computação em LLMs. Será que isso viabiliza data centers mais eficientes? 🤔
![WillieJones]()
La arquitectura PrfaS de Moonshot y Tsinghua parece una gran solución para los cuellos de botella en centros de datos con LLMs. ¡Esperamos ver resultados concretos pronto! 📈
![WillieRamirez]()
Moonshot AI und Tsinghua zeigen mit PrfaS einen vielversprechenden Ansatz gegen Rechenengpässe bei LLMs. Spannend zu sehen, wie diese Kooperation die Datenzukunft gestaltet! 🌐
![MateoAdams]()
대규모 LLM 추론에 필요한 컴퓨팅 파워를 위해 청화대와 Moonshot이 제안한 PrfaS 아키텍처가 흥미롭네요. 데이터 센터 병목 현상을 해결할 열쇠가 될지 주목됩니다! 🚀
![HarryNelson]()
Moonshot AI 和清華大學提出的 PrfaS 架構看起來很有潛力，特別是針對 LLM 推理時的算力瓶頸。不知道實際部署後效能如何，持續關注中！💻✨
As large language models (LLMs) demand more computational power during reasoning, conventional service architectures encounter bottlenecks. Moonshot AI, in collaboration with a research team from Tsinghua University, has introduced a novel architecture called Pre-filling as a Service (PrfaaS), aimed at overcoming the constraints of data centers and computing resources in LLM services.
The reasoning process of large language models is generally split into two phases: pre-filling and decoding. Pre-filling is computationally intensive, where the model processes input and creates a key-value cache (KVCache). Decoding is memory bandwidth-intensive, generating outputs sequentially. Traditional setups require both stages to occur within the same data center, leading to constraints in computation and bandwidth.
PrfaaS enables efficient cross-data-center service by offloading pre-filling tasks to dedicated high-compute clusters, using a standard Ethernet network to transfer the generated KVCache to local decoding clusters. Research indicates that this architecture boosts processing performance, achieving a 54% increase in service throughput over traditional models. Real-world case studies also show lower latency and higher efficiency.
The PrfaaS architecture decouples the three main subsystems—computing, networking, and storage—managing each independently. A precise routing mechanism ensures efficient transmission of long requests, preventing congestion from uneven resource allocation seen in traditional approaches. Additionally, a dual-timescale scheduling mechanism adapts to varying traffic patterns, further optimizing resource usage.
As the demand for cross-data-center reasoning grows and new hardware continues to emerge, PrfaaS offers a promising solution for future AI applications.
Nvidia H200 China deal survives Trump-Xi summit, but not as expected
President Trump traveled to Beijing with Jensen Huang joining the delegation at the eleventh hour, departing two days later with vague remarks about potential chip export developments. No deal materialized. Since Trump first authorized sales in Decem
How to fix Core Web Vitals for SEO ranking?
Creating visually appealing assets is crucial in game development. Pixel art, with its charming retro aesthetic, remains a popular choice. With the advent of AI tools, generating pixel art has become more accessible than ever. This guide explores how
TechCrunch Disrupt 2026 breakout session agenda unveiled
TechCrunch Disrupt 2026 arrives at Moscone West in San Francisco, October 13–15, uniting 10,000+ founders, investors, operators, and tech leaders for three days of insight, networking, and forward-looking dialogue. Register now to save up to $100 on
2026 Latest Top-rated AI Illustration Tools: Our curated list features the best powerfully designed tools for creating stunning editorial scenes and eye-catching cover art quickly. Each option has gone through rigorous real-world tests, offering a free vs paid comparison to help you find the perfect match for your workflow. Try these must-try solutions today to unlock your AI edge and boost your creative productivity. Explore now at XIX.AI.
10 tools
xix.ai
2026 Latest Top-Rated Best AI Tutors for Exam Prep are here on XIX.AI! Our curated list features powerful game-changing tools that offer free vs paid comparison, real-world tests, and weekly updated rankings. They help boost your writing efficiency, simplify complex topic learning, and unlock your academic potential. Explore now to Discover your perfect exam prep tool and Start creating today to Unlock your AI edge!
10 tools
xix.ai
2026 Latest Best Top-Rated AI Budget Planning Tools for Personal Money Management are here on XIX.AI! This curated collection features powerful, game-changing solutions that help you conduct real-world tests, understand spending habits, and create personalized plans to boost productivity. Enjoy a free vs paid comparison along with detailed rankings. Must-try tools designed to simplify money management and unlock your AI edge. Explore now!
11 tools
xix.ai
2026 Latest Best Top-Rated AI Sleep Tracking Assistants curated for ultimate recovery. This fully updated weekly guide features powerful tools that deliver real-world tests, free vs paid comparison, and detailed rankings to help you find the must-try solution that boosts sleep quality and supports better bedtime habits. Explore now at XIX.AI to Unlock your AI edge.
10 tools
xix.ai
2026 Latest Best Top-Rated AI SQL Copilots for Product and Revenue Analytics! XIX.AI has curated a powerful, game-changing collection of must-try tools that undergo real-world tests through weekly updated rankings. These tools help users boost productivity significantly by accelerating data analysis, streamlining complex queries, and generating accurate insights to drive smarter business decisions. Explore now to Discover your perfect tool for unlocking your AI edge!
14 tools
xix.ai
2026 Latest Best Top-Rated AI Homework Helpers That Teach Core Concepts Not Just Answers Powerful Tools for Students Must-Try Curated List Weekly Updated Free vs Paid Comparison Real-World Tests Accurate Rankings to Help You Unlock Your Full Learning Potential Explore now to Discover your perfect tool for smarter studying.
10 tools
xix.ai
Интересная архитектура PrfaS от Moonshot и Цинхуа для решения проблем с вычислительной мощностью LLM. Надеюсь, это поможет оптимизировать центры обработки данных! 🚀
A arquitetura PrfaS proposta por Moonshot e Tsinghua é um passo interessante para resolver gargalos de computação em LLMs. Será que isso viabiliza data centers mais eficientes? 🤔
La arquitectura PrfaS de Moonshot y Tsinghua parece una gran solución para los cuellos de botella en centros de datos con LLMs. ¡Esperamos ver resultados concretos pronto! 📈
Moonshot AI und Tsinghua zeigen mit PrfaS einen vielversprechenden Ansatz gegen Rechenengpässe bei LLMs. Spannend zu sehen, wie diese Kooperation die Datenzukunft gestaltet! 🌐
대규모 LLM 추론에 필요한 컴퓨팅 파워를 위해 청화대와 Moonshot이 제안한 PrfaS 아키텍처가 흥미롭네요. 데이터 센터 병목 현상을 해결할 열쇠가 될지 주목됩니다! 🚀
Moonshot AI 和清華大學提出的 PrfaS 架構看起來很有潛力，特別是針對 LLM 推理時的算力瓶頸。不知道實際部署後效能如何，持續關注中！💻✨
