3 ways Meta's Llama 3.1 is an advance for Gen AI

On Tuesday, Meta pulled back the curtain on the latest addition to its Llama family of large language models (LLMs), introducing Llama 3.1. The company proudly touts Llama 3.1 as the first open-source "frontier model," a term typically reserved for the most advanced AI models out there.

Llama 3.1 comes in various sizes, but it's the behemoth "405B" that really turns heads. With a staggering 405 billion neural "weights," or parameters, it outmuscles other notable open-source models like Nvidia's Nemotron 4, Google's Gemma 2, and Mixtral. What's even more intriguing are the three key decisions the Meta team made in crafting this giant.

These decisions are nothing short of a neural network engineering masterclass, forming the backbone of how Llama 3.1 405B was built and trained. They also build on the efficiency gains Meta demonstrated with Llama 2, which showed promising ways to reduce the overall compute budget for deep learning.

First off, Llama 3.1 405B ditches the "mixture of experts" approach, which Google uses for its closed-source Gemini 1.5 and Mistral uses for Mixtral. This method involves creating different combinations of neural weights, some of which can be turned off to streamline predictions. Instead, Meta's researchers stuck with the tried-and-true "decoder-only transformer model architecture," a staple since Google introduced it in 2017. They claim this choice leads to a more stable training process.

Secondly, to boost the performance of this straightforward transformer-based model, Meta's team came up with a clever multi-stage training approach. We all know that balancing the amount of training data and compute can significantly impact prediction quality. But traditional "scaling laws," which predict model performance based on size and data, don't necessarily reflect how well a model will handle "downstream" tasks like reasoning tests.

So, Meta developed its own scaling law. They ramped up both the training data and compute, testing different combinations over multiple iterations to see how well the resulting model performed on those crucial downstream tasks. This meticulous process helped them pinpoint the sweet spot, leading to the choice of 405 billion parameters for their flagship model. The final training was powered by 16,000 Nvidia H100 GPU chips on Meta's Grand Teton AI server, with a complex system to run data and weights in parallel.

The third innovation lies in the post-training phase. After each training round, Llama 3.1 goes through a rigorous process guided by human feedback, similar to what OpenAI and others do to refine their models' outputs. This involves "supervised fine-tuning," where the model learns to distinguish between desirable and undesirable outputs based on human preferences.

Meta then throws in a twist with "direct preference optimization" (DPO), a more efficient version of reinforcement learning from human feedback, pioneered by Stanford University AI scholars this year. They also train Llama 3.1 to use "tools," like external search engines, by showing it examples of prompts solved with API calls, boosting its "zero-shot" tool use capabilities.

To combat "hallucinations," the team curates specific training data and creates original question-answer pairs, fine-tuning the model to answer only what it knows and refuse what it's unsure about.

Throughout the development, the Meta researchers emphasized simplicity, stating that high-quality data, scale, and straightforward approaches consistently delivered the best results. Despite exploring more complex architectures and training recipes, they found the added complexity didn't justify the benefits.

The scale of Llama 3.1 405B is a landmark for open-source models, typically dwarfed by their commercial, closed-source counterparts. Meta's CEO, Mark Zuckerberg, highlighted the economic advantages, noting that developers can run inference on Llama 3.1 405B at half the cost of using models like GPT-4o.

Zuckerberg also championed open-source AI as a natural progression of software, likening it to the evolution of Unix from proprietary to a more advanced, secure, and broader ecosystem thanks to open-source development.

However, as Steven Vaughan-Nichols from ZDNET points out, some details are missing from Meta's code posting on Hugging Face, and the code license is more restrictive than typical open-source licenses. So, while Llama 3.1 is kind of open source, it's not entirely there. Yet, the sheer volume of detail about its training process is a refreshing change, especially when giants like OpenAI and Google are increasingly tight-lipped about their closed-source models.