Demystifying Common AI Terms You've Heard

Artificial intelligence is reshaping our world, and in the process, it's inventing an entirely new vocabulary to describe how it works. Spend just a few minutes reading about AI, and you'll encounter terms like LLMs, RAG, RLHF, and dozens of others that can leave even seasoned tech professionals feeling out of their depth. This glossary is our effort to demystify that language. We update it regularly as the field advances, so consider it a living document, much like the AI systems it explains.

AGI

Artificial General Intelligence, or AGI, is a somewhat vague concept. Broadly, it refers to AI that is more capable than the average human at a wide range of tasks, if not most of them. OpenAI CEO Sam Altman once described AGI as the "equivalent of a median human you could hire as a co-worker." Meanwhile, OpenAI's charter defines it as "highly autonomous systems that outperform humans at most economically valuable work." Google DeepMind has a slightly different take, viewing AGI as "AI that's at least as capable as humans at most cognitive tasks." Confused? Don't worry—experts at the forefront of AI research often are, too.

AI Agent

An AI agent is a tool that leverages AI to perform a sequence of tasks autonomously on your behalf. It goes beyond what a basic chatbot can do, handling actions like filing expenses, booking travel or restaurant reservations, or even writing and maintaining code. However, as we've noted, this is an emerging field with many moving parts, so the term "AI agent" can mean different things to different people. The necessary infrastructure to fully realize its envisioned capabilities is still under development. At its core, the concept implies an autonomous system that may utilize multiple AI models to execute multi-step tasks.

API Endpoints

Think of API endpoints as "buttons" on the back of a software application that other programs can press to trigger specific actions. Developers use these interfaces to build integrations—for instance, allowing one app to pull data from another, or enabling an AI agent to control third-party services directly without manual human input. Most smart home devices and connected platforms have these hidden buttons available, even if everyday users never see or interact with them. As AI agents become more sophisticated, they are increasingly able to discover and use these endpoints independently, unlocking powerful—and sometimes surprising—possibilities for automation.

Chain of Thought

Given a simple question, the human brain can often answer without much conscious thought—questions like, "Which animal is taller, a giraffe or a cat?" But for more complex problems, you might need a pen and paper to work through intermediate steps. For example, if a farmer has chickens and cows with a total of 40 heads and 120 legs, you'd likely write out a simple equation to find the answer (20 chickens and 20 cows).

In AI, chain-of-thought reasoning for large language models involves breaking down a problem into smaller, intermediate steps to improve the final output's accuracy. This approach usually takes longer to produce an answer, but the result is more likely to be correct, especially in logic or coding contexts. Reasoning models are developed from traditional LLMs and optimized for this step-by-step thinking through techniques like reinforcement learning.

(See: Large Language Model)

This is a more specific concept than a general "AI agent." A coding agent is a specialized program that can autonomously take step-by-step actions to complete a software development goal. Instead of merely suggesting code for a human to review and insert, a coding agent can write, test, and debug code independently, handling the iterative, trial-and-error work that typically fills a developer's day. These agents can operate across entire codebases, identifying bugs, running tests, and deploying fixes with minimal human oversight. Think of it like hiring an incredibly fast intern who never sleeps or loses focus—though, as with any intern, a human still needs to review the final work.

Compute

While the term can have multiple meanings, "compute" generally refers to the essential computational power required for AI models to function. This processing capability fuels the AI industry, enabling the training and deployment of powerful models. The term is often shorthand for the hardware that provides this power—components like GPUs, CPUs, TPUs, and other infrastructure that forms the foundation of the modern AI ecosystem.

Deep Learning

A subset of self-improving machine learning where AI algorithms are built with a multi-layered, artificial neural network (ANN) structure. This allows them to identify more complex patterns and relationships than simpler machine learning systems, such as linear models or decision trees. The design of deep learning algorithms is inspired by the interconnected network of neurons in the human brain.

Deep learning models can identify key features in data on their own, without requiring human engineers to predefine those characteristics. This structure also supports algorithms that learn from mistakes and, through repetition and adjustment, improve their outputs over time. However, deep learning systems require vast amounts of data (millions of points or more) to deliver good results. They also typically take longer to train than simpler machine learning algorithms, leading to higher development costs.

(See: Neural Network)

Diffusion

Diffusion is the core technology behind many AI models that generate art, music, and text. Inspired by physics, diffusion systems gradually "destroy" the structure of data—such as photos or songs—by adding noise until nothing recognizable remains. In physics, diffusion is spontaneous and irreversible (sugar dissolved in coffee can't return to cube form). But AI diffusion systems aim to learn a "reverse diffusion" process to reconstruct the original data from noise, thereby gaining the ability to generate new data from scratch.

Distillation

Distillation is a technique for transferring knowledge from a large AI model (the "teacher") to a smaller one (the "student"). Developers send queries to the teacher model and record its outputs, sometimes comparing them to a dataset to assess accuracy. These outputs are then used to train the student model to approximate the teacher's behavior.

Distillation can create a smaller, more efficient model based on a larger one with minimal performance loss. This is likely how OpenAI developed GPT-4 Turbo, a faster version of GPT-4.

While all AI companies use distillation internally, some may have also used it to catch up to leading models. However, distilling knowledge from a competitor's model typically violates the terms of service for AI APIs and chat assistants.

Fine-Tuning

This refers to the additional training of an AI model to optimize its performance for a specific task or domain beyond its original training focus. This is typically done by feeding the model new, specialized data relevant to the target area.

Many AI startups begin with a large language model as a foundation for a commercial product. They then aim to enhance its utility for a specific sector or task by fine-tuning it with their own domain-specific knowledge and expertise, supplementing the model's earlier training cycles.

(See: Large Language Model [LLM])

GAN

A Generative Adversarial Network (GAN) is a type of machine learning framework that has driven significant advances in generative AI, particularly for creating realistic data—including tools for deepfakes. GANs involve two neural networks working against each other: one (the generator) creates an output based on its training data, and the other (the discriminator) evaluates that output.

The two models are essentially programmed in a contest. The generator tries to produce outputs convincing enough to fool the discriminator, while the discriminator works to spot artificially generated data. This adversarial process can optimize AI outputs to be more realistic without additional human intervention. GANs are most effective for narrow applications, like generating realistic photos or videos, rather than for general-purpose AI.

Hallucination

Hallucination is the AI industry's term for when models generate incorrect or fabricated information. It's a major challenge for AI quality and reliability.

Hallucinations can produce misleading generative AI outputs with potential real-world risks—such as harmful medical advice from a health query.

This problem of AI fabrication is thought to stem from gaps in training data. The prevalence of hallucinations is driving a trend toward more specialized, vertical AI models—domain-specific systems with narrower expertise—as a way to reduce knowledge gaps and minimize the risk of disinformation.

Inference

Inference is the process of running a trained AI model to make predictions or draw conclusions from new data. It's important to note that inference cannot happen without prior training; a model must first learn patterns from a dataset before it can effectively extrapolate from that knowledge.

Many types of hardware can perform inference, from smartphone processors to powerful GPUs to custom AI accelerators. However, not all hardware is equally capable. Very large models would take an impractically long time to run on a standard laptop compared to a cloud server equipped with high-end AI chips.

[See: Training]

Large Language Model (LLM)

Large Language Models (LLMs) are the AI models powering popular assistants like ChatGPT, Claude, Google's Gemini, Meta's Llama, Microsoft Copilot, and Mistral's Le Chat. When you interact with an AI assistant, you're engaging with an LLM that processes your request directly or with the help of various tools, such as web browsing or code interpreters.

LLMs are deep neural networks composed of billions of numerical parameters (or weights) that learn the relationships between words and phrases, creating a multidimensional representation of language—a kind of linguistic map.

These models are trained by analyzing patterns in billions of books, articles, and transcripts. When you prompt an LLM, it generates the most probable linguistic pattern that fits your input.

(See: Neural Network)

Memory Cache

Memory cache refers to a process that enhances inference (the AI's response generation). Essentially, caching is an optimization technique designed to make inference more efficient. AI relies on intensive mathematical calculations, each consuming power. Caching reduces the number of calculations a model must perform by storing specific computations for reuse in future queries. There are different types of memory caching, with KV (key-value) caching being a well-known example for transformer-based models. KV caching increases efficiency and speeds up responses by reducing the time and computational effort needed to generate answers.

(See: Inference)

Neural Network

A neural network is the multi-layered algorithmic structure that underpins deep learning and, more broadly, the generative AI boom following the rise of large language models.

Although the idea of drawing inspiration from the human brain's interconnected neurons for data processing algorithms dates back to the 1940s, it was the more recent proliferation of graphical processing units (GPUs)—driven by the video game industry—that truly unlocked this theory's potential. These chips proved exceptionally well-suited for training algorithms with many more layers than was previously possible, enabling neural network-based AI systems to achieve superior performance in areas like voice recognition, autonomous navigation, and drug discovery.

(See: Large Language Model [LLM])

Open Source

Open source refers to software—or, increasingly, AI models—whose underlying code is publicly available for anyone to use, inspect, or modify. In AI, Meta's Llama family of models is a prominent example; in operating systems, Linux is the classic historical parallel. The open-source approach allows global researchers, developers, and companies to build upon each other's work, accelerating progress and enabling independent safety audits that closed systems cannot easily provide. Closed source means the code is private—you can use the product but not see how it works, as with OpenAI's GPT models. This distinction has become a central debate within the AI industry.

Parallelization

Parallelization means performing many operations simultaneously rather than sequentially—like having ten employees work on different parts of a project at once instead of one person doing everything step-by-step. In AI, parallelization is fundamental to both training and inference. Modern GPUs are specifically designed to perform thousands of calculations in parallel, a key reason they became the industry's hardware backbone. As AI systems grow more complex and models larger, the ability to parallelize work across many chips and machines has become a critical factor in determining how quickly and cost-effectively models can be built and deployed. Research into better parallelization strategies is now a field of study in its own right.

RAMageddon

RAMageddon is the playful new term for a serious trend sweeping the tech industry: a growing shortage of random access memory (RAM) chips, which power nearly all the tech products we use daily. As the AI industry has exploded, major tech companies and AI labs—all competing to build the most powerful and efficient AI—are purchasing such vast quantities of RAM for their data centers that little is left for others. This supply bottleneck is driving up prices for the remaining stock.

The impact spans industries like gaming (where companies have raised console prices due to memory chip scarcity), consumer electronics (where shortages could cause the largest dip in smartphone shipments in over a decade), and general enterprise computing (as companies struggle to source enough RAM for their own data centers). Price surges are expected to continue until the shortage ends, but unfortunately, there's little indication that will happen soon.

Reinforcement Learning

Reinforcement learning is a training method where an AI system learns by trial and error, receiving rewards for correct actions—similar to training a pet with treats, except the "pet" is a neural network and the "treat" is a mathematical signal indicating success. Unlike supervised learning, where a model is trained on a fixed dataset of labeled examples, reinforcement learning allows a model to explore its environment, take actions, and continuously update its behavior based on feedback. This approach has proven especially effective for training AI to play games, control robots, and, more recently, sharpen the reasoning abilities of large language models. Techniques like reinforcement learning from human feedback (RLHF) are now central to how leading AI labs fine-tune their models to be more helpful, accurate, and safe.

Token

Human-machine communication presents obvious challenges: people use human language, while AI programs execute tasks through complex, data-informed algorithmic processes. Tokens bridge this gap. They are the fundamental building blocks of human-AI communication, representing discrete segments of data processed or produced by an LLM. They are created through tokenization, which breaks down raw text into digestible units a language model can understand, similar to how a compiler translates human language into binary code for a computer. In enterprise contexts, tokens also determine cost—most AI companies charge for LLM usage on a per-token basis, meaning the more a business uses, the more it pays.

Token Throughput

Again, tokens are the small chunks of text—often parts of words—that AI language models break language into for processing; they are roughly analogous to "words" for understanding AI workloads. Throughput refers to how much can be processed in a given time, so token throughput essentially measures how much AI work a system can handle simultaneously. High token throughput is a key goal for AI infrastructure teams, as it determines how many users a model can serve at once and how quickly each receives a response. AI researcher Andrej Karpathy has described feeling anxious when his AI subscriptions sit idle—echoing his grad-school concern over underutilized expensive hardware—a sentiment that captures why maximizing token throughput has become an obsession in the field.

Training

Developing machine learning AI involves a process called training. Simply put, this involves feeding data into a model so it can learn patterns and generate useful outputs. It's the process by which the system responds to characteristics in the data, adapting its outputs toward a desired goal—whether that's identifying images of cats or composing a haiku on demand.

Training can be expensive because it requires massive amounts of data, and the required volumes continue to trend upward. This is why hybrid approaches, like fine-tuning a rules-based AI with targeted data, can help manage costs without starting from scratch.

[See: Inference]

Transfer Learning

A technique where a pre-trained AI model is used as the starting point for developing a new model for a different, but typically related, task. This allows knowledge gained from previous training cycles to be reapplied.

Transfer learning can drive efficiency by shortcutting model development. It is also useful when data for the new task is limited. However, the approach has limitations. Models relying on transfer learning for generalized capabilities will likely require additional training on domain-specific data to perform well in their focused area.

(See: Fine-Tuning)

Weights

Weights are central to AI training, as they determine the importance (or "weight") assigned to different features (or input variables) in the training data, thereby shaping the model's output.

In other words, weights are numerical parameters that define what is most significant in a dataset for a given training task. They function by multiplying inputs. Model training typically begins with randomly assigned weights, which adjust as the model iterates to produce outputs that more closely match the target.

For example, an AI model trained on historical real estate data to predict housing prices might have weights for features like the number of bedrooms and bathrooms, property type (detached or semi-detached), parking availability, garage, etc. The weights the model assigns to each input reflect their influence on property value based on the dataset.

Validation Loss

Validation loss is a metric that indicates how well an AI model is learning during training—lower numbers are better. Researchers monitor it closely as a real-time report card, using it to decide when to stop training, adjust hyperparameters, or investigate potential issues. A key concern it helps identify is overfitting, a condition where a model memorizes its training data rather than learning generalizable patterns. Think of it as the difference between a student who truly understands the material and one who merely memorized last year's exam—validation loss helps reveal which path your model is taking.

This article is updated regularly with new information.