Glossary of common AI terms every AI Engineer must understand
This article is divided into 7 parts for better segregation of the terms and understanding.
Part 1: Building a model
Neural network. A neural network is a structure of math units in layers. The design copies, in a simple way, the connections between neurons in the brain. The idea is from the 1940s. GPUs made the idea practical because GPUs can do a very large quantity of calculations at the same time.
Deep learning. Deep learning is machine learning that uses a neural network with many layers. The model finds the important features in the data. Deep learning needs a large quantity of data, often millions of examples. Thus, it costs more and takes more time than simpler methods.
Weights. Weights are the numbers inside the neural network. Each weight sets how much one input changes the result. At the start of training, the weights are random. Training changes the weights until the output is close to the correct answer.
Why it is important to understand: When an AI provider (GPT, Claude) mentions that a model has “70 billion parameters,” the parameters are mostly weights. More weights usually means more capability and a higher cost to run.
Training. Training is the process that sends data into the model so that the model learns patterns. Training is the most expensive step in the life of an AI model.
Validation loss. Validation loss is a score that shows how well the model learns. Researchers measure it on data that the model did not see during training. A lower score is better. The score helps to find overfitting. Overfitting occurs when a model memorizes its training examples but cannot answer new questions correctly.
Reinforcement learning (RL). In reinforcement learning, the model takes an action and receives a signal that shows whether the action was good. Then the model adjusts. Reinforcement learning from human feedback (RLHF) uses ratings from people as the signal. AI labs use RLHF to make models more helpful, more accurate, and safer.
Fine-tuning. Fine-tuning is more training on a model that is already trained. The new data is specialized, for example, legal documents or support tickets.
Why it is important to understand: Fine-tuning can make a general model better at your specific task. It is usually much cheaper than training a new model.
Transfer learning. Transfer learning uses the knowledge of one trained model as the starting point for a related task. It saves time and helps when you have little data. The model usually still needs some training on data from the new domain.
Distillation. Distillation uses a “teacher” model and a “student” model. The large teacher model answers many requests. A team uses these answers to train a smaller and faster student model. All AI labs use distillation on their own models.
Part 2: What occurs when you use a model
Large language model (LLM). An LLM is an AI model such as ChatGPT, Claude, Gemini, Muse, and Kimi. It is a deep neural network with billions of weights. It learns the relationships between words from a very large quantity of text. When you send a prompt, the LLM writes the most probable text that fits the prompt.
Token. A token is the unit of text that an LLM reads and writes. A token is usually a word or a part of a word. Tokenization is the step that divides your text into tokens. AI providers charge per token. More tokens mean a larger AI bill.
Inference. Inference is the use of a trained model to make an answer. Training occurs first. Inference occurs each time that you send a prompt. Many types of chips can do inference. Large models need powerful hardware to answer quickly.
Why it is important to know the difference: Training is a one-time cost. Inference is a cost that continues for each user and each request.
Memory cache (KV cache). A memory cache keeps calculations (i.e., remembering the context and the previous input and output) that the model already did, so the model does not do them again. The most well-known type is the KV (key-value) cache in transformer models. It makes answers faster and cheaper.
Token throughput. Token throughput is the quantity of tokens that a system can process in a given time. It sets how many users a model can serve at the same time and how fast each user receives an answer. The famous AI researcher Andrej Karpathy said that he feels anxious when his AI subscriptions are not in use. This shows why infrastructure teams monitor throughput closely.
Compute. Compute is the processing power that AI systems need. The term often also means the hardware: GPUs or TPUs with CUDA or Tensors cores, CPUs, and data centers.
Parallelization. Parallelization means that a system does many calculations at the same time, not one after the other. GPUs are designed for this. This is the main reason why GPUs are the base of the AI industry.
Mixture of Experts (MoE). An MoE model has many small “expert” networks. A router in the model selects only a few experts for each request. So, a very large model can stay fast and cheap to run, because most of the model is not active at a given time. Mistral’s Mixtral is a known example. Many people think that the newer OpenAI GPT models use a similar design. OpenAI did not officially confirm this.
Part 3: How a model reasons
Chain of thought. With chain of thought, a model divides a problem into small steps before it gives the answer. It is slower. But the answer is more frequently correct, mostly for logic, math, and code.
Reasoning model. A reasoning model is an LLM that is optimized for chain of thought, mostly with reinforcement learning. The “thinking” text that some AI agents show before the answer is the chain of thought.
Opaque recurrence. With opaque recurrence, a model sends a query through its internal layers many times. It does not write its steps in readable language. This method is efficient. A small model can do the work of a larger model with less compute. But the method leaves fewer readable records. Safety researchers use readable chains of thought to find incorrect model behavior. Less visibility makes this work more difficult. OpenAI’s Astra model (September 2026) is the best-known example.
Why it is important: If your organization must audit AI decisions, readable reasoning is a control. A model that hides its reasoning is more difficult to audit.
Recurrent depth. Recurrent depth is the engineering name for the same method as opaque recurrence. “Opaque recurrence” is the name that shows the safety concern.
Neuralese. Neuralese is a hypothetical worst case. In this case, a model reasons only in internal numbers, not in human language. Then nobody can read how the model thinks. No released model does this today. OpenAI says that Astra keeps its chain of thought readable. But some safety researchers think that opaque recurrence is a first step in the direction of neuralese.
Part 4: From chatbot to agent
AI agent. An AI agent uses AI to do a series of tasks for you. Examples: book a table, send an expense report, or write and maintain code. A chatbot answers one question. An agent completes a task with many steps. The term can have different meanings for different people.
Coding agent. A coding agent is an agent for software development. It writes code, runs the tests, reads the errors, and repairs the code. It can do this across a full codebase.
Why it is important to understand: A coding agent is fast, but it can make errors. A person must always review the code before it goes into production.
API endpoints. An API endpoint is a connection point (usually a URL) that software gives to other software programs. Agents can find and use these endpoints. This is one of the many ways an agent execute tasks.
Model Context Protocol (MCP). MCP is an open standard. It lets AI models connect to external tools and data, for example files, databases, Slack, or Google Drive. With MCP, a developer does not have to write a custom connector for each pair of model and tool. Anthropic introduced MCP in 2024. Later, Anthropic gave it to the Linux Foundation. OpenAI, Google, and Microsoft use it now.
Why it is important: Each MCP connection gives the AI access to a system. Approve each connection in the same way that you approve access for a new employee.
Part 5: Other ways that AI makes content
Diffusion. Diffusion is the method used in many image, music, and video generators. During training, the system adds noise to real data, for example, a photo, until nothing is recognizable. Then the system learns to reverse this process. After training, it starts from random noise and removes noise until a new image appears. Diffusion models generate new content (like images, music, or video) by learning how to clean up random pixel noise.
GAN (Generative Adversarial Network). A GAN has two neural networks that compete. One network makes false data. The other network tries to identify the false data. Each network makes the other network better. The result is very realistic output. GANs work best for specific tasks, for example, realistic photos and videos (including deepfakes). They are not general-purpose AI.
Part 6: Where AI fails
Hallucination. A hallucination occurs when an AI model gives incorrect information as if it is a fact. It is a quality problem. In health or law, it is also a safety problem. One possible cause is gaps in the training data. This is one reason why companies make more specialized models for specific domains.
Why it is important to know: An answer that sounds correct is not always correct. Always examine facts, numbers, and sources before you use them.
Part 7: The large AI debates
AGI (artificial general intelligence). AGI is AI that is equal to or better than humans at most tasks. But experts do not agree on one definition:
- The OpenAI charter: “highly autonomous systems that outperform humans at most economically valuable work.”
- Google DeepMind: AI that is at least as capable as humans at most cognitive tasks.
- Sam Altman: the equivalent of a median human that you could hire as a co-worker.
Recursive self-improvement (RSI). RSI is a scenario where AI models improve themselves without human help, and their capability increases very quickly. Some people compare it to the “singularity.” Some startups now try to build RSI. Most of them say that it is the next research step, not a disaster for mankind.
Open source and closed source. With an open-source model, often the weights are public. Anyone can examine it, change it, and build on it. Meta’s Muse Glimmer and many Chinese models such as Kimi and GLM are common examples. With a closed-source model, such as the OpenAI GPT models, you can use the model, but you cannot examine it. This is one of the main debates in the AI industry.
Why it is important to you: An open model can run on your own infrastructure, so your data stays with you. A closed model is usually easier to start with, but you depend on the vendor.
RAMageddon. RAMageddon is the name for a shortage of memory chips (RAM). AI companies buy very large quantities of RAM for their data centers. So, the prices of RAM is increasing for other industries, for example, game consoles, smartphones, pen drives, and enterprise servers. At this time, there is no clear sign that the shortage will end soon.
When you can explain these five terms without notes, you can understand most AI discussions at work.