Dev Glossary

Batch Normalization

Batch Size

Machine Learning2 resources

Bias-Variance Tradeoff

BLEU Score

Backpropagation

Backpropagation works out how much each weight in a network contributed to the error, by applying the chain rule backwards through the layers.

Machine Learning

Batch Normalization

Batch normalization rescales the values flowing between layers using statistics from the current batch, which makes deep networks train faster and more reliably.

Machine Learning

Batch Size

Batch size is the number of training examples processed before the model updates its weights, trading noise and speed against gradient stability.

Machine Learning

Bias-Variance Tradeoff

The bias-variance tradeoff explains why models fail in two opposite directions: too simple to learn the pattern, or too sensitive to the particular data they saw.

Machine Learning

BLEU Score

BLEU scores machine translation by measuring how many word sequences the output shares with human reference translations.

Machine Learning

C

Chain-of-Thought Prompting

Chunking

Machine Learning3 resources

Classification

Confusion Matrix

Context Window

Convolutional Neural Network

Cosine Similarity

Cross-Validation

Chain-of-Thought Prompting

Chain-of-thought prompting asks a model to work through its reasoning before answering, which substantially improves accuracy on multi-step problems.

Machine Learning

Chunking

Chunking is the process of dividing long documents into smaller pieces so embedding models and retrieval systems can process them accurately and return focused results.

Machine Learning

Classification

Classification is predicting which category something belongs to, choosing from a fixed set of options learned from labelled examples.

Machine Learning

Confusion Matrix

A confusion matrix is a table that summarizes every classifier prediction against every actual class, making it easy to see which categories the model confuses.

Machine Learning

Context Window

The context window is the total amount of text a language model can hold in view at once, counted in tokens and shared between your input and its reply.

Machine Learning

Convolutional Neural Network

A convolutional neural network slides small filters across its input to detect local patterns, which is what made it the standard approach for images.

Machine Learning

Cosine Similarity

Cosine similarity measures how closely two vectors point in the same direction, ignoring their length, by computing the cosine of the angle between them.

Machine Learning

Cross-Validation

Cross-validation estimates how well a model will generalise by training it several times on different splits of the data and averaging the results.

Machine Learning

D

Data Augmentation

Decision Tree

Deep Learning

Diffusion Model

Dropout

Data Augmentation

Data augmentation creates new training examples by transforming existing ones, teaching the model which changes should not affect its answer.

Machine Learning

Decision Tree

A decision tree is a flowchart-like model that splits data on feature values, making a series of simple if-then decisions that end in a prediction.

Machine Learning

Deep Learning

Deep learning is machine learning using neural networks with many layers, which discover useful features from raw data instead of being told what to look for.

Machine Learning

Diffusion Model

A diffusion model generates images by starting from pure noise and removing a little of it at a time, having learned what noise looks like.

Machine Learning

Dropout

Dropout randomly switches off a fraction of a network's units during training, so the model cannot depend on any single one and generalises better.

Machine Learning

E

Embedding

E

Epoch

E

Embedding

An embedding is a list of numbers that represents the meaning of a piece of text, so that similar meanings end up with similar numbers.

Machine Learning

E

Epoch

An epoch is one complete pass through the whole training dataset, after which the model has seen every example exactly once.

Machine Learning

F

F1 Score

Machine Learning35 resources

Feature Engineering

Few-Shot Learning

Fine-tuning

Function Calling

F1 Score

The F1 score is a single number that combines precision and recall into one metric by taking their harmonic mean, so both must be reasonably high for the score to be good.

Machine Learning

Feature Engineering

Feature engineering is turning raw data into the inputs a model can actually learn from, by cleaning, transforming and combining it.

Machine Learning

Few-Shot Learning

Few-shot learning means giving a model a handful of examples in the prompt so it infers the pattern, without any training or weight changes.

Machine Learning

Fine-tuning

Fine-tuning is taking a model that is already trained and training it further on your own examples, so it adopts a particular style, format, or task.

Machine Learning

Function Calling

Function calling is the ability of a language model to request execution of a predefined external function with structured arguments, instead of trying to answer from its own parameters alone.

Machine Learning

G

Gated Recurrent Unit

Generative Adversarial Network

Gradient Boosting

Gradient Descent

Guardrails

Gated Recurrent Unit

A GRU is a simplified LSTM that uses two gates instead of three, trading a little expressiveness for faster training and fewer parameters.

Machine Learning

Generative Adversarial Network

A GAN trains two networks against each other: one generates fake data, the other tries to spot it, and the competition drives both to improve.

Machine Learning

Gradient Boosting

Gradient boosting builds decision trees sequentially, with each new tree correcting the residual errors left by the previous ones, producing a strong learner from many weak learners.

Machine Learning

Gradient Descent

Gradient descent is the method that trains most models, repeatedly nudging every parameter in the direction that reduces the loss.

Machine Learning

Guardrails

Guardrails are layered input filters and output validators that keep an AI system inside its intended operating boundaries by blocking harmful requests and unsafe responses.

Machine Learning

H

Hallucination

H

Hyperparameter Tuning

Machine Learning27 resources

H

Hallucination

A hallucination is when an AI model produces something that sounds plausible and is actually false, invented, or unsupported by its source material.

Machine Learning

H

Hyperparameter Tuning

Hyperparameter tuning is searching for the settings you choose rather than the model learns, such as learning rate, depth and regularization strength.

Machine Learning

I

Inference

I

Inference

Inference is running a trained model on new data to get a prediction, without changing the model at all.

Machine Learning

K

K-Means Clustering

K-Nearest Neighbors

Knowledge Distillation

KV Cache

K-Means Clustering

K-means clustering is an unsupervised algorithm that partitions data into k clusters by repeatedly assigning points to the nearest centroid and recomputing those centroids until they stop moving.

Machine Learning

K-Nearest Neighbors

K-nearest neighbors is a simple instance-based algorithm that classifies or predicts a new example by majority vote or averaging over its closest neighbors in feature space.

Machine Learning

Knowledge Distillation

Knowledge distillation trains a small model to imitate a large one, capturing much of its behaviour in something far cheaper to run.

Machine Learning

KV Cache

The KV cache stores key and value tensors from previous transformer attention layers so the model does not recompute them on every generated token, dramatically speeding up autoregressive inference.

Machine Learning

L

Large Language Model

Layer Normalization

Learning Rate

Long Short-Term Memory

Machine Learning9 resources

Loss Function

Large Language Model

A large language model is a neural network trained on huge amounts of text that predicts what text should come next, one token at a time.

Machine Learning

Layer Normalization

Layer normalization rescales values across the features of a single example rather than across a batch, which is why transformers use it instead of batch normalization.

Machine Learning

Learning Rate

The learning rate controls how big a step training takes on each update, and it is the single setting most likely to break training if it is wrong.

Machine Learning

Long Short-Term Memory

An LSTM is a recurrent network with gates that decide what to keep, forget and output, giving it much better memory over long sequences than a plain RNN.

Machine Learning

Loss Function

A loss function turns how wrong a model is into a single number, which training then works to make smaller.

Machine Learning

M

Machine Learning

AIML6 resources

Mixture of Experts

MLOps

Model Context Protocol

Model Drift

Model Interpretability

Multimodal Model

Machine Learning

Machine learning (ML) is a type of artificial intelligence (AI) that allows computers to learn from data and improve their performance over time.

AIML

Mixture of Experts

Mixture of Experts routes each input to a small subset of specialized subnetworks, so a model can be extremely large without activating the full parameter set on every pass.

Machine Learning

MLOps

MLOps is the set of practices that combines machine learning, DevOps, and data engineering to deploy, monitor, and maintain models in production reliably.

Machine Learning

Model Context Protocol

Model Context Protocol is an open standard that standardizes how AI applications connect to external data sources and tools, removing the need for one-off integrations between models and systems.

Machine Learning

Model Drift

Model drift is the gradual decline in a model's accuracy when the real-world data it sees in production starts to differ from the data it was trained on.

Machine Learning

Model Interpretability

Model interpretability is the ability to understand why a model made a particular decision, which matters wherever those decisions have to be justified.

Machine Learning

Multimodal Model

A multimodal model is a system that processes and reasons across multiple input types, such as text, images, audio, or video, instead of being limited to a single modality.

Machine Learning

N

Naive Bayes

N

Neural Network

Machine Learning5 resources

N

Naive Bayes

Naive Bayes applies Bayes theorem with a conditional independence assumption between features, making it fast, simple, and surprisingly effective on text and small datasets.

Machine Learning

N

Neural Network

A neural network is a model built from layers of simple units that each apply a weight to their inputs, learning those weights from examples.

Machine Learning

O

Overfitting

Overfitting

Overfitting is when a model learns its training data too well, memorising the noise instead of the real pattern, so it fails on new data.

Machine Learning

P

Perplexity

Precision and Recall

Principal Component Analysis

Prompt Engineering

Prompt Injection

Security0 resources

Perplexity

Perplexity measures how surprised a language model is by text, and lower means the model predicted it better.

Machine Learning

Precision and Recall

Precision measures how many of the items the model labeled as positive are actually positive, while recall measures how many of the actual positives the model found.

Machine Learning

Principal Component Analysis

Principal Component Analysis finds orthogonal axes of maximum variance in data and projects it onto fewer dimensions while preserving as much information as possible.

Machine Learning

Prompt Engineering

Prompt engineering is writing instructions for an AI model carefully enough that it gives you what you want reliably, not just once by luck.

Machine Learning

Prompt Injection

Prompt injection is an attack where text supplied to a model is crafted to override its instructions, making it ignore its intended behaviour.

Security

Q

Quantization

Q

Quantization

Quantization stores a model's numbers at lower precision, cutting memory and speeding up inference for a small loss in accuracy.

Machine Learning

R

Random Forest

Recurrent Neural Network

Machine Learning22 resources

Regression

Regularization

Reinforcement Learning

Reinforcement Learning from Human Feedback

Reranking

Retrieval Augmented Generation

Robotics

Emerging Tech0 resources

ROC AUC

ROUGE Score

Random Forest

Random Forest is an ensemble method that trains many decision trees on different subsets of data and features, then averages their predictions to produce a more stable and accurate result.

Machine Learning

Recurrent Neural Network

A recurrent neural network processes a sequence one step at a time, passing a hidden state forward so earlier inputs can influence later ones.

Machine Learning

Regression

Regression is predicting a number rather than a category, learning the relationship between inputs and a continuous value.

Machine Learning

Regularization

Regularization is any technique that deliberately constrains a model so it generalises better instead of memorising its training data.

Machine Learning

Reinforcement Learning

Reinforcement learning trains an agent to make decisions by letting it act in an environment and rewarding the outcomes, rather than showing it correct answers.

Machine Learning

Reinforcement Learning from Human Feedback

RLHF trains a model to prefer the responses people rate higher, which is how raw language models become assistants that follow instructions.

Machine Learning

Reranking

Reranking is a second-stage retrieval step that takes an initial set of candidate documents and reorders them by semantic fit to the query using a more powerful but slower cross-encoder model.

Machine Learning

Retrieval Augmented Generation

RAG is a pattern where you search your own documents for relevant passages and paste them into the prompt, so the model answers from real sources instead of memory.

Machine Learning

Robotics

The field combining mechanical engineering, electronics, and software to create machines that can perform tasks autonomously or semi-autonomously.

Emerging Tech

ROC AUC

ROC AUC measures a classifier's ability to separate classes across every possible threshold by summarizing its receiver operating characteristic curve into a single number.

Machine Learning

ROUGE Score

ROUGE scores summarisation by measuring how much of a human reference summary appears in the generated one.

Machine Learning

S

Semantic Search

Structured Output

Supervised Learning

Support Vector Machine

Synthetic Data

System Prompt

Semantic Search

Semantic search retrieves documents by meaning rather than keyword overlap, using embedding similarity to find results that match the intent behind a query.

Machine Learning

Structured Output

Structured output is any technique that forces a language model to return data matching a predefined schema, such as JSON, XML, or a custom grammar, instead of free-form prose.

Machine Learning

Supervised Learning

Supervised learning trains a model on examples where the right answer is already known, so it can predict the answer for new cases.

Machine Learning

Support Vector Machine

A support vector machine is a classifier that finds the hyperplane with the maximum margin between classes, using kernel methods to handle nonlinear boundaries when needed.

Machine Learning

Synthetic Data

Synthetic data is artificially generated data that mimics the statistical properties of real data, used to augment training sets, test edge cases, or protect privacy.

Machine Learning

System Prompt

A system prompt is a hidden instruction set at the start of a conversation that shapes how a model behaves throughout the entire interaction, before the user ever speaks.

Machine Learning

T

Temperature

Tokenization

Machine Learning3 resources

Training Data

Transfer Learning

Transformer

Temperature

Temperature is a sampling parameter that controls randomness in a language model's output by scaling the probability distribution before the next token is chosen.

Machine Learning

Tokenization

Tokenization is the process of splitting text into small pieces called tokens, which are the units a language model actually reads.

Machine Learning

Training Data

Training data is the set of examples a model learns from, and its quality sets the ceiling on how good the finished model can possibly be.

Machine Learning

Transfer Learning

Transfer learning starts from a model already trained on a large dataset and adapts it to your task, which needs far less data than training from scratch.

Machine Learning

Transformer

A transformer is the neural network design behind modern AI models, which reads a whole sequence at once using attention instead of stepping through it word by word.

Machine Learning

U

Underfitting

U

Unsupervised Learning

Underfitting

Underfitting is when a model is too simple to learn the pattern in its data, so it gets things wrong even on the examples it trained on.

Machine Learning

U

Unsupervised Learning

Unsupervised learning finds structure in data that has no labels, grouping similar things together or reducing complexity without being told what to look for.

Machine Learning

V

Variational Autoencoder

V

Vector Database

Databases0 resources

V

Variational Autoencoder

A VAE learns to compress data into a smooth probabilistic space and rebuild it, which lets you generate new samples by picking points in that space.

Machine Learning

V

Vector Database

A vector database stores embeddings and finds the ones closest in meaning to a query, rather than matching exact values the way a normal database does.

Databases

Z

Zero-Shot Learning