Clear definitions and mental models for core software engineering concepts.
Showing 88 of 206 topics. Switching field only changes what you see here, not your learning path.
An activation function applies a nonlinear transformation to a neuron's input before passing it forward, giving neural networks the ability to learn patterns that linear models cannot represent.
An AI agent is a system where a language model works toward a goal by choosing tools to use, running them, and deciding what to do next based on the results.
Attention is how a model decides which parts of the input matter most for each piece it is processing, weighing every position against every other.
Backpropagation works out how much each weight in a network contributed to the error, by applying the chain rule backwards through the layers.
Batch normalization rescales the values flowing between layers using statistics from the current batch, which makes deep networks train faster and more reliably.
Batch size is the number of training examples processed before the model updates its weights, trading noise and speed against gradient stability.
The bias-variance tradeoff explains why models fail in two opposite directions: too simple to learn the pattern, or too sensitive to the particular data they saw.
BLEU scores machine translation by measuring how many word sequences the output shares with human reference translations.
Chain-of-thought prompting asks a model to work through its reasoning before answering, which substantially improves accuracy on multi-step problems.
Chunking is the process of dividing long documents into smaller pieces so embedding models and retrieval systems can process them accurately and return focused results.
Classification is predicting which category something belongs to, choosing from a fixed set of options learned from labelled examples.
A confusion matrix is a table that summarizes every classifier prediction against every actual class, making it easy to see which categories the model confuses.
The context window is the total amount of text a language model can hold in view at once, counted in tokens and shared between your input and its reply.
A convolutional neural network slides small filters across its input to detect local patterns, which is what made it the standard approach for images.
Cosine similarity measures how closely two vectors point in the same direction, ignoring their length, by computing the cosine of the angle between them.
Cross-validation estimates how well a model will generalise by training it several times on different splits of the data and averaging the results.
Data augmentation creates new training examples by transforming existing ones, teaching the model which changes should not affect its answer.
A decision tree is a flowchart-like model that splits data on feature values, making a series of simple if-then decisions that end in a prediction.
Deep learning is machine learning using neural networks with many layers, which discover useful features from raw data instead of being told what to look for.
A diffusion model generates images by starting from pure noise and removing a little of it at a time, having learned what noise looks like.
Dropout randomly switches off a fraction of a network's units during training, so the model cannot depend on any single one and generalises better.
An embedding is a list of numbers that represents the meaning of a piece of text, so that similar meanings end up with similar numbers.
An epoch is one complete pass through the whole training dataset, after which the model has seen every example exactly once.
The F1 score is a single number that combines precision and recall into one metric by taking their harmonic mean, so both must be reasonably high for the score to be good.
Feature engineering is turning raw data into the inputs a model can actually learn from, by cleaning, transforming and combining it.
Few-shot learning means giving a model a handful of examples in the prompt so it infers the pattern, without any training or weight changes.
Fine-tuning is taking a model that is already trained and training it further on your own examples, so it adopts a particular style, format, or task.
Function calling is the ability of a language model to request execution of a predefined external function with structured arguments, instead of trying to answer from its own parameters alone.
A GRU is a simplified LSTM that uses two gates instead of three, trading a little expressiveness for faster training and fewer parameters.
A GAN trains two networks against each other: one generates fake data, the other tries to spot it, and the competition drives both to improve.
Gradient boosting builds decision trees sequentially, with each new tree correcting the residual errors left by the previous ones, producing a strong learner from many weak learners.
Gradient descent is the method that trains most models, repeatedly nudging every parameter in the direction that reduces the loss.
Guardrails are layered input filters and output validators that keep an AI system inside its intended operating boundaries by blocking harmful requests and unsafe responses.
A hallucination is when an AI model produces something that sounds plausible and is actually false, invented, or unsupported by its source material.
Hyperparameter tuning is searching for the settings you choose rather than the model learns, such as learning rate, depth and regularization strength.
Inference is running a trained model on new data to get a prediction, without changing the model at all.
K-means clustering is an unsupervised algorithm that partitions data into k clusters by repeatedly assigning points to the nearest centroid and recomputing those centroids until they stop moving.
K-nearest neighbors is a simple instance-based algorithm that classifies or predicts a new example by majority vote or averaging over its closest neighbors in feature space.
Knowledge distillation trains a small model to imitate a large one, capturing much of its behaviour in something far cheaper to run.
The KV cache stores key and value tensors from previous transformer attention layers so the model does not recompute them on every generated token, dramatically speeding up autoregressive inference.
A large language model is a neural network trained on huge amounts of text that predicts what text should come next, one token at a time.
Layer normalization rescales values across the features of a single example rather than across a batch, which is why transformers use it instead of batch normalization.
The learning rate controls how big a step training takes on each update, and it is the single setting most likely to break training if it is wrong.
An LSTM is a recurrent network with gates that decide what to keep, forget and output, giving it much better memory over long sequences than a plain RNN.
A loss function turns how wrong a model is into a single number, which training then works to make smaller.
Machine learning (ML) is a type of artificial intelligence (AI) that allows computers to learn from data and improve their performance over time.
Mixture of Experts routes each input to a small subset of specialized subnetworks, so a model can be extremely large without activating the full parameter set on every pass.
MLOps is the set of practices that combines machine learning, DevOps, and data engineering to deploy, monitor, and maintain models in production reliably.
Model Context Protocol is an open standard that standardizes how AI applications connect to external data sources and tools, removing the need for one-off integrations between models and systems.
Model drift is the gradual decline in a model's accuracy when the real-world data it sees in production starts to differ from the data it was trained on.
Model interpretability is the ability to understand why a model made a particular decision, which matters wherever those decisions have to be justified.
A multimodal model is a system that processes and reasons across multiple input types, such as text, images, audio, or video, instead of being limited to a single modality.
Naive Bayes applies Bayes theorem with a conditional independence assumption between features, making it fast, simple, and surprisingly effective on text and small datasets.
A neural network is a model built from layers of simple units that each apply a weight to their inputs, learning those weights from examples.
Overfitting is when a model learns its training data too well, memorising the noise instead of the real pattern, so it fails on new data.
Perplexity measures how surprised a language model is by text, and lower means the model predicted it better.
Precision measures how many of the items the model labeled as positive are actually positive, while recall measures how many of the actual positives the model found.
Principal Component Analysis finds orthogonal axes of maximum variance in data and projects it onto fewer dimensions while preserving as much information as possible.
Prompt engineering is writing instructions for an AI model carefully enough that it gives you what you want reliably, not just once by luck.
Prompt injection is an attack where text supplied to a model is crafted to override its instructions, making it ignore its intended behaviour.
Quantization stores a model's numbers at lower precision, cutting memory and speeding up inference for a small loss in accuracy.
Random Forest is an ensemble method that trains many decision trees on different subsets of data and features, then averages their predictions to produce a more stable and accurate result.
A recurrent neural network processes a sequence one step at a time, passing a hidden state forward so earlier inputs can influence later ones.
Regression is predicting a number rather than a category, learning the relationship between inputs and a continuous value.
Regularization is any technique that deliberately constrains a model so it generalises better instead of memorising its training data.
Reinforcement learning trains an agent to make decisions by letting it act in an environment and rewarding the outcomes, rather than showing it correct answers.
RLHF trains a model to prefer the responses people rate higher, which is how raw language models become assistants that follow instructions.
Reranking is a second-stage retrieval step that takes an initial set of candidate documents and reorders them by semantic fit to the query using a more powerful but slower cross-encoder model.
RAG is a pattern where you search your own documents for relevant passages and paste them into the prompt, so the model answers from real sources instead of memory.
The field combining mechanical engineering, electronics, and software to create machines that can perform tasks autonomously or semi-autonomously.
ROC AUC measures a classifier's ability to separate classes across every possible threshold by summarizing its receiver operating characteristic curve into a single number.
ROUGE scores summarisation by measuring how much of a human reference summary appears in the generated one.
Semantic search retrieves documents by meaning rather than keyword overlap, using embedding similarity to find results that match the intent behind a query.
Structured output is any technique that forces a language model to return data matching a predefined schema, such as JSON, XML, or a custom grammar, instead of free-form prose.
Supervised learning trains a model on examples where the right answer is already known, so it can predict the answer for new cases.
A support vector machine is a classifier that finds the hyperplane with the maximum margin between classes, using kernel methods to handle nonlinear boundaries when needed.
Synthetic data is artificially generated data that mimics the statistical properties of real data, used to augment training sets, test edge cases, or protect privacy.
A system prompt is a hidden instruction set at the start of a conversation that shapes how a model behaves throughout the entire interaction, before the user ever speaks.
Temperature is a sampling parameter that controls randomness in a language model's output by scaling the probability distribution before the next token is chosen.
Tokenization is the process of splitting text into small pieces called tokens, which are the units a language model actually reads.
Training data is the set of examples a model learns from, and its quality sets the ceiling on how good the finished model can possibly be.
Transfer learning starts from a model already trained on a large dataset and adapts it to your task, which needs far less data than training from scratch.
A transformer is the neural network design behind modern AI models, which reads a whole sequence at once using attention instead of stepping through it word by word.
Underfitting is when a model is too simple to learn the pattern in its data, so it gets things wrong even on the examples it trained on.
Unsupervised learning finds structure in data that has no labels, grouping similar things together or reducing complexity without being told what to look for.
A VAE learns to compress data into a smooth probabilistic space and rebuild it, which lets you generate new samples by picking points in that space.
A vector database stores embeddings and finds the ones closest in meaning to a query, rather than matching exact values the way a normal database does.
Zero-shot learning means asking a model to do a task with only an instruction and no examples, relying on what it learned during training.
