Log aggregation is the practice of collecting logs from many sources into a central store where they can be searched, filtered, correlated, and retained for debugging and compliance.
Log aggregation pipelines collect log events from servers, containers, network devices, and applications. Agents, sidecars, or stdout collectors ship logs to a central backend. Indexing and retention policies make logs searchable and durable.
That centralization is necessary because distributed systems make SSH and local files obsolete as debugging surfaces. When a request crosses five services, the relevant log lines are scattered across machines and containers. Aggregation joins those fragments into one searchable timeline, making root cause analysis possible instead of guesswork.
Think of it like this. Think of gathering receipts from every cashier in a mall into one accounting system. Searching one drawer gives you only part of the story; the central ledger shows the full transaction.
Agents or collectors read logs from files, stdout, journald, or API endpoints. They parse, enrich, and ship events to a backend. The backend indexes content, applies retention, and provides query interfaces. Structured logging makes filtering reliable; unstructured text requires parsing heuristics that break easily.
"Log everything at debug level forever." Storage and signal-to-noise ratio collapse. "Aggregation replaces context." Without request IDs, trace IDs, or service labels, logs remain siloed. "One tool handles all logs." Application logs, audit logs, and metrics often have different retention and compliance requirements.
Enables cross-service debugging and compliance retention, but storage and indexing costs grow with volume. Best paired with structured logging, sampling, and clear retention policies.