There’s a special irony in observability: the moment your systems get loud is the moment they produce the most telemetry — and the most telemetry is exactly what kills an unprotected Collector. An incident starts, spans and logs surge, the Collector’s memory climbs, the OOM killer does its job, and your monitoring dies precisely when you bought it for. The arsonist and the fire alarm, sharing a fuse box.
The memory_limiter processor exists to break that loop. It’s not optional tuning; on any Collector that matters, it should be as standard as the exporter.
What it actually does
The memory_limiter watches the Collector’s heap usage on a fixed interval and enforces two thresholds:
The soft limit. When heap usage crosses it, the processor starts refusing data — it returns errors to whatever feeds it. Upstream components that can retry (an OTLP receiver, which tells its client “try again”) retry; the data waits at the sender instead of accumulating in a dying Collector. Scrapers like the RabbitMQ receiver simply skip a scrape and pick up again at the next interval.
The hard limit (limit_mib). If memory keeps climbing past it, the processor additionally forces a garbage-collection cycle — the last resort before the OOM killer makes the decision for you.
The gap between the two is spike_limit_mib: soft limit = limit_mib - spike_limit_mib. The default spike is 20% of the hard limit, which is right for most estates; spiky traffic or a long check interval justify more headroom.
Refusing data sounds bad until you consider the alternative. Without a limiter, the Collector doesn’t degrade — it disappears, taking every pipeline with it, and returns with empty buffers and no memory of what it missed. Refusal is a brownout; OOM is a blackout. Choose the brownout.
How to configure it
Three rules cover nearly every deployment:
1. Put it first. memory_limiter must be the first processor in every pipeline, directly after the receivers — backpressure has to reach the sender before the Collector has invested memory in processing.
2. Size it against the container, not the host. Give the Collector’s container a memory limit, set limit_mib comfortably below it (the process runs ~50 MiB above the heap target), or use limit_percentage/spike_limit_percentage and let it size itself from the cgroup limit.
3. Set GOMEMLIMIT too. The official recommendation is both: the GOMEMLIMIT environment variable at ~80% of the hard limit makes the Go runtime itself collect garbage more aggressively as it approaches the ceiling. The two mechanisms cooperate; neither replaces the other.
A complete example: RabbitMQ metrics into Sluicio, with a seatbelt
This extends the pipeline from our RabbitMQ monitoring guide — same receiver, same exporter, now with memory protection and batching:
And the runtime side, in your container spec:
environment:
GOMEMLIMIT: "320MiB" # ~80% of limit_mib
deploy:
resources:
limits:
memory: 512MThe ordering in processors: [memory_limiter, batch] is the part people get wrong: the limiter guards the door, the batcher packs the boxes. Reverse them and the batcher happily accumulates memory the limiter never sees.
What happens under pressure with this config: if something floods the Collector past 320 MiB of heap, the RabbitMQ receiver’s scrapes start being refused — you lose a 60-second sample or two of queue metrics. Past 400 MiB, forced GC buys back headroom. Either way, the Collector survives, and the pipeline resumes on its own. Without it: OOM kill, restart, and a blind spot exactly as wide as your incident.
Watch the limiter itself
The Collector reports its own refusals as internal telemetry: otelcol_processor_refused_metric_points (and its spans/logs siblings). Ship those to Sluicio like any other metric and put a health check on them — occasional refusals during a spike are the system working; sustained refusals mean your Collector is undersized and quietly rationing your visibility. That’s a capacity conversation, not an incident.
Because that’s the real point of the memory limiter: it converts a crash you’d discover during your worst outage into a metric you can see on any calm Tuesday.