Prometheus, InfluxDB, Datadog or ELK — which, and for what
Most comparisons of telemetry backends are written by people who have run one of them. The result is a list of features and a conclusion that happens to match whatever the author already uses. Having operated all four in production, the more useful framing is that they solve different collection problems, and the decision should follow the shape of what you are measuring rather than the shape of the vendor’s pricing page.
Prometheus
Pull-based. A server scrapes endpoints on an interval, which means your services expose metrics and Prometheus comes to get them.
That model is excellent in Kubernetes, because service discovery is already solved — pods come and go, and the scrape config follows them. It is also excellent operationally: if a target is down, that fact is itself a signal, and there is no queue of buffered metrics from a dead process to reason about.
Where it strains is long retention and high cardinality. A metric with a label containing user IDs or request paths will multiply series count until the server struggles, and Prometheus does not solve this for you — it fails loudly and somewhat late. Long-term storage means adding Thanos or Mimir, which is another system to run.
Choose it for service-level metrics in a container platform. Do not choose it as a general time-series store for everything the business wants to graph.
InfluxDB with Telegraf
Push-based. Telegraf agents sit on hosts and devices, collect, and push to InfluxDB.
This is the better fit precisely where Prometheus is awkward: host metrics, network devices, telephony systems, anything behind a NAT, anything that cannot expose a scrape endpoint. Telegraf’s input plugin coverage is very broad, and getting data out of an appliance that speaks SNMP and nothing else is a solved problem there and a project in Prometheus.
The trade-off is operational. You are now running a database with its own retention policies, downsampling and storage characteristics, and the failure modes are the failure modes of a database. Push also means a broken agent is silent — absence of data is ambiguous in a way it is not with a scrape.
Choose it for infrastructure and device telemetry, and for anywhere the push model removes a networking problem.
Datadog
Genuinely good, and the fastest path from nothing to a working observability practice. The agent collects everything, the integrations work, the correlation between metrics, traces and logs is the best of the four, and you do not run any of it.
The problem is that the pricing model rewards exactly the behaviours that a good observability practice produces. Custom metrics are billed per series, so high-cardinality labels — the ones that make an incident investigable — are expensive. Log ingestion and indexing are billed separately, so teams start dropping logs to control spend, which is the opposite of the intent. Most Datadog cost problems I have seen were not waste; they were the platform working as designed against a team that did not know the shape of the bill.
Choose it when speed to value matters more than unit cost, and go in with a cardinality policy from day one rather than after the renewal conversation.
ELK
Strong for log search and analysis. When the question is “what happened in this request across six services”, a well-indexed Elasticsearch cluster answers it better than a metrics system ever will, because logs carry the context metrics deliberately discard.
The honest cost is operational. Running Elasticsearch well — shard sizing, index lifecycle, JVM tuning, cluster upgrades — is a real ongoing burden, and teams consistently underestimate it when comparing against a hosted bill. If nobody owns that work, the cluster degrades quietly until an incident finds it.
Choose it when log search is central and you have someone to own the cluster. Otherwise compare it honestly against hosted logging, counting engineer time.
The decision in practice
Most organisations end up with more than one, and that is fine. The failure is not having several — it is having several with no stated boundary, so the same data lands in two places and nobody knows which to trust during an incident.
The useful exercise is to write down, per signal type, which system is authoritative and what it is for. Metrics from Kubernetes workloads here. Host and device telemetry there. Logs here, with this retention. Traces there. Once that document exists, the overlap becomes visible and the bill usually falls without removing anything anyone needed.