Engineering for Reliability
How to set up Prometheus monitoring for your services
Twenty-three substantive episodes on SLOs, telemetry, alerting, investigation, and operations
Twenty-three 2021–2022 episodes connect user-facing reliability targets to metrics, logs, traces, and alerts. Examples show instrumentation, Prometheus collection, quota monitoring, GKE investigation, and observability cost controls.
Google Cloud Tech 1 of 23
In this collection Browse 23 summaries 1 of 23
Infrastructure metrics alone do not tell you how often an application fails or how long its users wait. Yuri, a site reliability engineer at Google, instruments a Python service to expose those measurements, then configures Google Managed Service for Prometheus to collect them 00:00–02:06(opens in a new tab).
Key Points Covered
- Record application behavior and expose it for collection: The Flask example increments total-request and failed-request counters and times a route to record a latency distribution. It exposes the resulting values on
/metrics, an endpoint the collector can scrape—read to collect measurements 01:03–02:06(opens in a new tab). - Tell the collector which application to read: In the demonstrated setup, a
PodMonitoringresource selects pods by their labels and specifies their metrics endpoint. Installing the collector alone does not complete the application's monitoring configuration 02:06–03:11(opens in a new tab). - Verify collection in stages: First the demo checks that the application exposes metrics, then queries the collector's own metrics to confirm collection is running. Finally it queries an application metric to prove that the selected workload's data reached the backend, where it can be used in charts and alerts 02:06–04:13(opens in a new tab).
Full video: https://www.youtube.com/watch?v=qyfOE_78nT0(opens in a new tab)