Engineering for Reliability
How to troubleshoot the Ops Agent
Twenty-three substantive episodes on SLOs, telemetry, alerting, investigation, and operations
Twenty-three 2021–2022 episodes connect user-facing reliability targets to metrics, logs, traces, and alerts. Examples show instrumentation, Prometheus collection, quota monitoring, GKE investigation, and observability cost controls.
In this collection Browse 23 summaries 3 of 23
When a virtual machine is running but its application logs or resource measurements are missing, the Ops Agent may be failing before that data reaches Google Cloud. Yuri, an SRE at Google Cloud, separates troubleshooting into installation, startup, collection, and delivery so that each check narrows the failure 00:00–03:06(opens in a new tab).
Key Points Covered
- Installation and startup are separate checks: In the demonstrated environment, unsupported operating systems, legacy-agent conflicts, or invalid configuration can stop the pipeline early 01:01–02:04(opens in a new tab).
- A running agent still needs working collection modules: The logging and monitoring components can fail to start if invalid credentials, a proxy configuration, or a network problem prevents them from reaching their APIs 02:04–03:06(opens in a new tab).
- Separate reading data from sending it: For missing logs, check the logging module, then its local metrics to see whether it can read the source logs. If collection works, inspect the module's own log for API errors such as missing permissions or exhausted quota. The metrics module's logs provide a similar way to check startup and delivery failures 03:06–04:10(opens in a new tab).
Full video: https://www.youtube.com/watch?v=Sd0iznXSVcc(opens in a new tab)