Serverless Expeditions — selected guide
Cloud Run scalability
Fourteen episodes on service choice, architecture, scaling, state, security, operations, and delivery
Serverless design and operations through concrete examples: choosing a runtime, ordering events, managing relational state, undoing failed workflows, protecting secrets, and deploying changes safely. The selection also includes two AI-agent examples covering tools and database access.
In this collection Browse 14 summaries 4 of 14
What happens when traffic rises faster than the running containers can handle it? Martin and V test Cloud Run with a short synthetic workload in December 2023, showing requests queue while new instances start. The test excludes a real database, so it demonstrates only the request-serving part of a system.
Key points
- The handler is only an approximation: It waits for one second and computes prime numbers to represent backend wait plus CPU work; the presenters explicitly exclude database scaling 00:00–02:08(opens in a new tab).
- The dashboard visualizes a custom test harness: The service emits Pub/Sub signals when an instance is ready and about to stop, so a new instance does not appear until startup completes 02:08–03:14(opens in a new tab).
- Scaling takes time, and each instance serves multiple requests: When the load generator increases from 10 to 2,000 request loops, requests queue until Cloud Run adds capacity. The presenter explains that each container handles multiple requests concurrently, so the instance count need not match the number of clients 03:14–05:21(opens in a new tab).
- Successful requests do not establish acceptable latency: At 30,000 load-generating loops—not 30,000 end users—the display still reports 100% success. That shows the test requests completed, not that another application's response-time target would be met. The presenters warn that a real database must keep up independently and that running more instances costs money 05:21–06:26(opens in a new tab).
Full video: https://www.youtube.com/watch?v=8YtsV8BNTqY(opens in a new tab)