There’s a moment in every SaaS company’s life when one customer changes everything. Here it was a shift management product being rolled out globally for a single, very large enterprise. That meant far more users, many more locations and time zones, and an expectation that the product simply works at every hour of every day, because somewhere in the world it’s always the start of a shift.
What held up fine for a handful of customers wasn’t going to hold up for that.
Where the risk really was
The product itself was sound. The risk was in everything around it:
- Releases still had manual steps, so every deploy depended on whoever happened to be running it.
- Environments for different customers had drifted apart.
- Support was reactive. Tickets arrived and got handled, but nobody could say how fast, or whether things were getting better.
Shipping more features wouldn’t fix any of that. The priority was the delivery system itself.
Pipeline first
Every change, whether application or infrastructure, had to take the same automated path to production: tests and security scanning, an immutable build tagged with the commit, a deploy to staging with smoke tests, a manual approval gate, then a health-checked deploy to production with automatic rollback.
The important properties:
- Immutable artifacts. The exact image that passed tests in staging is the one that goes to production. Nothing is rebuilt.
- One path to production. No “quick fix” straight onto a server.
- Health-checked rollouts. Kubernetes only moves traffic to new pods once they pass readiness checks, and a failed rollout reverts automatically.
The pipelines ran on both GitHub and GitLab, because different products and customers lived in different places. The standards were the same on both.
Automation end to end
Beyond deploys, we automated the operational work too: infrastructure provisioning through Terraform, configuration through Helm values held in code, and monitoring set up as part of the deploy rather than added afterwards. A new service or environment arrived with its dashboards and alerts already in place.
Performance under real load
Global rollouts reveal bottlenecks that test environments never do. We profiled the application under realistic load, found the hot paths (usually database queries and chatty service-to-service calls) and fixed them alongside the developers. Infrastructure people and application developers looked at the same telemetry, so the problem was “our problem” and not “your code” versus “your servers.”
Support you can measure
The support overhaul mattered as much as the engineering. We rebuilt the process around KPIs that customers actually feel:
- Time to first response and time to resolution, broken down by severity.
- Ticket themes, so recurring issues became engineering work instead of repeat tickets.
- Clear escalation paths from first-line support through to the engineer on call.
To extend capacity, we also set up an offshore development team, with the same pipeline, standards and review process as everyone else. That consistency is what made it work.
What I’d tell another team
- Fix the delivery system before scaling features. Releases are where big rollouts break.
- Build once, promote the same artifact. Never rebuild for production.
- Measure support like a product. Response and resolution times, and the themes behind them.
- Use the same standards everywhere. That’s what lets a distributed team behave like one team.