The question metrics didn't answer
The client offers a supply chain management SaaS that runs on two clouds, AWS and Azure. Infrastructure metrics said whether the servers were up, but didn't answer the question that mattered to the business: can a customer log in and use the tool right now?
It was my first project as a consultant, and the answer I found was to have someone using the platform all the time.
A robot that does what the customer does
In Elven Monitoring, I built a robot that repeats a real customer's path at regular intervals. It logs in, searches inside the platform and runs create, read, update and delete operations on records. If any step gets slow or fails, the robot raises an alert.
I set the same robot running in production and in staging. In staging, it catches regressions before they reach customers.
Two clouds, one place
I installed the Elven Monitoring agents on AWS and Azure so both clouds showed up in a single view. I centralized alerts in Slack, integrating CloudWatch and SNS, and used the opportunity to remove alerts that were just noise. I also ran the environment's load test.
What changed
The team now knows whether the platform works from the customer's point of view, not just whether the servers are up. Degradation in login, search or the main operations now raises an alert even when the infrastructure looks healthy. Staging problems started showing up before production, and alerts from both clouds ended up in a single channel, with less noise.
What I take from it
Monitoring infrastructure and monitoring the customer experience are different things. A robot that does what the customer does is the most direct way to measure whether the product is delivering value at that moment.