What the team lost
New Relic was switched off because of cost, and with it went the screens the development team used to find bottlenecks: endpoints and database operations ranked by total time consumed, average response time and throughput.
SigNoz was already in production and receiving traces, but its native interface only sorted by volume. For the team, that was a step back. The most expensive bottlenecks are often endpoints with little traffic and a lot of time spent, and those vanish in a ranking by volume.
Prove before building
Before creating a single panel, I went through a read-only analysis phase: an inventory of the environment, the OpenTelemetry collector configuration, the instrumentation and the trace database schema, 12 checks in total.
With manual queries, I compared the numbers against the old New Relic screens. The top 5 endpoints came out identical and in the same order, with 27.19% of the time in SigNoz against 26.24% in New Relic for the first one. I confirmed there was no sampling anywhere in the pipeline, so the numbers were exact. And no entry span was missing its route, which was better than New Relic, which lumped part of the traffic under a generic label. I also measured actual trace retention, which set the honest maximum window for the panels.
The source that seemed obvious
The analysis changed the design. The natural path would have been to use the ready-made APM metrics, but they were broken (more on that below), so the panels query the traces directly.
In the database view, the instrumentation agent named queries only by database name: over a million calls and 53% of database time fell under a single label, with no table. I extracted the table from the query text, and the view started matching New Relic's. What couldn't be identified went into its own panel, declared, instead of hidden.
Delivering panel by panel
I built and published through the API a dashboard with nine panels covering the six views requested, versioned as JSON. Every panel was validated end to end against the query API before publishing. Rollback was simply deleting the dashboard, with no side effects: no application, collector or retention setting was changed.
What nobody had asked for
The APM metrics had collapsed. The processor that generates per-operation latency and rate metrics has a limit of 2,048 operations per service, and the ORM's internal spans, with about 900 different names, blew through it. The result: 100% of the series had lost the endpoint dimension. I measured that filtering those spans out left 44% headroom within the current limit, and opened the fix with validation in staging before production.
I also found instrumentation defects that distorted the numbers, and neutralized them with filters in the panels before handing them to the development team. Four services had duplicated entry spans, and the native page showed double the real throughput. A database access library produced a span alongside the driver's, counting database time twice. And waiting for a pool connection showed up at the top of "database time" without being a query. Finally, a blind spot: the cache had no instrumentation at all, while New Relic showed thousands of calls per minute to it.
What changed
The six APM views exist again, in production, with proven parity and without the cost of the tool that was switched off. The dashboard quickly made the case for the request: endpoints with very little traffic consuming a good share of the application's time. And the team got numbers more reliable than the ones it had before.
What I take from it
In an observability migration, the first deliverable is the proof of parity, not the dashboard. If the new numbers don't match the old ones, the team stops trusting the tool, and a pretty panel with wrong numbers is worse than none. Measuring before building changed the design itself: the source that seemed obvious had been broken for days.