A missed disconnection notice isn’t a UX gap. It’s a customer who finds out their power is off when the lights don’t come on, not before.
I held design authority on this program from inside the team leading overall delivery and data integration, not from a seat on either team actually producing screens. Notifications were already shipping across six channels: an outage map, a desktop in-app banner on the customer portal, a separate mobile in-app banner inside the mobile app, email, SMS, and IVR. The split wasn’t arbitrary. The outage map only rendered on desktop, and email and SMS behaved differently enough by device that they needed separate channels to track. A seventh channel, push notifications, was just coming online with one feature live. Two vendors split the work, a dedicated design execution vendor and a platform vendor’s embedded designers who owned the design system. Neither reported to me. I still had to get both building to one evidence standard, and every notification that didn’t land meant a customer picking up the phone instead, at five to fifteen dollars a call.
ROLE
Design Lead and Design Process Owner, embedded in the lead delivery and data-integration team, directing design execution across two external vendor design teams with no direct reporting line
TIMELINE
2.5 years on account; primary focus on the portal and notification workstreams
TEAM
Design execution vendor: designers, business analysts; platform vendor: designers, developers, testers; test vendor: testers; scrum management vendor: scrum masters; my consulting team: developers, testers, business analysts on the core data feed
STAKEHOLDERS
Design execution vendor leadership, platform vendor design system owners, test vendor leadership, scrum management function, client product and process owners
TOOLS / SYSTEMS
Figma for mockups and prototypes, SAP-based core billing platform, legacy on-prem and AWS-migrated notification infrastructure
CONSTRAINTS
Design authority exercised without a direct reporting line into either execution design team, a concurrent data platform migration owned by my own team, regulatory notification mandates for safety and reliability reporting
No One Owned Whether It Landed
Two separate problems on this engagement turned out to be the same problem. One was a missing yardstick. The other was a missing process for making five vendor teams answer to it. Both came down to the same gap: notifications were shipping on schedule, but nobody had a shared way to say whether shipping on schedule meant anything to the person receiving them.
A Gap No One Had Closed Yet
Six channels were already shipping: an outage map limited to desktop, a desktop-only in-app banner on the customer portal, a separate mobile in-app banner, email, SMS, and IVR for accessibility. Push notifications were still coming online when I rolled off, with only one feature live. Every channel had a delivery team. None of them had a shared answer to the only question that mattered: did the notification actually reach someone in a state they could act on it?
Six channels live. One still coming online. Same threshold for all of them.
Product leadership had already defined one: a Notification Efficacy Ratio, channel views divided by confirmed deliveries, scoring every notification channel on a single, comparable scale. I worked with those product leads to put it to use, applying the ratio to design direction and content decisions across both design teams, even though neither one reported into my consulting team. A ratio above 0.50 meant the message was clear enough to act on without a follow-up call; below 0.49, the customer usually ended up on the phone anyway. That distinction carried a real cost, since call center interactions ran five to fifteen dollars each. A channel scoring below the line wasn’t a UX nitpick to me. It was an operating cost my content and channel decisions were directly accountable to.
A ratio above 0.50 meant the message was clear enough to act on without a follow-up call; below 0.49, the customer usually ended up on the phone anyway.
Getting Five Teams to Row Together
The harder problem wasn’t the measurement. It was getting five vendor teams, each running its own version of agile, to route design and research evidence through a shared process before a feature shipped.
I created the intake itself: cross-department accountability at the roadmap level, a shared Definition of Ready that made research findings part of sprint planning instead of an afterthought, and OKR readouts that let every team see its work against the same evidence standard, not just its own. Getting it live meant working closely with the product owner and scrum master to build buy-in, since none of it held up in front of the teams who’d actually run it without their support behind it.
231 in. 132 passed. The tiers are what kept it from becoming chaos.
The two teams actually producing screens had no reason to adopt any of it. Neither reported to me, and both already had their own definition of done. I got them to the table by making the evidence standard useful to them, not by asking them to comply with it. I showed the design execution team which patterns were generating the most retest failures before a feature even shipped. I showed the platform vendor’s designers where their own design system decisions were driving down the efficacy ratio on channels they owned. Neither wanted a mandate from a team they didn’t answer to. Both wanted fewer defects showing up in their own queue, and the evidence standard was the fastest way to get there.
None of that existed when I joined. I wasn’t installing a new tool on teams that answered to me. I was asking teams already under real delivery pressure, most of whom had no obligation to listen to me at all, to pause just long enough to align on a customer experience bar we hadn’t agreed on yet, without stalling a schedule five other stakeholders were counting on.
The intake held up under real volume. Across the channels already live, I ran the triage that adjudicated 231 open observations against that same shared evidence standard, closing 132 as passing retests. That’s not a proof-of-concept number. It’s what the process produced once five teams actually started running through it.
What the Intake Actually Proved
- The Notification Efficacy Ratio, applied to real channel and content decisions, gave every live channel a comparable score against a real cost driver: five to fifteen dollars per avoidable call.
- The intake process, cross-department accountability, a shared Definition of Ready, and OKR readouts tied to one evidence standard, is a real operational artifact that replaced ad hoc, team-by-team quality checks.
- 231 observations triaged against a five-tier prioritization model, with 132 closing as passing retests, is proof the intake operated under real production volume across all six live channels.
Where My Claim Stops
The Notification Efficacy Ratio itself belongs to product leadership. What I can stand behind is how I applied it to design and content decisions, not its invention. The intake process is mine, built and then validated through buy-in with the product owner and scrum master who had to run it day to day. Push notifications were still one category deep when I rolled off, so I’m not claiming a finished channel, only the four that were live and the process that held under all of them.
The Distinction I Look For Now
The correction here wasn’t proposing a metric or an intake process. It was recognizing that neither one meant anything until five teams who didn’t report to me agreed to run through it on their own. My first instinct, when I saw a channel underperforming its ratio, was to redesign the content myself and hand back a fix. What actually moved the ratio was slower: building enough shared evidence that the teams making channel and content decisions started reaching for the same standard without me in the room.
That’s the distinction I check for now before I propose a process. Am I building something I’ll have to keep enforcing, or something a team will keep using once I’m gone. That question matters most with the teams who were never going to answer to me in the first place.