When I’m investigating a product issue, the evidence rarely lives in one place. There are Jira tickets, Slack conversations, client feedback, and dashboards. Each tells part of the story.
I built a Claude Code plugin to gather that information, identify patterns, and help prepare a case for prioritizing a fix.
I’d already experimented with gathering scattered information for my weekly update. But an issue summary raises a different question: how much confidence should I place in what it finds?
I built a few checks into the output to help me answer that.
Count the impact, not just the reports
Ten tickets don’t necessarily mean ten affected clients. They might describe repeated problems for one client, several transactions, or the same issue reported by different teams.
I wanted the summary to state what it was counting: distinct clients, affected orders, or teams encountering the problem. Those measures answer different questions.
I also added a flag for cases where one person or team contributed most of the reports. That doesn’t make the issue unimportant. It tells me to investigate before interpreting report volume as evidence of a widespread problem.
Check whether the sources overlap
Finding an issue in two systems can feel reassuring. But those systems may contain the same underlying evidence.
A client-feedback tool and a dashboard might count the same people. Adding their numbers would inflate the apparent impact. Treating them as independent confirmation could make the conclusion seem stronger than it is.
The instructions call out that possibility. I still need to understand what each source measures before using the numbers in a prioritization discussion.
Make uncertainty visible
An empty result can mean there’s nothing to find. It can also mean the search used the wrong project name, missed a channel, or couldn’t access a relevant source.
I made the output state its search scope and list limitations: the date range, skipped sources, and missing information.
I also asked it to label inferred root causes and impact estimates as hypotheses. A convincing explanation needs checking before I repeat it as fact.
These checks don’t establish that the summary is correct. They give me specific things to inspect.
Before I use the output to argue for a fix, I want to know who is affected, whether the evidence overlaps, and what the search left out. A concise summary is useful only if I can still see where its conclusions came from.