Looking for a crisis agency? Then you’ve probably seen some pretty compelling case studies—big name brands, major wins and encouraging outcomes. All bright, shiny and safe.

But, please know, it’s all baloney.

Case studies are a legitimate marketing tool for communication agencies for lots of reasons but they have no place in demonstrating crisis management capabilities.

First, a key trait of any competent crisis counselor is discretion. You wouldn’t want someone entrusted with handling your most sensitive information discussing it loudly in a crowded restaurant. So why choose a counselor who leverages your crisis situation as their marketing tool?

Even when a client name is redacted, it only takes a quick Google search to identify specifics of the situation. And, more to the point, if a crisis counselor sells you on their skills using a previous client experience, what’s to say your situation won’t be front and center in their next new business pitch? It’s just bad practice.

Second, crisis outcomes aren’t linear. When looking at a case study for a marketing program, you can be reasonably confident that it was executed in an environment of known information. Plenty of data likely exists for previous results, expected outcomes, consumer behavior, target demographics, etc. and it’s all factored into the creation of a strategy.

Therefore, when you enact a program, you can track it through to meaningful and measurable outcomes, delivering a case study that means something.

The very nature of a crisis is that it exists—at least partially—in an environment of unknown information. There aren’t benchmarks for how the previous fifty situations like this were handled by the company or it wouldn’t be a crisis. Therefore, tracking outcomes to specific actions of a crisis counselor is immediately fraught.

Think about it like this: If a crisis agency is hired to address widespread media coverage about a negative event and the media coverage decreases by 70 percent in the four months after the event is that because of the skill of the crisis agency? Or did the media coverage just run its course? Did a larger and more newsworthy event supplant the event news? And why is 70 percent good—could it have been 80 or 90 percent with a more competent agency?

You get the point. There isn’t—there can’t be—a consistent metric. Any success is contextual and subject to interpretation by those defining it.

Third, crisis isn’t just anecdotal outcomes. It’s high pressure, late night calls, long strategy sessions, lots of emotions, constant worrying, legal implications, what ifs and whys, blame and vindication. Your counselor isn’t a vendor or partner. This is not transactional consulting. Your crisis is their crisis.

In a case study, how do you know the agency will lose sleep thinking and planning and running down every possible path? How do you know when they show up at your house to stave off media in your driveway, that they’ll also think about repercussions to your children? What tells you that they will put their vacation time, their weekends—even their reputation—on the line to protect yours? You won’t find this in a case study.

So, instead, choose your counselor based on these criteria:

• Ask about knowledge. Don’t ask whether an agency has worked on a DOJ investigation, but what have they learned from working on DOJ investigations.

• Ask about failure. Crisis is high stakes and everyone in the business has something they wish had gone differently. An agency’s willingness to talk about that—and the learnings from it—are key.

• Get legal counsel involved. Crisis counsel must align with legal strategy. If your lawyer and your crisis team are in conflict, it translates to higher billings and headaches all the way around.

• Ask for confidential references. While case studies are a hat trick, the personal and specific experience of a real person the agency has worked with is gold. Every agency should have a few of these individuals who are willing to speak to their expertise and recent support.

• Vet the team. Don’t assume that the people in the room during the sales process are the crisis team with which you’ll work. Ask: Whose mobile phone can I call at 2 a.m. Sunday morning? Who will be overseeing the work of the team? How many people will work on my business? In a crisis, smaller teams are generally more nimble, more connected and more effective. If a lot of people are involved, ask why.

• Go with your gut. If you don’t like your crisis counselor as a person, it’s going to make a bad situation worse. Find someone who you want to have a beer with. You may end up having a few.