Normally at these sorts of events protocol dictates that I have a sales department chaperone present at all times to make sure I use the correct fork for the shrimp cocktail, etc, and this was no exception.

My technical colleague and I riffed away, deftly interspersing witty-yet-topical infosec anecdotes with sales patter and doomsaying while we charmed the gathered CIOs with our analysis of the threat insiders posed to their organisations.

Now, you and I know that any sort of insider access is game-fuckin-over, but for the purposes of making the presentation more sales-friendly than a singe powerpoint slide saying, "you're all fucked, plz give us some money while you're still in business," we humoured them.

As I drew to a close, I looked around the audience, fruit platters on the table, a few shunned greasy pastries (they did have bacon, at least) and stewed coffee. I went for my concluding slide -- the last bit of useful information to be shared with the room before the sales drones would activate and attack.

When my sales-chaperone guy saw it he started twitching up the back -- it was off topic and he knows how I roll.

"People sometimes ask me 'Adam, if you were in a room with two dozen CIOs and you could tell them one thing, what would it be?'," I began.

They don't, by the way, but hey -- I get to use any sort of shabby segue I like when I'm clucking on my particular nest. So here's what I'd say.

"Security is hard. It's hard to buy and it's even harder to know if you've bought it. But you have to care, so you hire experts in this arcane field, just as you would any other technical niche. And if the expert says 'your stuff is broken,' then you know where you are. But if they say 'your stuff is great,' then you've got a problem. \t

"Is it really great, or are they awful? Did your expert have a bad day? Is he covering for the fact they just lost half their tech team to a competitor? Did they give you a junior guy, or a box ticker, or even worse, are they out to sell you kit? You don't know, because quality is opaque to someone who isn't an expert here.

"If you take one thing away from today, its that this stuff is hard, and the quality of my work is opaque to you. The only rational choice is not to trust me. So don't hire us. Hire Deloitte, or IBM or whoever you want. But next quarter, pick someone different. Rotate your audit providers. Use one now, another for the audit next quarter, maybe even two different parallel providers on a critical project. Pit us against each other and make us compete. Then at least you have relative quality metrics, which is more than you have at the moment."

Its true, you know -- I'd much rather be going into a pen test or an audit knowing that some high-priced big-5er has already been through with Nessus and Impact, picking off all the dumb shit that just wastes everyone's time to write up. There's no joy in savaging the poor fish in their nesting barrels. (Of course this assumes that the big-5er actually did spot all the low-hangers, which is, uhhh, Not My Experience.)

Yes, your domain controller is still vulnerable to MS06_040. No, you've never patched, your passwords are crap and you have 100,000 clear-text credit cards in /tmp on your RHEL3 box. I'd much rather write up a report about some point of entry that forced me to write a python script to exploit it -- at least then I get to use the Courier font non-ironically.

That's actually the best bit of advice I could ever give a CIO. Please, for the love of God, don't just pick one security supplier. Don't let them cut and paste you the same report every quarter. Get someone else in. Compare the reports, the findings, the quality of the write-ups and mitigation advice and ./sploit.py scripts attached.

Please. Please? I mean, we worked real hard writing them up for you. I know you've only got a 40-minute project meeting, and best to just glance at the summary-table and cross out everything rated less than 'ohmigod'. But please. Get someone else. Don't make me write the same report twice. Let me write a report that I know is going to make some big-5 infosec team look like the boxtickers they are. Please. Let me at them. You know I'm going to find their report on your \\\\fileserver anyway after I MS06_040 your win2k domain controller and have to resist the urge to open it up for the epic lulz that will be within.

Do it for the good of your shareholders. You owe it to them to get a second opinion on something as important as your security. Currently you may not know who is doing the quality work, but it wont take you long to find out -- all you have to do is shop around. You can't tell if you're getting quality, so make us all work to show you.

Hell, maybe we should give discounts to customers who provide us with their previous provider's reports after we've written ours. The lulz would be so worth it. I'll suggest it to salesguy. Well, I would if he wasn't too busy talking scoping with the douche bag who called me a wiseass.

Metlstorm is a New Zealand-based freelance security consultant. He's created several tools including Hai2IVR, Winlockpwn and SSH_Jack. He's also an organiser of the annual Kiwicon security conference in Wellington, New Zealand.