Market research is bunkum

HOW SERIOUSLY do you take IT market research? How come they get it so spectacularly wrong? How come mainframes never died? And Ethernet? Where is the billion dollar ASP market that Durlacher predicted for the UK, back in 1999? And how come I still had that report down the back of the cupboard?

Come to think of it, whatever happened to Durlacher?

Millions of dollars are spent on market research. Entire business strategies are based on the intelligence bound up in those heavyweight reports. (Mind you, most people only ever read the executive summary. The rest of it might as well be padded out with Chinese phone directories) But how accurate are they? How thorough is the research?

At the bottom end of the market, it's pretty ropey. Surely everyone knows that. The beauty of research, for a hack, is that if you make it up, no-one can disprove it. Because they don't know the answers themselves. Don't ask how I know but trust me, it goes on. They'll only ever query your invoice if your charts aren't pretty, or your presentation doesn't have enough cliches.

A woman at a large British telecoms company once asked, apropos the commissioning of a research project: “Before I go in this meeting can you tell me who our rivals are?”

But what about at the top end of the market? Surely the elite intelligence providers, the Gartners and the Butlers and the IDCs have the resources to do a thorough job?

Worrying news comes from a security source. The Gartner Group is arguably the most influential of all the research companies. It produces a Magic Quadrant report, which sums up the state of every important market in IT. An IT manufacturer's fate is influenced by their placing in the Magic Quadrant. When the report finally comes through, everyone nervously skims through to see if they've been categorised as a leader, a challenger, a visionary or (ahem) a niche player.

“If vendors make it into the upper quadrant, you can guarantee they will get a press release out about it before you can draw breath,” said one IT PR agency boss.

So what happened with the Network Intrusion Prevention Magic Quadrant report that Gartner produced last December. No mention of Countersnipe, and yet they've won awards, they're in the Ernst & Young Security Solution Directory and industry mags like SC Magazine have endorsed them. Why the exclusion?

They need to contact Vendor Relations, according to Greg Young, research vice president for Network Security at Gartner, in an email to Countersnipe.

So lets get this straight. Unless you hand your forms in, you're not on the radar? Surely the purpose of the world's top market research is to know the market. Do you rely on people mailing your intelligence to you?

“No we do not Nick. We look at all the players in the market,” insists Laurence Goasduff, senior PR manager for Gartner.

So why were they left out? The mystery deepens. µ