Over the last couple of weeks you may have spotted some news stories floating about claiming cybercrime costs society US$388bn annually, with Australia alone suffering A$4.6bn in yearly losses.

If the numbers are to be believed, these reports say, that means cybercrime costs us nearly as much as the global trade in illicit drugs. It's a sensational claim and makes an awesome headline, but any way you slice or dice the numbers they just simply don't stack up.

What's more frustrating is the most cursory analysis of these figures is enough to show they're fanciful. Yet lazy media outlets, spurred on by reports carried by the sycophantic technology media, happily parroted these claims as fact without doing the most basic checking.

Institutional fraud measurements here in Australia blow the claimed numbers out of the water immediately. Direct losses from all personal fraud in Australia, online and offline, is estimated at A$1bn a year. So where did the claim involving a A$4.6bn annual impact come from?

The numbers were dug out of a survey report released by Norton, the consumer division of Symantec, a company that makes computer security software.

Norton engaged an online survey company, Strategy One, to quiz around 20,000 people in 24 countries about their experience with cybercrime. It asked respondents to nominate both direct and indirect losses they experienced as a result of online crime.

Based on the responses Strategy One received, direct annual consumer losses extrapolate to USD$114bn a year. That includes losses that consumers experienced that were reimbursed, as is the case with the vast majority of credit card fraud.

That figure seems high enough, but where things get comical is when Norton throws indirect losses into the mix.

When asked to nominate how much these cybercrime experiences were worth in terms of "time lost," Strategy One extrapolated a figure of USD$274bn.

Now, here's where it gets into Twilight Zone territory. According to Norton, United Nations figures estimate the illicit trade in heroin, cocaine and marijuana is worth US$288bn a year. The total illicit drugs trade is worth, apparently, US$411bn per annum.

But if you add the USD$114bn figure for direct cybercrime losses to the USD$274bn "time lost" figure, you wind up with a total just under the figure for drug sales (USD$402bn).

The result is the claim that "cybercrime is... approaching the value of all global drug trafficking". You see what they did there? Voila! Instant headlines!

Norton is actually equating a fictional "time lost dollars" -- that never actually existed -- with actual dollars spent on marijuana. You'd think the marketroids were smoking the green stuff themselves when they came up with that comparison.

They must be regular users, too, because Norton put out a press release on September 11, 2009, claiming cybercrime actually eclipsed the global drug trade. The press release was titled: "Cyber Crime has Surpassed Illegal Drug Trafficking as a Criminal Moneymaker; 1 in 5 will be a victim".

As a moneymaker? Really? That's not even what Norton is claiming now!

If that wasn't loose enough, it seems obvious that if we included "time loss" figures stemming from the illicit drug trade the comparison would be blown apart immediately.

Just think of the harm being inflicted on Mexico right now by the drug cartels, not to mention narco-related drama in countries like Afghanistan. Then there's the money spent on the "War on Drugs," keeping drug dealers in prison and the productive capacity society loses to all those dope-smoking young males glued to their PlayStation 3s.

While we're doing things the Norton way, why don't we include lost income that unemployed heroin users could be making if they straighten up? It'll be a completely meaningless number, but as long as it's big, apparently, it gets a run.

Norton's definition of cybercrime is also fairly liberal. It defines "online harassment" as a cybercrime, along with being "approached online by a sexual predators". Online credit card fraud is defined as "someone made an unauthorized use of my credit/debit card, or card number, to fraudulently obtain my money or property".

The word "online" doesn't even appear in the definition. This question, as written, will catch all credit card fraud.

It gets better. In June two researchers from Microsoft, Dinei Floręncio and Cormac Herley, wrote a brilliant paper titled "Sex, Lies and Cyber Crime Surveys".

You can read it here, but the general thrust of the paper is self-nominated loss figures are notoriously unreliable.

Floręncio and Herley say surveys of sexual behaviour demonstrate people lie when asked to nominate how many sexual partners they've had. In a random sample, it stands that the average number of sexual partners for both men and women would be the same. But they're not.

Women generally tell the truth, albeit shaving the number slightly. Men also generally tell the truth. Unfortunately, some men, the research says, massively overstate their notches-on-the-bedpost quota, and that throws the reliability of the survey more or less out the window. Floręncio and Herley argue overstated loss figures cause the same problems in cybercrime surveys.

Cybercrime surveys that rely on self-nominated losses require multi-layer sampling and a sample size of up to several million respondents in order to be regarded as accurate, the Microsoft paper claims.

The problem boils down to concentration. Overstated losses can massively skew data sets. In a survey based on 1,000 respondents, Microsoft says, a claim of a single loss of $50,000 would translate to a loss figure of $10bn over the whole population. A nominated loss of $7,500 translates to $1.5bn.

So, they say, we should "discard" any survey that doesn't disclose both the median and mean figures for responses. These two figures give the reader an idea of how concentrated the responses are.

The Norton survey does not disclose these figures.

There are places we can look to find more trustworthy sources of information relating to crime and fraud. The Australian Bureau of Statistics, for example, estimates all personal fraud, both online and offline, costs Australians AUD$1bn a year (2007).

The Australian Payments Clearing Association (APCA) calculated Australian Issued Payment Instrument (credit store and debit cards, cheques etc) at around A$210m for 2010.

Sure, a body representing financial institutions might have an interest in understating these figures, but Norton's report claims direct cybercrime losses in Australia are A$1.8bn a year (roughly A$300 per household) with a further A$2.8bn in indirect losses!

Put simply, Norton's figures just do not look credible next to institutional measurements, and Microsoft's research tells us these types of cybercrime surveys are unreliable.

For its part, Norton says the discrepancies can be explained because much cybercrime goes unreported. "We are confident that the Norton Cybercrime Report is a valid representation of the current state of consumer cybercrime," the company wrote in response to questions.

It's understandable that a company with a vested interest in overstating a problem will release this sort of marketing material. It's another thing for the media to just run with it without questioning the numbers.