The idea was simple. We'd install a bunch of anti-virus products and see who could modify existing viruses to sneak them past detection engines. There'd be beer and banter, a fun afternoon. It wasn't really a scientific contest -- most of the functionality of the scanners was actually turned off. We'd only test the CLI-based signature and heuristic components of the suites.
I'm one of those poor, poor souls who's been forced to repeatedly deploy appalling, sub-standard, anti-virus shit in enterprise environments over the last few years. Sick of trying to fight a virtual wildfire armed only with the IT equivalent of a warm leaf of lettuce, my friend Rich and I decided to stage RaceToZero as a form of protest.
We'd show the world just how awful antivirus software had become. The world would finally understand our pain.
When we announced the contest, some AV commentators and journalists went virtually lost their minds. The first RaceToZero contest, held at DEFCON XVI in Las Vegas last year, was indeed a tad on the controversial side.
Some commentators seemingly expected the headless horseman of the Apocalypse to come riding through the casino when the contest began. Kasperky antivirus founder and CEO Eugene Kaspersky actually compared the Race To Zero with bank robbery and the distribution of narcotics to children. In the minds of some, we were showing the bad guys how to do stuff they couldn't have learned on their own.
Others were a tad friendlier. They saw RaceToZero for what it was -- a bit of fun designed to demonstrate the ineffectiveness of signature-based antivirus technology as a sole method of defence against modern threats.
Either way, we didn't expect the publicity we got last year. In the words of George Carlin, the whole thing turned into a "huge, prick-waving dick fight". A circus, if you will.
So we're doing it again.
To live up to our critics we had planned a HERF gun making contest (hai2EugeneK) but decided on slipping viruses past AV products again instead. The friendly team from OffensiveComputing.net provided the samples we used last year and this year will be taking over the running of the competition.
RaceToZero is still my baby, but I'm happy to send it off to temporary but loving foster care.
OffensiveComputing.net's extensive knowledge of malware, reverse engineering and all things anti* will definitely lift the contest to another level. It won't be as half-assed as last year, (it's more likely to be fully-assed) and may actually produce some results that can be seen as useful benchmarking for endpoint security products.
The Anti-Malware Testing Standards Organization (AMTSO) has published guidelines for dynamic testing and RaceToZero will stick to them.
That means getting all fancy and scientific. As much fun as the last contest was, we didn't really prove much. This time we're trying to create a methodology that might actually tell the people responsible for buying endpoint security something useful, like which products did better.
That's right, vendors, you really should be scared now. We're going to empirically show the world how useless you are, instead of just heavily implying it.
While this balanced, unbiased testing of behavioural AV engines is happening, there will be a live scoreboard so that contestants and spectators alike can see how well the teams are doing and how effective each engine is at detecting the threats.
Another upgrade to the contest is automated unpacking and analysis of samples submitted by contestants, which will be validated against the contest guidelines.
Over the coming weeks more information will become available on the RaceToZero Website and the DEFCON Forums, we look forward to seeing all past and future contestants in Vegas again this year!
bogan \\m/
Bogan is security engineer and researcher from .nz. He is also instrumental in the organisation of Kiwicon, New Zealand's real-deal security conference. In his spare time bogan likes cooking, wearing black and admiring a good burnout.
