The Inquirer-Home

Hunting metamorphic viruses

Defcon 2006 Look for dissimilarity
Thu Aug 10 2006, 09:55
ONE OF THE more interesting talks at Defcon 2006 was on hunting metamorphic viruses and trojans. The short summary is that the older engines are fairly readily detectable, but the new ones are a true bitch to find. That said, if you are careful, it can be done.

The speakers took four of the most common virus generation engines, NGVCK, G2, VLC32 and MPCGEN and ran them through the ringer. A handful of viruses were generated from each engine, and compared to each other and some closely functional low level code for similarity.

The results were about what you would expect. The normal programs scored about 60 per cent similarity, the best of the virus generators, NGVCK, scored 10%. The other virus kits were somewhere in the middle. While the viruses were somewhat similar to their brethren, one interesting thing is that they were quite different from the other kits.

The same programs then had freshly generated viruses run through a gauntlet of eTrust, Avast and AVG, with mixed results. G2, VLC32 and MPCGEN all had some output detected, but NGVCK, the newest of the four had none detected by any AV program. Not good.

The researchers then looked at the viruses using a technique from generics called Hidden Markov Models (HMM), basically looking at the structure and behavior rather than the things that make it up. If you think about it, there has to be a given set of functions for the program to work, and this is analogous to the 3D structure of a protein. You can look at the DNA or the physical structure, and the same techniques work for computer viruses.

With a bit of training, the HMM detection method got most of the viruses while never giving off a false positive. To make things better, they also nabbed the occasional virus that was not part of the training, so the techniques have broader applicability than was initially intended.

This detection method relies on dissimilarity rather than similarity. If something is random, it won't look like the normal programs. The viruses will be all over the map rather than clustered around the points where safe programs lie because they don't do the same things.

While this technique may have promise, the fact that the current tools caught none of the output from the latest kit is disturbing. The black hats appear to be winning the battle, never a good thing, but the war is far from over. It is only going to get more interesting from here. µ


Share this:

blog comments powered by Disqus
Subscribe to INQ newsletters

Sign up for INQbot – a weekly roundup of the best from the INQ

INQ Poll

Heartbleed bug discovered in OpenSSL

Have you reacted to Heartbleed?