Preliminary anti-virus testing comments
Published January 22nd, 2014 at 1:06 PM EST , modified January 25th, 2014 at 9:44 AM EST
I have been working on another round of testing of anti-virus apps. The last time I did this was one year ago, in January of 2013, so I decided it was probably time to repeat it. I have finished all the scanning, but still have a pile of work in front of me to get all the data tabulated. Still, this experience has been frustrating enough that I want to make some preliminary comments, before I have the full results in-hand.
I set out to test 24 different anti-virus apps, installing them each in a separate VM as I did last time. By the end of the process of downloading, installing and scanning with all these apps, I was thoroughly frustrated. The vendors of some of these “anti-virus” apps had best be glad that I’m not in their physical presence, as they would get a loud and lengthy dressing-down. At the moment, I’m ready to chew nails and spit bullets!
What is making me angriest at the moment are those apps that are essentially complete frauds. I get very upset when people are tricked into spending money on an app that they believe is protecting them, and it turns out that they would have been just as well protected if they had spent their money on something like a solitaire game instead! There’s a class of malware that behaves this way, after all…
As last time, MacScan was an utter failure, detecting only four of the nearly two hundred samples I threw at it. Two others, which I did not test last time, performed abysmally as well. Magician, an app that seems to be an imitation of the infamous MacKeeper, detected only one antique sample, while MaxSecureAntivirus detected absolutely none of them! (Links to these apps are purposefully not provided.) These apps live up to the prejudice that many people have against anti-virus apps, and contaminate the entire anti-virus community with their uselessness.
Other apps suffered from different problems. Bitdefender (the version from the Bitdefender web site, not the one from the App Store) would not start a scan at all… the buttons simply did not respond to clicks. Norton absolutely destroyed the performance of the system it was running in, and wouldn’t scan while disconnected from the network. (It kept insisting that it must update its definitions before scanning, despite the fact that, as far as the system it was running it was concerned, it had just been updated manually only minutes before.) Thus, I was unable to complete testing of either of these programs.
Less serious, but still a source of enormous frustration for me, is the issue of user interface. Some of these apps have truly atrocious user interfaces. Only six of them actually allowed me to save data on the details of what was detected… the others all simply displayed lists of infected files. In some cases, those lists showed woefully inadequate information, refusing to even give information on where the files resided or what malware they were identified as! The only way to save these bare dregs of information was through a series of screenshots (snap a picture, scroll down, snap a picture, scroll down, and so on). I am truly dreading the task of sifting through and organizing such low-quality data!
In all, this process has left an extremely bad taste in my mouth. I’m sure that some of these apps did a superb job of detecting the malware I threw at them, but until the data is tabulated, that’s impossible to see. All I’m really seeing at the moment is the bad and the ugly. The good – what little I suspect there will be of it – will come in the official results within a few days or so.
January 25, 2014 @ 9:39 am EST: Minor update, MacScan actually detected four of the samples, not two. Two of the samples it missed were inadvertently included as compressed archives, which caused them to be missed. This has been corrected, and those items re-scanned with all the anti-virus apps. Magician and MaxSecureAntivirus detection counts did not change.
Results will probably be posted on Monday.