The problem with website threat scanning
Published April 19th, 2013 at 4:38 PM EST , modified April 19th, 2013 at 4:58 PM EST
I’ve been blacklisted. Well, not me, per se, but The Safe Mac website has been, by a number of website threat scanning services. How that happened is a bit of a disturbing story, and I know that I, for one, will never look at such services in quite the same way again. We’ve had discussions of trust before. This time, I’m going to ask you to aim a critical eye at the services you may be counting on to keep you from visiting malicious websites.
It all started a little more than a month ago, when a reader alerted me to the fact that McAfee was blocking access to my site, calling it a malware site, and that WebSense was also categorizing it as malicious. VirusTotal can check a site against number of different threat engines, and sure enough, the VirusTotal page for The Safe Mac was showing that these engines were blacklisting me.
I sent off a few e-mails, saw those errors get corrected, and thought it was all over. Unfortunately, I was not to be so lucky. Last week, I was alerted to the fact that McAfee was still blocking my site, by someone who was understandably suspicious of me and the content on my site. Sure enough, McAfee’s site still had me flagged. When I checked VirusTotal again, now Fortinet had also flagged my site as malicious. I did a bit more scrambling around, sending e-mail messages and submitting web forms. From there, I kept a daily watch on VirusTotal, and a game much like Whack-A-Mole ensued. Each day, I’d be clean again with some site that had considered me malicious the day before, but a new site would pop up that said I was malicious. And all the while, although BitDefender called me clean, the Additional Information link on the VirusTotal page revealed that BitDefender claimed that my site “was seen to host badware at some point in time.”
All of this was extremely puzzling to me. Although one should never have the hubris to say one’s site is invulnerable, I’ve got my site locked down pretty tightly. I keep the blog software updated, use a strong password with a non-default administrative username and have it configured to block repeat failed login attempts. I’ve been over the site with a fine-toothed comb, and cannot find any evidence that it is, or has ever been, hacked. Site scanners that download files from a site and scan them, like Quttera, back me up on that. So, why was I being blacklisted?
Eventually I hit a company at which I had a contact, and so I asked him what was going on. The response I got back revealed that there is a channel of communication between many security companies, through which they share not only malware samples, but also URLs. Apparently, someone (he didn’t say who) submitted my site as potentially malicious on that mailing list. In addition to that, he told me:
The level of suspiciousness depends on many factors, the domain age being one of them, the popularity of the site, how many sources of suspect URLs we’ve seen it from and trustworthiness of those sources.
So, the newness of the URL was also working against me. (Although the site has been around for a while, at www.reedcorner.net, I only just switched over to thesafemac.com two months ago.) He was also able to confirm for me that the issue was a false positive.
In the end, all false positives have been corrected except one. (CleanMX.com is a German site, and Chrome has failed completely in its efforts to translate it, so I’ve had a hard time trying to find contact information. Perhaps a German-speaking reader could help with that? 🙂 ) However, my site’s reputation will probably never be as spotless as I’d like it to be. Ironically, reedcorner.net has a squeaky clean reputation, despite the fact that the content is exactly the same. (Both thesafemac.com and reed corner.net resolve to exactly the same IP address, in fact.)
The CleanMX issue is, unfortunately, not particularly unique, in a way. Although the other sites I had to deal with were written in English, that didn’t make it much easier to find a way to alert them to a problem with their results. In some cases, I searched for half an hour before finding a way to do so. This doesn’t make it easy for a site owner to address these kinds of false positives.
What lesson can we take away from all this? Think back to trust. Remember, trust has to be earned. When it comes to automated site scanning and blocking technology, you simply cannot count on being able to believe what one site tells you. As with all such attempts to hand off decision making to some kind of software, it can be useful in many cases, but is prone to error. If a particular tool tells you that a site is malicious, don’t take that for granted. Do some investigation on your own.
There are many different ways of checking a site’s reputation. If one tool gives you an unexpected result, check with another. Check with a site that bases their results on user feedback, such as Web of Trust. (Keep in mind that any user-based tool like this can be similarly prone to error, thanks to things like users “gaming” the system or posting negative results to intentionally harm a competing site’s reputation.) Check with Google’s Safe Browsing service, which is used automatically by Safari, Firefox and Chrome. Scan a site with something like Quttera, which actually scans the pages on your site for exploits right then and there. Use VirusTotal to scan a site and get the results of many different engines at one time.
And if you happen to notice that a site you think should be legitimate is being blocked by a few of these engines, consider alerting the site owner to the problem, so that it can be fixed.