The Rise of the Rogue AV Testers

By Costin RaiuRecently, I was sitting around with a number of colleagues from Kaspersky Lab, discussing everybody’s favorite subject: the state of anti-virus testing these days. During the talks, somebody brought up the name of a new, obscure testing organization in the Far East. Nobody else had ever heard of them and so my colleague Aleks Gostev jokingly called them a “rogue Andreas Marx.”

By Costin Raiu

Recently, I was sitting around with a number of colleagues from Kaspersky Lab, discussing everybody’s favorite subject: the state of anti-virus testing these days. During the talks, somebody brought up the name of a new, obscure testing organization in the Far East. Nobody else had ever heard of them and so my colleague Aleks Gostev jokingly called them a “rogue Andreas Marx.”

It then occurred to us that some of these new testing labs that have recently appeared mimic the tactics of Rogue AV products. What exactly do I mean? Well, as we know the rogue AV business model is based on selling a false sense of security; we professionals know it is fake, but the victims don’t. People buy a Rogue AV hoping it will solve their security problems, but the products don’t do anything at best, and at worst, install additional malware.

Rogue AV Testers are somehow similar in behavior. In their case, the business model is no longer based on a false sense of security but instead, on a false sense of insecurity. So, how do they operate? Well, it seems to start with a number of tests which look legitimate, and mimic real world conditions. Then, the tests slowly become more “complicated” and security products do worse and worse. Sometimes, the product that did best in the previous test suddenly becomes the worst in the group. In other cases, all products fail miserably. Finally, the main idea emerges: that all security products are bad and utterly useless. Hence, the false sense of insecurity is promoted through the tests: you are insecure, your money was misspent – beware! 

Going further, the rogue AV testers use various techniques such as not disclosing product names in published test results and attempting to sell theses results for exorbitant fees.
Here are some characteristics we identified as being specific to rogue AV testers, that can help you spot them:

  • They are not affiliated with any serious testing organization, such as AMTSO. Sometimes, the Rogue AV Testers could also show fake affiliations or even falsely display (say) the AMTSO logo on their website, in order to remove suspicion and doubt.
  • They publish free public reports, but charge money for the “full” reports. In general, the public reports should look as bad as possible for all the tested products, to maximize the profits from selling the full reports.
  • The public reports are full of charts that look complicated and intelligent, but sometimes reveal amusing mistakes.
  • They claim all AV (or security) products are useless. This is the foundation stone of any business based on the “false sense of insecurity”.
  • They charge for samples and methodology, usually very large sums of money, to make sure the flawed methodology and samples cannot be reviewed externally.Reputable testers will make samples and methodology available for free to the developers of the products they test, instead charge for the rights to publish the results in magazines or for the permission to use the results in marketing materials. Charging money for samples is a clear indication that something wrong is going on.

There are other characteristics, but I think everybody gets the point.

Just as Rogue AV (scareware) products exploded and became one of the most profitable categories of crimeware, I suspect Rogue AV testers will follow. In the process, they will also become an extremely profitable category. And of course, the worst of all, they will provide a strong, negative value to the entire IT security industry.


So, if you are trying to compare security solutions, I recommend sticking to established testing organizations such as Virus Bulletin, AV-TEST.ORG and AV-COMPARATIVES or reputable magazines, with a good history behind them. If in doubt, ask for AMTSO affiliations and finally, do not forget about the list of hints that can help you spot Rogue AV Testing behavior.

Do not become a victim of the Rogue AV Testers!

* Costin Raiu is the Director of Kaspersky Lab’s Global Research & Analysis Team (GReAT) . This essay was first published in the current issue of Virus Bulletin magazine.   

Suggested articles


  • Alex on

    Too bad there are no laws to prevent these unethical practices. Or are there any?

  • MARK on

    As usual Theatpost is ahead of the game. You guys make me look so good at the bar.....:)!

    I get invitations to ladies homes to "FIX" there computers, and occasionally other aspects of there life which are in need of "repair" or "adjustment". Mind you it's only on occasion, but hey Chicks need there computers working too and there only to glad to repay the favor.

    To date I have turned countless people on to Kapersky products. No point in using anything else.

    I also Use SPYHUNTER 4 (Enigma)and MAX REGISTRY CLEANER (Max Secure Products)

    There are still elements of unwanted spyware/malware that get attached , but now i have tools to deal with them quickly and effectively.

    I read threat post articles frequently. Thank you for the efforts you make.

  • Jack Daniel on

    So we should blindly trust the puppets of the AV industry?  Sorry Mr. Raiu, the "established" AV testing bodies test to a set of guidelines you know are arbitrarily limited and not fully relevant to the modern threat landscape.

    I do believe that some of the AV tests are borderline "rogue", but I also have seen enough from others that I have a lot more faith in them than in the idea of letting the industry grade itself.

  • Henrik on

    You make some interesting points Mr. Raiu, but I have to question your motives. I just looked up AMTSO and it is a bunch of vendors. Why is it credible? My experience is that when an industry cooperates to convince the public that it knows best, and campaigns to discredit third party oversight, it is usually hiding something. Tobacco and Banking come to mind. So what are you hiding?
  • Craig Kensek on

    It appears that the 1800's were for snake oil salesmen, the 1900's, for aluminum siding vendors, and now in the 21st century rogue AV testers. There are too many pop-up scams out there today selling "av software", as well.  People are welcome to comment to the AMTSO on the guidelines/recommendations that they publish. Other companies are welcome to join as well. Are guidelines suggested by surgeons suspect because it's surgeons who perform the operations? Henrik ignores the whole transparency issue. That (and sorry if the phrase makes people's eyes glaze over) - statistical validity.

  • Anonymous on

    While I do understand your sentiments, this pointing of fingers on who is credible and who is not kinda turns me off.

    Credibility should be proven, not stated.

  • Anonymous on

    Most organizations of this sort are vendor driven--and have a stake in keeping the reputation intact. Who else would? The government? They are usually so far behind trends in almost any area that their actions are laughable, if not harmful. Communications standards based on wired telecom????

    Associations that self-police are aware that consumer dissatisfaction is generally the driving force for imposed regulation, so it's in their interest to keep their peers in line.

  • Anonymous on

    people should be able to look after themselves i go to siteavisor and look at comments read blogs etc google i use malwarebytes for malware and microsoft secuirty essendals for viruses great combo and doesnt sow your computer down when you get one of these popups press controll alt delete and delete your internet explorer and run ccleaner (crap cleaner)

    the internet is getting worser and worser what happend to the old days where there was bank robbers and stuff its all this this is bad they cant be caught either also half of the world has no brain so they cant do nothing lol

    avoid pirated software depends

    avoid dangerous sites like piratebay or video sites like watchxonline if you view these sign in it helps

    do not be tricked the internet is a cruel thing

    i think why nohing being done about problems like this is because i dont think the police understand and stuff like that because there idiots

    also fag indian rang me up claiming to be microsoft the other day lol wanted me to download some antivirus and delete my apps i wish i could of recorded lol it would of been good fun pretending to be a dick head and knowing nothing about computers pretending that your doing it

Subscribe to our newsletter, Threatpost Today!

Get the latest breaking news delivered daily to your inbox.