HOME     SITEMAP     RSS     TWITTER     EMAIL    
Search:   

FollowSteph Follow Steph as he posts Blog Blazer Friday
 

The Implications of Your Decisions

Right and Wrong Decisions

Yesterday I came across an article on The Register about the anti-virus software AVG. To quote The Register:

In late February, AVG paired its updated anti-virus engine with a real-time malware scanner that vets search engine results before you click on them. If you search Google, for instance, this LinkScanner automatically visits each address that turns up on Google’s results page.

This has two very large implications, especially when you consider that more than 20 million people have downloaded AVG. And as you can imagine, it has to do with the bandwidth.

Let’s take an example of what happens when you go to Google and search for “latest movies”. In the past you would go to Google’s search page, enter in your search, and get some results. Then when you clicked on one of those search results you would go to that page. Simple, and it’s what you would expect.

However if you have the latest version of AVG installed something else happens. You start the same way, go to Google, enter in your search term (“latest movies”) and click on the Search button. However here is where things change for the worse. AVG looks at the search results and behind the scenes starts to download each and every search result webpage. This is without you having to view or visit the webpage, it’s all happening behind the scenes.

Why is this bad? Because most ISPs (Internet Service Providers) limit how much bandwidth you can use per month. It might be very high for some people, but now imagine that for every search you do, you visit 10 webpages (the default number of search results on most search engines). Even if you don’t look at any of the search results. You’ve basically increased your bandwidth consumption by ten times for every Google search!!! And not just Google, for all search engines, including Yahoo, MSN, etc. Your bandwidth usage has just significantly increased, by multiples.

But it gets worse. What if the webpages AVG decides to check out behind the scenes aren’t just simple pages but rather webpages rich in media that include videos, images, and so on? You’re bandwidth will be consumed in no time. And what if you have more than one computer on your network? What if you have 2-3 computers in your home? If everyone is searching at the same time you can imagine that your network will get slower because everyone is trying to load multiple webpages at the same time. And don’t even get me started on corporate networks. I can’t imagine the increased load on a corporate network with 10-1000 users!!! For that reason alone I suspect corporations will stop using AVG, the bandwidth usage is just too expensive.

And right now this is only officially happening for search engine results, but what if one day they decide to continue down this path and do this for every webpage? I’d hate to see the bandwidth usage on that decision! Most websites have more than 10 links on them! The front page of this blog probably has closer to a 100 links than 10 links on it. So instead of 10 times as much bandwidth, you’re looking at 10-100 times! You’ll be eating up your bandwidth cap each month. And corporate networks will crawl to a halt.

But what about the websites themselves? The websites you visit might also significantly slow down. Let’s take an example using the assumption that AVG visited every link on every page to amplify the issue (even without this assumption, on some search terms the increase in traffic can be very significant). When You go to my company’s website LandlordMax, the landing page (the first page) has at least 20 links. The navigation alone is about 10 links. That means our servers now would have to be able to handle 20 times as much capacity to handle the same number of users (assuming they all used AVG). That is, every person who comes to our website would not just download the first webpage, but instead they would download 20 webpages. And each time they clicked on a link, they would get another 20+ pages. We’d have to increase our server capacity by 20 times. That’s very expensive, and where do you think that cost would have to eventually be offset? Into the price.

AOL Search Results Rankings

But assuming it doesn’t get to this level, right now for every search term we get listed on the first page, we’ll basically get hit with a webpage request. So for example, if the search term “latest movies” get 10,000 searches a month (from people using AVG), and we’re listed anywhere on the first page, we’ll have to handle 10,000 webpage requests even though only a percentage of that search traffic will come to our site as shown in the above graph. Who’s going to pay for all that bandwidth? It’s certainly not AVG! It’s both you the user of AVG and the website owner.

It gets even more interesting, assuming bandwidth is free. In the example above, let’s say it’s now your website and you’re the 10th search result listing, at the very bottom of the first page. In this case you can expect to get about 3% of the traffic, or using our example of 10,000, 300 visitors a month. Now, with the new AVG system, you can expect this number to dramatically increase. Instead of just 3% or 300 visitors a month, you’ll get 100% of the traffic, a full 10,000 visitors a month. That will greatly skew your web metrics and webserver needs. In this example you’ve just increased your traffic (and hence server capacity needs) by almost 3 orders of magnitude! This will affect how you run your website/business. If nothing else, it will increase your costs which means it will have to be offset somehow (usually an increase in price for the customer).

Above this, a lot of the decisions for online businesses are derived from the web metrics. With this new system the metrics of your visitors are now completely useless. The 10,000 visitors a month mean nothing. They don’t represent the true scale of traffic. You’ve basically lost the ability to determine what works and what doesn’t. Although I won’t get into the details here, let’s just say that this means you can no longer correctly determine when and why you have a “real” increase in traffic. It’s all masked in a barrage of fake traffic. And because your web metrics are useless, your marketing now becomes guesses in the dark. Which then means more dollars have to be spent to make the same amount of money. Again this will eventually have to be offset into the price of the products/services on your website.

In any case, this appears to be a very good example of a lose-lose scenario. No one wins. Of course it’s not as drastic as I’m making it out to be, AVG only accounts for at most 20 million users. But it’s very important to realize that this is a significant amount of users. It’s enough that not only have consumers begun to take notice, but many websites are also starting to report significant increases in traffic and bandwidth due to this issue.

What’s really irritating about all this is AVG’s Chief of Research (Thompson) response according to the Register:

And if that causes problems for webmasters, Thompson says, so be it. “I don’t want to sound flip about this, but if you want to make omelets, you have to break some eggs.”

Be careful of the implications of your decision. This may result in a very large community backlash, especially when it’s backed by a large number of webmasters. We all remember the Intel processor fiasco of 1993, commonly referred to as one of the biggest technical blunders of all time.

And on that note I’ll leave you with some after-thoughts to ponder.

  • What happens if the search result returns another search result (this could be maliciously implemented)?
  • You’re now exposed for exploits from every website on the search results, not just the ones you visit.
  • What’s your monhtly bandwidth cap?
  • What will be the performance impacts on your machine for continually scanning all those extra webpages for viruses, spyware, and malware?


 
Like this article?

Comments:

  •     Bob
    · June 30th, 2008  · 4:17 pm  · Permalink

    That’s the very reason why I uninstalled AVG…
    am using NOD32 and am very happy of it.

  •     Steph
    · June 30th, 2008  · 4:57 pm  · Permalink

    Hi Bob,

    I agree. We’ve been using NOD32 for several years here with great success. Especially for high performance developer boxes, the performance is very good!

  •     Pat Bitton
    · July 4th, 2008  · 6:28 pm  · Permalink

    Following is AVG’s official response to LinkScanner concerns:

    We’d like to thank our web community for bringing these challenges to our attention, as building community trust and protecting all of our users is critical to us. We have modified the Search-Shield component of LinkScanner to only notify users of malicious sites; this modified version will be rolled out on July 9th 2008. As of this date. Search-Shield will no longer scan each search result online for new exploits, which was causing the spikes that webmasters addressed with us. However, it is important to note that AVG still offers full protection against potential exploits through the Active Surf-Shield component of our product, which checks every page for malicious content as it is visited but before it is opened.

  •     Steph
    · July 6th, 2008  · 2:28 pm  · Permalink

    Hi Pat,

    Thank you for the update. That’s great news! I look forward to seeing the update rolled out.

  •     Day 37 - RC1 + Ready To Begin Website | The Recursive ISV
    · July 7th, 2008  · 8:07 am  · Permalink

    […] “In late February, AVG paired its updated anti-virus engine with a real-time malware scanner that vet… […]

  •     Day 37 – RC1 + Ready To Begin Website | The Recursive ISV
    · August 1st, 2010  · 6:20 am  · Permalink

    […] “In late February, AVG paired its updated anti-virus engine with a real-time malware scanner that vet… […]

Write a reply:





 
FollowSteph RSS
FOLLOWSTEPH'S
RSS FEED!


SOFTWARE AND BOOKS BY STEPHANE GRENIER:

LandlordMax Property Management Software

LandlordMax is the EASIEST
Property Management
Software available!
Try it for free!

Real Estate Pigeon

Real Estate Pigeon
The place to ask and answer
all your real estate questions

Blog Blazers: 40 Top Bloggers Share Their Secrets to Creating a High-Profile, High-Traffic, and High-Profit Blog!

Blog Blazers is a book that
features secrets from the
Top 40 Bloggers on the web

How to Generate Traffic to Your Website ebook

How to Generate Traffic to
Your Website
is an ebook for
to you achieve success


 

FollowSteph
More resources from Stephane Grenier:
PUBLICATIONS
For people who work on the web
Blog Blazers
How to Generate Traffic to Your Website
 
SOFTWARE
The EASIEST Property Management Software available!
LandlordMax


Copyright @2003-2013
LandlordMax Property Management Software

Disclaimer: This is a personal blog about my thoughts, experiences and ideas. The contents of this blog are for informational purposes only. No content should be construed as financial, business, personal, or any other type of advice. Commenters, advertisers and linked sites are entirely responsible for their own content and do not represent the views of myself. All decisions involve risks and results are not guaranteed. Always do your own research, due diligence, and consult your own professional advisors before making any decision. This blog (including myself) assumes no liability with regard to results based on use of information from this blog. If this blog contains any errors, misrepresentations, or omissions, please contact me or leave a comment to have the content corrected.