.

web tutorials

 

Cloudflare and AdSense: page load times, RocketLoader, and problems

Over the past two months, I have been experimenting with the excellent new CloudFlare service, which combines site protection against hacking attacks with bandwidth protection and agressive, intelligent cacheing. I tried this both with a VPS and a very powerful dedicated server.

CloudFlare works by acting as your DNS server, and essentially becoming a proxy for your site. Any complex requests, e.g. for dynamic content, are passed along; any simple ones, or (depending on your settings) complex ones for repeatedly demanded pages, are handled straight from CloudFlare. In theory this should reduce page load times, and if you're using a poor or slow service, I suspect this is exactly what happens. I've been paying a premium for a SmartVPS and dedicated server with LiquidWeb, and so I generally, with one huge exception, have found CloudFlare to provide no speed benefit, as measured by Google Analytics and Google Webmaster Tools. These tools were my choice because they aggregate the experiences of thousands and thousands of users, from many different locations. Also, while they might or might not be particularly accurate, they are the tools Google seems to be using to decide whether your site’s too slow.

CloudFlare had a dramatic on page load speed — and by “dramatic” I mean “cutting by a third” — but only when the Pro version’s RocketLoader was active. RocketLoader™ makes your Javascripts all run asynchronously, so they can be parallel processed by the host computer. Since nearly every computer in use today has some sort of multiple-core, multiple-processor, or HyperThreading setup — even a Mac Mini has two physical cores in its base setup — this greatly reduces total page load time.

Here’s the problem with RocketLoader, and the reason why I no longer use it: it also cut my AdSense revenue by around 25%.

Now, there’s a caveat. My site is enabled to use special Javascripts with AdSense, as you may have noticed. I don't know the effect of CloudFlare on Google AdSense revenue under normal conditions, with their default scripts — but I’d bet it would be similar, because all Google ads, as far as I know, depend on Javascript, as do those of VibrantMedia, Kontera, GumGum, and VigiLink.

What happens with RocketLoader, as far as I can tell, is that the ads pop up a few seconds after the page itself loads. I don’t know why that would cut revenue, but it seems to do so. The site without RocketLoader seems to be just fine, but activate RocketLoader, and revenue gets slammed.

Another problem I had were “false positives.” I set the various options to their lowest-protection levels and still had issues with false positives. At this point I don’t like the page that’s presented to suspected spammers, but that’ll change in the future (this article was written in November 2011).

The final problem was a DNS issue affecting the CloudFlare network, which made it nearly impossible for people on the East Coast to reach my site one morning. This made me a bit iffy about adding a point of failure. Running DNS on my own site at least means that if the DNS server is down, the rest of the site probably is, too.

I am sure that CloudFlare will work on these issues, but at the moment, I've gone back to ordinary DNS. I might try Amazon Route53 next for DNS, because despite the “extra point of failure” issue, it is a lot faster to change DNS to route to a different server, than to change the definitive DNS server through my domain name registrar.

CloudFlare does have some serious benefits which probably outweigh the problems for some people, especially those who have different ad serving systems. For one thing, CloudFlare provides an incredible amount of bandwidth reduction — it can slash bandwidth usage to a fraction of its original, which is handy if you're up against ISP limits. (Personally, I never come near them, mostly because I tend to over-provision.) There is a protection against your site going down, which is one of the main reasons I signed up.

And the main raison d’être of CloudFlare remains a key strength — protection against the bad guys. I have a couple of sites that were being hit hard by spammers and scammers and hackers, and for those sites, CloudFlare would have been the ideal solution. I ended up closing one down and fixing the other one with other means, but simply using CloudFlare would have saved me a lot of time and trouble, and that may well be what I do next time.

Overall — I am keeping my eye on these guys, because they are moving forward, responding to customer complaints, and improving things. Also, I love their interface, ideal for one who has to wear eyeglasses.

Update, November 22. I went back to CloudFlare because it turned out that their various optimizations still improved my site speed by around 8% even without RocketLoader. I analyzed the page loads and it turns out that by cacheing and compressing the ad agency javascripts, they are saving quite a bit of time. Oddly, my own pages are not the issue here, it's the ad agencies. Google’s own PageSpeed process points to Google’s various scripts as breaking speed rules; but so do those of VibrantMedia. While I can optimize my own site, I cannot optimize the ads — unless I run them through CloudFlare! It’s the only service I know which can do that. So for the moment, I'm stuck with them ... as long as I want to reduce page load times without going through individual pages and removing photos, or going nuts with an alternative web server (e.g. nginx).