Cachefly Content Delivery Network

by R.I. Pienaar | Nov 2, 2006 | Usefull Things

Further to my previous post about webservers for data delivery I’ve been investigating some content delivery networks.
One that’s been getting a bit of bloggers love recently is Cachefly, on paper and on their pretty flash movies it seems pretty awesome. However when it comes time to deliver it falls pretty short.
From their site a few bits from their blurbs:

While traditional (or ‘first-generation’) CDN’s have used ‘DNS tricks’ to circumvent BGP (the internet’s core protocol), BestHop™ instead harnesses the power of BGP to determine how content can be delivered in the most efficient manner over CacheFly’s global CDN footprint. BestHop™ uses Anycast to instruct carriers routers to make connections to the best available point-of-presence for the end user. By combining anycast with our unprecidented international footprint, CacheFly has built the next-generation in Content Delivery Network.

So by using anycast they route you to a static ip – always the same one – and your ISPs routing tables figure out the nearest POP where your files live, this sounds great I had some doubts about Anycast but I’ve been informed that it does in fact work well even for TCP traffic.
Their network seems pretty extensive at the moment, they have a link on their site called “Out Network” that opens a popup, this shows an impressive world view of servers in San Jose, Phoenix, Chicago, Toronto, Ashburn, London, Amsterdam, Stockholm and finally Tokyo. These locations are a good footprint for my needs.
Some more blurps from their site:

Speed counts on the web, and by using CacheFly for your website you can deliver your website up to 10x faster than traditional hosting.

CacheFly makes it faster:

• CacheFly’s BestHop™ traffic management system delivers your content at blazing speeds by delivering your content from the ‘edge’ of the internet, placing it closer to your visitors.
• Your content is always available. Guaranteed. So there is no need to worry about losing customers because of an ineffective server or network issues.
• CacheFly provides you with instant scalability – Our technology means you’re website’s effectiveness will not be impaired by increased traffic. *Embrace* flash crowds. They mean more $$ now, not more headaches.

The load-time of your website affects your bottom line, period.

This sounds great, so I want to host my site furniture on it, buttons, logos and also user profile photos etc. I want the furnishings to load as fast as possible so that there are no perceived delay in the page drawing due to photos loading slowly one by one, and this seems to match my needs perfectly.
Finally some other claims from their site:

• No changes to your existing hosting solution
• Easy to implement, be live less than 10 minutes

So what’s the problem then? I signed up for their biggest account since I have about 1GB of jpg’s to host, the account lists 24 hour support, SCP, SFTP and rsync uploads as features along with some other stuff.
Once I opened the account and logged into the control panel I was immediately presented with only FTP upload details, no rsync or ssh based system. Turns out for this you have to open a support ticket which the auto reply claims will be actioned within a day, thats great, 10 minutes indeed.
As for delivery to nearest to your clients, well thats just bullshit to say the least, the way they do it is by hosting all your files in the states by default. If a file gets a lot of hits from a specific region they’ll then wait for some threshold to be reached then they’ll propagate your files internationally, in the words of their support people:

“in order for files to cached internationally, they need to be considered ‘hot’; which will take more than several requests”

I’ve now hit a specific file over 150 times from the UK and it is still being served from the states. So how do they serve it from a specific location given its anycast? Well I hit a webserver in the netherlands, this server then responds with a 302 and redirects me to another server – a non Anycast IP – in the states. So in effect every hit I take on my servers gets translated into 2 hits until they decide a file has hit some magic threshold, and its a varying threshold since a large file (512Kb) hit it early on after only 20 or so hits, a much smaller test file is still being served from the states after 300 UK based hits.
As for their network map showing servers in the UK? Not true at all. From their sales people:

Also, you should note that we are turning up our London POP next week, this will significantly increase the already superb performance of our product.

And another response after I queried some more:

We have routers in london now, however, they deliver traffic to our amsterdam location. Next week we’ll be adding delivery infrastructure, as well as deploying our 10G port to the LINX peering fabric.

So not even close to as advertised, I’ve had hits to files from South Africa, China, Korea, UK, US and AU and only 5 out of 351 were served from Amsterdam, UK traffic though accounted for 337 of the 351 hits.
Cachefly looks good on paper, this is only day 2 of my trial with them and even if they get my rsync access sorted out I can’t see how I can possibly go to phase 2 – actually sending traffic their way – if they can’t get my files actually served globally. I’ve asked them about this and am waiting on a response, it might still turn around and be a winner for now though I think Cachefly is ok for a reasonably popular podcast or a big RSS feed, but not for serving 6 million files per day as quick as possible which is what I need it for.
UPDATE: After waiting 2.5 days for their support to even respond to my request to enable rsync, I’m now closing my account. My recommendation is to avoid Cachefly as far as possible.