ClonePanel

Web hosting (I’m talking about virtual hosting, also called shared hosting) can be divided into two main categories - single server or clustered.Using a single server (eg. with the popular CPanel control panel) is a beautifully simple solution while everything is going well, but for a variety of reasons (badly written or exploited scripts, ddos attack, processing incoming spam…) the server can become overloaded. When this happens web pages load slowly or time-out, databases become unavailable, mail may be rejected or lost. How often this occurs, how severe the effects and how long the server takes to recover depend on the quality of the hosting service.

Clustered solutions distribute the work among a cluster of servers, with separate machines handling the job of web server, database server, mail server etc. This makes it much less likely that the whole cluster will become overloaded and for simple static web sites the reliability should be significantly better. But for dynamic web sites that rely on database access to create their pages the story may be different. If the cluster has one database server and it becomes overloaded or starts refusing connections then all database-driven sites across the whole cluster are rendered useless (this is of course many more sites than would be affected by a failure in the single-server scenario). As before, the quality of management is critical.

Finally, with both the single-server and the clustered hosting systems there are factors completely beyond anyone’s control. Although rare it’s not unknown for datacenters to suffer problems - lightning strikes, power failure, cooling failure, cable cuts etc. Or there can be connection problems like bad routes that leave some people unable to connect to sites hosted in a certain location even though the server may be working perfectly.

So if you want your web site and e-mail to work all the time, what’s the solution? In my view, the priorities are these:

1. Choose a reliable host. Shared hosting (whether single-server or clustered) is relatively cheap and finding a good quality hosting company doesn’t require spending a fortune. I expect uptime to be around 99.9% over a prolonged period - good hosts will be able to demonstrate a record of achieving this. Or if you’re looking for the best possible reliability and money is no object, get a fully-managed dedicated server (this eliminates one major source of problems - your fellow users!)
2. Choose a second reliable host. One in a different datacenter a long distance away from the first (another continent is good!). Use both hosts to provide dns and mail services for your domain - nameserver and mail-exchanger records allow for this so that if one is unreachable the other will be used automatically.
3. Plan for the worst-case scenario. What would you do if the server died and the backups were found to be corrupt? Or if the server got hacked and your site defaced, with backups being replaced by the modified pages? Sure it shouldn’t happen but it does. So keep your own backups (multiple copies), synchronize them to another host and check that they are good.

Will all that give you 100% uptime? No. The main limitation is dns propagation time - using failover you can update your nameservers almost instantly to point to your spare server but dns servers around the world will not update until their cached data expires. And while you can set a short TTL (time to live) on your records there’s no guarantee that all ISPs will respect it, in fact it’s pretty much guaranteed that some won’t!

So I regard dns failover to a hot-spare server as a rather imperfect emergency measure in the event of disaster or prolonged outage. But in that situation, you will be glad you have it!

ClonePanel aims to combine snapshot backups, dns control and server monitoring to provide exceptional levels of web site reliability and security against data loss. It’s specifically intended for use with virtual hosting accounts rather than dedicated servers - no root access is needed.

This is a discussion of the background history to indicate why ClonePanel has developed these key features.

As a web developer, I want to provide a comprehensive service to clients - not just creating their site but providing ongoing support and maintenance. This is good for me, offering a continuing revenue stream, and also good for the clients who might otherwise have to look for support from others (who would be nowhere near as good, naturally!)

Hosting is an important part of this service, but I don’t want to become a host myself for several reasons:

I suspect it would be hard to break into a market already full of hosting businesses, some offering excellent service, some selling at improbably low prices and even some offering good service for a very reasonable price.

Providing 24/7/365 cover really requires a staff of at least 5 people. I prefer to continue as a one-man-band and I don’t want to be dragged out of bed in the middle of the night to fix a server!

Most of my clients are small businesses with small-business web-sites. They don’t use a lot of disk-space, bandwidth or processing power, and as a result I don’t need even one dedicated server. It’s simply more cost-effective to lease space on a server (or two, for redundancy).

So I prefer to be a “value-added reseller”, purchasing hosting from carefully-selected suppliers and reselling to my clients. Some clients prefer to purchase their own hosting plan, giving them more independence - that’s fine too; I can recommend suitable options and if they want me to continue maintaining their web site they can give me the login details. So I get to sleep at night and even take weekends away (much appreciated by my wife!) without worrying about web servers.

But should I be worried? How reliable is my carefully-chosen host? What uptime does their server really achieve? I believe uptime guarantees are utterly worthless - at best I may get a refund of one month’s hosting fees but the consequences of prolonged downtime could be devastating for my business. There are some good monitoring services available but they often just ping the server - what if the web-server’s down, or the database, or the server is overloaded and timing out on dynamic pages…?

DNS (domain name service) is another grey area. CPanel is a good and very popular control panel but its standard setup with a single dns server on two different IP addresses is woefully inadequate. There are many independent dns services available but whenever I’ve tried their free service I’ve found the servers are often slow to respond, giving a significant delay on the first page of a new site - unacceptable (of course the paid services should be better but I’ve not been inspired to invest in them…) CPanel itself offers a clustering system but this can only be set up with root access.

Finally, and hardest to evaluate there’s the question of backups. There is nothing as valuable as data - mine and my clients’, so how best to protect it? Most hosts back-up their own servers but for liability reasons offer no guarantees. Is their backup system reliable? You’ll probably only find out after the disaster happens, so you’d better hope so! Personally, I want my own backups, but taking full backups manually from numerous different web sites is a tedious job, and simply automating the standard backup system can result in significant data transfer and server loads.

So in short, I wanted a system to do all of this:

Server monitoring

To include server-load, web server and database connection, with data-logging for analysis and summaries.

DNS clustering system

Making use of the DNS services included with CPanel but not restricted to a single server and without requiring root access. The CPanel automation utilities are usable here and only require standard reseller access to WHM.

Backups

File transfer using rsync seemed to overcome the problems of creating and transferring huge backup files. The backup is synchronized to the original once or twice each day and the only data transferred is that related to files that have changed. To reduce data transfer even further, only the different parts of the modified file are sent and all data transferred is compressed with gzip.

Well, that was the start. But out of this other opportunities presented themselves:

Snapshot backups

Want to retrieve that file you deleted three weeks ago? How about that article in the database, not sure when it went missing… A snapshot-type backup system keeps “snapshots” of the whole web-server file system from as many different times as you want. Thanks to Linux hard-links large numbers of such snapshots can be stored using hardly any more disk space than the latest copy plus the changed files.

Hot-spare server

Once you have a process in place to synchronize from the web-server to a backup it’s a small step to sync back from the backup to another web-server. So you can keep a second account on another hosting service complete, updated and ready to go at any time - just switch over the dns…

DNS failover

Hang on, we have control of the dns servers too! So why not automate the process of switching? When the monitoring system indicates a server is in trouble we can just divert visitors over to the hot-spare!

Ok, I know - it’s not that simple. Dns record caching and propagation means that for a time after switching the web site over some visitors will continue to get to the original server. And for dynamic sites like this one there’s the important issue of which database gets updated - the one on the original server or the hot-spare? During transfers from one host to another this is often solved by having both servers connect to the same database but obviously in this case that won’t work (no redundancy). But even if we need to make the site read-only until the problem can be sorted out, this is surely still better than having it offline altogether.

So with two reseller-type hosting accounts and a package combining snapshot backups, dns control and monitoring, there’s an opportunity to provide exceptional levels of web site reliability and security against data loss. ClonePanel aims to be that package.