Installation on hosts without SSH access
August 23rd, 2007I’ve been hearing good things about HSphere as a platform for a reseller account - clustered services, fully integrated billing and ticketing and the ability to resell both Windows and Linux hosting (not that I really want to offer Windows but it could be useful at some stage). But there’s a catch: I’ve yet to find a HSphere provider who’s willing to offer SSH access. This is rarely a problem for CPanel or DirectAdmin - most hosts using these panels seem to offer it on request, presumably having accepted that some users have a legitimate need for it and that any additional security risks are minimal (above the risks associated with allowing users to run PHP, perl and shell scripts via CGI or cron). But perhaps the situation is different for HSphere - I read somewhere that SSH is used for internal communications within the cluster and allowing user SSH could compromise this. In any event, the reality seems to be, if you want SSH access don’t choose an HSphere reseller account.
Now I’ve always considered SSH access a prerequisite for ClonePanel. The backup server uses an SSH connection to transfer files using rsync. But could it be done any other way? I thought this could be an interesting challenge (masochist that I am) so I signed up for a reseller account with a popular HSphere host and set out to discover what was possible.
Of course there are always other ways to do backup and restore. Most common is to tar and zip all files into a single backup, transfer that using ftp and unzip and untar at the other end. But I really want to avoid this if I can - rsync is just so much more efficient and I don’t want these regular backups to cause any perceptible load on the servers. So I started looking at how to use rsync without being able to make an SSH connection to the hosting server. If I can’t connect in, what about having the server connect out? I can run shell scripts through cron, the system gives me access to the usual linux / bash commands including ssh and rsync, so what’s to stop me connecting from the hosting server to my backup VPS?
Well, as it turns out the first thing stopping me is a firewall! They have it sewn up pretty tight, including outgoing connections on several high-numbered ports that I tried - it makes sense from the security point of view (don’t open up anything unless you have to) but it makes things tricky for me. But it turns out that there are at least two ports open, 80 and 443, allowing scripts on the server to connect to external web sites - this seems like a requirement for any hosting server where the clients may need to fetch information from the web or communicate with a payment gateway.
So if I set up the VPS to run sshd on one of these ports, the hosting server can connect to it. Of course this means that it’s not possible to run a regular webserver on the VPS (on the standard ports) but I don’t want it to be web-accessible anyway. So I set up the ssh private key and known_hosts file on the hosting server (normally ssh will update known_hosts with the other server’s RSA key on the first interactive connection, after prompting to ask whether you want to connect, but in this case it must be done manually - there is no interactive connection!) and after a few attempts managed a simple ‘ls’ command on the backup server.
So it is possible: an outbound SSH connection from a hosting server where inbound SSH isn’t allowed. The use of a non-standard port is a little inconvenient but not a major problem. However we are talking about passwordless remote access to the backup server from a web-server… This is something I’ve always tried to avoid - the webserver is relatively insecure (ie. accessible to anyone who wants to connect to it, and running scripts which, even if kept updated, just might contain a zero-day exploit). This is a tough call - something I’d prefer not to do but against that I really would like to extend the functionality of ClonePanel to hosting servers without SSH access.
My compromise is this: a separate install of ClonePanel under a different user, with the new install used only for accounts connecting to this hosting server. Still not ideal perhaps but at least all other accounts are kept totally separate by file permissions on each user’s home directory. In the worst-case scenario where the hosting server gets hacked the attacker could use this connection to the backup VPS to read anything from the user’s filesystem, but this is mostly what’s already available to him in the web-directories of the compromised server, and the open-source ClonePanel system itself. There is one potential issue: the private keys allowing access to the corresponding accounts on other servers, but since these keys are restricted to use from the ClonePanel system’s IP address even that doesn’t allow the attack to go any further.
So that’s how it works. All that remains is to set up a simple rsync transfer script on the hosting server similar to the sync script on the ClonePanel system and run it at intervals with a cron job. Here’s what I ended up with:
#!/bin/sh
# This program is intended to sync a slave server to the backup
# where SSH connections to the slave are not allowed.
# The program runs on the slave server (via cron)
# and connects from the slave to the backup server to sync the files.
# After syncing the databases are restored.
# Several constant values need to be filled in below.
# ClonePanel - Manages duplicate accounts on two or more webservers,
# including snapshot backups, monitoring and failover dns.
# Copyright (C)2006 Chris Cheers, Internet Lynx.
# Contact chris[at]clonepanel[dot]com.
# Internet Lynx, PO Box 7117, Mannering Park, NSW 2259, Australia
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
exec 2>&1
# Display errors
set -u
# Be strict about variable declaration
unset PATH
#avoid use of $PATH - limit script to system commands we choose
#start of constant definitions
RSYNC=/usr/bin/rsync
SSH=/usr/bin/ssh
MYSQL=/usr/bin/mysql
#most likely won't need to change these
REMOTEUSER=username
REMOTEHOST=backup.clonepanel.example.com
REMOTEPORT=443
REMOTEWEB=path/to/public_html
REMOTEDB=path/to/cp0.30_db
#details for backup server - change as required
AUTH=/hsphere/local/home/username/.ssh/private.key
#private key file giving access for REMOTEUSER to REMOTEHOST
HOMEDB=/hsphere/local/home/username/cp_db
HOMEWEB=/hsphere/local/home/username/example.com
#directories for local server - change as required
EXCLUDES=excludes.txt
#name of file containing filenames to exclude - one per line
#filenames of all files modified to suit this server
NUM_DATABASES=2
DATABASES=(username_db1 username_db2)
DATABASE_USERS=(username_dbuser1 username_dbuser2)
DATABASE_PASSWORDS=(secret1 secret2)
DATABASE_HOSTS=(dbhost1 dbhost2)
DATABASE_FILES=(username_db1 username_db2)
#end of constant definitions (stop editing here!)
#sync from backups
$RSYNC -avz --delete --exclude-from="$EXCLUDES" -e "$SSH -p $REMOTEPORT -i $AUTH" $REMOTEUSER@$REMOTEHOST:$REMOTEWEB/* $HOMEWEB/
#sync web files
$RSYNC -avz -e "$SSH -p $REMOTEPORT -i $AUTH" $REMOTEUSER@$REMOTEHOST:$REMOTEDB/* $HOMEDB/
#sync database dump files
#restore databases
for (( i = 0 ; i < $NUM_DATABASES ; i++ ))
do
db=${DATABASES[$i]}
db_user=${DATABASE_USERS[$i]}
db_pass=${DATABASE_PASSWORDS[$i]}
db_host=${DATABASE_HOSTS[$i]}
db_file=${DATABASE_FILES[$i]}
$MYSQL -u$db_user -h$db_host -p$db_pass $db <$HOMEDB/$db_file.sql
done
Conclusion: It seems to be possible, with some minor compromises. With more development the process could be better integrated into ClonePanel - perhaps using a similar technique to transfer a directory of special scripts which could then be run by the ClonePanel system as cgi.
More to follow as I find the time…