How to Upgrade Python on CentOS

If you’re running CentOS, you’re probably a few versions behind on Python.  Currently, the version packaged for CentOS 5 and 6 is 2.6.  Contrary to what the title of this post implies, you actually cannot safely upgrade Python on any Redhat distribution.  If you’re feeling brave, try this to see why:

yum remove python

Warning, do not pass a ‘-y’ to the above command.  All you’re doing here is viewing output.  Hit CTRL+C once it prompts to continue, and no changes will be made to your system.

If you ran the command above, you’ll see all the packages that depend on Python, most all of which will break (including Yum itself) if you alter the system-installed package or attempt to upgrade.  Therefore, if you need a newer version of Python, the only safe way to do this is to install it alongside the system version.

This example is for installing Python 2.7, but you can easily do similar steps for version 3.3, etc.  Just download your preferred version from

cd /usr/src


tar -xvzf Python-2.7.6.tgz

cd Python-2.7.6

./configure –prefix=/usr/local


make install


And that’s all that’s to it!  Now any scripts that require the alternate version should have the following shebang:



If you need to use pip to install modules for your alternate version, make sure to use the correct one:




Implementing SendGrid with Exim on cPanel

Setting up your cPanel server to send through a third-party mail server is very easy to do, if you understand the basics of how cPanel builds its Exim configs. You never want to edit your exim.conf file directly – your changes will be wiped out any time a cPanel update runs or someone makes a change via the Exim Configuration Editor in WHM.  Below is a quick guide on setting up a custom mail router with a provider like SendGrid, which in turn will route all outbound email through the external mail service.

The below instructions cover how to do this via command line.  If you prefer WHM, simply go to WHM -> Exim Configuration Manager -> Advanced Editor and alter the sections indicated.

First, open up your /etc/exim.conf.local file in an editor and look for the @AUTH@ section.  Modify it to look like this:


driver = plaintext
public_name = LOGIN
client_send = : <user> : <password>


Of course, replace <user> with your SendGrid username and <password> with your SendGrid account password.  If you already have something in the AUTH section, simply add this block of text below it.

Now look for the @PREROUTERS@ section, and modify it to look like this:


driver = manualroute
domains = ! +local_domains
transport = sendgrid_smtp
route_list = "* byname"
host_find_failed = defer


The last modification should be to the @TRANSPORTSTART@ section:


driver = smtp
hosts =
hosts_require_auth =
hosts_require_tls =


Now save the file, and apply the changes:


service exim restart


To test whether things are working, send an email out from your server, and look for it in /var/log/exim_mainlog. You should see something like this in your log entry:


2013-10-08 19:37:29 1VTjeS-0000Ac-O3 -> my@email R=send_via_sendgrid T=sendgrid_smtp [x.x.x.x] X=TLSv1:DHE-RSA-AES256-SHA:256


If you have SPF records, you’ll need to add the hostname of the SMTP server to the record itself to allow the third-party mail server to send email on behalf of your domain.  Sendgrid will provide the hostname you should use.




How to Easily Monitor Cluster Status

If you are running cPanel 11.28 or higher (which we hope you are, considering the current release at the time of this writing is 11.40!), you have the option in WHM to automatically disable DNS clustering if too many connection failures occur:


WHM cluster status options


While this can be a handy feature, if you have a high-capacity system that frequently makes DNS updates, this could create complications.  The easy way out is to of course select the third option to keep the cluster members online, however, this could slow down DNS changes (adding/removing domains, etc) if your cluster members are unavailable.  Choosing to receive notifications may be a viable option, but some administrators receive so much email to the WHM admin contact address that the email alerts may not be as visible as they need to be.


If you have a monitoring system that supports custom plugins, such as Nagios, you can monitor the status of the cluster simply be looking for a few files.  When DNS clustering is globally enabled, a file called /var/cpanel/useclusteringdns will exist, and all cluster members will be enabled by default. The status files for specific cluster members are stored conveniently in /var/cpanel/clusterqueue/status/, and will be named as follows:


  • $IP: This file always exists, and has binary contents.  If it contains all 1′s, everything is fine.  If it has 0′s in it, connection failures have occurred. Of course, $IP represents the actual IP address of the remote cluster member
  • $IP-down: This file will exists when a cluster member is down.  Deleting this file will re-activate the cluster member.

You can also monitor the following folders which may help ore-emptively identify a DNS problem or overload within your clustering system:


  • /var/cpanel/clusterqueue/retry/:  Contains a single file for each request to the cluster that has failed.  Having too many of these will indicate a problem reaching one or more members of the cluster
  • /var/cpanel/clusterqueue/requests/: Contains a single file for each request to the cluster that has not yet synchronized.  Having too many of these at once will indicate heavy activity on the server.  If this activity is unusual, it may need to be investigated.


Knowing what files to look for can allow you to write your own monitoring scripts and/or Nagios plugins to periodically check for clustering problems, or attempt to automatically fix them, if you don’t want to rely on email notifications.


How to Set up Multiple Shared IPs on a cPanel Server

There are some situations where you may need to set up multiple shared IP addresses on a server.  Some reasons may include grouping accounts per IP or lowering the effect of DDoS attacks if it becomes necessary to block traffic to a specific interface.  Whatever the reason, multiple shared IP addresses on a cPanel server is rather easy to set up.


First you’ll want to reserve the IP address via WHM -> Reserved IPs.  Alternatively, simply drop them into /etc/reservedips, separated by line, ie:

If you want to label the reserved IPs, set them up in /etc/reservedips: ip ip

… and so on

Now rebuild the IP pool:


This will prevent the IPs from being used as dedicated IPs for individual cPanel accounts, but for multiple accounts to be able to use them as shared IPs you’ll need to add them to a pool.  Simply use the /var/cpanel/mainips/root file to list each IP on its own line.  When root creates an account on the server, the first IP address in that file will be used.  If you want a reseller to have multiple shared IPs as well, simply replace ‘root’ with the reseller’s username.


To take this configuration further, you may want to have the main IP rotate automatically so each IP in the pool gets equal use.  We’re written a script to automate this task rather seamlessly via cron.


Click here to download the script


Installation steps:

wget -O /usr/bin/rotate_main_ips

chmod 755 rotate_main_ips

echo ‘*/30 * * * * /usr/bin/rotate_main_ips’ > /etc/cron.d/rotate_main_ips

service crond reload



Install Tomcat 7 on a cPanel Server

CPanel has soon promised that Tomcat 7 will be supported in a future EasyApache release. Until then, you can get easily get the support of Tomcat 7 with just a little bit of manual intervention.  I will mention that right now, Tomcat 7 is not supported by cPanel so there’s no guarantee that their integrated features for Tomcat will work as expected.


First, go ahead and run EasyApache and select the option to install Tomcat. This will indeed install Tomcat 5.5, but that’s OK.  Let EasyApache do the grunt work for you.


After it’s installed, you’ll need to put the Tomcat 7 files in place:

cd /usr/local/jakarta


tar -xvzf apache-tomcat-7.0.41.tar.gz

ln -sf ln -sf /usr/local/jakarta/apache-tomcat-7.0.41 /usr/local/jakarta/tomcat

Note:  The above example is based off of the latest version of Tomcat available at this time.  You can get the latest version at .


While still in the jakarta folder, compile the daemon:

cd ./tomcat/bin

tar -xvzf tomcat-native.tar.gz

cd commons-daemon-*

cd unix



cp jsvc ../../

Now kill off tomcat and restart it:

killall java


And add the latter command to /etc/rc.local:

sed ‘/starttomcat/d’ -i /etc/rc.local

echo ‘/scripts/restartsrv_tomcat’ >> /etc/rc.local





cPanel Update Fails with Exit Code 6400

Since the release of cPanel 11.32, the cPanel update process has extended OS checks to ensure that you are only updating cPanel on a compatible system. Specifically, you can no longer upgrade past version 11.32 on CentOS 4, or 11.30 on Redhat 9.  Trying to do so will verbosely inform you that the upgrade is not possible.

However, if you’re running CentOS or RHEL 5 or 6 and still either see this error or a cryptic “exited with code 6400″ message as shown below, it’s usually a simple fix.
Running `/usr/local/cpanel/scripts/updatenow --upcp --log=/var/cpanel/updatelogs/update.1xxxxxx.log --force` failed, exited with code 6400

Firstly, if you run the updatenow command directly, you’ll see the error in its true form:

/usr/local/cpanel/scripts/updatenow –upcp

Results in:

Only Distro versions 5 and 6 supported in this version of cPanel & WHM

But, as you’ve already established, you are running version 5 or 6, but upcp doesn’t seem to recognize that.  To fix this, simply edit /var/cpanel/sysinfo.config and change the value of rpm_dist_ver to the major OS version you are running. For example, if you run CentOS/RHEL 6, you would do:


Then attempt to run the upgrade again, and it should be fine.  As to how it got this way, well, we’re not entirely sure. From thumbing through the code of the updatenow script, it appears to be trying to use the rpm headers to capture the OS version.  If your RPM database is/was corrupted at one point, the update process may have put an ‘unknown’ in there instead:

if ( -e $sys_info_file && ( !$locked_values{'rpm_dist_ver'} && !$new_conf{'rpm_dist_ver'} or $new_conf{'rpm_dist'} eq 'unknown' ) ) {
print "ERROR: Refusing to update $sys_info_file since I cannot determine your distro's major version\n";
print "Please run: " . q{rpm -qf --queryformat '%{VERSION}\n' /etc/redhat-release} . "\n";

If your cPanel update eventually completed, you’re probably fine.  If you have RPM issues still, the quick and dirty way to fix this is as so:

rm -rf /var/lib/rpm/__db.00*
yum clean all

Note:  Don’t try to trick the system by setting rpm_dist_ver to something that you’re not running, especially if you’re trying to force cPanel to update on your CentOS 4 system.  We’ve tried it, and it doesn’t work.


Fixing cPanel Locale Errors

If you receive an “Internal Death” (generic cPanel speak for “something broke”) when accessing WHM and/or cPanel, the first thing you should do is check the cPanel error logs at /usr/local/cpanel/logs/error_log.  You may see something like this:

die [Internal Death while parsing [stdin] xxxxx] Read of CDB_File failed: Protocol error at /usr/local/cpanel/Cpanel/CPAN/Locale/ line 217.
Cpanel::CPAN::Locale::Maketext::maketext() called at /usr/local/cpanel/whostmgr/docroot/templates/menu/root.tmpl line 53

This is almost always an issue with the locale database, and tends to occur when a cPanel update failed due to outside conditions – such as a server running of our disk space, memory, etc during the update.  To fix this, simply move the locale files out of the way:

mv /var/cpanel/locale /var/cpanel/bad_locale

If you have custom locales installed, you can go ahead and move the files back in (unless, of course, the custom locales were what was causing the problem).

Then run:

/usr/local/cpanel/bin/build_locale_databases –force

If your cPanel update did indeed fail, it’s a good idea to run it again.


Restoring DNS Zones on a cPanel Server

In rare situations, the DNS zones located in /var/named might disappear. We’ve heard of this happening after certain bind package updates, or simply due to administrator error. In either case, it may be possible to restore or at least recreate the missing zones.  Even if the server in question does not act as a nameserver, certain functions within cPanel require the zones to exist locally.  These instructions are geared toward a hosting server (not solely a nameserver) that may or may not be clustered with a cPanel DNSONLY server.  For more information about clustering, see this article.

Restoring from the DNS cluster


Firstly, if your server is clustered with another cPanel server, your job is going to be easy. All you have to do is have cPanel sync the zones from one of the other clustered server(s).  Here’s a very simple Python script that will pull out a list of main, addon, and parked domains from the cPanel data and sync them back to the server.  For this to work, you’ll need to make sure your remote DNS cluster is set to “synchronize” status.  To do this, you can either:


1) Log into WHM -> Configure Cluster and set the cluster’s role to “Synchronize changes”  OR

2) Run the following command, replacing $ip with the IP address of your DNS cluster (noting also that ‘root’ implies that this is the cluster config for the server itself. If you’re a reseller, replace “root” with your reseller username)

echo -n “sync” > /var/cpanel/cluster/root/config/$ip-dnsrole

Now create a file on your server called, for example, and chmod it to 755.  Paste in the following contents:

#!/usr/bin/env python
import os
import subprocess
import yaml
data_dir = '/var/cpanel/userdata'
user_list = os.listdir('/var/cpanel/users')
for user in user_list:
    data_file = '%s/%s/main' % (data_dir, user)
    data_map = None
        f = open(data_file, 'r')
        data_map = yaml.load(f)
    if data_map is not None:
        domains = []
        for addon in data_map['addon_domains']:
        for domain in domains:
  ['/scripts/dnscluster', 'synczone', domain])

Now, simply run the script and your zones will be copied from the remote cluster to the local server.  If this server is standalone and/or is a nameserver that does not have another clustered server attached to it, you basically only have the choice of either restoring from a backup or recreating the zones.


Restoring from Backups


Assuming you use the cPanel backup feature and have the system files backed up, you can find them in your cPanel backup directory.  If you’re not sure where they are being stored, look in cPanel -> Configure Backup where the “Backup Destination” value is set. Or simply:

grep BACKUPDIR /etc/cpbackup.conf

Assuming the backup location is /backup, you will find the files in /backup/cpbackup/weekly/dirs (for a weekly backup – it could also be named ‘daily’ or ‘monthly’).

To restore, go to the dirs folder mentioned above and run:

tar -C / -xvf _var_named.tar.gz


Recreating Zones


If you don’t have backups, your only choice may be to recreate the zones.  To do this, you can use the adddns script provided by cPanel:

/scripts/adddns $domain [$Ip]

If the $ip parameter is not passed, the main shared IP will be used instead.  Keep in mind that if you recreate the zone using this method, a default zone using cPanel’s zone template will be used – so any DNS modifications that were previously made to the zones will be lost.  You can use a similar script to the one above to do all domains on your server:

#!/usr/bin/env python
import os
import subprocess
import yaml
data_dir = '/var/cpanel/userdata'
user_list = os.listdir('/var/cpanel/users')
for user in user_list:
    data_file = '%s/%s/main' % (data_dir, user)
    f = open(data_file, 'r')
    data_map = yaml.load(f)
    domains = []
    for addon in data_map['addon_domains']:
    for domain in domains:['/scripts/adddns', domain])
Fixes // Hacks // Mail // Misc

Exclude Domains from RBL Checks for Incoming Email

Update: This feature is automatically enabled as of cPanel

If you have RBL’s enabled globally on your server, there may be times when you want to keep certain domains from having their mail scanned against an RBL. While cPanel supports excluding sender IP addresses from these checks, some manual Exim modifications are needed to exclude recipient domains.

First, create a file that we’ll use to list the domains to exclude:

touch /etc/skiprbldomains

Next, you’ll need to apply a custom patch to one of the Exim perl modules used by cPanel.  You can view the patch here.


wget -O  /usr/local/cpanel/src/skip_rbl_domains.patch \

patch < /usr/local/cpanel/src/skip_rbl_domains.patch


service exim restart


Now to exclude domains from RBL checks, simply list them in /etc/skiprbldomains
When cPanel updates, this perl module will be overwritten.  In this case we do not recommend setting the file to be immutable or adding it to /etc/cpanelsync.exclude, but rather adding a simple script hook to re-apply the patch after every Exim update:


echo “patch < /usr/local/cpanel/src/skip_rbl_domains.patch” >> /scripts/posteximup

chmod  700 /scripts/posteximup


Note: The information used in this post was provided as a cumulative effort between our staff, one of our clients, and the cPanel developers.

Using Skeleton Directories in cPanel

The skeleton (aka “skel”) directory is one of the frequently ignored features on Linux servers, especially for hosting providers.  With the skel directory, you can easily configure a default set of files to be automatically copied into new user accounts, which is particularly useful for customizing the holding pages for new customer websites.


On a bare Linux server, the skeleton folder can be found in /etc/skel.  On cPanel servers, the directory will be in “cpanel3-skel” in the home directory of the user that owns the account being created.  For example, if you’re the root user and you’re creating an account on the server, that account’s default files will be copied from /root/cpanel3-skel.  If you’re a reseller, it will be in your home folder, ie /home/yourusername/cpanel3-skel.


Let’s say when new users are created on your server, you want their websites to automatically show a basic index.html page that says “coming soon”.  You may also want the user to have a customized .bashrc file for their SSH sessions.  If you are root, you’ll create the following files:


  • /root/cpanel3-skel/public_html/index.html
  • /root/cpanel3-skel/.bashrc


Now, when user accounts are created, their accounts will automatically contain .bashrc and public_html/index.html.