Large Account Migration: When Transfer Tool Fails (rsync Hybrid Method)

Every cPanel administrator eventually encounters it: the account that simply refuses to migrate. Whether it’s timing out at 80%, throwing cryptic errors, or hanging indefinitely, large account migrations can turn a routine server consolidation into a multi-day headache. Here’s the hybrid rsync approach I’ve refined over years of dealing with accounts that make the Transfer Tool surrender.

Why the Transfer Tool Fails on Large Accounts

Before diving into the solution, it’s worth understanding why WHM’s Transfer Tool—which works flawlessly for 95% of migrations—chokes on certain accounts.

The Transfer Tool packages the entire account into a compressed archive, transfers it as a single unit, and then extracts it on the destination. This all-or-nothing approach becomes problematic when:

• Account size exceeds available /tmp or /home space during packaging
• Network interruptions force a complete restart (no resume capability)
• PHP/HTTP timeouts kill the process mid-transfer
• Memory exhaustion occurs during compression of large mail stores
• MySQL dumps of massive databases timeout or corrupt

Accounts over 50GB are where I typically start seeing issues, though I’ve had 30GB accounts with millions of small files cause more problems than 200GB accounts with a handful of large databases.

The Hybrid Approach: Best of Both Worlds

The strategy is straightforward: use rsync to handle the heavy lifting of file transfers (with its resume capability and efficiency), then leverage cPanel’s native tools for the account metadata, databases, and final integration.

This gives you:

• Resumable transfers — network blip at 90%? Pick up where you left off
• Delta sync capability — only transfer changed files on subsequent runs
• Parallel transfers — move files while the account is still live
• Controlled cutover — minimize actual downtime to minutes

Prerequisites and Planning

On Both Servers

Ensure rsync is installed (it should be by default on any cPanel server):

which rsync
rsync --version

SSH Key Authentication

Set up passwordless SSH from the source to the destination server. On the source server:

ssh-keygen -t ed25519 -C "migration-key"
ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

Test the connection:

ssh [email protected] "hostname && echo 'SSH working'"

Document the Account

Before touching anything, gather account details on the source:

# Get account summary
whmapi1 accountsummary user=username

# Document disk usage breakdown
du -sh /home/username/
du -sh /home/username/mail/
du -sh /home/username/public_html/
find /home/username -type f | wc -l

# List databases
mysql -e "SHOW DATABASES LIKE 'username\_%';"

# Check for addon domains and subdomains
cat /etc/userdomains | grep username

# Document SSL certificates
whmapi1 fetch_ssl_certificates_for_user user=username

Save this information—you’ll need it for verification after migration.

Phase 1: Initial Bulk Sync (While Account is Live)

The beauty of this approach is that you can perform the initial sync while the account remains fully operational on the source server. This dramatically reduces actual downtime.

Create the Account Shell on Destination

On the destination server, create the account with the same package/settings but minimal data:

whmapi1 createacct \
    username=username \
    domain=primarydomain.com \
    plan=default \
    [email protected] \
    password=$(openssl rand -base64 32)

Alternatively, use WHM’s interface to create the account manually if you need to set specific options.

Rsync the Home Directory

From the source server, perform the initial sync:

rsync -avzP --delete \
    --exclude '.cagefs' \
    --exclude 'mail/new/*' \
    --exclude 'mail/cur/*' \
    --exclude '.cpanel/caches' \
    --exclude 'tmp/*' \
    --exclude 'access-logs' \
    --exclude 'logs' \
    -e "ssh -o StrictHostKeyChecking=no" \
    /home/username/ \
    root@destination:/home/username/

Flag breakdown:

• -a — archive mode (preserves permissions, timestamps, symlinks)
• -v — verbose output
• -z — compress during transfer
• -P — show progress and enable resume on interrupted transfers
• --delete — remove files on destination that don’t exist on source
• --exclude — skip directories that will be handled separately or are transient

For very large accounts, I add bandwidth limiting to avoid saturating the network:

rsync -avzP --delete --bwlimit=50000 \  # 50MB/s limit
    ...

Sync the Mail Directory Separately

Mail directories deserve special handling due to their structure and the volume of small files:

rsync -avzP --delete \
    -e "ssh -o StrictHostKeyChecking=no" \
    /home/username/mail/ \
    root@destination:/home/username/mail/

For accounts with enormous mail stores, consider using --inplace to modify files in-place rather than creating temp copies:

rsync -avzP --delete --inplace \
    /home/username/mail/ \
    root@destination:/home/username/mail/

Phase 2: Database Migration

Databases require careful handling to maintain consistency.

For Manageable Database Sizes (Under 10GB Total)

Use mysqldump with appropriate flags:

# On source, dump all user databases
for db in $(mysql -N -e "SHOW DATABASES LIKE 'username\_%';"); do
    echo "Dumping $db..."
    mysqldump --single-transaction --quick --routines --triggers \
        "$db" | gzip > "/home/username/${db}.sql.gz"
done

Then rsync the dumps and import on destination:

# On destination, import
for dump in /home/username/*.sql.gz; do
    db=$(basename "$dump" .sql.gz)
    echo "Importing $db..."
    mysql -e "CREATE DATABASE IF NOT EXISTS \`$db\`;"
    gunzip < "$dump" | mysql "$db"
    rm "$dump"
done

For Large Databases (10GB+)

Use mydumper/myloader for parallel dumps, or consider direct binary copy if you can afford brief MySQL downtime:

# Install mydumper if not present
yum install mydumper -y  # or compile from source

# Parallel dump
mydumper -u root -B username_largedb -o /home/username/db_backup/ -t 4

Transfer and restore:

rsync -avzP /home/username/db_backup/ root@destination:/home/username/db_backup/
# On destination:
myloader -u root -B username_largedb -d /home/username/db_backup/ -t 4

Recreate Database Users

Export and recreate MySQL users. On source:

mysql -N -e "SELECT CONCAT('CREATE USER IF NOT EXISTS \'',user,'\'@\'',host,'\' IDENTIFIED BY PASSWORD \'',authentication_string,'\';') FROM mysql.user WHERE user LIKE 'username%';" > /home/username/db_users.sql

mysql -N -e "SELECT CONCAT('GRANT ALL ON \`',REPLACE(db,'\\_%','_%'),'\`.* TO \'',user,'\'@\'',host,'\';') FROM mysql.db WHERE user LIKE 'username%';" >> /home/username/db_users.sql

Then rsync and execute on destination.

Phase 3: The Cutover

Now comes the critical part—the actual migration cutover. This is where downtime occurs, so be prepared and work quickly.

Step 1: Suspend the Account on Source

whmapi1 suspendacct user=username reason="Migration in progress"

This prevents new data from being written during final sync.

Step 2: Final Delta Sync

Run rsync one more time to catch any changes since the initial sync:

rsync -avzP --delete \
    --exclude '.cagefs' \
    --exclude '.cpanel/caches' \
    --exclude 'tmp/*' \
    -e "ssh -o StrictHostKeyChecking=no" \
    /home/username/ \
    root@destination:/home/username/

This should be fast since rsync only transfers changed files.

Step 3: Final Database Sync

If the database had writes since your initial dump, re-dump and import:

mysqldump --single-transaction --quick username_database | \
    ssh root@destination "mysql username_database"

Step 4: Fix Ownership and Permissions

On the destination server:

# Fix ownership
chown -R username:username /home/username/
chown username:mail /home/username/etc/*/shadow

# Restore cPanel metadata ownership
chown root:root /home/username/.cpanel

# Fix common permission issues
find /home/username/public_html -type d -exec chmod 755 {} \;
find /home/username/public_html -type f -exec chmod 644 {} \;
chmod 750 /home/username/public_html

Step 5: Rebuild cPanel Configuration

On destination, force cPanel to recognize all the account’s components:

# Rebuild user configuration
/scripts/updateuserdomains
/scripts/rebuildhttpdconf
/scripts/restartsrv_httpd

# Rebuild mail configuration
/scripts/builddovecotconf
/scripts/restartsrv_dovecot

# Ensure databases are mapped to user
for db in $(mysql -N -e "SHOW DATABASES LIKE 'username\_%';"); do
    uapi --user=username Mysql set_privileges_on_database database="$db" privileges=ALL
done

# Rebuild FTP configuration
/scripts/ftpupdate

# Update quotas
/scripts/fixquotas

Step 6: Verify SSL Certificates

Check if SSL certificates transferred correctly:

whmapi1 fetch_ssl_certificates_for_user user=username

If missing, you may need to re-issue via AutoSSL:

/usr/local/cpanel/bin/autossl_check --user=username

Phase 4: Verification Checklist

Before updating DNS and declaring victory, verify everything:

# Compare file counts
echo "Source files: $(ssh source-server 'find /home/username -type f | wc -l')"
echo "Dest files: $(find /home/username -type f | wc -l)"

# Compare disk usage
echo "Source size: $(ssh source-server 'du -sh /home/username/')"
echo "Dest size: $(du -sh /home/username/)"

# Test website locally (add destination IP to your /etc/hosts)
curl -I -H "Host: primarydomain.com" http://localhost/

# Verify databases
mysql -e "SHOW DATABASES LIKE 'username\_%';"

# Check cPanel login
echo "Test cPanel login at: https://$(hostname):2083"

# Verify email
ls -la /home/username/mail/

DNS Cutover and Cleanup

Once verified, update DNS to point to the new server. I recommend:

1. Lower TTL to 300 seconds 24-48 hours before migration
2. Update A records to new server IP
3. Monitor both servers for 24-48 hours
4. Unsuspend source account temporarily if issues arise
5. After confirmation, terminate source account

Troubleshooting Common Issues

“Permission denied” errors during rsync

Ensure the destination account exists and you’re running as root:

id username  # Should return user info
ls -la /home/username  # Should show correct ownership

Mail not delivering after migration

Check Dovecot index files:

find /home/username/mail -name "dovecot*" -delete
/scripts/restartsrv_dovecot

cPanel shows wrong disk usage

Force quota recalculation:

/scripts/fixquotas --user=username

Databases visible in MySQL but not in cPanel

Map databases to the user:

for db in $(mysql -N -e "SHOW DATABASES LIKE 'username\_%';"); do
    /usr/local/cpanel/bin/dbmaptool $db --type mysql --owner username
done

Cron jobs missing

Check source and recreate:

# On source
crontab -l -u username

# On destination, recreate or copy /var/spool/cron/username

Automation Script

For those doing this frequently, here’s a wrapper script I use:

#!/bin/bash
# hybrid-migrate.sh - Large account migration helper
# Usage: ./hybrid-migrate.sh username source-ip destination-ip

USERNAME=$1
SOURCE=$2
DEST=$3

if [ -z "$USERNAME" ] || [ -z "$SOURCE" ] || [ -z "$DEST" ]; then
    echo "Usage: $0 username source-ip dest-ip"
    exit 1
fi

echo "=== Phase 1: Initial Sync ==="
rsync -avzP --delete \
    --exclude '.cagefs' \
    --exclude 'mail/new/*' \
    --exclude 'mail/cur/*' \
    --exclude '.cpanel/caches' \
    --exclude 'tmp/*' \
    -e "ssh -o StrictHostKeyChecking=no" \
    root@${SOURCE}:/home/${USERNAME}/ \
    /home/${USERNAME}/

echo "=== Phase 1 Complete ==="
echo "Review sync results, then run with --final flag for cutover"

Conclusion

The hybrid rsync method has saved me countless hours on problematic migrations. While it requires more manual steps than the Transfer Tool’s one-click approach, the reliability and control are worth it for accounts that simply won’t cooperate with automated tools.

The key advantages: resumable transfers, minimal downtime through pre-sync, and granular control over each component of the migration. Once you’ve done it a few times, the process becomes second nature.

Have a migration horror story or a technique I missed? Drop a comment below—I’m always looking to refine this process further.