Using Apache 2.2 with mod_proxy_balancer to provide load balancing for backend web servers, and OpenBSD built-in CARP support for virtual IP failover.
All load-balancer binaries, config files, certs, etc., stored under path /usr/local/lb/, so swapping out a new box (assuming it is also running OpenBSD) only requires creating a new tarball from the current primary, copying it over, and changing appropriate hostnames in Apache config.
Key files (all relative to /usr/local/lb):Using Debian-provided Apache and PHP packages along with a built-from-source XCache PHP opcode cache. PHP session data is written locally, and affinity is insured using source-IP hashing.
Shared files (i.e., uploads, Horde CFS, etc.) are stored in NFS mount exported from database servers, backed by replicated filesystem. Per-vhost webroots are under /usr/local/www/vhosts/, but Apache config data is in usual Debian paths (/etc/apache2/sites-available/...).
Database servers (x2) running Debian Etch:(Note: Etch was required for full hardware support)
PostgreSQL 8.1 and UCARP installed from Debian repository; DRBD built from source using module-assistant against default Etch x86_64 kernel.
Created runlevel control scripts for UCARP and DRBD services, as well as virtual IP “promotion” and “demotion” scripts as ’/etc/scripts/vip-{up,down}.sh’
To force demotion of one server, just run ’/etc/init.d/ucarp stop’, which will shut down ucarpd, and (as a side-effect) run ’/etc/scripts/vip-down.sh’, killing Postgres and NFS, then unmounting the DRBD filesystem.
DRBD troubleshooting/state info is in ’/proc/drbd’.