Blog by Edo Frederix edofrederix@gmail.com RSS

Backend web server install: new kernel, lighttpd, memcached, php5, etc

July 25, 2010

Abstract

Our backend web servers are running with a few software packages. For efficiency, we do not want to use any virtualization. So every server is running its own OS with a custom kernel and has its own server software installs. Here a quick manual on how to perform all the installations.

Make sure you have a fresh default installation of debian 505 running. Also make sure you have ssh running. We compiled our own kernel, with a few tweaks in it. In case you're fine with the default kernel, skip this step.

cd /usr/src/
wget http://.../linux-headers-version.archi.deb
wget http://.../linux-image-version.archi.deb
dpkg -i linux-image-*.deb
dpkg -i linux-headers-*.deb
update-initramfs -k version -c
pico /boot/grub/menu.list

Now make sure that the file system is correct and that there is a initrd line in the new kernel entries. Make sure that this new entry points to the right kernel.

It's important that all backends are running on the same clock. They need to sync their time from the same ntpdate server

dpkg-reconfigure tzdata
apt-get install ntpdate
cd /etc/init.d/
wget http://.../ntpdate
chmod +x ntpdate
./ntpdate
crontab -e

In the crontab file, add something like 25 6 * * * /etc/init.d/ntpdate. This will run every morning at 6:25, and sync all the servers.

We compile Lighttpd from the latest stable source.

cd /usr/src/
wget http://download.lighttpd.net/lighttpd/releases-1.4.x/lighttpd-version.tar.gz
tar xzf lighttpd-version.tar.gz
cd lighttpd-version
apt-get install zlib1g zlib1g-dev libpcre3 libpcre3-dev make build-essential
./configure --without-bzip2
make
make install
mkdir /etc/lighttpd
cd /etc/lighttpd
wget http://.../lighttpd.conf
pico lighttpd.conf (edit if needed)
cd /etc/init.d/
wget http://.../lighttpd
chmod +x lighttpd

Next to that we install PHP5, with a PHP accelerator (APC). This last part is a requirement, to save around 50% resource usage.

apt-get install php5-cgi php5-memcache php5-dev php5-gd php5-mysql
apt-get install memcached
apt-get install php-pear
pecl install apc
pico /etc/php5/cgi/conf.d/apc.ini

In this APC configuration file, add something like the following lines to enable APC. This gives APC a shared memory size of 30MB. This can safely be increased, if you need it.

extension=apc.so
apc.enabled=1
apc.shm_size=30

Now, let's configure php to use memcache as session file host. This enables us to have persistant sessions over different webservers.

session.save_handler = memcache
session.save_path="tcp://10.0.0.67:11211?persistent=1&weight=1&timeout=1&retry_interval=15"

We can now start Lighttpd, running with PHP5 and APC. First, we need to create a home directory and some files. Also make sure to properly edit /etc/lighttpd/lighttpd.conf

mkdir /var/www/
cd /var/www/
wget http://.../index.html
wget http://.../phpinfo.txt
mv phpinfo.txt phpinfo.php
mkdir /home/myp2p/public_html
/etc/init.d/lighttpd

And then, there are a few sidekick features that we need to install on our server, including: postfix relaying, the right IP address for the domain "myp2p.eu", and an updated sysctl.

apt-get install postfix (install with "no configuration")
cd /etc/postfix/
wget http://.../main.cf
pico main.cf
pico /etc/hosts (add the line "77.247.179.66   myp2p.eu")
rm -fr sysctl.conf
wget http://.../sysctl.conf
sysctl -p

The servers sync their website through rsync from a master server. We need to add the script that does this, and make sure it will not be prompted by a password.

cd /home/myp2p/
wget http://.../sync.sh
chmod +x sync.sh
apt-get install rsync
su myp2p -c "ssh-keygen -t rsa" (Copy this key (/home/myp2p/.ssh/id_rsa.pub) to the core server)
./sync.sh
crontab -e (add the line "*/5 *  * * *     /home/myp2p/sync.sh" and the line "* *    * * *     find /home/myp2p/public_html/unicache/* -mmin +10 -exec rm -fr {} ;")