TheHippo

After leaving my last company I rented a small virtual server at 1und1. I took the XL vHost package which contains:

• 20 GB Raid 5 Storage
• 512 MB RAM, max 2 GB RAM
• 2 TB Traffic

At this time I thought it would be nice to install every thing by myself, so I choose a plain “Ubuntu 8.04 LTS 64 bit minimal” as operating system for my vHost. I installed Apache2, PHP5, MySQL and all the stuff I needed. It all works fine and I was really impressed by the speed of this virtual server.

Then I moved sportinleipzig.de, which I made during the time at my last company to this new server. This page does not have a very high traffic (200 unique visitors a day), but a lot of content (at least 5000 “static” pages). After 2 days the server crashed the first time, whereas crash means, that he is still alive, but does not respond to anything more then a simple ping. I restarted the server over the admin panel. Half a week later the server crashed the next time. Investigating the cause for these crashed I found out a few interesting things.

First a blamed mod_tora which I installed on this virtual server to check if it is ready to use for some production code. Then I deactivated eAccelerator, because I thought it could be possible that it has some memory leaks. I even deactivated Piwik which I use as a tracking system for all the visitors on the pages of this server. (It is very nice! If you need some visitor tracking, give it a try.) But the server still crashed ever few days.

Later I recognized that these crashes possibly occur when Google starts to crawl the page, as the page was not completely indexed anymore. So I started to do some load test on the server. A good page to start is loadimpact.com and the Apache command line tool ab, which both give you the possibility to run a lot of requests on you own server. Having a ssh session open and refreshing the status page of the Virtuozzo panel it became more and more clear. These kind of virtual server are very restricted how many resources they could consume and there a few limit that are easy to reach:

• kmemsize:
  

  The size of unswappable kernel memory allocated for the internal kernel structures for the processes of a particular VPS.

• shmpages:
  

  The total size of shared memory (including IPC, shared anonymous mappings and tmpfs objects) allocated by the processes of a particular VPS, in pages.

• numproc (maximum set to 128!):
  

  The maximal number of processes the VPS may create.

• numfile (maximum set to 8192, this is okay):
  

  The number of files opened by all VPS processes.

• numtcpsock and numothersock (both set to 720, means: no socket servers on virtual servers!):
  

  he number of TCP sockets (PF_INET family, SOCK_STREAM type). This parameter limits the number of TCP connections and, thus, the number of clients the server application can handle in parallel.

  The number of sockets other than TCP ones. Local (UNIX-domain) sockets are used for communications inside the system. UDP sockets are used, for example, for Domain Name Service (DNS) queries. UDP and other sockets may also be used in some very specialized applications (SNMP agents and others).

There are a lot of other limits set to the server, but the first three are these limits that made my server crash, when having tons of requests on the Apache.

I am using the Apache2 in the prefork mode (this is default if you install it via the Ubuntu repositories). This means the Apache spawns a few processes to handle the requests. If load raises more processes will be created, if load goes down unused processes will be destroyed. Forking new processes cost a little bit time so the Apache keeps some processes ready just in the case of load peak. During the time google crawls the page many processes where created to handle the load. The problem was that these processes consumed to much memory (probably this could be a memory leak in one of the PHP  scripts) and some of some became zombies, leading the virtual server to consume all resources and then crash.

The solution:
Edit you “/etc/apache2/apache2.conf”. Search for this section:

You could read in the comment of the config the impact of every parameter. The important one is the last one which is set to “0″ by default, which means that processes live as long as possible. Setting it to “500″, which is quite conservative, makes every process ends after serving 500 request. This also makes sure that all the allocated memory is freed. That causes that processes need to be created more often, so I also raised “MinSpareServers” and “MaxSpareServers”, which are the processes keept ready for load peak.
In conclusion my server spends a little more time on spawning new Apache processes, but this makes sure memory is cleaned more often and finally solved my problem!

P.S.: I don’t know if this is on all virtual servers, but the vHosts at 1und1 have a “/proc/”-entry for reading the limits of you server. You type into the terminal:

root@foo:~# cat /proc/user_beancounters

After leaving my last company I rented a small virtual server at 1und1. I took the XL vHost package which contains: 20 GB Raid 5 Storage 512 MB RAM, max 2 GB RAM 2 TB Traffic At this time I thought it would be nice to install every thing by myself, so I choose a plain "Ubuntu 8.04 ...