When you do some googling on having more than 4GB of RAM on a 32bit x86 Linux system you usually get a lot of responses like: it’s not possible, you won’t have it, you’d have only 3,5 or 3GB anyway, that there are 3/1, 2/2 and 1/3 splits but you’re not using it all, etc.. And the overall feeling I had from reading those (and actually hearing my colleagues at work) is that It’s not something that you should go with. That’s crazy and fortunately it’s not true. You can have more than 4GB RAM on 32bits and here’s how :-)
Due to some memory hungry apps on one of my Linux x86 servers (Ubuntu 8.04.1 32-bit) I decided to upgrade RAM from current 2GB to 4GB or more. That’s why last week I was first checking out if the motherboard and chipset can actually handle 4GB and more using dmidecode. They do, so now what?
Off to install more than 4GB of RAM on x86 Linux
Basically what you need to have is enabled support for PAE (Physical Address Extensions). This is a CPU feature that allows the Operating System to handle addressing space as long as 36 bits (instead of 32 bits) so it means a whopping 64GB. This is available in Intel Pentium Pro and above and in compatible AMD counterparts.
You also need that on the OS level. Linux kernel configuraton option that enables PAE is called HIGHMEM so you have to take care about that.
My very short procedure to get 4GB+ is (as tested with Ubuntu-server 8.04.1)
- First you gotta check your specs if the motherboard and chipset handle that.
- Now check your running kernel settings for highmem, eg:
$ grep “HIGHMEM” /boot/config-`uname -r`
# CONFIG_DEBUG_HIGHMEM is not set
# CONFIG_NOHIGHMEM is not set
# CONFIG_HIGHMEM4G is not set
The red lines are the ones that make you happy ;-) If you don’t have HIGHMEM enabled, unfortunatelly you need to recompile the kernel with these options set. I won’t go into details of recompiling as that’s a broad topic and distro-specific if you’re not going for the vanilla kernel.
- If you have it like that and HIGHMEM’s enabled then you can put your memory chips in and reboot and check with ‘free’ command, like:
$ free total used free shared buffers cached Mem: 6229504 5786976 442528 0 129332 4502492 -/+ buffers/cache: 1155152 5074352 Swap: 0 0 0
Here you can see a total of Mem at 6GB. Notably swap has been disabled here.
Things to consider
That seemed really easy but here’s a list of things you should know:
- Most servers have a few medium-sized apps, like: 6 apps eating a total of 5GB and that should work OK and that’s why we increase RAM, right? Of course there’s an issue of so-called split. If you have a 3G/1G split, one single process can address as much as 3GB. That’s actually the default in Ubuntu-server. This means that memory allocations for more would fail anyway. So in order to actually utilize vast amounts of memory the apps should also be operating in multiple processes or multiple threads. Like Apache, postgresql, bind9. And, sadly, unlike mysql.
- You can check your defined split at runtime with:
$ grep “VMSPLIT” /boot/config-`uname -r`
and look for something like: CONFIG_VMSPLIT_3G=y
- More and more Linux distributions have HIGHMEM enabled, but not all and you can’t expect always to have it.
- Example: In Ubuntu server you have it enabled and compiled in but in the desktop flavour you don’t. The easy way out is using a -server kernel but that way you will be missing on default CFQ I/O scheduler (possible to enable it though via kernel params at boot or at runtime via sysfs), using a different timer interrupt frequency and other features you might not be comfortable with.
The other way is to reconfig, recompile and install a new kernel
- With more RAM available you can surely rethink your swap space as you probably be needing less of that or even none and you could kill it completely to make sure you never swap.
- You can, as always, play with /proc/sys/vm/swappiness and set that eg, to 0.
- You may want to benchmark your system before and after breaking the 4GB barrier with PAE as some people say that this may cause degraded performance due to how highmem is mapped. But remember if you had heavy swapping you are better off with read/writes on memory than on disk.