Linux + Administrator = Linuxator

Linux Adminstrator’s blog

  • Blog Stats

    • 39,106 hits

Having more than 4GB of RAM on x86 Linux

Posted by Maciej Sołtysiak on November 7, 2008

When you do some googling on having more than 4GB of RAM on a 32bit x86 Linux system you usually get a lot of responses like: it’s not possible, you won’t have it, you’d have only 3,5 or 3GB anyway, that there are 3/1, 2/2 and 1/3 splits but you’re not using it all, etc.. And the overall feeling I had from reading those (and actually hearing my colleagues at work) is that It’s not something that you should go with. That’s crazy and fortunately it’s not true. You can have more than 4GB RAM on 32bits and here’s how :-)

Due to some memory hungry apps on one of my Linux x86 servers (Ubuntu 8.04.1 32-bit) I decided to upgrade RAM from current 2GB to 4GB or more. That’s why last week I was first checking out if the motherboard and chipset can actually handle 4GB and more using dmidecode. They do, so now what?

Off to install more than 4GB of RAM on x86 Linux

Basically what you need to have is enabled support for PAE (Physical Address Extensions). This is a CPU feature that allows the Operating System to handle addressing space as long as 36 bits (instead of 32 bits) so it means a whopping 64GB. This is available in Intel Pentium Pro and above and in compatible AMD counterparts.

You also need that on the OS level. Linux kernel configuraton option that enables PAE is called HIGHMEM so you have to take care about that.

My very short procedure to get 4GB+ is (as tested with Ubuntu-server 8.04.1)

  1. First you gotta check your specs if the motherboard and chipset handle that.
  2. Now check your running kernel settings for highmem, eg:
    $ grep “HIGHMEM” /boot/config-`uname -r`
    # CONFIG_DEBUG_HIGHMEM is not set
    CONFIG_HIGHMEM=y
    # CONFIG_NOHIGHMEM is not set
    # CONFIG_HIGHMEM4G is not set
    CONFIG_HIGHMEM64G=y
    The red lines are the ones that make you happy ;-) If you don’t have HIGHMEM enabled, unfortunatelly you need to recompile the kernel with these options set. I won’t go into details of recompiling as that’s a broad topic and distro-specific if you’re not going for the vanilla kernel.
  3. If you have it like that and HIGHMEM’s enabled then you can put your memory chips in and reboot and check with ‘free’ command, like:
    $ free
                 total       used       free     shared    buffers     cached
    Mem:       6229504    5786976     442528          0     129332    4502492
    -/+ buffers/cache:    1155152    5074352
    Swap:            0          0          0

    Here you can see a total of Mem at 6GB. Notably swap has been disabled here.

Things to consider

That seemed really easy but here’s a list of things you should know:

  • Most servers have a few medium-sized apps, like: 6 apps eating a total of 5GB and that should work OK and that’s why we increase RAM, right? Of course there’s an issue of so-called split. If you have a 3G/1G split, one single process can address as much as 3GB. That’s actually the default in Ubuntu-server. This means that memory allocations for more would fail anyway. So in order to actually utilize vast amounts of memory the apps should also be operating in multiple processes or multiple threads. Like Apache, postgresql, bind9. And, sadly, unlike mysql.
  • You can check your defined split at runtime with:
    $ grep “VMSPLIT” /boot/config-`uname -r`
    and look for something like: CONFIG_VMSPLIT_3G=y
  • More and more Linux distributions have HIGHMEM enabled, but not all and you can’t expect always to have it.
  • Example: In Ubuntu server you have it enabled and compiled in but in the desktop flavour you don’t. The easy way out is using a -server kernel but that way you will be missing on default CFQ I/O scheduler (possible to enable it though via kernel params at boot or at runtime via sysfs), using a different timer interrupt frequency and other features you might not be comfortable with.
    The other way is to reconfig, recompile and install a new kernel
  • With more RAM available you can surely rethink your swap space as you probably be needing less of that or even none and you could kill it completely to make sure you never swap.
  • You can, as always, play with /proc/sys/vm/swappiness and set that eg, to 0.
  • You may want to benchmark your system before and after breaking the 4GB barrier with PAE as some people say that this may cause degraded performance due to how highmem is mapped. But remember if you had heavy swapping you are better off with read/writes on memory than on disk.

Posted in Hardware, Kernel | Tagged: , , , , , | 1 Comment »

Using dmidecode to find out what memory chips you have

Posted by Maciej Sołtysiak on October 28, 2008

Every once in a while admins need to add more RAM to the server. If you don’t have the exact specs handy (not everyone has a CMDB to do a quick lookup) you need to somehow get the crucial information using software. Here’s how I do it using dmidecode.

The story

For starters it’s good to know your motherboard details. dmidecode can output data from many sections called DMI types. Actually if you take a look into man 8 dmidecode you’ll see that there are 39 of those, including things like Power Supply, OEM Strings, Processor, Chassis or Cache. There also is a type called Base Board Information. Its numeric type is 2 and it is actually very easy to output just that with the following command:

root@dns:~# dmidecode -t 2
# dmidecode 2.9
SMBIOS 2.4 present.

Handle 0x0005, DMI type 2, 16 bytes
Base Board Information
        Manufacturer: Intel
        Product Name: S3000AH
        Version: D40859-208
        Serial Number: AZAY73900054
        Asset Tag: Not Specified
        Features:
                Board is a hosting board
                Board is replaceable
        Location In Chassis: Not Specified
        Chassis Handle: 0x0000
        Type: Motherboard
        Contained Object Handles: 0

Here we can see that I have an Intel S3000AH motherboard. With that I can find out the exact specifications and see that it supports:

Dual memory-channel, four DIMM slots for DDR2
533/667 MHz
Unbuffered ECC/non-ECC DIMMs (8 GB Max)

So far so good, I know my limits, let’s sniff around some more for Physical Memory Array (type 16):

root@dns:~# dmidecode -t 16
# dmidecode 2.9
SMBIOS 2.4 present.

Handle 0x0022, DMI type 16, 15 bytes
Physical Memory Array
        Location: System Board Or Motherboard
        Use: System Memory
        Error Correction Type: Single-bit ECC
        Maximum Capacity: 8 GB
        Error Information Handle: Not Provided
        Number Of Devices: 4

This confirmed the ECC type, Maximum Capacity and number of memory banks/slots. Good! Now let’s see what chips are actually in there? Let’s look for Memory Device types.

root@dns:~# dmidecode -t 17
# dmidecode 2.9
SMBIOS 2.4 present.

Handle 0x0023, DMI type 17, 27 bytes
Memory Device
        Array Handle: 0x0022
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 1024 MB
        Form Factor: DIMM
        Set: None
        Locator: J8J1
        Bank Locator: CHAN A DIMM 1
        Type: DDR2
        Type Detail: Synchronous
        Speed: 533 MHz (1.9 ns)
        Manufacturer: 0x7F98000000000000
        Serial Number: 0x813625B6
        Asset Tag: Unknown
        Part Number: 0x393930353332312D3030312E4130334C4600

Handle 0x0025, DMI type 17, 27 bytes
Memory Device
        Array Handle: 0x0022
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: DIMM
        Set: None
        Locator: J8J2
        Bank Locator: CHAN A DIMM 2
        Type: DDR2
        Type Detail: None
        Speed: Unknown
        Manufacturer: NO DIMM
        Serial Number: NO DIMM
        Asset Tag: NO DIMM
        Part Number: NO DIMM

Handle 0x0026, DMI type 17, 27 bytes
Memory Device
        Array Handle: 0x0022
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 1024 MB
        Form Factor: DIMM
        Set: None
        Locator: J9J1
        Bank Locator: CHAN B DIMM 1
        Type: DDR2
        Type Detail: Synchronous
        Speed: 533 MHz (1.9 ns)
        Manufacturer: 0x7F98000000000000
        Serial Number: 0x82363EB6
        Asset Tag: Unknown
        Part Number: 0x393930353332312D3030312E4130334C4600

Handle 0x0028, DMI type 17, 27 bytes
Memory Device
        Array Handle: 0x0022
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: DIMM
        Set: None
        Locator: J9J2
        Bank Locator: CHAN B DIMM 2
        Type: DDR2
        Type Detail: None
        Speed: Unknown
        Manufacturer: NO DIMM
        Serial Number: NO DIMM
        Asset Tag: NO DIMM
        Part Number: NO DIMM

Now here we have 4 sections. First two are Channel A and the other two are Channel B. This output shows I have one 1GB 533 MHz DDR in the first bank of each channel, totalling 2GB of RAM. Now I can go shopping!

Conclussion

When you are in need of information about hardware specs on a Linux box, you can use dmidecode to fetch all you need without actually having to be near the machine or have any documentation. All the details I needed with RAM in this story could be easily provided by one dmidecode -t 2,16,17. Hope you find that useful. I did :-)

Posted in Hardware | Tagged: , | 18 Comments »

Approaching Linux kernel 2.6.27

Posted by Maciej Sołtysiak on September 27, 2008

In my last post about SLOC metrics, I measured it takes around 83 days for a new Linux kernel version to come up and right now I think we’re around a week from a new 2.6.27. Kernel 2.6.26 was released on 13 July 2008, so adding 83 days to that we get 10 October 2008 – that’s just next week!

With Linus releasing RC7 a week ago on 21st September, I think the release date will actually be as I predicted, but we’ll see about that. But that’s just one thing. I also forecasted that 2.6.27 will have 6,445 million lines of code – as opposed to 6,015 million lines of 2.6.26. That’s 300 thousand SLOC. Huge number, I personally find it hard to belive that it will be that much, but hey, the numbers said so last time I checked. I haven’t been checking the size of the recent RCs so it’ll be nice for me to check it out once 2.6.27 hits the repository. Anyway stay tuned as I’ll be posting new metrics for the 2.6.27 when it gets here :-) Which is soon!

Posted in Kernel | Tagged: , , , | Leave a Comment »

5 things you didn’t know about linux kernel code metrics

Posted by Maciej Sołtysiak on July 22, 2008

Recently Greg Kroah Hartman showed some very interesting Linux kernel development stats. I decided to do some too and the result are 5 cool things you probably didn’t know about the kernel code ;-)

These aren’t anything I’ve seen so far about the kernel.

Greg’s stats

First, let’s quickly summarize Greg’s findings related to kernel size (he also did a lot of work on who’s contributing. I’ll skip this here). Daily average (based on data from 2007-2008 period) is:

  • 4300 lines added, 1800 lines removed, 1500 lines modified
  • 3.69 changes per hour.

Greg also says that the kernel is growing 10% each year, with a current 9.2 million lines of which the biggest part are the drivers (55%). The core kernel is about 5%.

Those numbers seemed a little bit odd to me. Especially the 9 million lines. I wanted to check it myself. What I found out was that Greg wasn’t counting the pure source lines of code (SLOC), but all lines, he didn’t exclude comments and blank lines. That is why my metrics differ from his. It’s funny that the Wikipedia article on SLOC gives 5.2 million for 2.6.0 kernel, which also seems incorrect.

My stats

I started with writing a small script that:

  1. downloads a 2.6.0 kernel, analyzes it using SLOCCount written by David Wheeler
  2. patches to one step newer kernel and analyzes it using the same tool.
  3. goes to number 2 until patches run out at 2.6.26

Just in case I also used a different tool called cloc to analyze the same code. Word of comment on tools used is at the end of this post.

This script ate 477MB of disk space with tarballs and bzipped patches.

1. The kernel has just reached 6 millions lines with 2.6.26!

Linux kernel lines of code

Linux kernel lines of code

Yes, indeed, with 2.6.26 we’ve reached over 6 million lines of code. You can see that on the chart on the right (click for a normal size version).

Both SLOCCount and CLOC show similar results. What is interesting here is that:

  • there’s over a million of blank lines,
  • and a million lines of comments (which are of course important too),
  • the code shows a faster-than-linear growth characteristic,
  • if current speed is maintained I predict we might get 7 million with 2.6.30 and 8 million with 2.6.33, just look at the forecast.
Linux kernel lines of code forecast

Linux kernel lines of code forecast

2. It takes about 83 days (2¾ months) for a new kernel release!

As Greg Kroah-Hartman says, the current release scheme is solid and we’re getting an average of around 80-83 days between releases. That stability was starting around 2006 while the first 2.6 releases were more frequent and buggy. Here’s a graph and a table showing the numbers for the stable release cycle.

Days between releases of Linux  kernel

Days between releases of Linux kernel

3. Number of files in the project continues to grow faster than linear

This means that not only the size of current code grows but lots of new things come around. And this is true. Think of virtualization infrastructures, wireless, new architectures (eg. OLPC was merged recently).

File in Linux kernel source

File in Linux kernel source

First, look at the sheer number of files and how much they weigh in MBs. To the right, blue line represents all files in the directory, green line shows the number files that were analyzed by SLOCCount and CLOC. Not all files are analyzed because not all contain code. Anyway this give an idea of how many files people put in the source code.

Size of Linux kernel source code

Size of Linux kernel source code

Size is growing very rapidly too. Recent kernels grow with an average 6,3 MB. The record winning kernel is 2.5.25 which gained a whopping 13MB. If you take the 83 day lifecycle this means that it was gaining around 80 kB per day! (It’s not just code, documentation adds up to the numbers)

Top 8 directories in the kernel source

Top 8 directories in the kernel source

It is quite educating to look at exactly what’s growing per directory in therms of SLOC. If you take a look at top 8 directories within the kernel you can notice that:

  • drivers (/drivers) are a huge part and grow very quickly
  • arch (/arch) started growing around 2.6.5
  • network (/net) started growing around 2.6.13
  • filesystems (/fs) do not grow that much but they have their bursts like with 2.6.16, 2.6.19, where bulks of code where merged
  • network (/net) which stareted growing at 2.6.16, now outgrew sound (/sound)

I also did a graph with bottom with the LOC, but personally I don’t see it particuraly amusing, but here goes:

The rest of the directories

The rest of the directories

4. Daily SLOC added are over 1000 and this metric is also growing

LOC/day growth between versions

LOC/day growth between versions

The daily growth of SLOC for given releases varies of course. There are quite big differences between versions, however what can be certainly stated is that the trend is growing. Both the lower and upper bounds are at higher values with each new kernel.

Not incredily interesting but still, a metric is a metric and you can compare with other projects and your own programs ;-)

5. Language breakdown of 2.6.26 using CLOC

2 different reports.
== CLOC ==
-----------------------------------------------------
Language          files     blank   comment      code
-----------------------------------------------------
C                 10195    921822    976772   4709722
C/C++ Header       9400    203125    321830   1096551
Assembler          1005     36250     42921    225549
make               1005      4569      5350     15238
Perl                 19      1157      1256      6092
yacc                  5       437       318      2919
Bourne Shell         48       404      1205      2623
C++                   1       205        58      1496
lex                   5       225       248      1395
HTML                  2        58         0       367
NAnt scripts          1        83         0       290
Lisp                  1        63         0       218
Python                2        41        37       208
ASP                   1        33         0       136
awk                   2        14         7        98
Bourne Again Shell    2         7        17        34
XSLT                  1         0         1         7
-----------------------------------------------------
SUM:              21695   1168493   1350020   6062943
-----------------------------------------------------

== SLOCCount ==
ansic:		5780304	(96.08%)
asm:		218132	(3.63%)
perl:		6075	(0.10%)
cpp:		3242	(0.05%)
yacc:		2919	(0.05%)
sh:		2609	(0.04%)
lex:		1825	(0.03%)
python:		331	(0.01%)
lisp:		218	(0.00%)
pascal:		116	(0.00%)
awk:		96	(0.00%)

Word of comment on tools used

SLOCCount is very fast, CLOC is very slow (crunching over 10 hours with CLOC). The results of SLOC are similar, there’s a difference around 1% between them, so It’s neglible. The output results were processed and put into a CSV file and processed with JpGraph. Why JpGraph? Because I wanted to try it out, just that :-)

Cheers!

Links

  1. Linux Kernel Development Stats from Greg Kroah Hartman
  2. Greg Kroah Hartman on the Linux Kernel
  3. David Wheeler’s SLOCCount
  4. CLOC – Count Lines of Code
  5. JpGraph – Graph creating library for PHP

Posted in Kernel | Tagged: , , | 14 Comments »

 
Follow

Get every new post delivered to your Inbox.