c - Virtual Memory size on Linux

Question

I'm trying to understand in depth the virtual memory management on OS Linux.

I don't really understand how the OS determine the size of the VM for a process.

I know that an 32-bits x86 OS can give up to 3 GB of vm adress space... Is it always true ?

In my case, i have about 110 MB physical memory with and 32-bits Linux, and my main process have a vm adress space about 660 MB. However, only 50 MB are in physical memory (the RSS of my process), so my physical RAM isn't full. The rest is free and almost the whole is used by page cache. That's seems to be a normal behaviour.

If i check /proc/my_process_PID/smap, there are several 8 MB anonymous VMA.

My actual problem is that i need to make a additionnal 10 MB malloc in the code, but unfortunately OOM-Killer kills my process (out-of-memory) ... I think there are no more free available pages in the vm for the Heap, isn't it ? Is there an huge memory leak somewhere ?

Why the OS doesn't extend my process vm size so ?

For information the vm size is unlimited : ulimit -v : unlimited

Answer 1

You can have 3GB of virtual memory per process (approximately, on many 32-bit Linux), and keep on creating new processes taking up gigabytes upon gigabytes of virtual memory. There's a small overhead in the kernel, but virtual memory is very cheap. The amount of address space you're using is probably not important, and it probably won't trigger the OOM killer.

However, your system only has so much RAM. As you start using pages in your address space (writing to them), the kernel is forced to find physical RAM to map them to. If there's no physical RAM, the kernel can evict other pages from RAM -- either swap them out or discard them. But if it can't evict any pages, then it triggers the OOM killer.

Running out of address space will cause malloc to return NULL on my system, instead of triggering the OOM killer.

It sounds like your process is simply using too much RAM. RSS isn't the amount of memory your process uses, it's just the amount that's in physical RAM right now. If your process has a memory leak and keeps growing, RSS will eventually stop growing -- because for every new page you use, the kernel will evict one page from your process.

Try using a memory profiler, like Valgrind. This will help you sort out what memory you should be worried about (mallocs) and what memory you can ignore (shared libraries and other memory mapped files). The kernel (and /proc) won't give you enough detail.