c - malloc() in Linux - "there is no guarantee that the memory really is available"?

Question

I'm making a game where the world is divided into chunks of data describing the world. I keep the chunks in a dynamically allocated array so I have to use malloc() when initializing the world's data structures.

Reading the malloc() man page, there is a Note as follows:

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer. For more information, see the description of /proc/sys/vm/overcommit_memory and /proc/sys/vm/oom_adj in proc(5), and the Linux kernel source file Documentation/vm/overcommit-accounting.

If Linux is set to use optimistic memory allocation then does this mean it doesn't always return the full amount of memory I requested in the call to malloc()?

I read that optimistic memory allocation an be disabled by modifying the kernel, but I don't want to do that.

So is there a way to check whether the program has allocated the requested amount?

Answer 1

This is not something you need to deal with from an application perspective. Users who don't want random processes killed by the "OOM killer" will disable overcommit themselves via

echo "2" > /proc/sys/vm/overcommit_memory

This is their choice, not yours.

But from another standpoint, it doesn't matter. Typical "recommended" amounts of swap are so ridiculous that no reasonable amount of malloc is going to fail to have physical storage to back it. However, you could easily allocate so much (even with forced MAP_POPULATE or manually touching it all) to keep the system thrashing swap for hours/days/weeks. There is no canonical way to ask the system to notify you and give an error if the amount of memory you want is going to bog down the system swapping.

The whole situation is a mess, but as an application developer, your role in the fix is just to use malloc correctly and check for a null return value. The rest of the responsibility is on distributions and the kernel maintainers.