Overcommit works by doubly allocating the same memory pointer to two applications and hoping one of them doesn’t actually write to that memory. it’s surprisingly common allegedly.

Anyway that’s why Torvald’s own git project uses the memset (…, size) function. Not to ensure it’s a certain default value…but to force the kernel into an OOM kill straight away…or not, rather than resulting in a hard to trace call stack.

To write portable code it is wise to check for a NULL upon a malloc return; however, on Linux, with the optimistic allocator enabled (which it is by default), malloc will never return a NULL. Since the memory page is allocated on a write to the pointer, the out-of-memory condition will cause the kernel to kill the process.

The proper action to take in an OOM condition is pretty much up to the application. Some applications can simply be restarted, but a restart is not the right course of action for health care or mission critical applications. I like getting a backtrace of an offending application, so I just let most of my applications just die to get a core file.

Am I missing something here?

You are wrong because SQLite does not return NULL to the user. It returns NULL to the other routines in the SQLite library which then gracefully handle it, sometimes returning the SQLITE_NOMEM error code. In general a developer using SQLite would never have any reason to call sqlite3_malloc directly. Additionally SQLite has several uses of memory such as as a cache, keeping an ongoing transaction in memory as much as possible before taking a lock on the database, scratch memory etc. It will never leak memory. You can read all the details at http://www.sqlite.org/malloc.html

djbdns is similar: dnscache allocates all of its cache memory up front, but needs some extra temporary memory for resolving a query and processing DNS response packets. If any of these requests fail, it will give up on just that individual query, clean up the temporary memory allocated, and print a warning message about it.

“”By the time malloc fails, the system is just plain fubared. I hate deeply convoluted error handling code that checks the return code of malloc….

“* Then goes through horrible contortions to attempt to recover sanity.
“* And has _never_ been tested.
“* And God help it if it either directly, or anywhere else on it’s call graph, invokes malloc again! Which it will if it tries to printf anything!

“So what is the correct solution?

“Wrap malloc in a function that checks the return code, and if it fails uses statically pre-allocated space to log a stack trace, then system error and reboot!

“But that sounds just like what you are ranting against! Hear me out to find the correct solution.

“What has gone wrong is you have exceeded the design load for that system. Suppose you were told to design a ferry to carry cars. But you couldn’t get management to decide how many cars it should carry. So you designed it so you could load any number of cars.

“When the ferry starts sinking and taking on water, on every gunwale there is a “water coming in” detector. Attached to each of those detectors is an amazingly complex one-off uniquely-designed gadget which you can’t test without sinking a loaded ferry (for many highly customised loads), that flings the last few cars into the water!

“One solution is to tell management to get their butts into gear and actually _decide_ on the designed for carrying capacity of the ferry. And then design mechanisms to only allow that many on.

“Your customers are OK with knowing that their system can only handle a finite load.”

http://linux-mm.org/OOM_Killer

Your error recovery code will never run. In case of real OOM error, your process may be killed (or another process will be killed to allocate memory for you).

Oh, and this may happen outside of malloc(). You get pointer to memory that isn’t allocated yet (page will be allocated when you access it).

If you wrote more posts about the inner workings of common projects, I for one wouldn’t complain!