Kernel OOM from cache not being dropped

Sorry if this has been discussed. I did a search on the forum and did not find the answer. If your search-fu is better than mine, please point me in the right direction.

Here's the problem. Embedded system, no swap. Cache eats up all available RAM during heavy usage. This is not a problem. As soon as user-space program allocates memory, cache gives it up. Everything works as advertised.

However, there's a 3rd party device driver that for certain operations, tries to allocate a 5th order page of physical memory. Well buddyinfo shows there are no 5th order page available -- OOM.

But, as soon as I force a drop cache, plenty becomes available, and the operation proceeds as normal.

So it would seem that when allocating virtual memory, cache is dropped to make that happen. When allocating physical memory, cache doesn't get dropped? That doesn't make sense, because then a full cache will make kernel drivers that require contiguous physical memory to fail, which I think is worse than having slow disk access.

Kernel is 2.6.36, memory compaction is enabled.

Perhaps there's some tuning parameter that I have to set?

Thanks!

It sounds like a driver bug to me. The driver should detect the OOM situation and ask the kernel to release adequate cache. I'm not sure what the instructions are for that. You could make the argument that the kernel should deal with that, but the answer to that is "it depends"... :-) It depends upon what mechanisms the driver is using to allocate memory, and what errors it is returning to the kernel itself.

Of course, not knowing what device and driver you are using, and not having any of the associated source code to examine, this is all just speculative.