Find the answer to your Linux question:
Results 1 to 2 of 2
First, I believe this is the correct forum for this issue, but if I am incorrect, I would greatly appreciate having this post moved to where it does belong. I'm ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Sep 2012
    Posts
    2

    Issue with oom-killer on SuSE Server - request assistance in solving


    First, I believe this is the correct forum for this issue, but if I am incorrect, I would greatly appreciate having this post moved to where it does belong.

    I'm tasked with helping to maintain our primary in-office production server. For two or three months, we had been having problems with the oom-killer activating on an almost nightly basis. A stopgap was instituted by carefully laying a pathway around our apparent culprit (the postgresql database setup). However, this feels like I've tried to put a bandage on a spurting wound; it makes the symptom go away, but the real problem is still there.

    I would appreciate any information on how I could solve this problem, either as direct changes I could make, or as links to further resources I might be able to use to discover what I am looking for.

    System information:
    Code:
    output from cat /etc/issue: Welcome to SUSE Linux Enterprise Server 11 SP1  (i586) - Kernel \r (\l).
    
    Output from uname -r: 2.6.32.46-0.3-pae
    
    Output from free command:
                 total       used       free     shared    buffers     cached
    Mem:      65248448   39242872   26005576          0      71640   39044220
    -/+ buffers/cache:     127012   65121436
    Swap:      2104472          0    2104472


    Sample log - taken from the blowup the first night I became aware of this issue. Information below is directly as reported in /var/log/warn:
    Code:
    Jul 26 23:38:09 CUSTDB kernel: [232078.493223] Comparifile.pl invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0
    Jul 26 23:38:09 CUSTDB kernel: [232078.493246] Pid: 14972, comm: Comparifile.pl Tainted: G           X 2.6.32.46-0.3-pae #1
    Jul 26 23:38:09 CUSTDB kernel: [232078.493252] Call Trace:
    Jul 26 23:38:09 CUSTDB kernel: [232078.493285]  [<c0206921>] try_stack_unwind+0x1b1/0x1f0
    Jul 26 23:38:09 CUSTDB kernel: [232078.493300]  [<c02058ef>] dump_trace+0x3f/0xe0
    Jul 26 23:38:09 CUSTDB kernel: [232078.493313]  [<c020652b>] show_trace_log_lvl+0x4b/0x60
    Jul 26 23:38:09 CUSTDB kernel: [232078.493324]  [<c0206558>] show_trace+0x18/0x20
    Jul 26 23:38:09 CUSTDB kernel: [232078.493337]  [<c0532ce9>] dump_stack+0x6d/0x74
    Jul 26 23:38:09 CUSTDB kernel: [232078.493352]  [<c029f2cf>] oom_kill_process+0x9f/0x2c0
    Jul 26 23:38:09 CUSTDB kernel: [232078.493364]  [<c029f9d3>] out_of_memory+0xf3/0x170
    Jul 26 23:38:09 CUSTDB kernel: [232078.493376]  [<c02a2a6c>] __alloc_pages_slowpath+0x41c/0x520
    Jul 26 23:38:09 CUSTDB kernel: [232078.493388]  [<c02a2c88>] __alloc_pages_nodemask+0x118/0x120
    Jul 26 23:38:09 CUSTDB kernel: [232078.493401]  [<c02cad05>] alloc_pages_current+0x65/0xc0
    Jul 26 23:38:09 CUSTDB kernel: [232078.493414]  [<c04d5313>] tcp_sendmsg+0x883/0x980
    Jul 26 23:38:09 CUSTDB kernel: [232078.493427]  [<c0493241>] sock_sendmsg+0xe1/0x120
    Jul 26 23:38:09 CUSTDB kernel: [232078.493437]  [<c049365e>] sys_sendto+0x11e/0x170
    Jul 26 23:38:09 CUSTDB kernel: [232078.493446]  [<c04936e7>] sys_send+0x37/0x40
    Jul 26 23:38:09 CUSTDB kernel: [232078.493455]  [<c0493e53>] sys_socketcall+0x203/0x2c0
    Jul 26 23:38:09 CUSTDB kernel: [232078.493465]  [<c02030c4>] sysenter_do_call+0x12/0x22
    Jul 26 23:38:09 CUSTDB kernel: [232078.493507]  [<ffffe424>] 0xffffe424
    Jul 26 23:38:12 CUSTDB kernel: [232078.493516] Node 0 DMA per-cpu:
    Jul 26 23:38:12 CUSTDB kernel: [232078.493523] CPU    0: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:12 CUSTDB kernel: [232078.493529] CPU    1: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:12 CUSTDB kernel: [232078.493534] CPU    2: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:12 CUSTDB kernel: [232078.493538] CPU    3: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:12 CUSTDB kernel: [232078.493543] CPU    4: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:12 CUSTDB kernel: [232078.493548] CPU    5: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:12 CUSTDB kernel: [232078.493553] CPU    6: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:12 CUSTDB kernel: [232078.493557] CPU    7: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:12 CUSTDB kernel: [232078.493562] CPU    8: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493567] CPU    9: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493572] CPU   10: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493577] CPU   11: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493581] CPU   12: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493586] CPU   13: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493591] CPU   14: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493596] CPU   15: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493600] CPU   16: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493605] CPU   17: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:14 CUSTDB kernel: [232078.493610] CPU   18: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493615] CPU   19: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493619] CPU   20: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493624] CPU   21: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493629] CPU   22: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493633] CPU   23: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493638] CPU   24: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493643] CPU   25: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493648] CPU   26: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493652] CPU   27: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:16 CUSTDB kernel: [232078.493657] CPU   28: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493662] CPU   29: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493666] CPU   30: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493671] CPU   31: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493676] CPU   32: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493681] CPU   33: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493685] CPU   34: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493690] CPU   35: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493695] CPU   36: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493700] CPU   37: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:17 CUSTDB kernel: [232078.493705] CPU   38: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493709] CPU   39: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493714] CPU   40: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493719] CPU   41: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493724] CPU   42: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493728] CPU   43: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493733] CPU   44: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493738] CPU   45: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493743] CPU   46: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493747] CPU   47: hi:    0, btch:   1 usd:   0
    Jul 26 23:38:18 CUSTDB kernel: [232078.493751] Node 0 Normal per-cpu:
    Jul 26 23:38:22 CUSTDB kernel: [232078.493758] CPU    0: hi:  186, btch:  31 usd: 182
    Jul 26 23:38:22 CUSTDB kernel: [232078.493763] CPU    1: hi:  186, btch:  31 usd: 164
    Jul 26 23:38:22 CUSTDB kernel: [232078.493768] CPU    2: hi:  186, btch:  31 usd:  65
    Jul 26 23:38:22 CUSTDB kernel: [232078.493773] CPU    3: hi:  186, btch:  31 usd:  52
    Jul 26 23:38:22 CUSTDB kernel: [232078.493777] CPU    4: hi:  186, btch:  31 usd:  27
    Jul 26 23:38:22 CUSTDB kernel: [232078.493782] CPU    5: hi:  186, btch:  31 usd: 177
    Jul 26 23:38:22 CUSTDB kernel: [232078.493787] CPU    6: hi:  186, btch:  31 usd:  22
    Jul 26 23:38:22 CUSTDB kernel: [232078.493791] CPU    7: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:22 CUSTDB kernel: [232078.493796] CPU    8: hi:  186, btch:  31 usd:  28
    Jul 26 23:38:22 CUSTDB kernel: [232078.493801] CPU    9: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:23 CUSTDB kernel: [232078.493805] CPU   10: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:23 CUSTDB kernel: [232078.493810] CPU   11: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:23 CUSTDB kernel: [232078.493815] CPU   12: hi:  186, btch:  31 usd:  37
    Jul 26 23:38:23 CUSTDB kernel: [232078.493819] CPU   13: hi:  186, btch:  31 usd:   4
    Jul 26 23:38:23 CUSTDB kernel: [232078.493824] CPU   14: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:23 CUSTDB kernel: [232078.493828] CPU   15: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:23 CUSTDB kernel: [232078.493833] CPU   16: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:23 CUSTDB kernel: [232078.493838] CPU   17: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:23 CUSTDB kernel: [232078.493842] CPU   18: hi:  186, btch:  31 usd:  11
    Jul 26 23:38:23 CUSTDB kernel: [232078.493847] CPU   19: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:29 CUSTDB kernel: [232078.493851] CPU   20: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:29 CUSTDB kernel: [232078.493856] CPU   21: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:29 CUSTDB kernel: [232078.493861] CPU   22: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:29 CUSTDB kernel: [232078.493865] CPU   23: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:29 CUSTDB kernel: [232078.493870] CPU   24: hi:  186, btch:  31 usd: 118
    Jul 26 23:38:29 CUSTDB kernel: [232078.493874] CPU   25: hi:  186, btch:  31 usd:  72
    Jul 26 23:38:29 CUSTDB kernel: [232078.493879] CPU   26: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:29 CUSTDB kernel: [232078.493884] CPU   27: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:29 CUSTDB kernel: [232078.493888] CPU   28: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:29 CUSTDB kernel: [232078.493893] CPU   29: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493898] CPU   30: hi:  186, btch:  31 usd:  28
    Jul 26 23:38:32 CUSTDB kernel: [232078.493902] CPU   31: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493907] CPU   32: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493911] CPU   33: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493916] CPU   34: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493921] CPU   35: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493925] CPU   36: hi:  186, btch:  31 usd: 180
    Jul 26 23:38:32 CUSTDB kernel: [232078.493930] CPU   37: hi:  186, btch:  31 usd: 137
    Jul 26 23:38:32 CUSTDB kernel: [232078.493935] CPU   38: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493939] CPU   39: hi:  186, btch:  31 usd: 176
    Jul 26 23:38:32 CUSTDB kernel: [232078.493944] CPU   40: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493948] CPU   41: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493953] CPU   42: hi:  186, btch:  31 usd:  96
    Jul 26 23:38:32 CUSTDB kernel: [232078.493958] CPU   43: hi:  186, btch:  31 usd:  33
    Jul 26 23:38:32 CUSTDB kernel: [232078.493962] CPU   44: hi:  186, btch:  31 usd:  29
    Jul 26 23:38:32 CUSTDB kernel: [232078.493967] CPU   45: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493971] CPU   46: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493976] CPU   47: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.493980] Node 0 HighMem per-cpu:
    Jul 26 23:38:32 CUSTDB kernel: [232078.493986] CPU    0: hi:  186, btch:  31 usd: 177
    Jul 26 23:38:32 CUSTDB kernel: [232078.493991] CPU    1: hi:  186, btch:  31 usd: 109
    Jul 26 23:38:32 CUSTDB kernel: [232078.493995] CPU    2: hi:  186, btch:  31 usd:  69
    Jul 26 23:38:32 CUSTDB kernel: [232078.494000] CPU    3: hi:  186, btch:  31 usd:  21
    Jul 26 23:38:32 CUSTDB kernel: [232078.494005] CPU    4: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494010] CPU    5: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494015] CPU    6: hi:  186, btch:  31 usd:   4
    Jul 26 23:38:32 CUSTDB kernel: [232078.494019] CPU    7: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494024] CPU    8: hi:  186, btch:  31 usd:  15
    Jul 26 23:38:32 CUSTDB kernel: [232078.494029] CPU    9: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494034] CPU   10: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494038] CPU   11: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494043] CPU   12: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494048] CPU   13: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494053] CPU   14: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494057] CPU   15: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494062] CPU   16: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494067] CPU   17: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494072] CPU   18: hi:  186, btch:  31 usd:  15
    Jul 26 23:38:32 CUSTDB kernel: [232078.494077] CPU   19: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:32 CUSTDB kernel: [232078.494081] CPU   20: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494086] CPU   21: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494091] CPU   22: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494096] CPU   23: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494100] CPU   24: hi:  186, btch:  31 usd:  15
    Jul 26 23:38:33 CUSTDB kernel: [232078.494105] CPU   25: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494110] CPU   26: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494115] CPU   27: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494119] CPU   28: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494124] CPU   29: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494129] CPU   30: hi:  186, btch:  31 usd:  40
    Jul 26 23:38:33 CUSTDB kernel: [232078.494134] CPU   31: hi:  186, btch:  31 usd:  27
    Jul 26 23:38:33 CUSTDB kernel: [232078.494138] CPU   32: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494143] CPU   33: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494148] CPU   34: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494153] CPU   35: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494157] CPU   36: hi:  186, btch:  31 usd:  59
    Jul 26 23:38:33 CUSTDB kernel: [232078.494162] CPU   37: hi:  186, btch:  31 usd:  83
    Jul 26 23:38:33 CUSTDB kernel: [232078.494167] CPU   38: hi:  186, btch:  31 usd:  29
    Jul 26 23:38:33 CUSTDB kernel: [232078.494172] CPU   39: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494177] CPU   40: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494181] CPU   41: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494186] CPU   42: hi:  186, btch:  31 usd:  70
    Jul 26 23:38:33 CUSTDB kernel: [232078.494191] CPU   43: hi:  186, btch:  31 usd: 106
    Jul 26 23:38:33 CUSTDB kernel: [232078.494196] CPU   44: hi:  186, btch:  31 usd:  75
    Jul 26 23:38:33 CUSTDB kernel: [232078.494201] CPU   45: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494205] CPU   46: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494210] CPU   47: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494214] Node 1 HighMem per-cpu:
    Jul 26 23:38:33 CUSTDB kernel: [232078.494220] CPU    0: hi:  186, btch:  31 usd:  33
    Jul 26 23:38:33 CUSTDB kernel: [232078.494225] CPU    1: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494230] CPU    2: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494234] CPU    3: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494239] CPU    4: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494244] CPU    5: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494249] CPU    6: hi:  186, btch:  31 usd:  25
    Jul 26 23:38:33 CUSTDB kernel: [232078.494254] CPU    7: hi:  186, btch:  31 usd:  44
    Jul 26 23:38:33 CUSTDB kernel: [232078.494258] CPU    8: hi:  186, btch:  31 usd:  28
    Jul 26 23:38:33 CUSTDB kernel: [232078.494263] CPU    9: hi:  186, btch:  31 usd:  31
    Jul 26 23:38:33 CUSTDB kernel: [232078.494268] CPU   10: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494273] CPU   11: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494278] CPU   12: hi:  186, btch:  31 usd:  17
    Jul 26 23:38:33 CUSTDB kernel: [232078.494282] CPU   13: hi:  186, btch:  31 usd:  29
    Jul 26 23:38:33 CUSTDB kernel: [232078.494287] CPU   14: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494292] CPU   15: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494297] CPU   16: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494301] CPU   17: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494306] CPU   18: hi:  186, btch:  31 usd:   8
    Jul 26 23:38:33 CUSTDB kernel: [232078.494311] CPU   19: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494315] CPU   20: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:33 CUSTDB kernel: [232078.494320] CPU   21: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494325] CPU   22: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494330] CPU   23: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494335] CPU   24: hi:  186, btch:  31 usd:   2
    Jul 26 23:38:34 CUSTDB kernel: [232078.494339] CPU   25: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494344] CPU   26: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494348] CPU   27: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494353] CPU   28: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494358] CPU   29: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494363] CPU   30: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494367] CPU   31: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494372] CPU   32: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494377] CPU   33: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494382] CPU   34: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494386] CPU   35: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494391] CPU   36: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494396] CPU   37: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494401] CPU   38: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494405] CPU   39: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494410] CPU   40: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494415] CPU   41: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494420] CPU   42: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494424] CPU   43: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494429] CPU   44: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494434] CPU   45: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494439] CPU   46: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494444] CPU   47: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494448] Node 2 HighMem per-cpu:
    Jul 26 23:38:34 CUSTDB kernel: [232078.494453] CPU    0: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494458] CPU    1: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494462] CPU    2: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494467] CPU    3: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494472] CPU    4: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494477] CPU    5: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494482] CPU    6: hi:  186, btch:  31 usd:   5
    Jul 26 23:38:34 CUSTDB kernel: [232078.494487] CPU    7: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494491] CPU    8: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494496] CPU    9: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494501] CPU   10: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494506] CPU   11: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494510] CPU   12: hi:  186, btch:  31 usd:  48
    Jul 26 23:38:34 CUSTDB kernel: [232078.494515] CPU   13: hi:  186, btch:  31 usd:  34
    Jul 26 23:38:34 CUSTDB kernel: [232078.494520] CPU   14: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494525] CPU   15: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494529] CPU   16: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494534] CPU   17: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494539] CPU   18: hi:  186, btch:  31 usd:   4
    Jul 26 23:38:34 CUSTDB kernel: [232078.494544] CPU   19: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494548] CPU   20: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494553] CPU   21: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:34 CUSTDB kernel: [232078.494558] CPU   22: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494563] CPU   23: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494567] CPU   24: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494572] CPU   25: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494577] CPU   26: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494581] CPU   27: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494586] CPU   28: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494591] CPU   29: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494596] CPU   30: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494600] CPU   31: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494605] CPU   32: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494610] CPU   33: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494615] CPU   34: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494619] CPU   35: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494624] CPU   36: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494629] CPU   37: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494634] CPU   38: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494639] CPU   39: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494643] CPU   40: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494648] CPU   41: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494653] CPU   42: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494657] CPU   43: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494662] CPU   44: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494667] CPU   45: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494672] CPU   46: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494677] CPU   47: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494681] Node 3 HighMem per-cpu:
    Jul 26 23:38:35 CUSTDB kernel: [232078.494686] CPU    0: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494691] CPU    1: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494695] CPU    2: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494700] CPU    3: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494705] CPU    4: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494710] CPU    5: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494715] CPU    6: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494719] CPU    7: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494724] CPU    8: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494729] CPU    9: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494734] CPU   10: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494738] CPU   11: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494743] CPU   12: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494748] CPU   13: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494753] CPU   14: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494757] CPU   15: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494762] CPU   16: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494767] CPU   17: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494772] CPU   18: hi:  186, btch:  31 usd:  90
    Jul 26 23:38:35 CUSTDB kernel: [232078.494776] CPU   19: hi:  186, btch:  31 usd:  56
    Jul 26 23:38:35 CUSTDB kernel: [232078.494781] CPU   20: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494786] CPU   21: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494791] CPU   22: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:35 CUSTDB kernel: [232078.494795] CPU   23: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494800] CPU   24: hi:  186, btch:  31 usd:  16
    Jul 26 23:38:36 CUSTDB kernel: [232078.494805] CPU   25: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494810] CPU   26: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494814] CPU   27: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494819] CPU   28: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494824] CPU   29: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494828] CPU   30: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494833] CPU   31: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494838] CPU   32: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494843] CPU   33: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494847] CPU   34: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494852] CPU   35: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494857] CPU   36: hi:  186, btch:  31 usd:  40
    Jul 26 23:38:36 CUSTDB kernel: [232078.494862] CPU   37: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494867] CPU   38: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494871] CPU   39: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494876] CPU   40: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494881] CPU   41: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494886] CPU   42: hi:  186, btch:  31 usd:  15
    Jul 26 23:38:36 CUSTDB kernel: [232078.494890] CPU   43: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494895] CPU   44: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494900] CPU   45: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494905] CPU   46: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494909] CPU   47: hi:  186, btch:  31 usd:   0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494924] active_anon:17768 inactive_anon:1251 isolated_anon:0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494927]  active_file:4037537 inactive_file:2268506 isolated_file:0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494930]  unevictable:0 dirty:1 writeback:0 unstable:0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494932]  free:9927201 slab_reclaimable:43193 slab_unreclaimable:7277
    Jul 26 23:38:36 CUSTDB kernel: [232078.494935]  mapped:6357 shmem:2350 pagetables:616 bounce:0
    Jul 26 23:38:36 CUSTDB kernel: [232078.494942] Node 0 DMA free:3468kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB active_file:16kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15796kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:5672kB slab_unreclaimable:56kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:5 all_unreclaimable? no
    Jul 26 23:38:36 CUSTDB kernel: [232078.494966] lowmem_reserve[]: 0 857 16100 16100
    Jul 26 23:38:36 CUSTDB kernel: [232078.494975] Node 0 Normal free:3624kB min:3712kB low:4640kB high:5568kB active_anon:0kB inactive_anon:0kB active_file:648kB inactive_file:484kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:877816kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:167100kB slab_unreclaimable:29052kB kernel_stack:5440kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1311 all_unreclaimable? yes
    Jul 26 23:38:36 CUSTDB kernel: [232078.495001] lowmem_reserve[]: 0 0 121946 121946
    Jul 26 23:38:36 CUSTDB kernel: [232078.495009] Node 0 HighMem free:2086260kB min:512kB low:17016kB high:33524kB active_anon:51496kB inactive_anon:4916kB active_file:9036536kB inactive_file:4546360kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15609140kB mlocked:0kB dirty:4kB writeback:0kB mapped:19716kB shmem:8084kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:1864kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
    Jul 26 23:38:36 CUSTDB kernel: [232078.495035] lowmem_reserve[]: 0 0 0 0
    Jul 26 23:38:36 CUSTDB kernel: [232078.495043] Node 1 HighMem free:9324924kB min:512kB low:17972kB high:35436kB active_anon:11040kB inactive_anon:8kB active_file:3969668kB inactive_file:3335940kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:16514064kB mlocked:0kB dirty:0kB writeback:0kB mapped:1708kB shmem:296kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:244kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
    Jul 26 23:38:36 CUSTDB kernel: [232078.495068] lowmem_reserve[]: 0 0 0 0
    Jul 26 23:38:36 CUSTDB kernel: [232078.495076] Node 2 HighMem free:13723212kB min:512kB low:17972kB high:35436kB active_anon:5264kB inactive_anon:68kB active_file:2082736kB inactive_file:830996kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:16514064kB mlocked:0kB dirty:0kB writeback:0kB mapped:2788kB shmem:920kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:116kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
    Jul 26 23:38:36 CUSTDB kernel: [232078.495101] lowmem_reserve[]: 0 0 0 0
    Jul 26 23:38:36 CUSTDB kernel: [232078.495108] Node 3 HighMem free:14567316kB min:512kB low:17292kB high:34076kB active_anon:3272kB inactive_anon:12kB active_file:1060544kB inactive_file:360244kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15869920kB mlocked:0kB dirty:0kB writeback:0kB mapped:1216kB shmem:100kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:240kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
    Jul 26 23:38:36 CUSTDB kernel: [232078.495134] lowmem_reserve[]: 0 0 0 0
    Jul 26 23:38:36 CUSTDB kernel: [232078.495141] Node 0 DMA: 61*4kB 43*8kB 16*16kB 8*32kB 3*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3468kB
    Jul 26 23:38:36 CUSTDB kernel: [232078.495162] Node 0 Normal: 630*4kB 146*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3688kB
    Jul 26 23:38:36 CUSTDB kernel: [232078.495182] Node 0 HighMem: 27*4kB 47011*8kB 28784*16kB 8845*32kB 2717*64kB 583*128kB 287*256kB 1081*512kB 79*1024kB 1*2048kB 2*4096kB = 2086372kB
    Jul 26 23:38:36 CUSTDB kernel: [232078.495204] Node 1 HighMem: 42*4kB 44*8kB 53*16kB 21*32kB 8*64kB 4*128kB 4*256kB 3*512kB 1*1024kB 2*2048kB 2274*4096kB = 9325048kB
    Jul 26 23:38:36 CUSTDB kernel: [232078.495224] Node 2 HighMem: 45*4kB 52*8kB 36*16kB 24*32kB 7*64kB 4*128kB 3*256kB 2*512kB 1*1024kB 2*2048kB 3348*4096kB = 13723220kB
    Jul 26 23:38:36 CUSTDB kernel: [232078.495244] Node 3 HighMem: 27*4kB 27*8kB 36*16kB 18*32kB 4*64kB 3*128kB 2*256kB 3*512kB 294*1024kB 376*2048kB 3294*4096kB = 14567492kB
    Jul 26 23:38:36 CUSTDB kernel: [232078.495266] 6308398 total pagecache pages
    Jul 26 23:38:36 CUSTDB kernel: [232078.495270] 7 pages in swap cache
    Jul 26 23:38:36 CUSTDB kernel: [232078.495275] Swap cache stats: add 109, delete 102, find 28830/28831
    Jul 26 23:38:36 CUSTDB kernel: [232078.495280] Free swap  = 2104228kB
    Jul 26 23:38:36 CUSTDB kernel: [232078.495283] Total swap = 2104472kB
    Jul 26 23:38:37 CUSTDB kernel: [232079.281645] No available memory (MPOL_BIND): kill process 14972 (Comparifile.pl) score 0 or a child
    Jul 26 23:38:37 CUSTDB kernel: [232079.281652] Killed process 14972 (Comparifile.pl)
    I have already found the recommendation for, and tried, setting /proc/sys/vm/overcommit_memory to 2; it does not seem to have any effect. If a process is running against our database for too long, the oom-killer still strikes. The most common culprits are the associated slony replication package, and the weekly full-database vacuum.

    I have been informed on another forum that Node 0 Normal is a probable issue here, as it has a very limited size showing. Given that it's displayed 'free' value in the above log is underneath the displayed 'min' value, I am inclined to agree, but I am no expert.

    My first question is this: Is there a reasonable way to look at the status of the system memory during normal run time, formattable in a manner similar to the above error log's display? I'd like to be able to see what Node 0 Normal looks like under ordinary circumstances.

    My second question is this, I guess: If anyone else has any better ideas than I do as to how to handle the oom-killer issue, on this system, would you please be willing to present them to me?

    Thank you for your assistance,
    EnderX

  2. #2
    Just Joined!
    Join Date
    Sep 2012
    Location
    Finland
    Posts
    96
    This page will give you some insight to your problem: (copy and paste into your browser)
    linux-mm.org/OOM_Killer

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •