Find the answer to your Linux question:
Results 1 to 10 of 10
Like Tree1Likes
  • 1 Post By elija
Hi all, I have a machine that I use to build images for use on a plug computer. Since it takes a little while to build these images I started ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Linux User jkwilborn's Avatar
    Join Date
    Nov 2012
    Posts
    285

    Lots of unused memory!


    Hi all, I have a machine that I use to build images for use on a plug computer. Since it takes a little while to build these images I started watching the system monitor to see how the resources were being used. I was quite surprised to see that only about 1.7% of my memory was being used. So I watched more closely and sure enough it ranged from 1.1 to 1.7%.

    I have 32 GB of memory on this thing and I wish to use it, so does anyone know what to change or tweak to make this happen? I've changed 'swappiness' to 10 but no change in memory usage. Since I'm doing compiles, linkage and scripts I would think it would run much faster using more memory.

    Should I post this elsewhere?

    Any suggestions?

    Thanks

    Jack

  2. #2
    Linux User zenwalker's Avatar
    Join Date
    Feb 2010
    Location
    Inland Empire
    Posts
    298
    How large are your image files and what distro and software is being used to process them?
    Also, knowing specifics about hardware and if images are multi-band, 32-bit color, etc would be useful.
    Personally, I woud be grateful to use only approcimately 550MB of my RAM to obtain the image desired!
    Why the concern?
    "What you think about me is none of my business"
    _______________________________________________
    mint 17 xfce | HandyLinux | Slacko

  3. #3
    Penguin of trust elija's Avatar
    Join Date
    Jul 2004
    Location
    Either at home or at work or down the pub
    Posts
    3,651
    Maybe that's all it needs. Does the free command show any swap usage while (or after) creating the images?
    "I used to be with it, then they changed what it was.
    Now what was it isn't it, and what is it is weird and scary to me.
    It'll happen to you too."

    Grandpa Simpson



    The Fifth Continent

  4. $spacer_open
    $spacer_close
  5. #4
    Linux User jkwilborn's Avatar
    Join Date
    Nov 2012
    Posts
    285
    I'm using Debian Linux, 3.2.0-4-686-pae, and it states that all the build tools are up to date, but I don't know specifics. The resultant image is right around 2GB as we are building a Debian Linux Image to run on the plug-computer so it's Wheezy that we are building. It does swap a lot, but I'm not sure where to get specifics on any of these items. I've watched the 'system monitor' to get some idea of what's going on and I consistently have 31+ GB of memory free. Seems it should be using more than that and swapping less?

    The run usually takes a half hour or so, but I have lots of ram, and it's not being used. I was just wondering if there are ways to allow more ram or restrict it so I can check these and see if they are reducing the ram usage and letting it swap. The swappiness didn't seem to make any difference. Maybe I should set it to 1 or 0 and see what happens?

    I've been in computers many years and when they tune them they always seem to set them up to have little left during the run, which seems right to me as using the most memory and less swapping. Since ram is so much faster and I have it installed, I wish to use it and I'm finding that the system tools are not showing much use if any.

    That's my concerns,. If you want anything specific, you need to address how to obtain what you desire. I'm relatively new on Linux and finding some specifics on this is an area I have little experience with.

    Thanks all,

    Jack

    This is an adendum, I set the monitor tools to show swap and it looks like it's not really swapping anything in or out. I can't believe that can do all of this in 1/2 MB! Maybe they can. I still think there must be an option to use more memory or a limit size that can be changed. I guess that's why there is no difference when I set swappiness to 10!

    Jack
    Last edited by jkwilborn; 05-16-2013 at 04:11 PM.

  6. #5
    Penguin of trust elija's Avatar
    Join Date
    Jul 2004
    Location
    Either at home or at work or down the pub
    Posts
    3,651
    A couple of possibilities:

    1. Turn your swap off by running
    Code:
    swapoff -a
    as root. You can turn it back on using the
    Code:
    swapon -a
    command, also as root.

    2. Build your images in /tmp which you can mount in RAM.

    As root, edit /etc/fstab and add tmpfs /tmp tmpfs size=4096m 0 0 to the end of it on a blank line. You can change the 4096 to any size you want. This allocates 4GB. Once you have saved the file run
    Code:
    mount -a
    as root.

    Building your images to /tmp will effectively be building them to RAM; with the added bonus of everything else written to /tmp also being blindingly fast! On the down side, you will have to copy or move the image elsewhere once it is built. It might still be quicker than writing the image directly to hard disk.

    Back up your fstab before making any changes as you can completely break your system by getting it wrong. With a backup, it is nothing a live CD can't fix!
    zenwalker likes this.
    "I used to be with it, then they changed what it was.
    Now what was it isn't it, and what is it is weird and scary to me.
    It'll happen to you too."

    Grandpa Simpson



    The Fifth Continent

  7. #6
    Linux User jkwilborn's Avatar
    Join Date
    Nov 2012
    Posts
    285
    Nice, I like that. I will give that a try and see if I can see an decrease in time required. It was right in front of me and I didn't even think about a ramdisk.

    Thanks...

    Jack

  8. #7
    Linux User jkwilborn's Avatar
    Join Date
    Nov 2012
    Posts
    285
    Created the swap file as you suggested. It is much quicker, went from about 40 minutes to little less than 30 for a build. But as usual I seem to have gobs of memory left over that is never used.

    I just scanned an 8 X 10 color photo that produces a 1.4 GB tiff file. When I try to open the file I get an error from some packages that it fails because of lack of memory. I'm not exactly sure of what it's saying, but it does not let me view the image. I do have a listing of the /proc/meminfo if that is of any help to anyone. I don't seem to see anything that tells me what's happening except that it seems to know I have 32 GB. Does anybody see anything that would tip them off as to why usage of main memory is shown as low? The highest is usually only a couple of percent.

    I have heard of people rebuilding the kernel with options showing a different memory limit, but mine is being shown by the OS as all there so something must be limiting it use somewhere. This is the only OS I've ever had this problem with! It would be nice to be able to use some of the applications to edit these large tiff or jpeg files (or even view them!)


    Code:
    # cat /proc/meminfo
    MemTotal:       33153104 kB
    MemFree:        27243036 kB
    Buffers:          159596 kB
    Cached:          4986188 kB
    SwapCached:            0 kB
    Active:          4757288 kB
    Inactive:         997656 kB
    Active(anon):     702120 kB
    Inactive(anon):   208536 kB
    Active(file):    4055168 kB
    Inactive(file):   789120 kB
    Unevictable:          16 kB
    Mlocked:              16 kB
    HighTotal:      32516352 kB
    HighFree:       26894936 kB
    LowTotal:         636752 kB
    LowFree:          348100 kB
    SwapTotal:      46387192 kB
    SwapFree:       46387192 kB
    Dirty:                72 kB
    Writeback:             0 kB
    AnonPages:        609176 kB
    Mapped:           127868 kB
    Shmem:            301496 kB
    Slab:             108240 kB
    SReclaimable:      79204 kB
    SUnreclaim:        29036 kB
    KernelStack:        3064 kB
    PageTables:         7860 kB
    NFS_Unstable:          0 kB
    Bounce:                0 kB
    WritebackTmp:          0 kB
    CommitLimit:    62963744 kB
    Committed_AS:    2711244 kB
    VmallocTotal:     122880 kB
    VmallocUsed:       32996 kB
    VmallocChunk:      65380 kB
    HardwareCorrupted:     0 kB
    AnonHugePages:         0 kB
    HugePages_Total:       0
    HugePages_Free:        0
    HugePages_Rsvd:        0
    HugePages_Surp:        0
    Hugepagesize:       2048 kB
    DirectMap4k:       10232 kB
    DirectMap2M:      901120 kB
    Thanks for any suggestions and have a happy Memorial day in memory of those that gave the ultimate sacrifice.

    Jack

  9. #8
    Penguin of trust elija's Avatar
    Join Date
    Jul 2004
    Location
    Either at home or at work or down the pub
    Posts
    3,651
    Is this a 64 bit os, 32 bit PAE or 32 bit? Also how are you opening the image?
    "I used to be with it, then they changed what it was.
    Now what was it isn't it, and what is it is weird and scary to me.
    It'll happen to you too."

    Grandpa Simpson



    The Fifth Continent

  10. #9
    Linux User jkwilborn's Avatar
    Join Date
    Nov 2012
    Posts
    285
    eliija, thanks for reading and asking, it is a 64 bit OS and I'm just double clicking on the file in a file browser. I've tried invoking it via the command line and get the same, I believe.

    I would have thought that if I were to look at memory with a 32 bit system it would not show the complete 32 GB in the /proc/meminfo file, but don't know. My system is Wheezy with "3.2.0-4-686-pae" as probably a better identifier...

    Thanks for any suggestions you may have. I've looked at a number of boards, but it appears that questions like it have been asked with little or no replies.

    Jack

  11. #10
    Penguin of trust elija's Avatar
    Join Date
    Jul 2004
    Location
    Either at home or at work or down the pub
    Posts
    3,651
    OK, 3.2.0-4-686-pae is a 32 bit pae kernel. Pae allows a 32 bit kernel to address upto 64GB of RAM but an individual process is still limited to 4GB which could explain the lack of memory use. A 64 bit kernel is usually identified as x86_64.
    "I used to be with it, then they changed what it was.
    Now what was it isn't it, and what is it is weird and scary to me.
    It'll happen to you too."

    Grandpa Simpson



    The Fifth Continent

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •