Virtual Memory

I have a doubt about the linux virtual memory concept.
What does it actually mean when you say that the linux kerenel has 1 gb of virtual memory space for itself? Does it mean that if I kalloc for more than 1GB in code , will it fail? I also read that kernel memory is not pageable and any memory alloced will directly consume "Physical RAM". Please clear this concept for me.

Virtual memory means that process will believe that is has that amount of the memory to use. The responsible for managing this idea is the kernel together the MMU HW.

Every process on Linux OS when created receives a 4GB Virtual Memory space no matter if you actually has this memory available or not. Virtual memory is a concept to make the usage of available memory better and to protect one process to access another process memory.

The kernel is like another process in the system so it will receive a Virtual Memory space of 4GB. When the kernel is compiled you can define how you will split this 4GB space. There are options like 3GB for executing user space process and 1GB for kernel itself.

BR,
Leví

Thanks levi_silva , please bear with me.

Quote:

---

There are options like 3GB for executing user space process and 1GB for kernel itself

---

. This is a little confusing to me .What does it imply ? Does it mean that each process has 1Gb virtual mempry set aside for using function calls from Kernel and the rest 3Gb for themselves?? I understand the theory and concept of virtual memory, but this 3 gb and 1 gb partiition is confusing to me.

Kernel executes in two ways: Doing its internal tasks and making things in behalf of user space programs.

When you split the kernel in 3GB for user and 1GB for kernel means that kernel process has 3GB of virtual memory to handle request from user space process like the system calls. This space is used by the kernel to handle all user process request. How share that and how use it better is a internal task of the kernel.

This 1GB is used to save kernel internal structures and to handle tasks like handling HW interrupts and scheduler tasks.

If you have a system that has a lot of HW interrupts you can choose 3GB of kernal space and 1GB of user. In the desktop versions, normally you will use 3 on user and 1 in kernel because it is a system designed for users :)

BR,
Leví

@levi_silva Thanks a lot for your reply , one more doubt this 3gb and 1gb partition is for entire kernel and userspace or is it per user space process ?
Example : Firefox has 3gb virtual and all the system calls used by firefox has 1gb virtual and one more process say a word-process also has 3gb virtual plus the 1 gb virtual for its system calls.

Or

Is it like the entire linux kernel has 4gb virtual memory and out of which 1gb is used by kernel and rest are used by user-process?

Example: If you are using 32-bit machine.
With 32-bit we can have 4GB of physical memory (RAM).
With 32-bit we can have 4GB of virtual memory (actually resides of disk, kernel swaps data in/out whenever needed).

Normally, In this 4GB of memory (physical/virtual) the Linux memory management divides 3parts for user space and 1part for kernel space. This partition configuration can be modified as Levi said (you need to figure it out).
So, we can see that all user space programs collectively use 3GB of memory (physical/virtual).
You need to understand that kernel is not a program. It is a system, which has many components memory management, scheduler, etc. All those components collectively use 1GB of memory (physical/virtual).

For instance, firefox is an application running in user space. Just an example, If this program uses any of the file operations like fopen/mmap/fclose a file. The corresponding system call (like for fopen the system call is sys_open) on that particular file will be operated in kernel (now this program becomes task in kernel and the information about this process will be in task_struct), and kernel provides the information required by user program. So, the firefox when it allocates memory dynamically it uses the user space memory (virtual). When the program is executed then the data is swapped in to physical memory for the use of processor.
kmalloc also allocates the memory in virtual memory, whenever the processor runs the memory is mapped to physical memory by using PAGE_SHIFT... blah..blah..blah..

I hope, the concept will be more clear if you could go through memory management in LDD3.

PS: kmalloc's total memory available is only 128KB. If the kernel reaches to this point kmalloc will fail to allocate memory. You can change this limitation but not recommended. Instead, kernel offers other type of services like get_free_pages() etc.

Hey,

"Example : Firefox has 3gb virtual and all the system calls used by firefox has 1gb virtual and one more process say a word-process also has 3gb virtual plus the 1 gb virtual for its system calls."

All created process get 4GB of virtual memory. In the above example, Firefox will have 4GB of virtual address and word-processor will also have 4GB of virtual memory. This is a good feature of virtual memory. Seems that we have more physical memory available that we really have.

"Is it like the entire linux kernel has 4gb virtual memory and out of which 1gb is used by kernel and rest are used by user-process?"

Linux Kernel is also a process so it will have a 4GB of virtual memory. Kernel is the essential component of the Linux because it is responsible to manage all the things in the OS. Kernel uses this 1GB of virtual memory to handle internal tasks. The 3GB that are left are used by kernel to manage all the user process. All user process "share" this 3GB.

This is the theory involved. To see how it works in the real world you will have to hack kernel code and study the Memory Management subsystem.

As tutika said, LDD3 is good a book. I also recommend Linux Kernel Development from Robert Love.

"@levi_silva Thanks a lot for your reply" -> Lets share and discuss Linux internals :)

Have a good day, It's 09 AM here in Brazil =)
Leví

Ha got it now !!! thanks and Yes Lets share and discuss Linux internals :)

I was always told that virtual memory operates like this: the kernel tracks down all the code and data that a process will require (inside the executable, in libraries and elsewhere), adds it up and divides it into pages. Then it copies individual pages into core when the cpu requests them by raising a page fault.

But I was also told that the whole kernel must be in core all the time because there is nothing "behind" the kernel that could copy kernel pages from virtual memory into core. So how can the kernel itself use virtual memory?

When kernel is using CPU to do some schedule for example. When it needs to access his own mapped resources it can call its Memory Management subsystem to get this information. I hope it answers your question. :)