How memory are actually used?

Hello,

I am running Apache HTTPD 1.3.37 with mod_php, when using the top
command, I see:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12584 web 15 0 142m 12m 9176 S 8 0.3 2:07.01 httpd
12586 web 15 0 142m 12m 9168 S 8 0.3 2:08.85 httpd
12589 web 15 0 141m 10m 7376 S 8 0.3 2:06.83 httpd
12591 web 15 0 142m 11m 7420 S 8 0.3 1:59.64 httpd
12594 web 15 0 142m 11m 7444 S 8 0.3 2:06.25 httpd
12585 web 15 0 141m 12m 9200 S 6 0.3 2:03.83 httpd

Assume all the httpd process is show above...

1. Is the total memory currently used is : 12+12+10+11+11+12 = 68M ?
2. Is the max. memory even allocated by is: 142+142+141+142+142+141 = 850M


Thanks.

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

howard chen wrote:
> Hello,
>
> I am running Apache HTTPD 1.3.37 with mod_php, when using the top
> command, I see:
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 12584 web 15 0 142m 12m 9176 S 8 0.3 2:07.01 httpd
> 12586 web 15 0 142m 12m 9168 S 8 0.3 2:08.85 httpd
> 12589 web 15 0 141m 10m 7376 S 8 0.3 2:06.83 httpd
> 12591 web 15 0 142m 11m 7420 S 8 0.3 1:59.64 httpd
> 12594 web 15 0 142m 11m 7444 S 8 0.3 2:06.25 httpd
> 12585 web 15 0 141m 12m 9200 S 6 0.3 2:03.83 httpd
>
> Assume all the httpd process is show above...
>
> 1. Is the total memory currently used is : 12+12+10+11+11+12 = 68M ?
> 2. Is the max. memory even allocated by is: 142+142+141+142+142+141 = 850M
>
>
> Thanks.
>

Welcome to the wonderful world of Linux memory. The memory reported in
the VIRT/RES column cannot simply be added up to calculate the total
memory usage because some memory may be shared. When a process is
fork()'d (to create a child process), the memory between the parent and
child process will be shared. Once a portion of the memory is changed by
the parent or child, that portion of the memory will become distinct
between the two processes. For example:

* Parents using 10MB
* A child process is created
* Child now shows usage of 10MB, although total usage is only 10MB, not
20MB, due to shared memory
* Child process overwrites 5MB of the memory. Now that memory is unique
to the child, so it will be separate
* Memory usage is now 10MB (parent) + 5MB (child) = 15MB (theoretically)

This is just a very basic example, since the true details behind the
memory allocation is more complex. There is a python script available
that will try to give you a more accurate calculation of memory usages
available here:

http://www.pixelbeat.org/scripts/ps_mem.py


--
Justin Pasher

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

On Wed 17 Sep 2008, howard chen wrote:
> I am running Apache HTTPD 1.3.37 with mod_php, when using the top
> command, I see:
>
> * PID USER * * *PR *NI *VIRT *RES *SHR S %CPU %MEM * *TIME+ *COMMAND
> 12584 web * * * 15 * 0 *142m *12m 9176 S * *8 *0.3 * 2:07.01 httpd
> 12586 web * * * 15 * 0 *142m *12m 9168 S * *8 *0.3 * 2:08.85 httpd
> 12589 web * * * 15 * 0 *141m *10m 7376 S * *8 *0.3 * 2:06.83 httpd
> 12591 web * * * 15 * 0 *142m *11m 7420 S * *8 *0.3 * 1:59.64 httpd
> 12594 web * * * 15 * 0 *142m *11m 7444 S * *8 *0.3 * 2:06.25 httpd
> 12585 web * * * 15 * 0 *141m *12m 9200 S * *6 *0.3 * 2:03.83 httpd
>
> Assume all the httpd process is show above...
>
> 1. Is the total memory currently used is : 12+12+10+11+11+12 = 68M ?
> 2. Is the max. memory even allocated by is: 142+142+141+142+142+141 =
> 850M

No and no.

How to interprete these figures depend a bit on your operating system.
But in general UNIX systems try to share memory between processes.
There are several ways how that can be achieved. In one way a program
module (executable binary, shared lib) is divided into sections. Each
section is marked by at compile time if it can be shared between
processes or not. Code sections are usually read-only during program
execution hence they can be shared. If your programs use a shared libc
for example all the code in this lib is shared between all processes
that use the lib. But a program module does not have to be loaded
completely into memory to do some work. Code or data pages that are not
needed are not loaded.

A second way to achieve memory sharing is called copy-on-write. When a
process forks both processes at first share all their memory. Now one
of them starts writing to a specific area of its memory. Only at this
time the operating system allocates a personal copy of that piece of
memory for that process. It also does not copy the whole process memory
but only a small chunk (page). So the more both processes write to
their memory the more memory they consume together. But they probably
never consume together twice the amount of memory one of them has
consumed while it has run alone.

So what you see as VIRT is the amount of mem that process would consume
if it runs completely alone and if all of its segments are loaded into
memory. RES is the part of VIRT that is actually allocated. Both say
nothing about sharing. SHR says how much of RES is shared between at
least 2 processes by the first method (not copy-on-write). But it does
not say among how much or which processes it is shared.

Hence you can't simply add up these numbers.

A better way to judge memory consumption on Linux is /proc/$PID/smaps
and tools that use it. /proc/$PID/clear_refs on recent kernels is also
interesting in that regard. To get a general impression how much memory
my apache needs you can start vmstat with a small parameter, stop the
apache and look how it affects the memory related columns.

Torsten

--
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org