Discussion:
Why process memory starts so high up in virtual space with FreeBSD malloc?
(too old to reply)
Yuri
2008-12-01 21:28:48 UTC
Permalink
I am compiling the following program:

#include <stdlib.h>
main() { printf("0x%x\n", malloc(1)); }

in 32-bit 7.1-PRERELEASE and get 0x28201100 which is ~673MB of 4GB
address space or 16%.

When I run the same program with the google malloc (from
devel/google-perftools)
I get much lower value 0x80aa0e8 which is ~135MB of 4GB address space or
~3%.

Why FreeBSD memory allocator wastes such a high percentage of the memory
space?

Yuri
Giorgos Keramidas
2008-12-01 22:43:26 UTC
Permalink
Post by Yuri
#include <stdlib.h>
main() { printf("0x%x\n", malloc(1)); }
You should probably use printf("%p", ptr) to print pointers :)
Post by Yuri
in 32-bit 7.1-PRERELEASE and get 0x28201100 which is ~673MB of 4GB
address space or 16%.
When I run the same program with the google malloc (from
devel/google-perftools)
I get much lower value 0x80aa0e8 which is ~135MB of 4GB address space or
~3%.
Why FreeBSD memory allocator wastes such a high percentage of the
memory space?
The FreeBSD malloc(3) implementation can use either mmap() or sbrk() to
obtain memory from the system. It does not 'waste a high percentage of
memory' but it simply maps only high addresses (with an unmapped 'hole'
in lower addresses).

You can tune FreeBSD malloc(3) to prefer either sbrk() or mmap() by
setting the 'D' and 'M' malloc options. The following program prints
different initial allocation results when these two malloc() options are
configured differently:

% cat -n foo.c
1 #include <stdio.h>
2 #include <stdlib.h>
3
4 int
5 main(void)
6 {
7 char *p;
8
9 p = malloc(1);
10 if (p == NULL)
11 return EXIT_FAILURE;
12
13 printf("%p\n", malloc(1));
14 free(p);
15 return EXIT_SUCCESS;
16 }
%

For example:

% env MALLOC_OPTIONS='Dm' ./foo
0x8101102
% env MALLOC_OPTIONS='dM' ./foo
0x28201102
%

More details about the 'D', 'M' and other malloc() options should be
available in the manpage for your release.

- Giorgos
Yuri
2008-12-01 22:57:23 UTC
Permalink
Post by Giorgos Keramidas
The FreeBSD malloc(3) implementation can use either mmap() or sbrk() to
obtain memory from the system. It does not 'waste a high percentage of
memory' but it simply maps only high addresses (with an unmapped 'hole'
in lower addresses).
But the hole it leaves with MALLOC_OPTIONS='dM' is way larger than the
one left by 'Dm' option. Usually malloc will keep allocating addresses
higher
than this initial value and will never come back and fill some parts of
this gap.
Therefore "wasting" this space.

Yuri
Dan Nelson
2008-12-01 23:37:58 UTC
Permalink
Post by Giorgos Keramidas
The FreeBSD malloc(3) implementation can use either mmap() or
sbrk() to obtain memory from the system. It does not 'waste a high
percentage of memory' but it simply maps only high addresses (with
an unmapped 'hole' in lower addresses).
But the hole it leaves with MALLOC_OPTIONS='dM' is way larger than
the one left by 'Dm' option. Usually malloc will keep allocating
addresses higher than this initial value and will never come back and
fill some parts of this gap. Therefore "wasting" this space.
Have you actually verified that space is wasted? Note that the default
for malloc is "DM", which lets malloc use both break- and mmap-based
memory. Depending on the libc version, one or the other will be
preferred, but both will be used if necessary. Also, unless you have
tuned your kernel, you may only be able to allocate 512MB of memory via
break() (the kern.maxdsize tunable lets you adjust this). Here's what
I get with a simple test program on a month-old 7.1-PRE (before the
preference got switched to mmap):

$ MALLOC_OPTIONS=Dm ./malloctest
Malloced 535822336 bytes. First: 0x8080000, Last: 0x27e80000
$ MALLOC_OPTIONS=dM ./malloctest
Malloced 2542796800 bytes. First: 0x28200000, Last: 0xbfa00000
$ MALLOC_OPTIONS=DM ./malloctest
Malloced 3078619136 bytes. First: 0x8080000, Last: 0xbfa00000
$ ./malloctest
Malloced 3078619136 bytes. First: 0x8080000, Last: 0xbfa00000

So using only break(), I can allocate 511 MB. Using only mmap(), I can
allocate 2.36 GB. Using both (the default) I can allocate 2.86 GB.
--
Dan Nelson
***@allantgroup.com
Dan Nelson
2008-12-01 23:43:12 UTC
Permalink
Here's what I get with a simple test program on a month-old 7.1-PRE
Gah. silly mailing-list attachment stripper.

#include <stdlib.h>
#include <stdio.h>

int main(void)
{
size_t malloced = 0;
size_t chunksize = 1024*1024;
void *first = NULL;
void *last = NULL;
void *ptr;

while ((ptr = malloc(chunksize)) != NULL) {
if (first == NULL)
first = ptr;
last = ptr;
malloced += chunksize;
}
printf("Malloced %zu bytes. First: %p, Last: %p\n", malloced, first, last);
exit(0);
}
--
Dan Nelson
***@allantgroup.com
Giorgos Keramidas
2008-12-02 11:42:20 UTC
Permalink
Post by Giorgos Keramidas
The FreeBSD malloc(3) implementation can use either mmap() or sbrk() to
obtain memory from the system. It does not 'waste a high percentage of
memory' but it simply maps only high addresses (with an unmapped 'hole'
in lower addresses).
But the hole it leaves with MALLOC_OPTIONS='dM' is way larger than the
one left by 'Dm' option. Usually malloc will keep allocating addresses
higher than this initial value and will never come back and fill some
parts of this gap. Therefore "wasting" this space.
The 'D' and 'M' options set what malloc() will _prefer_, they do not
force malloc() to use _only_ the particular type of memory space. As
Dan explained in another post, both memory types will be used if there
is need for more address space.

Continue reading on narkive:
Loading...