malloc related parameters can be changed with mallopt (3)
.
For example, you can change the threshold for using mmap inside malloc to 1MB by doing the following:
mallopt(M_MMAP_THRESHOLD, 1024*1024);
Some of the parameters that can be set with mallopt can also be changed with environment variables.
For example, the M_MMAP_THRESHOLD
mentioned above can also be set with the MALLOC_MMAP_THRESHOLD_
environment variable.
$ export MALLOC_MMAP_THRESHOLD_=1048576
$ ./a.out
Environment variables can be applied without changing the program, which is useful when changing system defaults. Changing mallopt with this environment variable uses glibc's Tunables mechanism.
In this article, the flow of setting glibc tunables
Make a note of the MALLOC_MMAP_THRESHOLD_
environment variable as an example.
If you have any mistakes, I would appreciate it if you could point them out.
The environment is as follows.
$ arch
x86_64
$ uname -r
5.4.0-42-generic
$ lsb_release -d
Description: Ubuntu 18.04.4 LTS
$ dpkg -l | grep libc-bin
ii libc-bin 2.27-3ubuntu1.2 amd64 GNU C Library: Binaries
MALLOC_MMAP_THRESHOLD_
Flow that reflects environment variablesAuxiliary vector can be obtained with getauxval (3)
,
Auxiliary information of the executable file passed from the ELF loader of the kernel.
In the loader, get the ʻAT_SECURE information from the auxiliary vector and If ʻAT_SECURE
is true, the global variable __libc_enable_secure
will be set.
Here, ʻAT_SECURE is a name that is a little misleading, This means that the current process needs to run securely. More specifically, ʻAT_SECURE
is set in any of the following cases.
--Programs with set-user-ID or set-group-ID valid --Programs with capabilities
glibc-2.27/elf/dl-sysdep.c
ElfW(Addr)
_dl_sysdep_start (void **start_argptr,
void (*dl_main) (const ElfW(Phdr) *phdr, ElfW(Word) phnum,
ElfW(Addr) *user_entry, ElfW(auxv_t) *auxv))
{
...
for (av = GLRO(dl_auxv); av->a_type != AT_NULL; set_seen (av++))
switch (av->a_type)
{
...
case AT_SECURE:
__libc_enable_secure = av->a_un.a_val;
break;
...
}
__tunables_init (_environ);
...
(*dl_main) (phdr, phnum, &user_entry, GLRO(dl_auxv));
return user_entry;
}
tunable_list []
from the value of the environment variableIn __tunables_init
, if the item intunable_list []
is included in the environment variable,
The value of the environment variable is reflected by tunable_initialize ()
.
However, if __libc_enable_secure
is set (= ʻAT_SECUREis set), Processing is performed according to the following three types of
security_level`.
--SXID_ERASE
: ʻAT_SECURE, delete it from the environment variable so that the child process cannot read it. --Ignore if
SXID_IGNORE: ʻAT_SECURE
(remains in environment variable)
--NONE
: Always reflect
glibc-2.27/elf/dl-tunables.c
void
__tunables_init (char **envp)
{
...
while ((envp = get_next_env (envp, &envname, &len, &envval,
&prev_envp)) != NULL)
{
...
for (int i = 0; i < sizeof (tunable_list) / sizeof (tunable_t); i++)
{
tunable_t *cur = &tunable_list[i];
...
if (__libc_enable_secure)
{
if (cur->security_level == TUNABLE_SECLEVEL_SXID_ERASE)
{
/* Erase the environment variable. */
...
}
if (cur->security_level != TUNABLE_SECLEVEL_NONE)
continue;
}
tunable_initialize (cur, envval);
break;
}
}
}
}
The security_level
of each item in the tunable is defined in the .list
file along with the environment variable name and type.
glibc-2.27/elf/dl-tunables.list
glibc {
malloc {
...
mmap_threshold {
type: SIZE_T
env_alias: MALLOC_MMAP_THRESHOLD_
security_level: SXID_IGNORE
}
...
}
}
Automatically generate dl-tunable-list.h
from dl-tunables.list
by scripts / gen-tunables.awk
tunable_list []
is defined there.
dl-tunable-list.h
static tunable_t tunable_list[] attribute_relro = {
...
{TUNABLE_NAME_S(glibc, malloc, mmap_threshold), {TUNABLE_TYPE_SIZE_T, 0, SIZE_MAX}, {}, NULL, TUNABLE_SECLEVEL_SXID_IGNORE, "MALLOC_MMAP_THRESHOLD_"},
...
};
tunable_list []
to mp_
at the first mallocFrom now on, it's not about tunable, but about malloc parameters. Also, the execution timing is not the loader, but the first time malloc is called after entering the normal main function.
The first time the program calls the memory allocation function, malloc_hook_ini
is called and
Call ptmalloc_init
from there.
glibc-2.27/malloc/hooks.c
static void *
malloc_hook_ini (size_t sz, const void *caller)
{
__malloc_hook = NULL;
ptmalloc_init ();
return __libc_malloc (sz);
}
In ptmalloc_init
, getmmap_threshold
from the tunable_list set in 2.
Set to the structure mp_
that holds the malloc parameters together.
glibc-2.27/malloc/arena.c
static void
ptmalloc_init (void)
{
...
TUNABLE_GET (mmap_threshold, size_t, TUNABLE_CALLBACK (set_mmap_threshold));
...
}
glibc-2.27/elf/dl-tunables.c
void
__tunable_get_val (tunable_id_t id, void *valp, tunable_callback_t callback)
{
tunable_t *cur = &tunable_list[id];
switch (cur->type.type_code)
{
...
case TUNABLE_TYPE_SIZE_T:
{
*((size_t *) valp) = (size_t) cur->val.numval;
break;
}
...
if (cur->initialized && callback != NULL)
callback (&cur->val);
}
The actual setting is the do_set_mmap_threshold
function.
If you look here, there is actually an upper limit for the set value,
You can see that dynamic threshold control is disabled when setting mmap_threshold
.
glibc-2.27/malloc/malloc.c
static inline int
__always_inline
do_set_mmap_threshold (size_t value)
{
/* Forbid setting the threshold too high. */
if (value <= HEAP_MAX_SIZE / 2)
{
LIBC_PROBE (memory_mallopt_mmap_threshold, 3, value, mp_.mmap_threshold,
mp_.no_dyn_threshold);
mp_.mmap_threshold = value;
mp_.no_dyn_threshold = 1;
return 1;
}
return 0;
}
Now it's sunny and the value of the MALLOC_MMAP_THRESHOLD_
environment variable will be used to determine if to mmap
.
I'm happy.
glibc-2.27/malloc/malloc.c
static void *
sysmalloc (INTERNAL_SIZE_T nb, mstate av)
{
...
if (av == NULL
|| ((unsigned long) (nb) >= (unsigned long) (mp_.mmap_threshold)
&& (mp_.n_mmaps < mp_.n_mmaps_max)))
{
...
mm = (char *) (MMAP (0, size, PROT_READ | PROT_WRITE, 0));
}
}
When setting the malloc parameter with an environment variable, it is better to check whether the setting is properly reflected by the debugger.
Following the operation of malloc (Environment Variables) --Qiita mallopt(3) - Linux manual page The GNU C Library getauxval(3) - Linux manual page
Recommended Posts