Using jemalloc to Optimize Memory Allocation

Introduction

This document describes the proper steps to install libjemalloc.so that is optimized for a user's Linux system. For any questions regarding this document, please contact Sentieon support at support@sentieon.com.

Background

jemalloc is a memory allocator, optimized for high memory allocation performance and fewer memory fragments in multi-thread scenarios. For more general information about jemalloc, please refer to https://github.com/jemalloc/jemalloc

Sentieon recommends using jemalloc to improve memory management and overall performance in Sentieon applications, especially Sentieon bwa-mem.

Installation

Sentieon recommends installing a pre-built libjemalloc.so. Users may need root access to complete the installation.

RHEL/CentOS 8.x

yum install epel-release
yum install jemalloc

By default, the libjemalloc.so is installed at

/usr/lib64/libjemalloc.so.2

RHEL/CentOS 7.x

yum install epel-release
yum install jemalloc

By default, the libjemalloc.so is installed at

/usr/lib64/libjemalloc.so.1

Ubuntu 20.04

apt update
apt install libjemalloc2

By default, the libjemalloc.so is installed at

/usr/lib/x86_64-linux-gnu/libjemalloc.so.2

Ubuntu 18.04

apt update
apt install libjemalloc1

By default, the libjemalloc.so is installed at

/usr/lib/x86_64-linux-gnu/libjemalloc.so.1

Other systems without a prebuilt package

Please refer to INSTALL.md on the jemalloc GitHub page, https://github.com/jemalloc/jemalloc, for more information on how to build and install jemalloc.

Loading jemalloc in a Sentieon pipeline

The LD_PRELOAD environment variable can be used to load the jemalloc library in Sentieon at run time.

For example, on a CentOS 8.x system, you can use the following code to set this environment variable before running the Sentieon tools:

export LD_PRELOAD=/usr/lib64/libjemalloc.so.2