Primers › Java

Java heap size and container memory

A web service written in Java runs in a container with a 4 GiB memory limit. Someone checks its heap and finds a maximum of 1 GiB, a quarter of what the container was given, and wonders whether the other 3 GiB are going to waste. They aren’t, or not all of them. The 1 GiB is a default with a rule behind it, and most of the rest is spoken for.

This primer covers how the JVM picks its heap size inside a container, where the rest of the memory goes, and how to read what a running service is really using. When a Java pod keeps restarting, why a pod keeps restarting covers telling a memory problem from the other causes.

A few terms

• JVM: the Java Virtual Machine, the program that runs Java code. Each Java service is one JVM process.
• Heap: the area of memory where the JVM keeps the objects a program creates. When no code refers to an object anymore, the garbage collector frees its space.
• Garbage collector (GC): the part of the JVM that finds unused objects and frees them. Some collectors pause the whole application while they work.
• Container limit: a ceiling on how much memory a container may use, enforced by the Linux kernel. A container that goes over it can be killed [1].
• MiB and MB: a mebibyte (MiB) is 1,048,576 bytes and a megabyte (MB) is 1,000,000. In Kubernetes a limit written 2G is 2,000,000,000 bytes and 2Gi is 2,147,483,648, so the suffix matters [1].

Where you run into a JVM

The JVM doesn’t run Java source code. It runs class files, a compiled format, and doesn’t care which language produced them, so other languages compile to it too [5]. Kotlin is one: it targets the JVM, and Kotlin services can use Java frameworks like Spring Boot [6]. Anything built in Java or another JVM language brings a JVM with it when it runs. Some common examples:

• Web services and APIs, often built with Spring Boot, a Java framework for standalone services [7]. These are the services most teams write themselves and package into containers.
• Jenkins, the build server. Its controller is a Java program, and it won’t start on a Java version it doesn’t support [8].
• Elasticsearch, the search engine. It ships with its own copy of Java and sizes its own heap, which comes up again below [9].

Where the default heap comes from

The heap needs an upper bound, and the JVM picks one when nobody sets it. By default it takes an initial heap of 1/64 of the available memory and a maximum heap of 1/4 [10]. The flag behind the maximum is -XX:MaxRAMPercentage, whose default is 25 percent [4].

Inside a container, “available memory” means the container’s limit, not the machine’s. That’s because of container support, which is on by default on Linux and lets the JVM read the container’s memory and CPU limits [4]. Without it, a JVM on a 64 GiB node would size its heap as if it had the whole node.

For the 4 GiB container:

1. Memory the JVM sees: 4,096 MiB.
2. Initial heap: 4,096 ÷ 64 = 64 MiB.
3. Maximum heap: 4,096 × 0.25 = 1,024 MiB, or 1 GiB.

An explicit setting wins over the default. -Xmx2g sets the maximum heap directly, and -XX:MaxRAMPercentage=60 changes the fraction [4]. The percentage is usually the better choice in a container, because if someone later raises the container limit, the heap grows with it. A fixed -Xmx stays where it was.

In a container, a pod, or on its own server

The same JVM runs in all three places. Two things differ between them: what “available memory” means to the JVM, and how many other programs share that memory:

Containers and pods are the common case for services written today, and they suit the default well: the JVM is alone with its limit, and the limit was chosen for it. On a server of its own, the default is less of a fit. A JVM that’s the only real program on a 64 GiB server takes 16 GiB of heap and leaves most of the machine idle, and four JVMs on one server would each claim 16 GiB with nothing left between them. On servers, the heap is usually set explicitly instead. Elasticsearch, for example, sizes its heap automatically and says that anyone overriding it should keep the heap to no more than half of the memory available, with all the Elasticsearch processes on one host or in one container counted together, because it needs the rest for memory outside the heap [9].

Why only a quarter

The heap is only part of what a JVM uses. The container limit has to cover all of this [13], [4]:

• the heap
• metaspace, where class definitions live. By default it has no upper limit at all.
• the code cache, which holds compiled code, up to 240 MB by default
• one stack per thread
• direct buffers and other native memory

The heap is a quarter of the budget. The rest is for everything else the process needs, and whatever is left is the margin before the limit.

A heap set equal to the container limit leaves nothing for the non-heap parts, so the process crosses the limit and gets killed even though the heap itself isn’t full. That’s the reason for the cautious default. Teams often raise the fraction to 50 or 75 percent once they’ve measured how much non-heap memory their service really uses.

The two ways of running out look different:

Reading the value a running service uses

The command most people reach for first starts a brand new JVM:

kubectl exec <pod> -- java -XX:+PrintFlagsFinal -version | grep -E 'MaxHeapSize|InitialHeapSize'

That prints what a fresh JVM would choose in this container, and ignores any flags the running service was started with. To ask the running service itself, use jcmd, which comes with the JDK. It takes the process ID, and when Java is the container’s main process, that’s PID 1 [14]:

kubectl exec <pod> -- jcmd 1 VM.flags
kubectl exec <pod> -- jcmd 1 GC.heap_info

VM.flags prints the flags in effect and their values, including MaxHeapSize and which collector is in use. GC.heap_info shows how much of the heap is in use right now [14]. Compare the two before blaming the heap: a service using 300 MiB of a 1 GiB heap has plenty of room left.

Flags are often set through the JAVA_TOOL_OPTIONS environment variable instead of the command line. The JVM prints that variable’s whole value to its error output every time it starts, as Picked up JAVA_TOOL_OPTIONS: … [15], so anything secret put there, like a trust store password, ends up in the logs.

To see what the variable is set to, print it from inside the container [16]:

kubectl exec <pod> -- printenv JAVA_TOOL_OPTIONS

That shows the value the container was given. A startup script can still change it before launching Java, so the JVM’s own line in the logs is the surer record of what it used. --previous reads the logs of the run before the last restart [17]:

kubectl logs <pod> | grep 'Picked up JAVA_TOOL_OPTIONS'
kubectl logs <pod> --previous | grep 'Picked up JAVA_TOOL_OPTIONS'

Which garbage collector you get

When nobody chooses a collector, the JVM picks one based on the machine. It treats a machine as server-class if it sees two or more processors and at least 1,792 MB of memory, and gives server-class machines the G1 collector. Everything else gets the Serial collector [10].

In a container, the processor count comes from the CPU limit, rounded up: the JVM divides the container’s CPU quota by its period and takes the ceiling [18]. So a limit of 500m (half a CPU) or 1000m both look like one processor, and 1500m looks like two. A container limited to one CPU or less gets Serial no matter how much memory it has.

Serial uses a single thread for all its work. It suits single-processor machines and applications with small data sets, up to about 100 MB [19]. That’s fine for a small service, and it also means a full collection stops the whole application, on a CPU that may already be throttled. A long pause like that is one of the ways a healthy service fails its health checks, which why a pod keeps restarting covers.

References

Every source cited across the primers is collected on one page.