This post highlights three performance tuning techniques for Red Hat Enterprise Linux (6.2) and OpenJDK (1.6): TCP send / receive windows, large page memory, and i-CMS.
These techniques are particularly applicable to data grids or to any distributed system that stores a lot of data.
I just finished a technical white paper on JBoss Data Grid (JDG) performance factors, and these techniques were necessary for JDG to perform at its best.
Set the maximum size of the TCP send / receive windows of the accordingly.
Example: 640K Send Window / 25MB Receive Window
echo 655360 > /proc/sys/net/core/wmem_max echo 26214400 > /proc/sys/net/core/rmem_max
Configure large page memory. More information about large page memory can be found here.
Example: A server with 16GB of memory with 12.5GB of it reserved for large page memory.
echo 17179869184 > /proc/sys/kernel/shmmax echo 6400 > /proc/sys/vm/nr_hugepages echo 500 > /proc/sys/vm/hugetlb_shm_group
Update the resource limits.
jboss soft memlock unlimited jboss hard memlock unlimited
Set the JVM options.
Note: The heap size (12GB) is less than the amount of memory reserved for large page memory (12.5GB) as the heap is not the only memory used by the JVM.
-Xms12G -Xmx12G -XX:+UseLargePages
Use the Concurrent Mark Sweep (CMS) garbage collector in Incremental mode (i-CMS).