Linux kernel tuning for Apache Hadoop & Spark


Settings for all Linux OS levels

sysctl -w vm.swappiness=5
sysctl -w vm.dirty_background_ratio=5
sysctl -w vm.dirty_ratio=40
sysctl -w vm.dirty_writeback_centisecs=500
sysctl -w vm.dirty_expire_centisecs=100

Additional settings for Redhat 6.x or greater

Disable Transparent Hugh Pages

echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

Leave a comment