首页 > 其他 > 详细

Hadoop Kernel tunning

时间:2015-12-16 09:24:09      阅读:237      评论:0      收藏:0      [点我收藏+]
/etc/security/limits.conf

@ochadoop soft nofile 102642
@ochadoop hard nofile 102642
@ochadoop soft nproc unlimited
@ochadoop hard nproc unlimited
@ochadoop soft memlock unlimited
@ochadoop hard memlock unlimited

/etc/sysctl.conf

#尽可能少地让应用把内容交换到硬盘
echo "vm.swappiness = 0" >> /etc/sysctl.conf
#可以让应用在请求内存的时候使用交换
echo vm.overcommit_memory = 1 >> /etc/sysctl.conf

Mount options:disabling file access time

The real problem with maintaining access time (or
atime as it’s commonly called) is that every time a file is read, the metadata needs to be
updated. That is, for each read, there’s also a mandatory write. This is relatively expensive at scale and can negatively impact the overall performance of Hadoop

sample:

LABEL=/ / ext3 noatime,nodiratime 1 1
LABEL=/data/1 /data/1 ext3 noatime,nodiratime 1 2
LABEL=/data/2 /data/2 ext3 noatime,nodiratime 1 2
LABEL=/data/3 /data/3 ext3 noatime,nodiratime 1 2
LABEL=/data/4 /data/4 ext3 noatime,nodiratime 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SWAP-sda2 swap swap defaults 0 0

Hadoop Kernel tunning

原文:http://www.cnblogs.com/huaxiaoyao/p/5050033.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!