首页 > 其他 > 详细

swapniess and overcommit and other hadoop optimization

时间:2015-08-30 06:33:58      阅读:279      评论:0      收藏:0      [点我收藏+]

限制产生的文件大小,单位KB

ulimit -f 1
dd if=/dev/zero of=1g.txt bs=1023 count=1 可以
dd if=/dev/zero of=1g.txt bs=1024 count=1 可以
dd if=/dev/zero of=1g.txt bs=1025 count=1 不可以
ulimit -f ulimit

限制cpu的使用时间

ulimit -t 1

echo ‘ls -aRl /’ > 1.sh && bash 1.sh 发现程序运行一会儿后自动被kill

firefoxbug overcommit的解释

http://www.firefoxbug.com/index.php/archives/2800/
http://blog.csdn.net/houjixin/article/details/46412557
http://linuxstorm.blog.51cto.com/5218690/1609334
cat /proc/sys/vm/overcommit_memory
echo 0 > /proc/sys/vm/overcommit_memory
cat /proc/sys/vm/overcommit_memory
http://blog.csdn.net/metaxen/article/details/7188689
http://www.51testing.com/html/87/43487-204904.html
http://www.vpsee.com/2013/10/how-to-configure-the-linux-oom-killer/

动态链接库

http://m.blog.csdn.net/blog/zhangmiaoping23/45675119 # 写的非常棒

swapniess 更新

http://www.slideshare.net/technmsg/improving-hadoop-performancevialinux?next_slideshow=1

http://www.pythian.com/blog/what-exactly-is-swappiness/
http://www.linuxvox.com/2009/10/what-is-the-linux-kernel-parameter-vm-swappiness/
http://lwn.net/Articles/100978/ 不同的内核对swapniess的解释也不一样

处理

cat /proc/sys/vm/swappiness
echo 10 > /proc/sys/vm/swappiness
echo "vm.swapniess = 10" >> /etc.sysctl.conf

THP

cat /sys/kernel/mm/redhat_transparent_hugepage/defrag
cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

网卡升级

巨型帧打开后,可以提升10%~20%

page cache

echo 1 > /proc/sys/vm/drop_caches
dd if=/dev/zero of=512M-file count=512000 bs=1k
time cat 512M-file &>/dev/null
echo 1 > /proc/sys/vm/drop_caches
time cat 512M-file &>/dev/null

版权声明:本文为博主原创文章,未经博主允许不得转载。

swapniess and overcommit and other hadoop optimization

原文:http://blog.csdn.net/vbaspdelphi/article/details/48099319

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!