客户一套核心业务集群一节点主机本地硬盘损坏，由于本地磁盘空间较小，当初部分文件系统并没有做镜像，导致一块硬盘损坏的同时
部分文件系统完全损坏，包括oracle集群及rdbms的安装目录，事已至此，我们只能通过将集群中该节点的信息清理掉，
并将节点重新通过addNode的方式添加进集群中
对于如何清理，我们不详细叙述，因为一节点已经损坏，集群完全无法启动，所以我们只需要在正常节点，删除
一节点的vip等基本信息，将oraInventory中关于一节点的的集群及rdbm注册信息删除
本章主要说明在addNode过程中，我们碰到的几个问题
export IGNORE_PREADDNODE_CHECKS=Y ----忽略预检
/grid/database/11.2.0/oui/bin/addNode.sh "CLUSTER_NEW_NODES={icdb1}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={icdb1-vip}" -silent
开始正常的添加集群操作
此处开始第一次报错：
Copying to remote nodes (Tuesday, May 5, 2015 9:06:53 PM CST)
.....JVMDUMP006I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" - please wait.
JVMDUMP032I JVM requested Snap dump using ‘/grid/database/11.2.0/oui/bin/Snap.20150505.210707.48300266.0001.trc‘ in response to an event
JVMDUMP010I Snap dump written to /grid/database/11.2.0/oui/bin/Snap.20150505.210707.48300266.0001.trc
JVMDUMP032I JVM requested Heap dump using ‘/grid/database/11.2.0/oui/bin/heapdump.20150505.210707.48300266.0002.phd‘ in response to an event
JVMDUMP010I Heap dump written to /grid/database/11.2.0/oui/bin/heapdump.20150505.210707.48300266.0002.phd
JVMDUMP032I JVM requested Java dump using ‘/grid/database/11.2.0/oui/bin/javacore.20150505.210707.48300266.0003.txt‘ in response to an event
JVMDUMP010I Java dump written to /grid/database/11.2.0/oui/bin/javacore.20150505.210707.48300266.0003.txt
icdb2:/home/grid$more /oracle/oraInventory/logs/oraInstall2015-05-05_09-06-28PM.err
Exception in thread "Thread-62" java.lang.OutOfMemoryError
at java.io.PrintStream.write(PrintStream.java:496)
at java.io.PrintStream.print(PrintStream.java:636)
at oracle.sysman.oii.oiic.OiicSilentInstallPhaseProgressListener.displayProgress(OiicSilentInstallPhaseProgressListener.java:372)
at oracle.sysman.oii.oiic.OiicSilentInstallPhaseProgressListener.addBytes(OiicSilentInstallPhaseProgressListener.java:341)
at oracle.sysman.oii.oiic.OiicSilentInstallPhaseProgressListener.addPercentage(OiicSilentInstallPhaseProgressListener.java:555)
at oracle.sysman.oii.oiix.OiixProgressUpdator$OiixProgressUpdatingWorker.run(OiixProgressUpdator.java:199)
at java.lang.Thread.run(Thread.java:811)
看基础的报错信息，似乎也很明确：
Exception in thread "Thread-62" java.lang.OutOfMemoryError
类似java内存不足，导致内存溢出报错
但是我们似乎并不知道该如何调整java内存信息，在这种时候，我们检索MOS文档搜索在addNode中关于java.lang.OutOfMemoryError 错误，以下文档：
Addnode.sh Fails With PRKC-1025 and Java.Lang.OutOfMemoryError (文档 ID 1085893.1)
Increase JRE_MEMORY_OPTIONS=" -mx1024m" or greater value in the oraparam.ini located in: $GRID_HOME/oui/
For 12.1.0.x release, the same change is also required in the addnode_oraparam.ini under $GRID_HOME/addnode/ directory.
该文档明确说明了修改java内存的位置及建议修改的大小
修复该错误后，我们正常进行添加过程，在结尾时候出现报错：
Copying to remote nodes (Tuesday, May 5, 2015 9:21:31 PM CST)
...............................................................................................WARNING:Error while copying directory /grid/database/11.2.0 with exclude file list ‘/tmp/OraInstall2015-05-05_09-21-07PM/installExcludeFile.lst‘ to nodes ‘icdb1‘. [PRKC-PRCF-2015 : One or more commands were not executed successfully on one or more nodes : <null>]
---------------------------------------------------------------------------------- icdb1:
PRCF-2023 : The following contents are not transferred as they are non-readable.
Directories: Files:
1) /grid/database/11.2.0/bin/diag/asm/user_root/host_3759066517_80/alert/log.xml
2) /grid/database/11.2.0/bin/diag/asm/user_root/host_3759066517_80/lck/AM_1096102193_3488045378.lck
3) /grid/database/11.2.0/bin/diag/asm/user_root/host_3759066517_80/lck/AM_1096102262_3454819329.lck
4) /grid/database/11.2.0/bin/diag/asm/user_root/host_3759066517_80/lck/AM_1744845641_3861997533.lck
5) /grid/database/11.2.0/bin/diag/asm/user_root/host_3759066517_80/lck/AM_3216668543_3129272988.lck
上述文件仅仅是集群产生的日志类文件，由于权限为root:system导致grid并没有copy相应的权限。对于以上错误，我们忽略该错误即可
------------------------------------------------------------------------------------
<版权所有，文章允许转载，但必须以链接方式注明源地址，否则追究法律责任!>
原博客地址：http://blog.itpub.net/23732248/
原作者：应以峰 (frank-ying)
-------------------------------------------------------------------------------------
一次增加节点时碰到的问题
原文：http://blog.itpub.net/23732248/viewspace-1653855/
