(1) operation history:
SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt assembly
按官方教程失败，但去掉红色的部分，很容易编译成功，这条指令会带来什么？
修改为
SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt clean compile package assembly
正常通过
(2) 启动测试程序
export YARN_CONF_DIR=$HADOOP_HOME SPARK_JAR=./assembly/target/scala-2.9.3/spark-assembly-0.8.1-incubating-hadoop2.2.0.jar ./spark-class org.apache.spark.deploy.yarn.Client --jar ./examples/target/scala-2.9.3/spark-examples-assembly-0.8.1-incubating.jar --class org.apache.spark.examples.JavaSparkPi --args yarn-standalone --num-workers 3 --master-memory 2g --worker-memory 2g --worker-cores 1
(3) 结果
... 14/02/27 00:25:57 INFO yarn.Client: Application report from ASM: application identifier: application_1393415612957_0004 appId: 4 clientToAMToken: null appDiagnostics: appMasterHost: 192.168.0.105 appQueue: default appMasterRpcPort: 0 appStartTime: 1393431898882 yarnAppState: RUNNING distributedFinalState: UNDEFINED appTrackingUrl: jiahuili-MBP.local:8088/proxy/application_1393415612957_0004/ appUser: admin 14/02/27 00:25:58 INFO yarn.Client: Application report from ASM: application identifier: application_1393415612957_0004 appId: 4 clientToAMToken: null appDiagnostics: appMasterHost: 192.168.0.105 appQueue: default appMasterRpcPort: 0 appStartTime: 1393431898882 yarnAppState: FINISHED distributedFinalState: SUCCEEDED appTrackingUrl: jiahuili-MBP.local:8088/proxy/application_1393415612957_0004/A appUser: admin
check http://localhost:8088/cluster/ for task details
Ref:
(1) http://spark.incubator.apache.org/docs/latest/running-on-yarn.html
(2) http://blog.csdn.net/macyang/article/details/17632645 -- about yarn
(3) http://dongxicheng.org/framework-on-yarn/build-spark-on-hadoop-2-yarn/
operation note for "spark with yarn",布布扣,bubuko.com
operation note for "spark with yarn"
原文：http://www.cnblogs.com/enyun/p/3570357.html