spark sql 能够通过thriftserver 访问hive数据，默认spark编译的版本是不支持访问hive，因为hive依赖比较多，因此打的包中不包含hive和thriftserver,因此需要自己下载源码进行编译，将hive，thriftserver打包进去才能够访问，详细配置步骤如下：
?
1、下载源码
?
2、下载Maven,并配置
此配置简单，就略过
?
3、使用maven进行打包：
打包命令：?
mvn -Pyarn -Dhadoop.version=2.3.0-cdh5.0.0 -Phive -Phive-thriftserver -DskipTests clean package
?
上面的hadoop.version可以根据自己的需要设置相应的版本
?
注：在windows环境下编译时，由于编译需要的内存很大，因此要设置一下maven使用的内存数据，
进入MAVEN_HOME/bin/ 目录，修改mvn.bat文件，定位到下面一行位置，此行已被注释
?
@REM set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000
?
?
然后在下一行增加下面内容，并保存
set MAVEN_OPTS= -Xms1024m -Xmx1024m -XX:MaxPermSize=512m?
?
编译时可能会失败，这是由于网络中断的原因，多试几次就能成功
?
?
4、将SPARK_SRC_HOME\assembly\target\scala-2.10\spark-assembly-1.4.0-hadoop2.3.0-cdh5.0.0.jar 文件替换掉SPARK_HOME/lib 目录下的??spark-assembly*.jar文件
?
SPARK_SRC_HOME：为spark源码路径
SPARK_HOME：spark安装路径
?
5、cp HIVE_HOME/conf/hive-site.xml SPARK_HOME/conf/
将hive安装目录下的hive-site.xml文件复制到spark的conf目录下
?
6、进入SPARK_HOME/bin/ 运行 ./spark-sql --master spark://masterIp:7077
?
?
7、查询hive表数据
?
?
? ? ? ? ?> select * from test2;
15/07/16 14:07:13 INFO ParseDriver: Parsing command: select * from test2
15/07/16 14:07:13 INFO ParseDriver: Parse Completed
15/07/16 14:07:20 INFO BlockManagerInfo: Removed broadcast_0_piece0 on 192.168.209.140:51413 in memory (size: 1671.0 B, free: 267.3 MB)
15/07/16 14:07:20 INFO BlockManagerInfo: Removed broadcast_0_piece0 on 192.168.209.141:35827 in memory (size: 1671.0 B, free: 267.3 MB)
15/07/16 14:07:21 WARN HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
15/07/16 14:07:21 INFO deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
15/07/16 14:07:22 INFO MemoryStore: ensureFreeSpace(377216) called with curMem=0, maxMem=280248975
15/07/16 14:07:22 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 368.4 KB, free 266.9 MB)
15/07/16 14:07:22 INFO MemoryStore: ensureFreeSpace(32203) called with curMem=377216, maxMem=280248975
15/07/16 14:07:22 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 31.4 KB, free 266.9 MB)
15/07/16 14:07:22 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 192.168.209.141:35827 (size: 31.4 KB, free: 267.2 MB)
15/07/16 14:07:22 INFO SparkContext: Created broadcast 1 from processCmd at CliDriver.java:423
15/07/16 14:07:23 INFO FileInputFormat: Total input paths to process : 1
15/07/16 14:07:23 INFO SparkContext: Starting job: processCmd at CliDriver.java:423
15/07/16 14:07:23 INFO DAGScheduler: Got job 1 (processCmd at CliDriver.java:423) with 2 output partitions (allowLocal=false)
15/07/16 14:07:23 INFO DAGScheduler: Final stage: ResultStage 1(processCmd at CliDriver.java:423)
15/07/16 14:07:23 INFO DAGScheduler: Parents of final stage: List()
15/07/16 14:07:23 INFO DAGScheduler: Missing parents: List()
15/07/16 14:07:23 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[5] at processCmd at CliDriver.java:423), which has no missing parents
15/07/16 14:07:23 INFO MemoryStore: ensureFreeSpace(7752) called with curMem=409419, maxMem=280248975
15/07/16 14:07:23 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 7.6 KB, free 266.9 MB)
15/07/16 14:07:23 INFO MemoryStore: ensureFreeSpace(4197) called with curMem=417171, maxMem=280248975
15/07/16 14:07:23 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 4.1 KB, free 266.9 MB)
15/07/16 14:07:23 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 192.168.209.141:35827 (size: 4.1 KB, free: 267.2 MB)
15/07/16 14:07:23 INFO SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:874
15/07/16 14:07:23 INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 1 (MapPartitionsRDD[5] at processCmd at CliDriver.java:423)
15/07/16 14:07:23 INFO TaskSchedulerImpl: Adding task set 1.0 with 2 tasks
15/07/16 14:07:24 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, 192.168.209.139, ANY, 1435 bytes)
15/07/16 14:07:24 INFO TaskSetManager: Starting task 1.0 in stage 1.0 (TID 2, 192.168.209.140, ANY, 1435 bytes)
15/07/16 14:07:24 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 192.168.209.140:51413 (size: 4.1 KB, free: 267.3 MB)
15/07/16 14:07:24 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 192.168.209.140:51413 (size: 31.4 KB, free: 267.2 MB)
15/07/16 14:07:25 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 192.168.209.139:44770 (size: 4.1 KB, free: 267.3 MB)
15/07/16 14:07:30 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 192.168.209.139:44770 (size: 31.4 KB, free: 267.2 MB)
15/07/16 14:07:34 INFO TaskSetManager: Finished task 1.0 in stage 1.0 (TID 2) in 10767 ms on 192.168.209.140 (1/2)
15/07/16 14:07:39 INFO DAGScheduler: ResultStage 1 (processCmd at CliDriver.java:423) finished in 15.284 s
15/07/16 14:07:39 INFO StatsReportListener: Finished stage: org.apache.spark.scheduler.StageInfo@9aa2a1b
15/07/16 14:07:39 INFO DAGScheduler: Job 1 finished: processCmd at CliDriver.java:423, took 15.414215 s
15/07/16 14:07:39 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 15287 ms on 192.168.209.139 (2/2)
15/07/16 14:07:39 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool?
15/07/16 14:07:39 INFO StatsReportListener: task runtime:(count: 2, mean: 13027.000000, stdev: 2260.000000, max: 15287.000000, min: 10767.000000)
15/07/16 14:07:39 INFO StatsReportListener: ? ? 0% ? ? ?5% ? ? ?10% ? ? 25% ? ? 50% ? ? 75% ? ? 90% ? ? ? 95% ? ? 100%
15/07/16 14:07:39 INFO StatsReportListener: ? ? 10.8 s ?10.8 s ?10.8 s ?10.8 s ?15.3 s ?15.3 s ?15.3 s ? ?15.3 s ?15.3 s
15/07/16 14:07:39 INFO StatsReportListener: task result size:(count: 2, mean: 15663.000000, stdev: 15.000000, max: 15678.000000, min: 15648.000000)
15/07/16 14:07:39 INFO StatsReportListener: ? ? 0% ? ? ?5% ? ? ?10% ? ? 25% ? ? 50% ? ? 75% ? ? 90% ? ? ? 95% ? ? 100%
15/07/16 14:07:39 INFO StatsReportListener: ? ? 15.3 KB 15.3 KB 15.3 KB 15.3 KB 15.3 KB 15.3 KB 15.3 KB ? 15.3 KB 15.3 KB
15/07/16 14:07:39 INFO StatsReportListener: executor (non-fetch) time pct: (count: 2, mean: 72.195332, stdev: 18.424154, max: 90.619485, min: 53.771178)
15/07/16 14:07:39 INFO StatsReportListener: ? ? 0% ? ? ?5% ? ? ?10% ? ? 25% ? ? 50% ? ? 75% ? ? 90% ? ? ? 95% ? ? 100%
15/07/16 14:07:39 INFO StatsReportListener: ? ? 54 % ? ?54 % ? ?54 % ? ?54 % ? ?91 % ? ?91 % ? ?91 % ? ? ?91 % ? ?91 %
15/07/16 14:07:39 INFO StatsReportListener: other time pct: (count: 2, mean: 27.804668, stdev: 18.424154, max: 46.228822, min: 9.380515)
15/07/16 14:07:39 INFO StatsReportListener: ? ? 0% ? ? ?5% ? ? ?10% ? ? 25% ? ? 50% ? ? 75% ? ? 90% ? ? ? 95% ? ? 100%
15/07/16 14:07:39 INFO StatsReportListener: ? ? ?9 % ? ? 9 % ? ? 9 % ? ? 9 % ? ?46 % ? ?46 % ? ?46 % ? ? ?46 % ? ?46 %
111
222
333
444
555
666
777
888
999111
222
333
444
555
666
777
888
999111
............
?
Time taken: 26.523 seconds, Fetched 889 row(s)
15/07/16 14:07:39 INFO CliDriver: Time taken: 26.523 seconds, Fetched 889 row(s)
spark-sql> ? ?
?
由于我是自己笔记本上的虚拟机上运行，同时只有两个work节点，所以运行会慢一点，但不影响功能
?
看到有的同学会要求先启动thriftserver，经测试可以不需要手动启动，在运行./spark-sql时会自动启动
同时hive-site.xml也不需要修改任何配置
?
?
原文：http://daizj.iteye.com/blog/2227658