spark基础练习（未完)

1、filter
val rdd = sc.parallelize(List(1,2,3,4,5))
val mappedRDD = rdd.map(2*_)
mappedRDD.collect
val filteredRDD = mappedRdd.filter(_>4)
filteredRDD.collect

(上述完整写法)
val filteredRDDAgain = sc.parallelize(List(1,2,3,4,5)).map(2 * _).filter(_ > 4).collect

2、wordcount
val rdd = sc.textfile("/data/README.md")
rdd.count
rdd.cache
val wordcount = rdd.flatMap(_.split(‘、‘)).map(_,1).reduceByKey(_+_)
wordcount.collect
wordcount.saveAsTextFile("/data/result")

3、sort
val== rdd.flatMap(_split(‘ ‘)).map((_,1)).reduceByKey(_+_).map(x => (x._2,x._1)).sortByKey(false).map(x => (x._2,x._1)).saveasTextFile("/data/resultsorted")

4、union
val rdd1 = sc.parallelize(List((‘a‘,1),(‘b‘,1)))
val rdd2 = sc.parallelize(List((‘c‘,1),(‘d‘,1)))
val result = rdd1 union rdd2
result.collect
(join 同理)

spark基础练习（未完)

原文：http://www.cnblogs.com/yangsy0915/p/4876262.html