首页 > 其他 > 详细

How to convert matrix to RDD[Vector] in spark

时间:2017-07-21 11:32:52      阅读:446      评论:0      收藏:0      [点我收藏+]

The matrix is generated from SVD, and I am using the results from SVD to do clustering analysis.

 if your clustering only supports RDD as its input, here‘s how you can do the transformation

  def toRDD(sc :SparkContext,m: Matrix): RDD[Vector] = {
        val columns: Iterator[Array[Double]] = m.toArray.grouped(m.numRows)
//        val rows: Seq[Array[Double]] = columns.toSeq // Skip this if you want a column-major RDD.
        val rows: Seq[Seq[Double]] = columns.toSeq.transpose // Skip this if you want a column-major RDD.
        val vectors: Seq[DenseVector] = rows.map(row => new DenseVector(row.toArray))
        sc.parallelize(vectors)

  

How to convert matrix to RDD[Vector] in spark

原文:http://www.cnblogs.com/xiaoma0529/p/7216802.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!