首页 > 其他 > 详细

机器学习笔记(Washington University)- Clustering Specialization-week four

时间:2017-06-02 00:34:42      阅读:393      评论:0      收藏:0      [点我收藏+]

1. Probabilistic clustering model

  • (k-means) Hard assignments do not tell the full story, capture the uncertainty
  • k-means only considers the cluster center, not good for overlapping clusters,disparate cluster size,different shaped cluster
  • learn weights on dimensions
  • can learn cluster-specific weights on dimensions

 

2. Gaussian distribution

1-D gaussian is fully specified by mean μ and variance σ2.

2-D gaussian is fully specified by mean μ vector and covariance matrix Σ.

技术分享

 

 thusly our mixture model of gaussian is defined by 

k, μk, Σk

 

3. EM(Expectation maximization)

what if we knew the cluster parameters {πk, μk, Σk} ?

compute responsibilites:

技术分享

 

rik is the responsibility cluster k takes for observation i.

p is the probability of assignment to cluster k, given model parameters and observaed value.

πk is the initial probability of being from cluster k.

N is the gaussian model.

 

what if we knew the cluster soft assignments rij ?

技术分享

 

The procedure for the iterative algorithm:

1. initialize

2. estimate cluster responsibilities given current parameter estimates(E-step)

3. maximize likelihood given soft assignments

 

Notes:

EM is a coordinate-ascent algorithm

EM converges to a local mode

There are many ways to initialize the EM algorithm and it is important for convergence rates and quality of local mode

  • random choose k centroids
  • pick center sequentially like in k-means++
  • initilize k-means solution
  • grow mixture model by splitting until k clusters are formed

prevent overfitting

  • Do not let the variance goes down to zero, add small amount to diagonal of covariance estimate

 

机器学习笔记(Washington University)- Clustering Specialization-week four

原文:http://www.cnblogs.com/climberclimb/p/6931296.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!