从公式的角度理解L2和L1正则

L2正则

$$C=C_0+\frac{\lambda}{n}\sum_{i=1}^n{w_i^2}$$

$$\frac{\partial C}{\partial w}=\frac{\partial C_0}{\partial w}+\frac{\lambda}{2n}w$$

\begin{equation}w\to w‘=w-\eta\frac{\partial C}{\partial w}=\left(1-\frac{\eta\lambda}{n}\right)w-\eta\frac{\partial C_0}{\partial w}\label{g1}\end{equation}

L1正则

 $$C=C_0+\frac{\lambda}{n}\sum_{i=1}^n{|w_i|}$$

$$\frac{\partial C}{\partial w}=\frac{\partial C_0}{\partial w}+\frac{\lambda}{n}\textrm{sgn}(w)$$

$$\textrm{sgn}(w)=\left\{\begin{matrix}1 & \textrm{if}\;w\geqslant 0\\0 & \textrm{if}\;w<0\end{matrix}\right.$$

$$w \to w-\frac{\eta\lambda}{n}\textrm{sgn}(w)-\eta\frac{\partial C_0}{\partial w}=w\pm\frac{\eta\lambda}{n}-\eta\frac{\partial C_0}{\partial w}$$

从公式的角度理解L2和L1正则

原文：https://www.cnblogs.com/zhangchaoyang/p/12902212.html