机器学习的一些笔记

$$h_\theta(x) = \theta_0 + \theta_1 x$$

$$\min {1 \over 2n}\sum_{i = 1}^n [h_\theta(x_i) - y_i]^2$$

$$h_\theta(x) = \theta_0 + \theta_1 x$$

$$\theta_0, \theta_1$$

$$J(\theta_0, \theta_1) = {1 \over 2n} \sum_{i = 1}^n [h_\theta(x_i) - y_i]^2$$

$$\min J(\theta_0, \theta_1)$$

$$\theta_j := \theta_j - \alpha {\partial J(\theta_0, \theta_1)\over \partial \theta_j} \quad (\text{for j = 0 and j = 1})$$

$$\begin{aligned} {\partial J(\theta_0, \theta_1)\over \partial \theta_0} =& {\displaystyle \partial \left({1 \over 2n} \sum_{i = 1}^n [\theta_0 + \theta_1 x_i - y_i]^2 \right) \over \partial \theta_0} \\ =& {1 \over 2n} \cdot {\displaystyle \partial \left(\sum_{i = 1}^n [\theta_0 + \theta_1 x_i - y_i]^2 \right) \over \partial \theta_0} \\ =& {1 \over 2n} \cdot {\displaystyle \partial \left(\sum_{i = 1}^n [\theta_0^2 + 2(\theta_1 x_i - y_i)\theta_0 + (\theta_1 x_i - y_i)^2 ] \right) \over \partial \theta_0} \\ =& {1 \over n} \sum_{i = 1}^n [\theta_0 + (\theta_1 x_i - y_i)] \\ =& {1 \over n} \sum_{i = 1}^n [h_\theta(x_i) - y_i] \\ \end{aligned}$$

$$\begin{aligned} {\partial J(\theta_0, \theta_1)\over \partial \theta_1} =& {\displaystyle \partial \left({1 \over 2n} \sum_{i = 1}^n [\theta_0 + \theta_1 x_i - y_i]^2 \right) \over \partial \theta_1} \\ =& {1 \over 2n} {\displaystyle \partial \left(\sum_{i = 1}^n [\theta_0 + \theta_1 x_i - y_i]^2 \right) \over \partial \theta_1} \\ =& {1 \over 2n} {\displaystyle \partial \left(\sum_{i = 1}^n [\theta_0^2 + 2(\theta_1 x_i - y_i)\theta_0 + (\theta_1 x_i - y_i)^2 ] \right) \over \partial \theta_1} \\ =& {1 \over 2n}\sum_{i = 1}^n [2\theta_0 x_i + 2(\theta_1 x_i - y_i) x_i ] \\ =& {1 \over n}\sum_{i = 1}^n [\theta_0 x_i + (\theta_1 x_i - y_i) x_i ] \\ =& {1 \over n}\sum_{i = 1}^n (\theta_0 + \theta_1 x_i - y_i) x_i \\ =& {1 \over n} \sum_{i = 1}^n [h_\theta(x_i) - y_i]x_i \\ \end{aligned}$$

$$\begin{aligned} \theta_0 :=& \theta_0 - \alpha {1 \over n} \sum_{i = 1}^n [h_\theta(x_i) - y_i] \\ \theta_1 :=& \theta_1 - \alpha {1 \over n} \sum_{i = 1}^n [h_\theta(x_i) - y_i]x_i \end{aligned}$$

$$f(x + \delta x) \approx f(x) + \nabla f(x) \cdot \delta x + \delta x \cdot H \cdot \delta x$$

$$f(x + \delta x) \approx f(x) + \nabla f(x) \cdot \delta x$$

$$f(x + \delta x) - f(x) \approx \nabla f(x) \cdot \delta x \leqslant 0$$

$$f(x_1, x_2) = x_1^2 + 2 x_2^2 + x_1$$

$$W_t = W_{t - 1} - \eta \cdot \Delta W$$

$$v_t =\beta v_{t - 1} + (1 - \beta) g_w$$

$$\hat{v}_t = {v_t \over 1 - \beta^t}$$

$$w_t = w_{t - 1} - \eta \cdot \hat{v}_t$$

$$\Delta W_t = {\partial J(W_{t - 1} + \gamma V_{t - 1}) \over \partial W}$$

$$W_t = W_{t - 1} - {\eta \over \sqrt{S_t} + \varepsilon} \cdot \Delta W_t$$

$$S_t = S_{t - 1} + \Delta W_t \cdot \Delta W_t$$

$$\Delta W_t = {\partial J(W_{t - 1}) \over \partial W}$$