Chain Rule

Definition

One Variable Case

The chain rule is a formula that express the derivative of the composition of two differentiable functions in terms of the derivatives of each. $(f\circ g{)}^{\prime }(x)=f^{\prime }(g(x))g^{\prime }(x)$ In Leibniz’s notation, if $y=f(u)$ and $u=g(x)$, then $\frac{dy}{dx}=\frac{dy}{du}\frac{du}{dx}$

Multivariable Case

Consider a function $f(g_1,g_2,\dots ,g_m):{\mathbb{R}}^m\to \mathbb{R}$ where $g_i(x_1,x_2,\dots ,x_n):{\mathbb{R}}^n\to \mathbb{R}\forall i=1,2,\dots ,m$. The partial derivatives of $f$ with respect to $x_j$ is given by the formula $\frac{\partial f}{\partial x_j}=\sum _{i=1}^m\frac{\partial f}{\partial g_i}\frac{\partial g_i}{\partial x_j},\forall j=1,\dots ,n$

Composition

Suppose differentiable functions $f:D\subset {\mathbb{R}}^n\to {\mathbb{R}}^m$ and $g:D^{\prime }\subset {\mathbb{R}}^m\to {\mathbb{R}}^k$ and assume that for each $\mathbf{x}\in D$, $f(\mathbf{x})\in D^{\prime }$. Then $g\circ f$ is differentiable and the gradient of the composite function is defined as $\nabla (g\circ f)(\mathbf{x})=\nabla g(f(\mathbf{x}))\cdot \nabla f(\mathbf{x})$ here $\nabla g(f(\mathbf{x}))$ is $k\times m$, and $\nabla f(\mathbf{x})$ is $m\times n$ Jacobian matrices, and the result $\nabla (g\circ f)$ is $k\times n$ matrix.

Link to original