Differentiation

Derivatives

Definition: Suppose $f(x_1,\dots ,x_n):U\subset {\mathbb{R}}^n\to \mathbb{R}$ the partial derivative of $f$ with respect to variable $x_i$ is $\frac{\delta f}{\delta x_i}=\underset{h\to 0}{\lim }\frac{f(x_1,\dots ,x_i+h,\dots ,x_n)-f(x_1,\dots x_n)}{h}$

Definition: $f:U\subset {\mathbb{R}}^n\mapsto {\mathbb{R}}^m$ is differentiable at $x_0$ if $\underset{x\to x_0}{\lim }\frac{\Vert f(x)-f(x_0)-T(x_0)(x-x_0)\Vert }{\Vert x-x_0\Vert }=0$
What is $T(x_0)$? It is some matrix $\in {\mathbb{M}}_{m\times n}$
More precisely, say $f(x_1,\dots x_n)=(f_1(x_1,\dots ,x_n),\dots f_m(x_1,\dots ,x_n))$

$$T(x_0)=\left(\begin{array}{cccc}\frac{\delta f_1}{\delta x_1}& \frac{\delta f_1}{\delta x_2}& \dots & \frac{\delta f_1}{\delta x_n}\\ \frac{\delta f_2}{\delta x_1}& \frac{\delta f_2}{\delta x_2}& \dots & \frac{\delta f_2}{\delta x_n}\\ ⋮& ⋮& ⋮& ⋮\\ \frac{\delta f_m}{\delta x_1}& \frac{\delta f_m}{\delta x_2}& \dots & \frac{\delta f_m}{\delta x_n}\end{array}\right)=\mathbf{D}f(x_0)$$

Example: $f:{\mathbb{R}}^2\mapsto {\mathbb{R}}^2,f(x,y)=(e^{x+y}+y,y^{2}x)$
$\mathbf{D}f(x)=\left(\begin{array}{cc}{e}^{x+y}& e^{x+y}+1\\ y^2& 2xy\end{array}\right)$

Definition: The gradient of a multivariable single-valued function is $\overrightarrow{\nabla f(x)}=\left(\frac{\delta f}{\delta x_1},\dots ,\frac{\delta f}{\delta x_n}\right)$

Theorem: Say $f:U\subset {\mathbb{R}}^n\mapsto {\mathbb{R}}^m$ is differentiable at $x_0$, then $f$ is also continuous at $x_0$
Theorem: $f:U\subset {\mathbb{R}}^n\mapsto {\mathbb{R}}^m$, suppose $\frac{\delta f_i}{\delta x_j}$ for all $1\le i\le m$ and $1\le j\le n$ exists and are continuous at $x_0$, then $f$ is differentiable

Paths and Curves

Definition: $C:[a,b]\subset \mathbb{R}\mapsto {\mathbb{R}}^n$ is a path
A curve is a set of points which belongs to a range for some path $C$

The tangent vector along a curve $c:[a,b]\mapsto {\mathbb{R}}^3,c(t)=(x(t),y(t),z(t))$ at point $t_0$ is $c^{\prime }(t_0)=(x^{\prime }(t_0),y^{\prime }(t_0),z^{\prime }(t_0))$

The chain rule applies here, $f:{\mathbb{R}}^2\mapsto {\mathbb{R}}^2,c:[a,b]\mapsto {\mathbb{R}}^2$ then $(f\circ c{)}^{\prime }(t)=\mathbf{D}f(c(t))\cdot c^{\prime }(t)$

Higher Order Derivatives

Observation: $f:{\mathbb{R}}^2\to \mathbb{R}$, if $\frac{\partial f}{\partial x},\frac{\partial f}{\partial y}$ are continuous at $(x_0,y_0)$ then $f$ is differentiable at $(x_0,y_0)$
Then, if these partial derivatives have their own continuous partial derivatives, then $f$ is a twice differential function, $f\in C^2$

Higher order partial derivatives work the same as regular ones

Weird observation, after a few examples it looks like $\frac{{\partial }^{2}f}{\partial x\partial y}=\frac{{\partial }^{2}f}{\partial y\partial x}$!

Theorem: If $f(x,y)$ is twice differentiable then the mixed partial derivatives $\frac{{\partial }^{2}f}{\partial x\partial y}=\frac{{\partial }^{2}f}{\partial y\partial x}$ are equal

Proof (sort of):
$\frac{{\partial }^{2}f}{\partial x\partial y}=\underset{h\to 0}{\lim }\frac{\frac{\partial }{\partial y}f(x+h,y,z)-\frac{\partial }{\partial y}f(x,y,z)}{h}=\underset{h\to 0}{\lim }\frac{\underset{l\to 0}{\lim }\frac{f(x+h,y+l,z)-f(x+h,y,z)}{l}-\underset{l\to 0}{\lim }\frac{f(x,y+l,z)-f(x,y,z)}{l}}{h}=\underset{h,l\to 0}{\lim }\frac{f(x+h,y+l,z)-f(x+h,y,z)-f(x,y+l,z)+f(x,y,z)}{hl}$
$\frac{{\partial }^{2}f}{\partial y\partial x}=\underset{l\to 0}{\lim }\frac{\frac{\partial }{\partial x}f(x,y+l,z)-\frac{\partial }{\partial x}f(x,y,z)}{l}=\underset{l\to 0}{\lim }\frac{\underset{h\to 0}{\lim }\frac{f(x+h,y+l,z)-f(x,y+l,z)}{h}-\underset{h\to 0}{\lim }\frac{f(x+h,y,z)-f(x,y,z)}{h}}{l}=\underset{h,l\to 0}{\lim }\frac{f(x+h,y+l,z)-f(x+h,y,z)-f(x,y+l,z)+f(x,y,z)}{hl}$
One issue with this is that the limit switching operation is not strictly allowed