Floating Point Numbers

Due to practical considerations, computers do not do exact floating point math
In most programming situations, this is not an issue, but our calculations in numerical methods can be affected

IEEE 754 (1985) is the modern floating point number specification

$\mathcal{F}(k,B,N_{\text{min}},N_{\text{max}})=\pm (\frac{d_1}{B}+\frac{d_2}{B^2}+\cdots +\frac{d_k}{B^k})\cdot B^n$
$=\pm 0.d_1{d}_2\dots d_k\cdot B^n$
$=\pm d_1{d}_2\dots d_k\cdot B^{n-k}$
where $0\le d_i<B$, $d_1>0$, $N_{\text{min}}\le n\le N_{\text{max}}$

In these equations, $d_i{d}_j$ refers to concatenation

The smallest number that can be stored is $B^{N_{\text{min}}-1}$ and the largest is $(1-B^{-k})B^{N_{\text{max}}}$

How do we round a real number to a floating point number? Set it to the closest floating point number, and if it’s equidistant then set it to whichever ends in an even digit (and if it’s outside bounds then we get an overflow or underflow)

Suppose $\tilde{r}$ approximates $r$, the absolute error is $|\tilde{r}-r|$ and the relative error is $\frac{|\tilde{r}-r|}{|r|}$

We define machine accuracy ${ϵ}_m=\frac{1}{2}{B}^{1-k}$ (also known as unit roundoff) as the smallest positive floating point number that when added to 1 produces a different number

The standard is called double precision which takes 64 bits, $B=2$, $k=53$, $N_{\text{min}}=-1021$, $N_{\text{max}}=1024$

Theorem: If $x\in \mathbb{R}$ and $|x|$ lies between the smallest and largest positive numbers in $\mathcal{F}$ then $\text{fl}(x)=x(1+\delta )$, $|\delta |<{ϵ}_m$ or in other words, the relative number is less than ${ϵ}_m$

How do we do operations within $\mathcal{F}$? Our floating point numbers are not closed under regular operations, so we must define floating point operations $\oplus ,\ominus ,\otimes ,\oslash$

Essentially, we just round the result of each operation to keep the result within $\mathcal{F}$, and on an individual basis these operations behave well in terms of error

However, we must be mindful that errors can propagate and sometimes become problematic