Skip to main content

Section 5.5 Richardson Extrapolation and Romberg Integration

We have seen two different types of quadrature formulas here, both Newton-Cotes and Gaussian Quadrature evaluate the function on a fixed number of points, whereas the composite Newton-Cotes formulas refine the interval by breaking it up into subintervals first. This increases the accuracy of the method at the expense of increasing the number of operations.
In a manner similar to that of acceleration rootfinding as in Section sect:accel:conv, we will see in this section that we can use the Newton-Cotes composite formulas and an acceleration to speed up the rate of convergence of quadrature formulas.

Subsection 5.5.1 Richardson Extrapolation

Suppose that we have a method that has a predictable error formula. For example, let \(S\) be the exact answer and \(A(h)\) be the approximation that depends on \(h\text{.}\) Thus consider an approximation such that the error \(S-A(h)\) can be expressed,
\begin{equation} S - A(h) = k_{1} h + k_{2} h^{2} + k_{3} h^{3} + \cdots,\tag{5.5.1} \end{equation}
for some constants \(k_{1}, k_{2}, \ldots\text{.}\) This method is an \(O(h)\) method since
\begin{equation*} \begin{aligned}\lim_{h \rightarrow 0}\frac{S-A(h)}{h}\amp = \frac{k_{1} h + k_{2} h^{2} + k_{3} h^{3}+\cdots}{h}= k_{1}\end{aligned} \end{equation*}
is non-zero.
Richard extrapolation exploits knowledge of the order of a method to improve its order of convergence. If we evaluate (5.5.1) at \(h\) and \(h/2\) we get,
\begin{align} S\amp = A(h) + k_{1} h + k_{2} h^{2} + k_{3} h^{3} + \cdots \tag{5.5.2}\\ \amp \qquad \text{and} \notag\\ S\amp = A\biggl(\frac{h}{2}\biggr) + k_{1} \biggl(\frac{h}{2}\biggr) + k_{2} \biggl(\frac{h}{2}\biggr)^{2} + k_{3} \biggl(\frac{h}{2}\biggr)^{3} + \cdots \tag{5.5.3} \end{align}
Multiplying (5.5.3) by 2 and subtracting (5.5.2) results in
\begin{equation} 2S-S = S = 2 A \biggl(\frac{h}{2}\biggr) - A(h) + \hat{k}_{2} h^{2} + \hat{k}_{3} h^{3} + \cdots\tag{5.5.4} \end{equation}
where \(\hat{k}_{2}\) is a function of \(k_{2}\) and \(\hat{k}_{3}\) is a function of \(k_{3}\text{.}\) The \(h\) term has canceled. Also, the exact form of \(\hat{k}_{2}(k_{2})\) and \(\hat{k}_{3}(k_{3})\) does not matter.
For convenience, we label \(A_{1}(h) = A(h)\) and
\begin{equation*} A_{2}(h)= 2A\biggl(\frac{h}{2}\biggr) - A(h) = 2A_{1}\biggl(\frac{h}{2}\biggr) - A_{1}(h), \end{equation*}
and thus (5.5.4) can be written as:
\begin{equation} S = A_{2}(h) + \hat{k}_{2} h^{2} + \hat{k}_{3} h^{3} + \cdots\tag{5.5.5} \end{equation}
and evaluating (5.5.5) at \(h/2\) results in
\begin{align} S\amp = A_{2}\biggl(\frac{h}{2}\biggr) + \hat{k}_{2} \biggl(\frac{h}{2}\biggr)^{2} + \hat{k}_{3} \biggl(\frac{h}{2}\biggr)^{3} + \cdots \notag\\ \amp = A_{2} \biggl(\frac{h}{2}\biggr) + \frac{1}{4}\hat{k}_{2} h^{2} + \frac{1}{8}\hat{k}_{3} h^{3} + \cdots \tag{5.5.6} \end{align}
and similar to that above if we multiply (5.5.6) by 4 and subtract (5.5.5) the result is
\begin{equation*} \begin{aligned}3 S\amp = 4 A_{2} \biggl(\frac{h}{2}\biggr) - A_{2}(h) -\frac{1}{2}\hat{k}_{3} h^{3} +O(h^{4})\end{aligned} \end{equation*}
and dividing by 3 results in
\begin{equation*} \begin{aligned}S\amp = \frac{4}{3}A_{2} \biggl(\frac{h}{2}\biggr) - \frac{1}{3}A_{2}(h) -\frac{1}{6}\hat{k}_{3} h^{3} +O(h^{4})\end{aligned} \end{equation*}
and define \(A_{3}\) as the first two term on the right hand side or
\begin{equation*} \begin{aligned}A_{3}(h)\amp = \frac{4}{3}A_{2} \biggl(\frac{h}{2}\biggr) - \frac{1}{3}A_{2}(h)\end{aligned} \end{equation*}
If we continue this process, then we can define the following:
\begin{equation} A_{n}(h) = \frac{2^{n-1}}{2^{n-1}-1}A_{n-1}\biggl(\frac{h}{2}\biggr) - \frac{1}{2^{n-1}-1}A_{n-1}(h)\tag{5.5.7} \end{equation}
and \(A_{n}(h)\) is an order \(O(h^{n})\) approximation of \(S\text{.}\)
The values of \(A_{n}\) can be created in a table as follows:
\(O(h)\) \(O(h^{2})\) \(O(h^{3})\) \(O(h^{4})\) \(O(h^{5})\) \(\cdots\)
\(A_{1}(h)\)
\(A_{1}(h/2)\) \(A_{2}(h)\)
\(A_{1}(h/4)\) \(A_{2}(h/2)\) \(A_{3} (h)\)
\(A_{1}(h/8)\) \(A_{2}(h/4)\) \(A_{3} (h/2)\) \(A_{4}(h)\)
\(A_{1}(h/16)\) \(A_{2}(h/8)\) \(A_{3}(h/4)\) \(A_{4}(h/2)\) \(A_{5}(h)\)
By building up such a table, the values to the right and down are more accurate. Generally, the last diagonal element will be the most accurate.
The next example shows that we can use this method to improve the \(O(h)\) forward difference formula to approximate the derivative.

Example 5.5.1.

Create a Richardson Extrapolation table to \(O(h^{5})\) to find \(f'(0)\) for \(f(x) = \sin x\) using the \(O(h)\) forward difference formula in (5.1.1). Use \(h=\pi/2\text{.}\)
In this case we build the table above, row by row. We will need the following:
\begin{equation*} \begin{aligned}A_{1}(h)\amp = A_{1}\biggl(\frac{\pi}{2}\biggr) = \frac{f(\pi/2)-f(0)}{\pi/2}= \frac{1}{\pi/2}= \frac{2}{\pi}\approx 0.636619772367581 \\ A_{1}(h/2)\amp = A_{1} \biggl(\frac{\pi}{4}\biggr) = \frac{f(\pi/4)-f(0)}{\pi/4}= \frac{\sqrt{2}/2}{\pi/4}\approx 0.900316316157106 \\ A_{1}(h/4)\amp = A_{1} \biggl(\frac{\pi}{8}\biggr) = \frac{f(\pi/8)-f(0)}{\pi/8}\approx 0.974495358404433 \\ A_{1}(h/8)\amp = A_{1} \biggl(\frac{\pi}{16}\biggr) = \frac{f(\pi/16)-f(0)}{\pi/16}\approx 0.993586851144206 \\ A_{1}(h/16)\amp = A_{1} \biggl(\frac{\pi}{32}\biggr) = \frac{f(\pi/32)-f(0)}{\pi/32}\approx 0.998394393035618\end{aligned} \end{equation*}
Next, we create the table:
\(O(h)\) \(O(h^{2})\) \(O(h^{3})\) \(O(h^{4})\) \(O(h^{5})\)
0.63661977
0.90031631 1.16401285
0.97449535 1.04867440 1.01022825
0.99358685 1.01267834 1.00067965 0.99931556
0.99839439 1.00320193 1.00004313 0.99995219 0.99999464
where the first column in \(A_{1}(h/2^{k})\) as seen above and the \(j\)th column is \(A_{j}(h/2^{k})\text{,}\) calculated from (5.5.7) and using the column before it.
As you can tell, the last row, last column is the most accurate value of \(f'(0)\text{,}\) valid to 5 digits. This was based using the first column, of which the most accurate in that column was only 2 digits accurate.

Subsection 5.5.2 Richardson Extrapolation for other error schemes

The Richard Extrapolation explained above relied on the fact that the error between the actual value and the approximation were successive powers of \(h\) as seen in (5.5.1). If instead, we had the error structure:
\begin{align} S - A(h) \amp = k_{1} h^{2} + k_{2} h^{4} + k_{3} h^{6} + \cdots \notag\\ \amp = \sum_{n=0}^{\infty}k_{n} h^{2n} \tag{5.5.8} \end{align}
is another common error structure and here we will examine how to apply Richardson Extrapolation for such a situation. Solving above for \(S\) and evaluating it at \(h\) and \(h/2\text{,}\) the result is
\begin{align} S \amp = A(h) + k_{1} h^{2} + k_{2} h^{4} + k_{3} h^{6} + \cdots \tag{5.5.9}\\ S\amp = A\biggl(\frac{h}{2}\biggr) + k_{1}\biggl(\frac{h}{2}\biggr)^{2} + k_{2} \biggl(\frac{h}{2}\biggr)^{4} + k_{3} \biggl(\frac{h}{2}\biggr)^{6} + \cdots \tag{5.5.10} \end{align}
and taking a linear combination of these to eliminate the \(h^{2}\) term one takes 4 times (5.5.10) and subtracts (5.5.9) to result in
\begin{equation*} \begin{aligned}3S\amp = 4 A\biggl(\frac{h}{2}\biggr) - A(h) -\frac{3}{4}k_{2} h^{4} -\frac{15}{16}k_{3} h^{6} + \cdots\end{aligned} \end{equation*}
Defining \(A_{1}(h)=A(h)\) and then
\begin{equation*} \begin{aligned}A_{2}(h)\amp = \frac{4}{3}A_{1}\biggl(\frac{h}{2}\biggr) - \frac{1}{3}A_{1}(h)\end{aligned} \end{equation*}
Similarly, define
\begin{equation} A_{n}(h) = \frac{4^{n-1}}{4^{n-1}-1}A_{n-1}\biggl(\frac{h}{2}\biggr) - \frac{1}{4^{n-1}-1}A_{n-1}(h).\tag{5.5.11} \end{equation}
In performing Richardson extrapolation in this case, a table is create with the first column of \(A_{1}(h/2^{n})\text{,}\) the numerical approximation for the given step size. The remaining columns are filled in with (5.5.11).
In the next section, we will see how to apply Richardson extrapolation in this case to that of integration.

Subsection 5.5.3 Romberg Integration

The technique of Richardson Extrapolation can be applied to any situation in which the error formula can be written as powers of \(h\text{.}\) Romberg integration uses this with composite trapezoid rule to develop a fast-converging quadrature scheme.
It can be shown that composite trapezoid rule can be written as
\begin{equation*} \begin{aligned}\int_{a}^{b} f(x) \, dx\amp = \frac{h}{2}\biggl( f(x_{0}) + 2 \sum_{i=1}^{n-1}f(x_{i}) + f(x_{n}) \biggr) + \sum_{i=0}^{\infty}k_{i} h^{2i}\end{aligned} \end{equation*}
which fits the error scheme from (5.5.8). Thus Romberg integration creates a table based on the composite trapezoid rule and Richardson extrapolation.
The first column in a table will be the composite trapezoid rule. The \(m\)th row in this column will be the rule with \(2^{m-1}\text{.}\) Define \(h_{k}=(b-a)/2^{k-1}\text{.}\)
\begin{align*} R_{1,1}\amp = \frac{h_{1}}{2}\bigl( f(a)+f(b) \bigr) = \frac{b-a}{2}\bigl(f(a)+f(b) \bigr) \\ R_{2,1}\amp = \frac{h_{2}}{2}\bigl(f(a) +f(b) + 2 f(a+h_{2}) \bigr) \\ \amp = \frac{b-a}{4}\biggl( f(a) + f(b) + 2 f\biggl(a+ \frac{b-a}{2}\biggr) \biggr) \notag \\\amp = \frac{1}{2}\bigl( R_{1,1}+ h_{1} f(a+ h_{2}) \bigr) \\ R_{3,1}\amp = \frac{h_{3}}{2}\bigl(f(a)+f(b) + 2 [ f(a+h_{3}) +f(a+2h_{3}) + f(a+3h_{3})]\bigr) \\ \amp = \frac{b-a}{8}\bigl( f(a) + f(b) + 2 f(a+h_{2}) + 2 [ f(a+h_{3}) + f(a+3h_{3})] \bigr) \notag \\\amp = \frac{1}{2}\bigl( R_{2,1}+ h_{2} [f(a+h_{3}) + f(a+3h_{3})]\bigr) \end{align*}
and extending this in general leads to
\begin{equation} R_{n,1} = \frac{1}{2}\biggl( R_{n-1,1}+ h_{n-1}\sum_{k=1}^{2^{n-1}}f(a+(2k-1)h_{n}) \biggr).\tag{5.5.12} \end{equation}
Using this formula along with Richardson extrapolation, which applied in this case to (5.5.11) results in
\begin{equation} R_{n,j+1} = \frac{4^{j-1}}{4^{j-1}-1}R_{n,j}- \frac{1}{4^{j-1}-1}R_{n-1,j-1}\tag{5.5.13} \end{equation}
Typically a table is created using 1) \(R_{n,1}\) which is the trapezoid rule given in (5.5.12) and 2) the relation in (5.5.13). We will see this in the following example.

Example 5.5.2.

Perform Romberg integration to evaluate \(\int_{0}^{\pi/2}\cos x \, dx\text{.}\) Create a table of 4 rows.
Solution.
The first column, needs to be filled with composite trapezoid rule as explained above. That is, for \(R_{1,1}\) apply (eq:romberg:first). The other values of \(R_{n,1}\) is found using (eq:romberg:first:col).
\begin{equation*} \begin{aligned}R_{1,1}\amp = \frac{\pi/2}{2}\bigl( f(0) + f(\pi/2)\bigr) \approx 0.785398163397448 \\ R_{2,1}\amp = \frac{1}{2}\bigl(R_{1,1}+ \frac{\pi}{2}f(\pi/4) \bigr) \approx 0.948059448968520 \\ R_{3,1}\amp = \frac{1}{2}\bigl(R_{2,1}+ \frac{\pi}{4}[f(\pi/8)+f(3\pi/8)]\bigr) \approx 0.987115800972775 \\ R_{4,1}\amp = \frac{1}{2}\bigl(R_{3,1}+ \frac{\pi}{8}[f(\pi/16)+f(3\pi/16)+f(5\pi/16)+f(7\pi/16)]\bigr) \\\amp \approx 0.996785171886170 \\\end{aligned} \end{equation*}
Next, using (5.5.13), the columns are filled from left to right to get:
n \(R_{n,1}\) \(R_{n,2}\) \(R_{n,3}\) \(R_{n,4}\)
1 0.7853981633
2 0.9480594489 1.0022798774
3 0.9871158009 1.0001345849 0.9999915654
4 0.9967851718 1.0000082955 0.9999998762 1.0000000081
The result in \(R_{4,4}\) has an absolute error of about \(10^{-8}\text{,}\) which is remarkable in that the best approximation in the first column is only to 2 digits.
Generally Romberg integration is performed in a manner that builds up a table until the desired accuracy is found. That is, the table is created row by row until two consecutive numbers on the diagonal are found within the desired level of error. The following example shows this.

Example 5.5.3.

Use Romberg integration to find \(\int_{0}^{1} e^{-x^2}\, dx\) to within \(10^{-5}\text{.}\)
First, we start by calculating \(R_{1,1}\) and \(R_{2,1}\) using (eq:romberg:first) and (eq:romberg:first:col)
\begin{equation*} \begin{aligned}R_{1,1}= \frac{1}{2}\bigl( f(0) + f(1) \bigr) \approx 0.683939720585721 \\ R_{2,1}= \frac{1}{2}\bigl(R_{1,1}+ f(1/2) \bigr) \approx 0.731370251828563 \\\end{aligned} \end{equation*}
Then use (eq:romberg) to find \(R_{2,2}\text{:}\)
\begin{equation*} \begin{aligned}R_{2,2}= \frac{4}{3}R_{2,1}- \frac{1}{3}R_{1,1}= 0.747180428909510\end{aligned} \end{equation*}
Since \(|R_{2,2}-R_{1,1}| \gt 10^{-5}\text{,}\) another row of the table is built using (eq:romberg:first:col)
\begin{equation*} \begin{aligned}R_{3,1}= \frac{1}{2}\bigl(R_{2,1}+ \frac{1}{2}[f(1/4)+f(3/4)]\bigr) \approx 0.742984097800381\end{aligned} \end{equation*}
Then \(R_{3,2}\) and \(R_{3,3}\) are computed using (eq:romberg)
\begin{equation*} \begin{aligned}R_{3,2}\amp = \frac{4}{3}R_{3,1}-\frac{1}{3}R_{2,1}= 0.746855379790987 \\ R_{3,3}\amp = \frac{16}{15}R_{3,2}-\frac{1}{15}R_{2,2}= 0.746833709849753\end{aligned} \end{equation*}
Since \(|R_{3,3}-R_{2,2}|\approx 3 \cdot 10^{-4}\text{,}\) this is still not within the desired error. At least one more row must be calculated.
\begin{equation*} \begin{aligned}R_{4,1}\amp = \frac{1}{2}\bigl(R_{3,1}+ \frac{1}{4}[f(1/8)+f(3/8)+f(5/8)+f(7/8)]\bigr) \\\amp \approx 0.745865614845695\end{aligned} \end{equation*}
Then \(R_{4,2}\text{,}\) \(R_{4,3}\) and \(R_{4,4}\) are computed:
\begin{equation*} \begin{aligned}R_{4,2}\amp = \frac{4}{3}R_{4,1}-\frac{1}{3}R_{3,1}= 0.746826120527465 \\ R_{4,3}\amp = \frac{16}{15}R_{4,2}-\frac{1}{15}R_{3,2}= 0.746824169909898 \\ R_{4,4}\amp = \frac{64}{63}R_{4,3}-\frac{1}{63}R_{3,3}= 0.746824018482282\\\end{aligned} \end{equation*}
and now \(|R_{4,4}-R_{3,3}|= 9 \cdot 10^{-6}<10^{-5}\text{,}\) so we can stop here. The approximate value of the integral is \(R_{4,4}\text{.}\)