Eli Bendersky's website - Math

Notes on Fourier series

2026-05-27T19:30:00-07:00

The trigonometric Fourier series is a beautiful mathematical theory that shows how to decompose a periodic function into an infinite sum of sinusoids. These are my notes on the subject, with some examples and the connection to linear algebra in Hilbert space.

Coefficients of Fourier series

Let’s assume that is a well-behaved 2L-periodic [1] function and that we can find coefficients a_n and b_n such that:

\[f(x)=\sum_{n=0}^{\infty}\left(a_n cos\frac{n\pi x}{L}+b_n sin\frac{n\pi x}{L}\right)\]

Then we say that the Fourier series on the right-hand side converges to . We’ll talk more about the assumptions mentioned above and convergence in the next section.

Note that when n=0, the sum becomes just ; therefore it’s customary to write the series starting with n=1, with a separate constant component (which is the function's average over one period). To make computations nicer, this constant is typically called a_0 / 2, so:

\[f(x)=\frac{a_0}{2}+\sum_{n=1}^{\infty}\left(a_n cos\frac{n\pi x}{L}+b_n sin\frac{n\pi x}{L}\right)\]

Our goal is to find the coefficients a_n and b_n that satisfy this equation. We’ll do this in three steps.

Step 1: Integrate both sides of the equation between -L and L [2].

\[\int_{-L}^{L}f(x)dx=\int_{-L}^{L}\frac{a_0}{2}dx+\sum_{n=1}^{\infty}\bigg (\int_{-L}^{L}a_n cos\frac{n\pi x}{L}dx+\int_{-L}^{L}b_n sin\frac{n\pi x}{L}dx\bigg )\]

Per Appendix A, all integrals within the sum are zero, so we’re left with:

\[\int_{-L}^{L}f(x)dx=\int_{-L}^{L}\frac{a_0}{2}dx=\bigg[\frac{x\cdot a_0}{2}\bigg]_{-L}^{L}=a_0\cdot L\]

And thus we find :

\[a_0=\frac{1}{L}\int_{-L}^{L}f(x)dx\]

Step 2: Multiply both sides by cos\frac{m\pi x}{L} (m is a positive integer constant) and integrate between -L and L.

\[\begin{aligned} \int_{-L}^{L}f(x)cos\frac{m\pi x}{L}dx&=\int_{-L}^{L}\frac{a_0}{2}cos\frac{m\pi x}{L}dx\\ &+\sum_{n=1}^{\infty}\bigg (\int_{-L}^{L}a_n cos\frac{n\pi x}{L}cos\frac{m\pi x}{L}dx+\int_{-L}^{L}b_n sin\frac{n\pi x}{L}cos\frac{m\pi x}{L}dx\bigg ) \end{aligned}\]

Looking at the right-hand side, the first integral is zero per Appendix A, and the last integral is zero per Appendix B. We’re left with:

\[\int_{-L}^{L}f(x)cos\frac{m\pi x}{L}dx=\sum_{n=1}^{\infty}\int_{-L}^{L}a_n cos\frac{n\pi x}{L}cos\frac{m\pi x}{L}dx\]

Per Appendix B, the integral on the right is zero for all n\neq m, and L for n=m. Therefore, we can write:

\[\int_{-L}^{L}f(x)cos\frac{m\pi x}{L}dx=a_m\cdot L\]

Recall that m is an arbitrary integer, just like ; for consistency, we’ll replace m by and isolate a_n:

\[a_n=\frac{1}{L}\int_{-L}^{L}f(x)cos\frac{n\pi x}{L}dx\]

Step 3: Hopefully it’s clear where this is going now; multiply both sides by sin\frac{m\pi x}{L} and integrate between -L and L. Using a very similar reasoning to step 2, we’ll end up with:

\[b_n=\frac{1}{L}\int_{-L}^{L}f(x)sin\frac{n\pi x}{L}dx\]

We’ve just found a way to calculate all the coefficients of our Fourier series for :

\[f(x)=\frac{a_0}{2}+\sum_{n=1}^{\infty}\left(a_n cos\frac{n\pi x}{L}+b_n sin\frac{n\pi x}{L}\right)\]

Where:

\[\begin{aligned} a_0&=\frac{1}{L}\int_{-L}^{L}f(x)dx\\ a_n&=\frac{1}{L}\int_{-L}^{L}f(x)cos\frac{n\pi x}{L}dx\\ b_n&=\frac{1}{L}\int_{-L}^{L}f(x)sin\frac{n\pi x}{L}dx \end{aligned}\]

Conditions on f and convergence of Fourier series

The previous section discusses Fourier series for a function that is well-behaved - but what does that mean? The full answer would lead us deep into analysis, which I’d like to avoid here. So I’ll keep it brief.

We typically assume that is square integrable, which is denoted as L^2. Moreover, we assume that the function is piecewise smooth: each segment of the function has continuous derivatives. A very simple example of a piecewise smooth function is f(x)=|x|. Another is the triangular wave function used in the example below.

These conditions hold for pretty much any reasonable function we want to approximate using Fourier series, so they aren’t a serious burden.

For a function that satisfies these conditions, it’s guaranteed to have a Fourier series that pointwise converges to it. This means that at every continuous point of , the Fourier series converges to it exactly; at every jump point, the Fourier series converges to the mid-point of the jump.

Cosine and Sine series

Sometimes, additional properties of the function can help us simplify the Fourier series for it. If f_e(x) is an even function, then we know that:

\[b_n=\frac{1}{L}\int_{-L}^{L}f(x)sin\frac{n\pi x}{L}dx=0\]

Because the function inside the integral is odd, and integrating an odd function over a symmetric interval results in 0.

Therefore, the Fourier series for such f_e(x) is a cosine series:

\[f_e(x)=\frac{a_0}{2}+\sum_{n=1}^{\infty}a_n cos\frac{n\pi x}{L}\]

With coefficients and a_n given as before.

Similarly if f_o(x) is an odd function, then its and a_n are 0, and its Fourier series is a sine series:

\[f_o(x)=\sum_{n=1}^{\infty}b_n sin\frac{n\pi x}{L}\]

Fourier series for a non-periodic function defined on an interval

So far we’ve been talking about 2L-periodic functions that can be faithfully represented by Fourier series. But what if we have a non-periodic function defined on a finite interval?

E.g. suppose we have f(x)=x on the interval [0,L]. Can we approximate it with a Fourier series?

Yes! First, we have to make a choice of how to extend the function to the negative interval [-L,0]. Then, we simply repeat the function every 2L - this is called a periodic extension. Note that the Fourier series calculation only cares about the range [-L,L]. The resulting series will approximate the generated periodic function in its entirety, and in particular will also converge to it in the [0,L] interval (except maybe the endpoints, depending on the mode of extension).

There are several natural ways to extend a function defined on [0,L] into the interval [-L,0] [3]:

Direct periodic repetition: we simply repeat every L: f(x+L)=f(x)\ \forall x.
Even extension: f(|x|)
Odd extension: when x\ge 0 and -f(-x) when x<0.

Here’s an example of extending our sample function f(x)=x onto the full interval [-L,L] and then repeating it periodically every 2L:

Note that the Fourier series for these extended functions will be different. However, they will all converge to in the interval [0,L]. Typically, even and odd extensions have the benefit of producing either cosine or sine series, correspondingly (as discussed in the previous section).

We’ve seen that Fourier series work well for periodic functions and also non-periodic functions defined on a finite domain (because we can extend these periodically). But what about aperiodic functions defined on the entire real line? This is where we’ll have to leave Fourier series behind and move on to their generalization - the Fourier transform; this will be a topic for a separate post.

Example

Let’s take the following triangular function t(x) [4]:

t(x) is periodic with period 4. We can define it by starting with a formula on the interval [0,2]:

\[t(x)= \begin{cases} x & 0 \leq x \leq 1 \\ 2-x & 1 < x \leq 2 \\ \end{cases}\]

Then making an odd extension into [-2,0] and repeating it periodically. Now we can go ahead to calculate its Fourier coefficients.

Since this function is odd, we know that we’ll get a sine series, as a_n are going to be 0 for all . Let’s calculate b_n; in our case L=2 (half the period).

\[b_n=\frac{1}{2}\int_{-2}^{2}t(x)sin\frac{n\pi x}{2}dx\]

Since t(x) is odd and so is the sine, we’re integrating an even function over a symmetric interval. Therefore, we only have to integrate on the positive half of the range and multiply the result by two:

\[b_n=\int_{0}^{2}t(x)sin\frac{n\pi x}{2}dx\]

Let’s set k=\frac{n\pi}{2}:

\[b_n=\int_{0}^{2}t(x)sin(kx)dx\]

And split up the integral for the different segments of t(x):

\[b_n=\int_{0}^{1}x\cdot sin(kx)dx+\int_{1}^{2}(2-x)sin(kx)dx\]

The first integral, by the method described in Appendix C:

\[I_1=\int_{0}^{1}x\cdot sin(kx)dx=\bigg[\frac{-x cos(kx)}{k}+\frac{sin(kx)}{k^2} \bigg]_{0}^{1}=\frac{sin(k)}{k^2}-\frac{cos(k)}{k}\]

The second integral can also be split into two:

\[I_2=\int_{1}^{2}2sin(kx)dx - \int_{1}^{2}x\cdot sin(kx)dx\]

The first of these is trivial to calculate; the second can once again use Appendix C. After some tedious but straightforward calculations [5] we’ll get:

\[I_2=\frac{cos(k)}{k}+\frac{sin(k)-sin(2k)}{k^2}\]

Adding I_1+I_2, we get:

\[\begin{aligned} b_n=I_1+I_2&=\frac{sin(k)}{k^2}-\frac{cos(k)}{k}+\frac{cos(k)}{k}+\frac{sin(k)-sin(2k)}{k^2}\\ &=\frac{2sin(k)-sin(2k)}{k^2} \end{aligned}\]

Now let’s substitute k=\frac{n\pi}{2} back. This makes sin(2k) zero because the sine of an integer multiple of \pi is always zero:

\[b_n=\frac{2sin \frac{n\pi}{2}}{\left (\frac{n\pi}{2}\right )^2}=\frac{8sin \frac{n\pi}{2}}{n^2\pi^2}\]

We have b_n, so the Fourier series for our t(x) is:

\[t(x)=\sum_{n=1}^{\infty}\frac{8}{n^2\pi^2}sin\frac{n\pi}{2}sin\frac{n\pi x}{2}\]

Note that for even values of , sin \frac{n\pi}{2} is zero, so only the odd terms remain:

\[t(x)=\frac{8}{\pi^2}\bigg[ sin\frac{\pi x}{2}-\frac{1}{3^2} sin\frac{3\pi x}{2}+\frac{1}{5^2}sin\frac{5\pi x}{2}-\cdots\bigg]\]

Here’s an interactive chart showing how the series t(x) converges to our triangular function. You can set the number of terms in the Fourier series and see the effect (red line). Note that all even coefficients are zero so it will look the same for as for n-1 when is odd.

n (terms in the Fourier series)

Compact formula using a single phase-shifted sinusoid

We’ve written the Fourier series for as follows so far:

\[f(x)=\frac{a_0}{2}+\sum_{n=1}^{\infty}\left(a_n cos\frac{n\pi x}{L}+b_n sin\frac{n\pi x}{L}\right)\]

We can rewrite this in a somewhat more compact form, using a single sinusoid with a configurable phase at each :

\[f(x)=\frac{a_0}{2}+\sum_{n=1}^{\infty}q_n\cdot cos\left(\frac{n\pi x}{L}+\theta_n\right)\]

Based on Appendix D, q_n and \theta_n can be computed as follows:

\[\begin{aligned} q_n&=\sqrt{a_n^2+b_n^2}\\ \theta_n&=\operatorname{atan2}(-b_n,a_n) \end{aligned}\]

When Fourier series are used in the context of signal processing, this formulation is easier to reason about because it represents the magnitude and phase shift of each harmonic of in the frequency domain [6]

Complex Fourier series

It should not come as a surprise that the Fourier series, being a combination of trigonometric functions, can also be represented with complex exponential functions.

Specifically, we’ll show that our can be approximated as follows:

\[f(x)=\sum_{n=-\infty}^{\infty}C_n\cdot e^{in\pi x/L}\]

Let’s calculate C_n. We proceed in a manner similar to before, by multiplying both sides of the equation by e^{-im\pi x/L} and taking an integral in the range [-L,L]:

\[\begin{aligned} \int_{-L}^{L}f(x)e^{-im\pi x/L}dx&=\sum_{n=-\infty}^{\infty}\int_{-L}^{L}C_n\cdot e^{in\pi x/L}e^{-im\pi x/L}dx\\ &=\sum_{n=-\infty}^{\infty}\int_{-L}^{L}C_n\cdot e^{i(n-m)\pi x/L}dx \end{aligned}\]

By Appendix A, the sum elements are all zero when n\neq m. When n=m, we get:

\[\int_{-L}^{L}f(x)e^{-im\pi x/L}dx=\int_{-L}^{L}C_m\cdot 1 \cdot dx=2LC_m\]

Therefore, renaming m to (since it’s just an arbitrary integer constant):

\[C_n=\frac{1}{2L}\int_{-L}^{L}f(x)e^{-in\pi x/L}dx\]

We’ve found an alternative formulation to Fourier series, using complex exponentials instead of trigonometric functions. While this was a direct derivation, another way to achieve the same result is to use the Euler Formula to derive:

\[\begin{aligned} cos\theta&=\frac{e^{i\theta}+e^{-i\theta}}{2}\\ sin\theta&=\frac{e^{i\theta}-e^{-i\theta}}{2i} \end{aligned}\]

And substitute these into the original Fourier series formula. I’ll leave this as an exercise for the diligent reader; eventually, the result will be the same. Moreover, it’s possible to show a direct correspondence between a_n, b_n and C_n, for n>0:

\[\begin{aligned} C_0&=\frac{a_0}{2}\\ C_n&=\frac{a_n-ib_n}{2}\\ C_{-n}&=\frac{a_n+ib_n}{2}\\ \end{aligned}\]

Note that C_{-n}=C_n^* when both a_n and b_n are real (which is the case for a real-valued ). This helps explain why the complex formulation has negative frequencies in the sum; when the function is actually real, each negative frequency is paired up with a positive frequency and the result is real [7]:

\[\begin{aligned} C_n e^{in\pi x/L}+C_{-n} e^{-in\pi x/L}&=C_n e^{in\pi x/L}+C_n^* e^{-in\pi x/L}\\ &=C_n e^{in\pi x/L}+\left(C_{n} e^{in\pi x/L}\right)^*\\ &=2\operatorname{Re}\bigg(C_{n} e^{in\pi x/L}\bigg) \end{aligned}\]

So, for a real function we only need to account for positive frequencies:

\[f(x)=C_0+\sum_{n=1}^{\infty}2\operatorname{Re}\bigg(C_{n} e^{in\pi x/L}\bigg)\]

We can take it further. C_n is a complex number, so let’s represent it in polar form as C_n=\frac{q_n}{2} e^{i\theta_n} (the factor of half will make sense soon). Then:

\[\begin{aligned} \operatorname{Re}\bigg(C_{n} e^{in\pi x/L}\bigg)&=\operatorname{Re}\bigg(\frac{q_n}{2} e^{i\theta_n}e^{in\pi x/L}\bigg)\\ &=\frac{q_n}{2}\operatorname{Re}\bigg(e^{i(n\pi x/L + \theta_n)}\bigg)\\ &=\frac{q_n}{2} cos\bigg(\frac{n\pi x}{L}+\theta_n\bigg) \end{aligned}\]

And substituting back into the sum:

\[f(x)=C_0+\sum_{n=1}^{\infty}q_n cos\bigg(\frac{n\pi x}{L}+\theta_n\bigg)\]

This is precisely the compact formulation from the previous section!

Fourier orthogonal basis in Hilbert space

The most beautiful aspect of Fourier theory is that it doesn’t just happen to work by chance, and is deeply connected to linear algebra. Please read my post on Hilbert space before proceeding.

The space of real-valued square integrable functions L^2 forms a Hilbert space, in which we can define the inner product (assuming real functions):

\[\langle f,g \rangle=\int_{-L}^{L}f(x)g(x) dx\]

We’ve demonstrated that the family of functions:

\[1,\qquad cos\frac{n\pi x}{L},\qquad sin\frac{n\pi x}{L}\]

Are all mutually orthogonal, because their pairwise inner products are zero! We’ve also shown that any function in L^2 can be represented as a weighted sum of these functions:

\[f(x)=\frac{a_0}{2}+\sum_{n=1}^{\infty}\left(a_n cos\frac{n\pi x}{L}+b_n sin\frac{n\pi x}{L}\right)\]

So these functions form a basis for L^2. When we think of these functions as vectors (in an infinite Hilbert space), much of what we did in this post starts feeling like "normal" linear algebra. For example, when we have a set of basis vectors and we want to know how to represent some vector in this basis, we usually find the coefficients by projecting it onto the basis. E.g. with a basis vector e_1, the coefficient of :

\[c=\frac{\langle v, e_1\rangle}{\langle e_1, e_1\rangle}\]

Similarly, when we calculate the coefficient b_n for some function , we project onto the basis vector sin\frac{n\pi x}{L} by calculating:

\[b_n=\frac{\langle f(x), sin\frac{n\pi x}{L}\rangle}{\langle sin\frac{n\pi x}{L}, sin\frac{n\pi x}{L}\rangle}\]

From Appendix B, we know that the denominator is L, and we’ve just denoted:

\[\langle f(x), sin\frac{n\pi x}{L}\rangle=\int_{-L}^{L}f(x)sin\frac{n\pi x}{L}dx\]

So we get:

\[b_n=\frac{1}{L}\int_{-L}^{L}f(x)sin\frac{n\pi x}{L}dx\]

Which should look familiar!

This is the core linear-algebra idea behind Fourier series: the functions 1, cos\frac{n\pi x}{L}, and sin\frac{n\pi x}{L} play the role of orthogonal basis vectors, while the Fourier coefficients are coordinates of in this basis. The integral formulas for a_n and b_n are not mysterious tricks; they are projections, just like dot products with basis vectors in ordinary Euclidean space.

Fourier series therefore let us decompose a function into independent orthogonal directions, much like decomposing a vector into its , , and z components.

Appendix A: Integrals of sinusoids

For any integer n\neq 0 and an arbitrary constant L, we have:

\[\begin{aligned} \int_{-L}^{L}cos\frac{n\pi x}{L}dx&=\bigg[\frac{L}{n\pi}sin\frac{n\pi x}{L}\bigg]_{-L}^{L}\\ &=\frac{L}{n\pi}(sin(n\pi)-sin(-n\pi))=0 \end{aligned}\]

Similarly:

\[\begin{aligned} \int_{-L}^{L}sin\frac{n\pi x}{L}dx&=\bigg[\frac{-L}{n\pi}cos\frac{n\pi x}{L}\bigg]_{-L}^{L}\\ &=\frac{-L}{n\pi}\left(cos(n\pi)-cos(-n\pi)\right)=0 \end{aligned}\]

Using these, we can calculate the integral of a complex exponential function for an integer n\neq 0:

\[\begin{aligned} \int_{-L}^{L}e^{in\pi x/L}dx=\int_{-L}^{L}\bigg[ cos\frac{n\pi x}{L} +i\cdot sin\frac{n\pi x}{L}\bigg] dx=0 \end{aligned}\]

Appendix B: Integrals of products of sinusoids

We’ll start with the product of two sines, for any positive integers m and :

\[ss=\int_{-L}^{L}sin\frac{m\pi x}{L}\cdot sin\frac{n\pi x}{L}dx\]

Using the trigonometric identity for a product of sines, we can write:

\[\begin{aligned} ss&=\frac{1}{2}\int_{-L}^{L}\bigg(cos\frac{(m-n)\pi x}{L}-cos\frac{(m+n)\pi x}{L}\bigg)dx\\ &=\frac{1}{2}\int_{-L}^{L}cos\frac{(m-n)\pi x}{L}dx-\frac{1}{2}\int_{-L}^{L}cos\frac{(m+n)\pi x}{L}dx \end{aligned}\]

Now let’s focus on two different scenarios, m\neq n and m=n.

If m\neq n, then each of the integrals constituting ss are 0 (see on Appendix A), so ss=0.

If m=n, then the second integral is still 0, but the first one isn’t:

\[\begin{aligned} ss&=\frac{1}{2}\int_{-L}^{L}cos\frac{0\pi x}{L}dx\\ &=\frac{1}{2}\int_{-L}^{L}1dx=L \end{aligned}\]

Therefore:

\[ss=\int_{-L}^{L}sin\frac{m\pi x}{L}\cdot sin\frac{n\pi x}{L}dx= \begin{cases} L & m = n \\ 0 & m \neq n \end{cases}\]

We can use exactly the same approach to show that:

\[cc=\int_{-L}^{L}cos\frac{m\pi x}{L}\cdot cos\frac{n\pi x}{L}dx= \begin{cases} L & m = n \\ 0 & m \neq n \end{cases}\]

One more variant to cover:

\[sc=\int_{-L}^{L}sin\frac{m\pi x}{L}\cdot cos\frac{n\pi x}{L}dx\]

Since sine is an odd function and cosine is an even function, their product is an odd function. And the integral of an odd function over a symmetric interval is 0 (see this post for more details).

Therefore:

\[sc=\int_{-L}^{L}sin\frac{m\pi x}{L}\cdot cos\frac{n\pi x}{L}dx=0\]

Appendix C: A useful integral

Let’s calculate the indefinite integral:

\[I=\int x\cdot sin(kx) dx\]

For some constant k. We’ll use integration by parts:

\[\int u\cdot dv =u\cdot v - \int v\cdot du\]

Here u=x, so du=dx. Also dv=sin(kx), so v=-\frac{cos(kx)}{k}.

Putting it together:

\[I=\frac{-x\cdot cos(kx)}{k}+\int \frac{cos(kx)}{k} dx=\frac{-x\cdot cos(kx)}{k}+\frac{sin(kx)}{k^2}\]

Appendix D: Sinusoid with phase as a sum of sin and cos

Let’s take a general sinusoid with magnitude q, frequency and phase :

\[s(x)=q\cdot cos(wx+\theta)\]

We’re going to show that s(x) can be represented as a sum of a sine and a cosine with no phase. This is related to my earlier post on the sum of same-frequency sinusoids.

Let’s start by expanding s(x) using a trigonometric identity:

\[s(x)=q\cdot cos(\theta)cos(wx)-q\cdot sin(\theta)sin(wx)\]

Now we’ll denote: a=q\cdot cos(\theta) and b=-q\cdot sin(\theta), so:

\[s(x)=a\cdot cos(wx)+b\cdot sin(wx)\]

We have a and b in terms of q and , but what about the other way around?

Let’s take the equations:

\[\begin{aligned} a&=q\cdot cos(\theta)\\ b&=-q\cdot sin(\theta) \end{aligned}\]

Square both of them and add together:

\[\begin{aligned} a^2+b^2&=q^2\cdot(cos^2(\theta)+sin^2(\theta))=q^2\\ &\Rightarrow q=\sqrt{a^2+b^2} \end{aligned}\]

Now we’ll take the equations for b and a and divide one by the other:

\[\begin{aligned} \frac{b}{a}&=\frac{-sin(\theta)}{cos(\theta)}\\ &\Rightarrow\theta=\operatorname{atan2}(-b,a) \end{aligned}\]

Where the atan2 function is careful to take into account the sign of both numerator and denominator. Also it’s worth mentioning that is determined up to additions of 2\pi.

To conclude, for any q, and :

\[q\cdot cos(wx+\theta)=a\cdot cos(wx)+b\cdot sin(wx)\]

With the aforementioned conversion formulas for a, b.

[1]	A function is called periodic if there exists some constant P>0 such that f(x+P)=f(x)\ \forall x. In our case, we denote the period as P=2L to make presentation and calculations neater.

[2]	Since f(x)\in L^2, we know that it’s integrable on a finite interval

[3]	There’s also an infinite number of less natural ways to extend the function; they will still work, but may make the calculation needlessly complicated

[4]	I chose this one because the more common ones like a square wave and a sawtooth wave have many derivations available online, including on Wikipedia.

[5]	Try it for yourself - it’s a good exercise.

[6]	When q_n=0 for some , the corresponding \theta_n can be chosen arbitrarily.

[7]	For a complex-valued , this conjugate symmetry no longer holds in general. a_n and b_n may be complex, so C_n and C_{-n} need not be conjugates.

Scaling, stretching and shifting sinusoids

2026-05-02T07:17:00-07:00

This is a brief and simple [1] explanation of how to adjust the standard sinusoid sin(x) to change its amplitude, frequency and phase shift. More precisely, given the general function:

\[s(x)=A\cdot sin(w\cdot x+\theta)\]

We’ll see how adjusting the parameters , and affect the shape of s(x). Each section below covers one of these aspects mathematically, and you can use the demo at the bottom to experiment with the topic visually.

Scaling

Scaling is conceptually the simplest change; we adjust to increase or decrease the amplitude (maximal height) of s(x). Setting A=2 will make the value twice as large (in both the positive and negative direction) as the original function.

Stretching

Stretching changes the frequency of sin(x), which is inverse proportional to its period. The baseline function sin(x) has a period of 2\pi, meaning it repeats every 2\pi. In other words, sin(x)=sin(x+2\pi) for any .

If we set w=2, we get sin(2x). This function repeats itself twice as fast as sin(x), because is multiplied by 2 before being fed into the sinusoid. If changes by \pi, the sinusoid’s input changes by 2\pi. Therefore, the period of sin(2x) is \pi, the period of sin(4x) is \frac{\pi}{2} and so on. [2]

More generally, the period of sin(wx) is \frac{2\pi}{w}. Play with the demo below to see this in action, by changing and observing how the waveform changes.

If we know the period p we want, we can easily calculate the that gives us this period:

\[p=\frac{2\pi}{w} \implies w=\frac{2\pi}{p}\]

Shifting

The final parameter we discuss is ; it’s called the phase of the sinusoid. In the baseline sin(x), . The sinusoid is 0 at x=0, achieves its positive peak at x=\frac{\pi}{2}, crosses 0 again at x=\pi, negative peak at x=\frac{3\pi}{2} and returns to its original position at x=2\pi where the repetition begins.

By adding a non-zero , we don’t affect the sinusoid’s amplitude or frequency, but we do shift it right or left along the axis. For example, suppose we use the function sin(x+\theta) with \theta=\frac{\pi}{2}. Then when x=0, we have sin(\frac{\pi}{2}), so the sinusoid is already at its positive peak; at x=\frac{\pi}{2}, the sinusoid crosses 0 into the negatives, etc. Everything happens earlier (by exactly the value of \theta=\frac{\pi}{2}) than in the baseline sinusoid. In other words, we’ve shifted the function left by \frac{\pi}{2}. Similarly, when is negative, everything happens later, and the function is shifted right.

Putting it all together

We’ve now gone over all the parameters for the function:

\[s(x)=A\cdot sin(w\cdot x+\theta)\]

controls the scaling factor (amplitude).
is the frequency and controls the repetition period
controls the phase - how much the sinusoid is shifted left or right

Use the demo below to adjust these parameters and observe their effect on the sinusoid:

A ω θ

[1]	The math level of this post is high-school, at best. My main goal here is to test how to integrate interactive demos into my blog posts.

[2]

This can be a bit counter-intuitive at first; we scale by 2, but the period scales by half. Why? The reason is that affects the sinusoid’s domain, while the period is a property of its range. Therefore, an inverse relation is reasonable, once we put more thought into it. In fact, is often called the angular frequency of the sinusoid, and frequency is inverse proportional to the period.

Notes on Lagrange Interpolating Polynomials

2026-02-28T18:58:00-08:00

Polynomial interpolation is a method of finding a polynomial function that fits a given set of data perfectly. More concretely, suppose we have a set of n+1 distinct points [1]:

\[(x_0,y_0), (x_1, y_1), (x_2, y_2)\cdots(x_n, y_n)\]

And we want to find the polynomial coefficients {a_0\cdots a_n} such that:

\[p(x)=a_0 + a_1 x + a_2 x^2 + \cdots + a_n x^n\]

Fits all our points; that is p(x_0)=y_0, p(x_1)=y_1 etc.

This post discusses a common approach to solving this problem, and also shows why such a polynomial exists and is unique.

Showing existence using linear algebra

When we assign all points (x_i, y_i) into the generic polynomial p(x), we get:

\[\begin{aligned} p(x_0)&=a_0 + a_1 x_0 + a_2 x_0^2 + \cdots a_n x_0^n = y_0\\ p(x_1)&=a_0 + a_1 x_1 + a_2 x_1^2 + \cdots a_n x_1^n = y_1\\ p(x_2)&=a_0 + a_1 x_2 + a_2 x_2^2 + \cdots a_n x_2^n = y_2\\ \cdots \\ p(x_n)&=a_0 + a_1 x_n + a_2 x_n^2 + \cdots a_n x_n^n = y_n\\ \end{aligned}\]

We want to solve for the coefficients a_i. This is a linear system of equations that can be represented by the following matrix equation:

\[{\renewcommand{\arraystretch}{1.5}\begin{bmatrix} 1 & x_0 & x_0^2 & \dots & x_0^n\\ 1 & x_1 & x_1^2 & \dots & x_1^n\\ 1 & x_2 & x_2^2 & \dots & x_2^n\\ \vdots & \vdots & \vdots & \ddots &\vdots \\ 1 & x_n & x_n^2 & \dots & x_n^n \end{bmatrix} \begin{bmatrix} a_0\\ a_1\\ a_2\\ \vdots\\ a_n\\ \end{bmatrix}= \begin{bmatrix} y_0\\ y_1\\ y_2\\ \vdots\\ y_n\\ \end{bmatrix} }\]

The matrix on the left is called the Vandermonde matrix. This matrix is known to be invertible (see Appendix for a proof); therefore, this system of equations has a single solution that can be calculated by inverting the matrix.

In practice, however, the Vandermonde matrix is often numerically ill-conditioned, so inverting it isn’t the best way to calculate exact polynomial coefficients. Several better methods exist.

Lagrange Polynomial

Lagrange interpolation polynomials emerge from a simple, yet powerful idea. Let’s define the Lagrange basis functions l_i(x) (i \in [0, n]) as follows, given our points (x_i, y_i):

\[l_i(x) = \begin{cases} 1 & x = x_i \\ 0 & x = x_j \quad \forall j \neq i \end{cases}\]

In words, l_i(x) is constrained to 1 at and to 0 at all other x_j. We don’t care about its value at any other point.

The linear combination:

\[p(x)=\sum_{i=0}^{n}y_i l_i(x)\]

is then a valid interpolating polynomial for our set of n+1 points, because it’s equal to at each (take a moment to convince yourself this is true).

How do we find l_i(x)? The key insight comes from studying the following function:

\[l'_i(x)=(x-x_0)\cdot (x-x_1)\cdots (x-x_{i-1}) \cdot (x-x_{i+1})\cdots (x-x_n)= \prod_{\substack{0\leq j \leq n \\ j \neq i}}(x-x_j)\]

This function has terms (x-x_j) for all j\neq i. It should be easy to see that l'_i(x) is 0 at all x_j when j\neq i.

What about its value at , though? We can just assign into l'_i(x) to get:

\[l'_i(x_i)=\prod_{\substack{0\leq j \leq n \\ j \neq i}}(x_i-x_j)\]

And then normalize l'_i(x), dividing it by this (constant) value. We get the Lagrange basis function l_i(x):

\[l_i(x)=\frac{l'_i(x)}{l'_i(x_i)}=\prod_{\substack{0\leq j \leq n \\ j \neq i}}\frac{x-x_j}{x_i-x_j}\]

Let’s use a concrete example to visualize this. Suppose we have the following set of points we want to interpolate: (1,4), (2,2), (3,3). We can calculate l'_0(x), l'_1(x) and l'_2(x), and get the following:

Note where each l'_i(x) intersects the axis. These functions have the right values at all x_{j\neq i}. If we normalize them to obtain l_i(x), we get these functions:

Note that each polynomial is 1 at the appropriate and 0 at all the other x_{j\neq i}, as required.

With these l_i(x), we can now plot the interpolating polynomial p(x)=\sum_{i=0}^{n}y_i l_i(x), which fits our set of input points:

Polynomial degree and uniqueness

We’ve just seen that the linear combination of Lagrange basis functions:

\[p(x)=\sum_{i=0}^{n}y_i l_i(x)\]

is a valid interpolating polynomial for a set of n+1 distinct points (x_i, y_i). What is its degree?

Since the degree of each l_i(x) is , then the degree of p(x) is at most . We’ve just derived the first part of the Polynomial interpolation theorem:

Polynomial interpolation theorem: for any n+1 data points (x_0,y_0), (x_1, y_1)\cdots(x_n, y_n) \in \mathbb{R}^2 where no two x_j are the same, there exists a unique polynomial p(x) of degree at most that interpolates these points.

We’ve demonstrated existence and degree, but not yet uniqueness. So let’s turn to that.

We know that p(x) interpolates all n+1 points, and its degree is . Suppose there’s another such polynomial q(x). Let’s construct:

\[r(x)=p(x)-q(x)\]

That do we know about r(x)? First of all, its value is 0 at all our , so it has n+1 roots. Second, we also know that its degree is at most (because it’s the difference of two polynomials of such degree). These two facts are a contradiction. No non-zero polynomial of degree \leq n can have n+1 roots (a basic algebraic fact related to the Fundamental theorem of algebra). So r(x) must be the zero polynomial; in other words, our p(x) is unique \blacksquare.

Note the implication of uniqueness here: given our set of n+1 distinct points, there’s only one polynomial of degree \leq n that interpolates it. We can find its coefficients by inverting the Vandermonde matrix, by using Lagrange basis functions, or any other method [2].

Lagrange polynomials as a basis for P_n(\mathbb{R})

The set P_n(\mathbb{R}) consists of all real polynomials of degree \leq n. This set - along with addition of polynomials and scalar multiplication - forms a vector space.

We called l_i(x) the "Lagrange basis" previously, and they do - in fact - form an actual linear algebra basis for this vector space. To prove this claim, we need to show that Lagrange polynomials are linearly independent and that they span the space.

Linear independence: we have to show that

\[s(x)=\sum_{i=0}^{n}a_i l_i(x)=0\]

implies a_i=0 \quad \forall i. Recall that l_i(x) is 1 at , while all other l_j(x) are 0 at that point. Therefore, evaluating s(x) at , we get:

\[s(x_i)=a_i = 0\]

Similarly, we can show that a_i is 0, for all \blacksquare.

Span: we’ve already demonstrated that the linear combination of l_i(x):

\[p(x)=\sum_{i=0}^{n}y_i l_i(x)\]

is a valid interpolating polynomial for any set of n+1 distinct points. Using the polynomial interpolation theorem, this is the unique polynomial interpolating this set of points. In other words, for every q(x)\in P_n(\mathbb{R}), we can identify any set of n+1 distinct points it passes through, and then use the technique described in this post to find the coefficients of q(x) in the Lagrange basis. Therefore, the set l_i(x) spans the vector space \blacksquare.

Interpolation matrix in the Lagrange basis

Previously we’ve seen how to use the \{1, x, x^2, \dots x^n\} basis to write down a system of linear equations that helps us find the interpolating polynomial. This results in the Vandermonde matrix.

Using the Lagrange basis, we can get a much nicer matrix representation of the interpolation equations.

Recall that our general polynomial using the Lagrange basis is:

\[p(x)=\sum_{i=0}^{n}a_i l_i(x)\]

Let’s build a system of equations for each of the n+1 points (x_i,y_i). For :

\[p(x_0)=\sum_{i=0}^{n}a_i l_i(x_0)\]

By definition of the Lagrange basis functions, all l_i(x_0) where i\neq 0 are 0, while l_0(x_0) is 1. So this simplifies to:

\[p(x_0)=a_0\]

But the value at node is , so we’ve just found that a_0=y_0. We can produce similar equations for the other nodes as well, p(x_1)=a_1, etc. In matrix form:

\[{\renewcommand{\arraystretch}{1.5}\begin{bmatrix} 1 & 0 & 0 & \dots & 0\\ 0 & 1 & 0 & \dots & 0\\ 0 & 0 & 1 & \dots & 0\\ \vdots & \vdots & \vdots & \ddots &\vdots \\ 0 & 0 & 0 & \dots & 1 \end{bmatrix} \begin{bmatrix} a_0\\ a_1\\ a_2\\ \vdots\\ a_n\\ \end{bmatrix}= \begin{bmatrix} y_0\\ y_1\\ y_2\\ \vdots\\ y_n\\ \end{bmatrix} }\]

We get the identity matrix; this is another way to trivially show that a_0=y_0, a_1=y_1 and so on.

Appendix: Vandermonde matrix

Given some numbers \{x_0 \dots x_n\} a matrix of this form:

\[V= {\renewcommand{\arraystretch}{1.5}\begin{bmatrix} 1 & x_0 & x_0^2 & \dots & x_0^n\\ 1 & x_1 & x_1^2 & \dots & x_1^n\\ 1 & x_2 & x_2^2 & \dots & x_2^n\\ \vdots & \vdots & \vdots & \ddots &\vdots \\ 1 & x_n & x_n^2 & \dots & x_n^n \end{bmatrix} }\]

Is called the Vandermonde matrix. What’s special about a Vandermonde matrix is that we know it’s invertible when are distinct. This is because its determinant is known to be non-zero. Moreover, its determinant is [3]:

\[\det(V) = \prod_{0 \le i < j \le n} (x_j - x_i)\]

Here’s why.

To get some intuition, let’s consider some small-rank Vandermonde matrices. Starting with a 2-by-2:

\[\det(V)=\det\begin{bmatrix} 1 & x_0 \\ 1 & x_1 \\ \end{bmatrix}=x_1-x_0\]

Let’s try 3-by-3 now:

\[\det(V)=\det {\renewcommand{\arraystretch}{1.5}\begin{bmatrix} 1 & x_0 & x_0^2 \\ 1 & x_1 & x_1^2 \\ 1 & x_2 & x_2^2 \\ \end{bmatrix} }\]

We can use the standard way of calculating determinants to expand from the first row:

\[\begin{aligned} \det(V)&=1\cdot(x_1 x_2^2 - x_2 x_1^2)-x_0(x_2^2-x_1^2)+x_0^2(x_2 - x_1)\\ &=x_1 x_2^2 - x_2 x_1^2 - x_0 x_2^2+x_0 x_1^2+x_0^2 x_2 - x_0^2 x_1\\ \end{aligned}\]

Using some algebraic manipulation, it’s easy to show this is equivalent to:

\[\det(V)=(x_2-x_1)(x_2-x_0)(x_1-x_0)\]

For the full proof, let’s look at the generalized n+1-by-n+1 matrix again:

Recall that subtracting a multiple of one column from another doesn’t change a matrix’s determinant. For each column k>1, we’ll subtract the value of column k-1 multiplied by from it (this is done on all columns simultaneously). The idea is to make the first row all zeros after the very first element:

\[V= {\renewcommand{\arraystretch}{1.5}\begin{bmatrix} 1 & 0 & 0 & \dots & 0\\ 1 & x_1 - x_0 & x_1^2 - x_1 x_0& \dots & x_1^n - x_1^{n-1} x_0\\ 1 & x_2 - x_0 & x_2^2 - x_2 x_0& \dots & x_2^n - x_2^{n-1} x_0\\ \vdots & \vdots & \vdots & \ddots &\vdots \\ 1 & x_n - x_0 & x_n^2 - x_n x_0& \dots & x_n^n - x_n^{n-1} x_0\\ \end{bmatrix} }\]

Now we factor out x_1-x_0 from the second row (after the first element), x_2-x_0 from the third row and so on, to get:

\[V= {\renewcommand{\arraystretch}{1.5}\begin{bmatrix} 1 & 0 & 0 & \dots & 0\\ 1 & x_1 - x_0 & x_1(x_1 - x_0)& \dots & x_1^{n-1}(x_1 - x_0)\\ 1 & x_2 - x_0 & x_2(x_2 - x_0)& \dots & x_2^{n-1}(x_2 - x_0)\\ \vdots & \vdots & \vdots & \ddots &\vdots \\ 1 & x_n - x_0 & x_n(x_n - x_0)& \dots & x_n^{n-1}(x_n - x_0)\\ \end{bmatrix} }\]

Imagine we erase the first row and first column of . We’ll call the resulting matrix .

\[W= {\renewcommand{\arraystretch}{1.5}\begin{bmatrix} x_1 - x_0 & x_1(x_1 - x_0)& \dots & x_1^{n-1}(x_1 - x_0)\\ x_2 - x_0 & x_2(x_2 - x_0)& \dots & x_2^{n-1}(x_2 - x_0)\\ \vdots & \vdots & \ddots &\vdots \\ x_n - x_0 & x_n(x_n - x_0)& \dots & x_n^{n-1}(x_n - x_0)\\ \end{bmatrix} }\]

Because the first row of is all zeros except the first element, we have:

\[\det(V)=\det(W)\]

Note that the first row of has a common factor of x_1-x_0, so when calculating \det(W), we can move this common factor out. Same for the common factor x_2-x_0 of the second row, and so on. Overall, we can write:

\[\det(W)=(x_1-x_0)(x_2-x_0)\cdots(x_n-x_0)\cdot \det {\renewcommand{\arraystretch}{1.5}\begin{bmatrix} 1 & x_1 & x_1^2 & \dots & x_1^{n-1}\\ 1 & x_2 & x_2^2 & \dots & x_2^{n-1}\\ \vdots & \vdots & \vdots & \ddots &\vdots \\ 1 & x_n & x_n^2 & \dots & x_n^{n-1} \end{bmatrix} }\]

But the smaller matrix is just the Vandermonde matrix for \{x_1 \dots x_{n}\}. If we continue this process by induction, we’ll get:

\[\det(V) = \prod_{0 \le i < j \le n} (x_j - x_i)\]

If you’re interested, the Wikipedia page for the Vandermonde matrix has a couple of additional proofs.

[1]	The -es here are called nodes and the -s are called values.

[2]	Newton polynomials is also an option, and there are many other approaches.

[3]	Note that this means the product of all differences between x_j and where is strictly smaller than j. That is, for n=2, the full product is (x_2-x_1)(x_2-x_0)(x_1-x_0). For an arbitrary , there are \frac{n(n-1)}{2} factors in total.

Notes on Linear Algebra for Polynomials

2026-02-25T18:34:00-08:00

We’ll be working with the set P_n(\mathbb{R}), real polynomials of degree \leq n. Such polynomials can be expressed using n+1 scalar coefficients a_i as follows:

\[p(x)=a_0+a_1 x + a_2 x^2 + \cdots + a_n x^n\]

Vector space

The set P_n(\mathbb{R}), along with addition of polynomials and scalar multiplication form a vector space. As a proof, let’s review how the vector space axioms are satisfied. We’ll use p(x), q(x) and r(x) as arbitrary polynomials from the set P_n(\mathbb{R}) for the demonstration. Similarly, a and b are arbitrary scalars in .

Associativity of vector addition:

\[p(x)+[q(x)+r(x)]=p(x)+q(x)+r(x)=[p(x)+q(x)]+r(x)\]

This is trivial because addition of polynomials is associative [1]. Commutativity is similarly trivial, for the same reason:

Commutativity of vector addition:

\[p(x)+q(x)=q(x)+p(x)\]

Identity element of vector addition:

The zero polynomial 0 serves as an identity element. \forall p(x)\in P_n(\mathbb{R}), we have 0 + p(x) = p(x).

Inverse element of vector addition:

For each p(x), we can use q(x)=-p(x) as the additive inverse, because p(x)+q(x)=0.

Identity element of scalar multiplication

The scalar 1 serves as an identity element for scalar multiplication. For each p(x), it’s true that 1\cdot p(x)=p(x).

Associativity of scalar multiplication:

For any two scalars a and b:

\[a[b\cdot p(x)]=ab\cdot p(x)=[ab]\cdot p(x)\]

Distributivity of scalar multiplication over vector addition:

For any p(x), q(x) and scalar a:

\[a\cdot[p(x)+q(x)]=a\cdot p(x)+a\cdot q(x)\]

Distributivity of scalar multiplication over scalar addition:

For any scalars a and b and polynomial p(x):

\[[a+b]\cdot p(x)=a\cdot p(x) + b\cdot p(x)\]

Linear independence, span and basis

Since we’ve shown that polynomials in P_n(\mathbb{R}) form a vector space, we can now build additional linear algebraic definitions on top of that.

A set of k polynomials p_k(x)\in P_n(\mathbb{R}) is said to be linearly independent if

\[\sum_{i=1}^{k}a_i p_i(x)=0\]

implies a_i=0 \quad \forall i. In words, the only linear combination resulting in the zero vector is when all coefficients are 0.

As an example, let’s discuss the fundamental building blocks of polynomials in P_n(\mathbb{R}): the set \{1, x, x^2, \dots x^n\}. These are linearly independent because:

\[a_0 + a_1 x + a_2 x^2 + \cdots a_n x^n=0\]

is true only for zero polynomial, in which all the coefficients a_i=0. This comes from the very definition of polynomials. Moreover, this set spans the entire P_n(\mathbb{R}) because every polynomial can be (by definition) expressed as a linear combination of \{1, x, x^2, \dots x^n\}.

Since we’ve shown these basic polynomials are linearly independent and span the entire vector space, they are a basis for the space. In fact, this set has a special name: the monomial basis (because a monomial is a polynomial with a single term).

Checking if an arbitrary set of polynomials is a basis

Suppose we have some set polynomials, and we want to know if these form a basis for P_n(\mathbb{R}). How do we go about it?

The idea is using linear algebra the same way we do for any other vector space. Let’s use a concrete example to demonstrate:

\[Q=\{1-x, x, 2x+x^2\}\]

Is the set Q a basis for P_2(\mathbb{R})? We’ll start by checking whether the members of Q are linearly independent. Write:

\[a_0(1-x)+a_1 x + a_2(2x+x^2)=0\]

By regrouping, we can turn this into:

\[a_0 + (a_1-a_0+2a_2)x+a_2 x^2=0\]

For this to be true, the coefficient of each monomial has to be zero; mathematically:

\[\begin{aligned} a_0&=0\\ a_1-a_0+2a_2&=0\\ a2&=0\\ \end{aligned}\]

In matrix form:

\[\begin{bmatrix} 1 & 0 & 0\\ -1 & 1 & 2\\ 0 & 0 & 1\\ \end{bmatrix} \begin{bmatrix}a_0\\ a_1\\ a_2\end{bmatrix} =\begin{bmatrix}0\\ 0\\ 0\end{bmatrix}\]

We know how to solve this, by reducing the matrix into row-echelon form. It’s easy to see that the reduced row-echelon form of this specific matrix is I, the identity matrix. Therefore, this set of equations has a single solution: a_i=0 \quad \forall i [2].

We’ve shown that the set Q is linearly independent. Now let’s show that it spans the space P_2(\mathbb{R}). We want to analyze:

\[a_0(1-x)+a_1 x + a_2(2x+x^2)=\alpha +\beta x + \gamma x^2\]

And find the coefficients a_i that satisfy this for any arbitrary , and \gamma. We proceed just as before, by regrouping on the left side:

\[a_0 + (a_1-a_0+2a_2)x+a_2 x^2=\alpha +\beta x + \gamma x^2\]

and equating the coefficient of each power of separately:

\[\begin{aligned} a_0&=\alpha\\ a_1-a_0+2a_2&=\beta\\ a2&=\gamma\\ \end{aligned}\]

If we turn this into matrix form, the matrix of coefficients is exactly the same as before. So we know there’s a single solution, and by rearranging the matrix into I, the solution will appear on the right hand side. It doesn’t matter for the moment what the actual solution is, as long as it exists and is unique. We’ve shown that Q spans the space!

Since the set Q is linearly independent and spans P_2(\mathbb{R}), it is a basis for the space.

Inner product

I’ve discussed inner products for functions in the post about Hilbert space. Well, polynomials are functions, so we can define an inner product using integrals as follows [3]:

\[\langle p, q \rangle = \int_{a}^{b} p(x) q(x) w(x) \, dx\]

Where the bounds a and b are arbitrary, and could be infinite. Whenever we deal with integrals we worry about convergence; in my post on Hilbert spaces, we only talked about L^2 - the square integrable functions. Most polynomials are not square integrable, however. Therefore, we can restrict this using either:

A special weight function w(x) to make sure the inner product integral converges
Set finite bounds on the integral, and then we can just set w(x)=1.

Let’s use the latter, and restrict the bounds into the range [-1,1], setting w(x)=1. We have the following inner product:

\[\langle p, q \rangle = \int_{-1}^{1} p(x) q(x) \, dx\]

Let’s check that this satisfies the inner product space conditions.

Conjugate symmetry:

Since real multiplication is commutative, we can write:

\[\langle p, q \rangle = \int_{-1}^{1} p(x) q(x) \, dx =\int_{-1}^{1} q(x) p(x) \, dx=\langle q, p \rangle\]

We deal in the reals here, so we can safely ignore complex conjugation.

Linearity in the first argument:

Let p_1,p_2,q\in P_n(\mathbb{R}) and a,b\in \mathbb{R}. We want to show that

\[\langle ap_1+bp_2,q \rangle = a\langle p_1,q\rangle +b\langle p_2,q\rangle\]

Expand the left-hand side using our definition of inner product:

\[\begin{aligned} \langle ap_1+bp_2,q \rangle&=\int_{-1}^{1} (a p_1(x)+b p_2(x)) q(x) \, dx\\ &=a\int_{-1}^{1} p_1(x) q(x) \, dx+b\int_{-1}^{1} p_2(x) q(x) \, dx \end{aligned}\]

The result is equivalent to a\langle p_1,q\rangle +b\langle p_2,q\rangle.

Positive-definiteness:

We want to show that for nonzero p\in P_n(\mathbb{R}), we have \langle p, p\rangle > 0. First of all, since p(x)^2\geq0 for all , it’s true that:

\[\langle p, p\rangle=\int_{-1}^{1}p(x)^2\, dx\geq 0\]

What about the result 0 though? Well, let’s say that

\[\int_{-1}^{1}p(x)^2\, dx=0\]

Since p(x)^2 is a non-negative function, this means that the integral of a non-negative function ends up being 0. But p(x) is a polynomial, so it’s continuous, and so is p(x)^2. If the integral of a continuous non-negative function is 0, it means the function itself is 0. Had it been non-zero in any place, the integral would necessarily have to be positive as well.

We’ve proven that \langle p, p\rangle=0 only when p is the zero polynomial. The positive-definiteness condition is satisfied.

In conclusion, P_n(\mathbb{R}) along with the inner product we’ve defined forms an inner product space.

Orthogonality

Now that we have an inner product, we can define orthogonality on polynomials: two polynomials p,q are orthogonal (w.r.t. our inner product) iff

\[\langle p,q\rangle=\int_{-1}^{1}p(x)q(x)\, dx=0\]

Contrary to expectation [4], the monomial basis polynomials are not orthogonal using our definition of inner product.

For example, calculating the inner product for 1 and x^2:

\[\langle 1,x^2\rangle=\int_{-1}^{1}x^2\, dx=\frac{x^3}{3}\biggr|_{-1}^{1}=\frac{2}{3}\]

There are other sets of polynomials that are orthogonal using our inner product. For example, the Legendre polynomials; but this is a topic for another post.

[1]	There’s a level of basic algebra below which we won’t descend in these notes. We could break this statement further down by saying that something like a_i x^i + a_j x^j can be added to b_i x^i + b_j x^j by adding each power of separately for any and j, but let’s just take it for granted.

[2]

Obviously, this specific set of equations is quite trivial to solve without matrices; I just want to demonstrate the more general approach. Once we have a system of linear equations, the whole toolbox of linear algebra is at our disposal. For example, we could also have checked the determinant and seen it’s non-zero, which means that a square matrix is invertible, and in this case has a single solution of zeroes.

[3]	And actually with this (or any valid) inner product, P_n(\mathbb{R}) indeed forms a Hilbert space, because it’s finite-dimensional, and every finite-dimensional inner product space is complete.

[4]	Because of how naturally this set spans P_n(\mathbb{R}). And indeed, we can define alternative inner products using which monomials are orthogonal.

LaTeX, LLMs and Boring Technology

2025-10-25T06:20:00-07:00

Depending on your particular use case, choosing boring technology is often a good idea. Recently, I've been thinking more and more about how the rise and increase in power of LLMs affects this choice.

By definition, boring technology has been around for a long time. Piles of content have been written and produced about it: tutorials, books, videos, reference manuals, examples, blog posts and so on. All of this is consumed during the LLM training process, making LLMs better and better at reasoning about such technology.

Conversely, "shiny technology" is new, and has much less material available. As a result, LLMs won't be as familiar with it. This applies to many domains, but one specific example for me personally is in the context of LaTeX.

LaTeX certainly fits the "boring technology" bill. It's decades old, and has been the mainstay of academic writing since the 1980s. When I used it for the first time in 2002 (for a project report in my university AI class), it was already very old. But people keep working on it and fixing issues; it's easy to install and its wealth of capabilities and community size are staggering. Moreover, people keep working with it, producing more and more content and examples the LLMs can ingest and learn from.

I keep hearing about the advantages of new and shiny systems like Typst. However, with the help of LLMs, almost none of the advantages seem meaningful to me. LLMs are great at LaTeX and help a lot with learning or remembering the syntax, finding the right packages, deciphering errors and even generating tedious parts like tables and charts, significantly reducing the need for scripting [1].

You can use LLMs either as standalone or fully integrated into your LaTeX environment; Overleaf has a built-in AI helper, and for local editing you can use VSCode plugins or other tools. I'm personally content with TeXstudio and use LLMs as standalone help, but YMMV.

There are many examples where boring technology and LLMs go well together. The main criticism of boring technology is typically that it's "too big, full of cruft, difficult to understand". LLMs really help cutting through the learning curve though, and all that "cruft" is very likely to become useful some time in the future when you graduate from the basic use cases.

To be clear: Typst looks really cool, and kudos to the team behind it! All I'm saying in this post is that for me - personaly - the choice for now is to stick with LaTeX as a "boring technology".

Appendix: some examples of LLMs being helpful with LaTeX

For finding the right math symbols, I rarely need to scan reference materials any longer. LLMs will easily answer questions like "what's that squiggly Greek letter used in math, and its latex symbol?" or "write the latex for Green's theorem, integral form". For the trickiest / largest equations, LLMs are very good at "here's a picture I took of my equation, give me its latex code" these days [2].
"Here's a piece of code and the LaTeX error I'm getting on it; what's wrong?"

This is made more ergonomic by editor integrations, but I personally find that LaTeX's error message problem is hugely overblown. 95% of the errors are reasonably clear, and serious sleuthing is only rarely required in practice. In that minority of cases, pasting some code and the error into a standalone LLM isn't a serious time drain.
Generating TikZ diagrams and plots. For this, the hardest part is getting started and finding the right element names, and so on. It's very useful to just ask an LLM to emit something initial and then tweak it manually later, as needed. You can also ask the LLM to explain each thing it emits in detail - this is a great learning tool for deeper understanding.

Recently I had luck going "meta" with this: when the diagram has repetitive elements, I may ask the LLM to "write a Python program that generates a TikZ diagram ...", and it works well.
Generating and populating tables, and converting them from other data formats or screenshots.
Help with formatting and typesetting (how do I change margins to XXX and spacing to YYY).

[1]

When it comes to scripting, I generally prefer sticking to real programming languages anyway. If there's anything non-trivial to auto-generate I wouldn't use a LaTeX macro, but would write a Python program to generate whatever I need and embed it into the document with something like \input{}.

Typst's scripting system may be marketed as "clean and powerful", but why learn yet another scripting language?

[2]

Ignoring LaTeX's equation notation and doing their own thing is one of the biggest mistakes Typst makes, in my opinion. LaTeX's notation may not be perfect, but it's near universal at this point with support in almost all math-aware tools.

Typst's math mode is a clear sign of the second system effect, and isn't even stable.