Eli Bendersky's website

How big are factorials?

2026-08-27T18:52:00-07:00

The other day, I found myself wondering how big 52! (52 factorial) is, and that led me to ponder how these could be estimated without a calculator or a computer.

It turns out there’s some fairly interesting math behind being able to estimate the size (number of digits) of a factorial reasonably accurately. This post will start by stating how to do the estimate, and if you’re curious you can read on for the math background.

Without further ado, the approximation is:

\[\text{number of digits in n!}\approx n\log_{10}\left(\frac{n}{e}\right)+2\]

As an example, let’s use my original question, by estimating this for 52!

Well, 52 divided by e is... 20-ish? And \log_{10}(20) is about 1.3 [1]; therefore our estimate comes out to:

\[\text{number of digits in 52!}\approx 52\cdot 1.3 +2 \approx69\]

The real answer is 68, so this is very close! In estimates like this - when you’re dealing with enormous numbers - being off by a couple of digits usually isn't a big deal.

The Gamma function

The Gamma function for real n>0 is defined [2] as:

\[\Gamma(n)=\int_{0}^{\infty}x^{n-1}e^{-x}dx\]

This integral does not have an analytic expression in the general case, but it does have a very useful property that we can take advantage of. Let’s see what \Gamma(n+1) is:

\[\Gamma(n+1)=\int_{0}^{\infty}x^{n}e^{-x}dx\]

And now use integration by parts with:

\[u=x^n\qquad v=-e^{-x}\]

Then:

\[du=nx^{n-1} dx\qquad dv=e^{-x}dx\]

So:

\[\begin{aligned} \Gamma(n+1)&=\int_{0}^{\infty}x^{n}e^{-x}dx\\ &=\left. -x^n e^{-x}\right|_{0}^{\infty}-\int_{0}^{\infty}-e^{-x}n x^{n-1}dx\\ &=n\int_{0}^{\infty}x^{n-1}e^{-x}dx \end{aligned}\]

But notice that the last integral is just \Gamma(n); therefore, we’ve shown that:

\[\Gamma(n+1)=n\Gamma(n)\]

Let’s also calculate \Gamma(1) - it’s a special case that has an analytical solution:

\[\Gamma(1)=\int_{0}^{\infty}e^{-x}dx=\left. -e^{-x}\right|_{0}^{\infty}=1\]

This helps establish an induction argument:

\[\begin{aligned} \Gamma(2)=1\cdot\Gamma(1)&=1!\\ \Gamma(3)=2\cdot\Gamma(2)&=2!\\ \Gamma(4)=3\cdot\Gamma(3)&=3!\\ \dots\\ \Gamma(n+1)=n\cdot\Gamma(n)&=n! \end{aligned}\]

In other words - the Gamma function is an interpolation of the factorial over all positive reals. Here’s a plot of the Gamma function over a small range; note that the y axis is log-scale because of the function’s fast growth:

Stirling’s approximation

You may have encountered Stirling’s approximation before:

\[n!\approx\sqrt{2\pi n}\cdot\left(\frac{n}{e} \right)^n\]

It’s a great approximation that works reasonably well even for small . This section is a brief overview of how Stirling’s formula is derived from the Gamma function.

Taking:

\[n!=\Gamma(n+1)=\int_{0}^{\infty}x^n e^{-x}dx\]

We’ll start by massaging the integrand a bit:

\[n!=\int_{0}^{\infty}x^n e^{-x}dx=\int_{0}^{\infty}e^{n \ln x} e^{-x}dx\]

And making a change of variables x=ny, which means that dx=ndy:

\[\begin{aligned} n!&=\int_{0}^{\infty}ne^{n\ln(ny) - ny}dy=n\int_{0}^{\infty}e^{n(\ln n+\ln y-y)}dy\\ &=ne^{n\ln n}\int_{0}^{\infty}e^{n(\ln y-y)}dy \end{aligned}\]

These steps make the integral amenable to applying Laplace’s method, which allows us to approximate definite integrals of the form:

\[\int_{a}^{b}e^{nf(x)}dx\]

Where is a twice-differentiable function and some large number. By Laplace’s method, such integrals can be approximated by:

\[\int_{a}^{b}e^{nf(x)}dx\approx\sqrt{\frac{2\pi}{n|f''(x_0)|}}e^{n f(x_0)}\]

Where is the global maximum of .

Let’s see how to apply this method [3] to the latest equation we have for n! (renaming the dummy integration variable back to x):

\[n!=ne^{n\ln n}\int_{0}^{\infty}e^{n(\ln x-x)}dx\]

In our case, f(x)=\ln x - x. It’s easy to show that this function is twice differentiable and has a global maximum at . Moreover:

\[\begin{aligned} f(x_0)&=-1\\ f''(x_0)&=-1\\ \end{aligned}\]

Substituting these into the proper places in Laplace’s approximation, we get:

\[\begin{aligned} n! &\approx ne^{n\ln n}\sqrt{\frac{2\pi}{n}}e^{-n}\\ &\approx \sqrt{2\pi n}\cdot e^{n(\ln n - 1)}\\ &\approx \sqrt{2\pi n}\cdot \left(\frac{n}{e}\right)^n\quad\blacksquare \end{aligned}\]

Number of digits from Stirling’s approximation

We can calculate the number of digits in n! by taking the base-10 logarithm of Stirling’s formula:

\[\begin{aligned} \text{number of digits in n!}&\approx \log_{10} \left(\sqrt{2\pi n}\cdot \left(\frac{n}{e}\right)^n\right)\\ &\approx \log_{10}\left(\sqrt{2\pi n}\right)+ \log_{10}\left(\frac{n}{e}\right)^n\\ &\approx \log_{10}\left(\sqrt{2\pi n}\right)+ n\cdot \log_{10}\left(\frac{n}{e}\right) \end{aligned}\]

Note that the first term is not multiplied by itself; therefore, as grows, it will become less and less noticeable. That said, it still adds a couple of digits - so you should take it into account if you want a more accurate approximation [4]

[1]	Mental tricks for calculating \log_{10} is a different topic, but it really helps to remember that \log_{10}2=0.3, \log_{10}3=0.5, and from here using the various logarithm laws to estimate multiples.

[2]	As a matter of fact, the Gamma function is defined for complex numbers, but for our purpose talking about the reals is sufficient.

[3]	As much as I’d like to dive into why this approximation works, the rabbit hole in this post is already deep enough!

[4]	It adds up to 2 extra digits as long as is less than 1600 or so, and may add more than 2 after that, though no more than 3 until n is 160000. It’s not clear why anyone would like to estimate the number of digits of 1600! (about 4450, in case you were wondering), let alone 160000!

Concurrent Servers: Part 8 - Go

2026-08-22T07:52:00-07:00

This is part 8 in a series of posts on writing concurrent network servers. In this part, we'll switch to Go and see how it tackles the challenges described earlier in the series.

All posts in the series:

Part 1 - Introduction
Part 2 - Threads
Part 3 - Event-driven
Part 4 - libuv
Part 5 - Redis case study
Part 6 - Callbacks, Promises and async/await
Part 7 - Rust
Part 8 - Go (this part)

This post assumes a basic familiarity with the Go programming language.

Sequential state machine server

As before, we'll start with a sequential server for the basic state machine protocol presented in part 1.

This is the main function:

func main() {
  port := "9090"
  if len(os.Args) >= 2 {
    port = os.Args[1]
  }
  log.Println("Serving on port", port)

  listener, err := net.Listen("tcp", ":"+port)
  if err != nil {
    log.Fatal("Error listening:", err)
  }
  defer listener.Close()

  for {
    conn, err := listener.Accept()
    if err != nil {
      log.Printf("Error accepting connection: %v", err)
      continue
    }
    log.Println("connection received from", conn.RemoteAddr())

    if err := server.ServeSerialProtocol(conn); err != nil {
      log.Printf("Error serving %v: %v", conn.RemoteAddr(), err)
    } else {
      log.Println("peer done", conn.RemoteAddr())
    }
  }
}

As in the previous parts, the server is "infinite"; it never stops serving new connections until it's explicitly killed.

This is the function implementing the protocol for a single client; it takes a net.Conn value that represents a socket with a client connected on the other end:

type processingState int

const (
  waitForMsg processingState = iota
  inMsg
)

// ServeSerialProtocol serves our serial protocol to a single TCP connection.
func ServeSerialProtocol(conn net.Conn) error {
  defer conn.Close()

  if _, err := conn.Write([]byte{'*'}); err != nil {
    return err
  }

  var state processingState = waitForMsg
  buf := make([]byte, 1024)
  for {
    n, err := conn.Read(buf)
    for _, b := range buf[:n] {
      switch state {
      case waitForMsg:
        if b == '^' {
          state = inMsg
        }
      case inMsg:
        if b == '$' {
          state = waitForMsg
        } else {
          var bb byte = byte(b) + 1
          if _, err := conn.Write([]byte{bb}); err != nil {
            return err
          }
        }
      }
    }

    // Check error after processing the bytes received along with it.
    if err != nil {
      if errors.Is(err, io.EOF) || errors.Is(err, net.ErrClosed) {
        return nil
      } else {
        return err
      }
    }
  }
}

One goroutine per client

Rather than directly exposing OS threads, the Go runtime implements its own M:N scheduling of lightweight goroutines on top of OS threads. Using goroutines in Go is cheap - both in terms of syntax and developer effort, and in terms of system resources.

Here's a version of our serial protocol server that serves clients concurrently by launching a goroutine for each client. The part of the code that's different from the previous sample is highlighted:

func main() {
  port := "9090"
  if len(os.Args) >= 2 {
    port = os.Args[1]
  }
  log.Println("Serving on port", port)

  listener, err := net.Listen("tcp", ":"+port)
  if err != nil {
    log.Fatal("Error listening:", err)
  }
  defer listener.Close()

  for {
    conn, err := listener.Accept()
    if err != nil {
      log.Printf("Error accepting connection: %v", err)
      continue
    }
    log.Println("connection received from", conn.RemoteAddr())

    go func() {
      if err := server.ServeSerialProtocol(conn); err != nil {
        log.Printf("Error serving %v: %v", conn.RemoteAddr(), err)
      } else {
        log.Println("peer done", conn.RemoteAddr())
      }
    }()
  }
}

The concurrent modification in this case is particularly simple because the server is infinite; there's no point waiting for these goroutines to finish (and hence no need for a sync.WaitGroup). The parameters for server.ServeSerialProtocol are lexically captured from the enclosing scope and its return value is handled by the surrounding closure.

Because goroutines are very cheap, this server is very unlikely to run out of resources due to launching too many goroutines; in fact, it will probably run out of something else - like file descriptors for sockets - first. However, sometimes it's still useful to limit the degree of concurrency - even in Go, and we'll discuss some approaches to do so in the following sections.

Limiting concurrency with a semaphore

Here are some scenarios in which it makes sense to limit the degree of concurrency in Go programs, even though goroutines are cheap to launch and operate:

Tasks may be compute intensive, and the CPU capacity of any server is inherently limited. If too many concurrent goroutines compete for limited CPUs, they will all make very little progress. It may make more sense to have fewer tasks that complete in a reasonable time.
Protecting potentially limited downstream resources, such as concurrent DB connections or other services. For example, if the server has to send requests to other services for each task, and these are rate-limited, concurrency will have to be carefully managed.
Security reasons when work is dictated by clients; malicious clients can overload and crash a service that's too eager to serve, making it unavailable for legitimate clients.

Let's switch to the primality testing server from part 4 for the rest of the post, because it represents a somewhat more realistic workload. As a reminder: the server receives numbers, simulates blocking by sleeping, and returns "prime" or "composite". The unbounded one-goroutine-per-client version looks almost identical to the previous code sample, except that the goroutine invocation calls another function:

go func() {
  if err := server.ServePrimeProtocol(conn); err != nil {
    log.Printf("Error serving %v: %v", conn.RemoteAddr(), err)
  } else {
    log.Println("peer done", conn.RemoteAddr())
  }
}()

Where ServePrimeProtocol is [1]:

// ServePrimeProtocol serves our prime protocol to a single TCP connection.
func ServePrimeProtocol(conn net.Conn) error {
  defer conn.Close()
  buf := make([]byte, 1024)

  for {
    n, readerr := conn.Read(buf)
    if n > 0 {
      // Parse the read buffer to an i64
      num, err := strconv.ParseInt(strings.TrimSpace(string(buf[:n])), 10, 64)
      if err != nil {
        return err
      }

      response := "composite"
      if isPrime(num, true) {
        response = "prime"
      }
      if _, err := conn.Write([]byte(response + "\n")); err != nil {
        return err
      }
    }

    if readerr != nil {
      if errors.Is(readerr, io.EOF) || errors.Is(readerr, net.ErrClosed) {
        return nil
      }
      return readerr
    }
  }
}

// isPrime returns true if n is prime, false otherwise. If delay is true, it
// will sleep for n milliseconds before calculating.
func isPrime(n int64, delay bool) bool {
  if delay {
    time.Sleep(time.Duration(n) * time.Millisecond)
  }

  if n < 2 {
    return false
  }
  if n%2 == 0 {
    return n == 2
  }
  for i := int64(3); i*i <= n; i += 2 {
    if n%i == 0 {
      return false
    }
  }
  return true
}

The simplest way to limit concurrency in Go is by using a counting semaphore, implemented with a channel:

func main() {
  port := "8070"
  if len(os.Args) >= 2 {
    port = os.Args[1]
  }
  log.Println("Serving on port", port)

  listener, err := net.Listen("tcp", ":"+port)
  if err != nil {
    log.Fatal("Error listening:", err)
  }
  defer listener.Close()

  maxConcurrency := runtime.NumCPU()
  sem := make(chan struct{}, maxConcurrency)

  for {
    conn, err := listener.Accept()
    if err != nil {
      log.Printf("Error accepting connection: %v", err)
      continue
    }
    log.Println("connection received from", conn.RemoteAddr())

    // Acquire a token from the semaphore to limit concurrency.
    sem <- struct{}{}

    go func() {
      // Return the token when done serving the connection.
      defer func() { <-sem }()

      if err := server.ServePrimeProtocol(conn); err != nil {
        log.Printf("Error serving %v: %v", conn.RemoteAddr(), err)
      } else {
        log.Println("peer done", conn.RemoteAddr())
      }
    }()
  }
}

The channel sem serves as a semaphore; note that it's a bounded channel with a maximal size. A token is acquired by sending to the channel, and released by receiving from the channel. When the channel is full, the send operation sem <- struct{}{} blocks until a token was removed by some other goroutine [2].

The type of the channel is struct{} which means "empty", or "no data". This is idiomatic in Go for channels that are used solely for their semantics, not to send/receive any actual data.

Worker pool

Since launching goroutines is cheap and limiting concurrency is easy as shown above, the "worker pool" pattern is often unnecessary for scenarios like our server. Still, it has occasional uses (such as when workers have to maintain some non-trivial state across tasks) so it's worth discussing it here.

Here's a variant of our primality testing server that uses a worker pool:

func worker(jobs <-chan net.Conn) {
  for conn := range jobs {
    if err := server.ServePrimeProtocol(conn); err != nil {
      log.Printf("Error serving %v: %v", conn.RemoteAddr(), err)
    } else {
      log.Println("peer done", conn.RemoteAddr())
    }
  }
}

func main() {
  port := "8070"
  if len(os.Args) >= 2 {
    port = os.Args[1]
  }
  log.Println("Serving on port", port)

  listener, err := net.Listen("tcp", ":"+port)
  if err != nil {
    log.Fatal("Error listening:", err)
  }
  defer listener.Close()

  // The channel is unbuffered, so it will block when there are no available
  // workers to accept a new connection.
  jobs := make(chan net.Conn)
  maxConcurrency := runtime.NumCPU()
  for i := 0; i < maxConcurrency; i++ {
    go worker(jobs)
  }

  for {
    conn, err := listener.Accept()
    if err != nil {
      log.Printf("Error accepting connection: %v", err)
      continue
    }
    log.Println("connection received from", conn.RemoteAddr())

    jobs <- conn
  }
}

A fixed number of worker goroutines is launched; these goroutines all receive "jobs" from the same channel. In the Accept loop, each client connection is sent to this channel as a new job and is picked up by the next available worker. As mentioned before, you would typically see a sync.WaitGroup somewhere to ensure clean shutdown of goroutines, but in our case it isn't necessary because we have a server that never exits.

Async?

Do programmers have to resort to async / event-driven programming in Go? In my experience, almost never. Go was designed from the bottom up to be suitable for large-scale concurrency; goroutines are very cheap to create, have a tiny memory footprint and switching happens very quickly, all in user space. Measurements I ran back in 2018 have shown switching times of ~170 ns, as compared to 1-2 microseconds for threads on Linux.

Moreover, Go already uses event-driven loops like epoll underneath for I/O. Goroutines that wait for I/O like sockets are effectively "parked" and consume no resources (beyond their small memory footprint); they are woken up by Go's runtime when their I/O descriptors are ready - this is very similar to how asynchronous programming works!

That said, some people certainly do try to stretch their resources even more with direct asynchronous programming in Go when millions of streams are handled concurrently. All I'll say is that this is very rare, and an overwhelming majority of users never have to do this.

Conclusion

In 2018, I wrote a post named Go hits the concurrency nail right on the head, and after several more years of active coding, I fully stand behind that statement. Go is extremely powerful and ergonomic for concurrent programs; while other environments go to great lengths to implement async-await style event loops in libraries, in Go it's already baked into the core language and runtime. You want event-driven I/O with very lightweight green threads that can also execute blocking tasks without worrying about the function coloring problem? Go has you covered.

Code

All the code for this post is available on GitHub.

[1]

Careful readers will note two issues with this code: (1) the protocol assumes the complete number is read from the socket in a single conn.Read call, and there's no framing - separation between distinct numbers; (2) the prime checking loop uses i*i which may overflow for large numbers.

These issues are consistent across all the versions of the prime server in C, Python, JavaScript and Rust in earlier parts, because my focus was on the simplest possible code to demonstrate a point about concurrency.

[2] Exercise: note that our semaphore protects the entire ServePrimeProtocol, meaning that a malicious (or incompetent) client can connect and idle without sending any requests, thus reducing our concurrent capacity by 1. Adjust the code to move the semaphore around the prime calculation itself, so only this part is bounded.

Concurrent Servers: Part 7 - Rust

2026-08-15T09:41:00-07:00

This is part 7 in a series of posts on writing concurrent network servers. In this part, we discuss how the challenges described in earlier parts are tackled in the Rust programming language.

All posts in the series:

Part 1 - Introduction
Part 2 - Threads
Part 3 - Event-driven
Part 4 - libuv
Part 5 - Redis case study
Part 6 - Callbacks, Promises and async/await
Part 7 - Rust (this part)
Part 8 - Go

Several years have passed since the previous parts were published. I've recently went over them to make sure the information presented is still relevant and all the code samples build and run using modern toolchains. I strongly recommend reviewing the previous parts before reading this one.

This post assumes a basic familiarity with the Rust programming language. It will only explain Rust constructs when we encounter code that wouldn't appear in an introductory book or tutorial.

Setting the baseline - a sequential state machine server

The first few parts in the series focused on a socket server that implements a simple state machine protocol. See part 1 for a complete description of the protocol. Let's start by showing how this protocol is implemented in a basic sequential Rust server:

use async_socket_server::serve_connection;
use std::net::TcpListener;

fn main() -> std::io::Result<()> {
    let port = match std::env::args().nth(1) {
        Some(s) => s,
        None => "9090".to_string(),
    };
    let addr = format!("127.0.0.1:{port}");

    let listener = TcpListener::bind(addr)?;
    println!("Serving on port {port}");

    loop {
        let (stream, addr) = listener.accept()?;
        println!("connection received from {}", addr);
        if let Err(e) = serve_connection(stream) {
            eprintln!("error serving connection: {}", e);
        } else {
            println!("peer done {addr}");
        }
    }
}

With the function serve_connection defined as:

pub enum ProcessingState {
    WaitForMsg,
    InMsg,
}

pub fn serve_connection(mut stream: TcpStream) -> std::io::Result<()> {
    stream.write_all(b"*")?;

    let mut state = ProcessingState::WaitForMsg;
    let mut buf = [0u8; 1024];
    loop {
        let n = stream.read(&mut buf)?;
        if n == 0 {
            // Connection closed by the client.
            break;
        }

        for byte in &buf[..n] {
            match state {
                ProcessingState::WaitForMsg => {
                    if *byte == b'^' {
                        state = ProcessingState::InMsg;
                    }
                }
                ProcessingState::InMsg => {
                    if *byte == b'$' {
                        state = ProcessingState::WaitForMsg;
                    } else {
                        let newbyte = byte.wrapping_add(1);
                        stream.write_all(&[newbyte])?;
                    }
                }
            }
        }
    }

    Ok(())
}

As a reminder, this server version is sequential because it accepts clients one by one; the main loop blocks on serve_connection until it's done (the client closes the connection), and only then goes back to accept the next client.

One thread per client

Clearly, handling clients one by one won't do. In part 2, we've discussed approaches that use OS threads to handle clients concurrently. Let's start with the unbounded one-thread-per-client solution in Rust:

use async_socket_server::serve_connection;
use std::{net::TcpListener, thread};

fn main() -> std::io::Result<()> {
    let port = match std::env::args().nth(1) {
        Some(s) => s,
        None => "9090".to_string(),
    };
    let addr = format!("127.0.0.1:{port}");

    let listener = TcpListener::bind(addr)?;
    println!("Serving on port {port}");

    loop {
        let (stream, addr) = listener.accept()?;
        println!("connection received from {}", addr);

        let res = thread::Builder::new().spawn(move || {
            if let Err(e) = serve_connection(stream) {
                eprintln!("error serving connection: {}", e);
            } else {
                println!("peer done {addr}");
            }
        });

        if let Err(e) = res {
            eprintln!("error spawning thread: {}", e);
        }
    }
}

The spawn method returns a Result<JoinHandle<T>>; on success, we allow the handle to be dropped at the end of the loop iteration. In Rust, this detaches the thread; we don't actually wait for it to complete. This is reasonable for our code sample, because the loop is infinite; it never terminates anyway. The potential for runaway threads is just one of the issues with the unbounded threads approach discussed in part 2. The solution is to use a fixed thread pool.

Thread pool

Before diving into the code, a quick note on the design: the thread pool is a fixed set of threads that await "jobs" and handle them to completion. In our case a "job" is serve_connection for a specific client. There are many ways to implement a thread pool; for our use case, I went with a set of threads that all get a shared channel to which the main thread sends jobs. A worker thread picks up the next job from the channel, serves it to completion, and goes back to waiting for the next job. Here's how this looks in code:

struct Job {
    stream: TcpStream,
    addr: SocketAddr,
}

fn worker(receiver: Receiver<Job>) {
    while let Ok(job) = receiver.recv() {
        if let Err(e) = serve_connection(job.stream) {
            eprintln!("error serving connection from {}: {}", job.addr, e);
        } else {
            println!("peer done {}", job.addr);
        }
    }
}

What is Receiver? It's a type from the crossbeam_channel crate:

use crossbeam_channel::{Receiver, bounded};

Rust's builtin channels in std are mpsc - multi producer, single consumer, but what we need for our job queue is a channel that supports multiple consumers (the worker threads). While std does have mpmc, this is an experimental API only available in nightly versions at the time of writing. Therefore, I've opted to include the crossbeam_channel crate that provides well-tested mpmc channels for this sample [1].

And here's the main function:

use async_socket_server::serve_connection;
use crossbeam_channel::{Receiver, bounded};
use std::{
    io,
    net::{SocketAddr, TcpListener, TcpStream},
    thread,
};

const NUM_WORKERS: usize = 256;
const JOB_QUEUE_CAPACITY: usize = NUM_WORKERS;

fn main() -> io::Result<()> {
    let port = match std::env::args().nth(1) {
        Some(s) => s,
        None => "9090".to_string(),
    };
    let addr = format!("127.0.0.1:{port}");

    let listener = TcpListener::bind(addr)?;
    println!("Serving on port {port}");

    let (tx, rx) = bounded::<Job>(JOB_QUEUE_CAPACITY);
    for _ in 0..NUM_WORKERS {
        let receiver = rx.clone();
        thread::Builder::new().spawn(move || worker(receiver))?;
    }
    drop(rx);

    loop {
        let (stream, addr) = listener.accept()?;
        println!("connection received from {addr}");

        // A full queue blocks this loop, preventing it from accepting more
        // connections until a worker becomes available.
        if tx.send(Job { stream, addr }).is_err() {
            return Err(io::Error::other("all connection workers stopped"));
        }
    }
}

Note that our job channel is bounded - it has a fixed size. This helps naturally implement a backpressure mechanism - if too many clients connect, the following clients will have to wait - the main loop blocks on tx.send and won't accept additional clients on the socket until jobs are cleared from the channel.

Asynchronous, event-driven server

In parts 4, 5 and 6 of the series we've discussed event-driven, or asynchronous servers. Let's see how it's done in Rust. Specifically, part 6 presented a gradation from callbacks to promises to async/await mechanisms; Rust supports all of these and - as you'd expect - modern code is usually written with async/await while hiding all the details of promises (called futures in Rust) underneath.

Without further ado, here's our simple state machine protocol in asynchronous Rust:

use async_socket_server::ProcessingState;
use tokio::io::{self, AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;

#[tokio::main]
async fn main() -> io::Result<()> {
    let port = match std::env::args().nth(1) {
        Some(s) => s,
        None => "9090".to_string(),
    };
    let addr = format!("127.0.0.1:{port}");

    let listener = TcpListener::bind(addr).await?;
    println!("Serving on port {port}");

    loop {
        let (socket, addr) = listener.accept().await?;
        println!("connection received from {addr:?}");

        tokio::spawn(async move {
            if let Err(e) = serve_connection_async(socket).await {
                eprintln!("error serving connection from {addr:?}: {e}");
            } else {
                println!("peer done {addr:?}");
            }
        });
    }
}

Rust takes an interesting approach to async programming: it supports some of its fundamental building blocks (like futures and the async and await keywords) in the core language, but leaves the actual async engine implementation (the thing that implements the event loop) to external crates. By far the most popular crate for async programming in Rust is is tokio, so that's what we're using here.

After reading the JS code in part 6, the Rust snippet above should appear fairly familiar, except perhaps the explicit tokio task "spawn". Instead of enqueuing a callback on the connection returned by listener.accept, the code spawns a tokio task, which can be seen as a green thread, and hence uses similar terminology [2]. These tasks must not issue blocking calls; therefore, they are supposed to use tokio's I/O utilities instead of the usual, blocking std utilities. In fact, we have to implement an async version of serve_connection to make this work:

async fn serve_connection_async(mut stream: tokio::net::TcpStream) -> std::io::Result<()> {
    stream.write_all(b"*").await?;

    let mut state = ProcessingState::WaitForMsg;
    let mut buf = [0u8; 1024];
    loop {
        let n = stream.read(&mut buf).await?;
        if n == 0 {
            // Connection closed by the client.
            break;
        }

        for byte in &buf[..n] {
            match state {
                ProcessingState::WaitForMsg => {
                    if *byte == b'^' {
                        state = ProcessingState::InMsg;
                    }
                }
                ProcessingState::InMsg => {
                    if *byte == b'$' {
                        state = ProcessingState::WaitForMsg;
                    } else {
                        let newbyte = byte.wrapping_add(1);
                        stream.write_all(&[newbyte]).await?;
                    }
                }
            }
        }
    }

    Ok(())
}

Note how similar this code is to serve_connection from earlier; the only real differences are the await calls on socket reads and writes [3], and the types involved. For example, instead of a std::net::TcpStream used in the synchronous samples, here we're using tokio::net::TcpStream. Tokio has an underlying dependency called mio to handle non-blocking APIs for all kinds of I/O. It wraps OS-specific event loops like epoll to do so efficiently.

Asynchronous primality testing servers

While most of the series has been using a simple state machine server as the driving example, part 6 switched focus to a server for primality testing which simulates long compute tasks. Let's see how this is done in Rust with tokio:

use tokio::io::{self, AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;

#[tokio::main]
async fn main() -> io::Result<()> {
    let port = match std::env::args().nth(1) {
        Some(s) => s,
        None => "8070".to_string(),
    };
    let addr = format!("127.0.0.1:{port}");

    let listener = TcpListener::bind(addr).await?;
    println!("Serving on port {port}");

    loop {
        let (socket, addr) = listener.accept().await?;
        println!("connection received from {addr:?}");

        tokio::spawn(async move {
            if let Err(e) = serve_client(socket).await {
                eprintln!("error serving connection from {addr:?}: {e}");
            } else {
                println!("peer done {addr:?}");
            }
        });
    }
}

async fn serve_client(mut stream: tokio::net::TcpStream) -> std::io::Result<()> {
    let mut buf = [0u8; 1024];

    loop {
        let n = stream.read(&mut buf).await?;
        if n == 0 {
            // Connection closed by the client.
            return Ok(());
        }

        // Parse read buf into u64.
        let num = std::str::from_utf8(&buf[..n])
            .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?
            .trim()
            .parse::<u64>()
            .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;

        let answer = if isprime(num, true) {
            "prime"
        } else {
            "composite"
        };
        stream
            .write_all((answer.to_string() + "\n").as_bytes())
            .await?;
    }
}

This code is very similar to the previous snippet conceptually; isprime is:

// Check if n is prime, returning a boolean. The delay parameter is optional;
// if true, the function will block for n milliseconds before computing the
// answer. This is useful for simulating a long-running computation.
fn isprime(n: u64, delay: bool) -> bool {
    if delay {
        std::thread::sleep(std::time::Duration::from_millis(n));
    }

    if n < 2 {
        return false;
    }
    if n % 2 == 0 {
        return n == 2;
    }

    let mut r = 3;
    while r * r <= n {
        if n % r == 0 {
            return false;
        }
        r += 2;
    }
    true
}

Note that this sample demonstrates a job that can block (simulated with a sleep in this case). This can be problematic in an async context, as the tokio documentation explains.

One potential solution would be to dispatch a blocking task to a separate thread pool and use tokio channels to communicate with it; this is similar to the approach we've taken in the thread pool sample above.

Part 6 also included a version of this server that caches data on a local Redis instance; the goal was to demonstrate the complexity of event-driven code when additional layers of callbacks are added and how async/await can help mitigate that. Here's our Rust version of this server, using the redis crate (that has a tokio component enabled explicitly to support async calls):

use redis::AsyncTypedCommands;
use redis::aio::MultiplexedConnection;
use tokio::io::{self, AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;

#[derive(Clone)]
struct AppState {
    redis_connection: MultiplexedConnection,
}

const REDIS_URL: &str = "redis://127.0.0.1";

#[tokio::main]
async fn main() -> io::Result<()> {
    let port = match std::env::args().nth(1) {
        Some(s) => s,
        None => "8070".to_string(),
    };
    let addr = format!("127.0.0.1:{port}");

    let app_state = AppState {
        redis_connection: {
            redis::Client::open(REDIS_URL)
                .map_err(io::Error::other)?
                .get_multiplexed_async_connection()
                .await
                .map_err(io::Error::other)?
        },
    };

    let listener = TcpListener::bind(addr).await?;
    println!("Serving on port {port}");

    loop {
        let (socket, addr) = listener.accept().await?;
        println!("connection received from {addr:?}");
        let app_state = app_state.clone();

        tokio::spawn(async move {
            if let Err(e) = serve_client(socket, app_state).await {
                eprintln!("error serving connection from {addr:?}: {e}");
            } else {
                println!("peer done {addr:?}");
            }
        });
    }
}

async fn serve_client(
    mut stream: tokio::net::TcpStream,
    mut app_state: AppState,
) -> std::io::Result<()> {
    let mut buf = [0u8; 1024];

    loop {
        let n = stream.read(&mut buf).await?;
        if n == 0 {
            // Connection closed by the client.
            return Ok(());
        }

        // Parse read buf into u64.
        let num = std::str::from_utf8(&buf[..n])
            .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?
            .trim()
            .parse::<u64>()
            .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;

        // Try the redis cache first. If found in cache, send the cached
        // answer.
        let cachekey = format!("primecache:{num}");
        match app_state.redis_connection.get(cachekey.clone()).await {
            Ok(Some(cached)) => {
                stream.write_all((cached.clone() + "\n").as_bytes()).await?;
                println!("cached num {num} is {cached}");
                continue;
            }
            Ok(None) => {
                // Not found in cache, continue to compute.
            }
            Err(error) => {
                return Err(io::Error::other(error));
            }
        }

        let answer = if isprime(num, true) {
            "prime"
        } else {
            "composite"
        };

        // Save answer in cache and send it to the client.
        app_state
            .redis_connection
            .set(cachekey, answer)
            .await
            .map_err(io::Error::other)?;
        stream
            .write_all((answer.to_string() + "\n").as_bytes())
            .await?;
    }
}

Notes:

Because of the function color problem, the redis crate has a connection constructor specifically for async: get_multiplexed_async_connection.
Here we have an example of shared state between tokio tasks - the Redis connection. Note that we don't require any particular synchronization because MultiplexedConnection is Clone; cloning it to different tasks is safe - and in fact that's what we do for each new task. There's no magic here; if you look inside MultiplexedConnection, you'll see that it already has all the synchronization mechanisms implemented internally, as needed.
Due to the magic of async/await, the code in serve_client is nice and linear. We simply await on the Redis call, and once it's back we continue with the rest of the handler. Since we're using an async Redis connection, in case waiting is required, control will be ceded to some other task that's not currently blocked on I/O.

In conclusion, while Rust provides excellent support for async programming, it doesn't solve its inherent issues like function colors and the need for careful separation between blocking and non-blocking tasks. These issues are typically surmountable with some extra care, and async programming with Tokio in Rust is very popular due to its performance benefits.

Code

All the code for this post is available on GitHub.

[1]

Rust makes a conscious design choice to keep its standard library minimal. In part 2, we've used Python's builtin stdlib thread pools; Rust also has several crates that implement thread pools, but I decided against using them because they obscure the underlying working of the code too much. Our sample implements a simple thread pool, but unfortunately still has to reach for an external crate to use a well-supported multi-consumer channel mechanism.

[2] tokio is quite sophisticated; it manages a thread pool with an efficient work-stealing mechanism, onto which new tasks are spawned. Therefore, all shared data must be protected with mutexes or other synchronization primitives, and Rust's traits like Send and Sync play an important role, where applicable.

[3]	Having to code this alternative just for the sake of async invocation is a classical example of the function color problem.

Relative velocity and closing speed

2026-08-03T20:01:00-07:00

In Physics simulations or game engines it’s sometimes useful to determine the speed with which two objects are approaching each other. This post will discuss the concept of closing speed, which is the normal component of the relative velocity of two objects.

Relative velocity and its components

Suppose we have objects A and B [1] with velocity vectors \vec{V_A} and \vec{V_B}. The relative velocity of B w.r.t. A is:

\[\vec{V}_{B|A}=\vec{V}_B-\vec{V}_A\]

Put differently, it’s the velocity of B in A’s frame of reference.

This relative velocity is a vector, and we can split it into orthogonal components. Obviously, the nature of such a split depends on the basis we want to use. We could look at the vector’s x an y components (we’ll be using - two dimensional space, but everything here applies to 3D as well), but for this post we’re interested in something slightly different:

We draw a line connecting the two objects. The component of \vec{V}_{B|A} in the direction of this line is called the normal component of relative velocity, while the component perpendicular to this direction is called the tangential component.

Computing the normal component

How do we find the component of a vector in the direction of a specific line? By using a vector projection!. We’ll represent the line by a vector, and find the projection of \vec{V}_{B|A} onto this vector.

The positions of A and B can also be seen as vectors: \vec{P}_A and \vec{P}_B. The line connecting them can then be expressed as the vector \vec{P}_B-\vec{P}_A:

All we need from this position difference vector is its direction, not its magnitude, however [2]. So we’ll use the unit vector of \vec{P}_B-\vec{P}_A, denoted as:

\[\widehat{P}=\frac{\vec{P}_B-\vec{P}_A}{|\vec{P}_B-\vec{P}_A|}\]

Finally, to find the projection of \vec{V}_{B|A} onto \widehat{P}, we compute [3]:

\[S_c=\vec{V}_{B|A}\cdot\widehat{P}=(\vec{V}_B-\vec{V}_A)\cdot\widehat{P}\]

Where the multiplication operator between the vectors is the dot product. Note that the result of the dot product is a scalar; therefore, the quantity S_c is called the closing speed - it expresses the rate at which the relative distance of the two objects is changing. If it’s positive, the objects are drifting farther apart; if it’s negative, the objects are getting closer together. Therefore the term "closing speed" may be slightly confusing; alternatively, this has been called a "signed separation speed", or "normal relative speed" [4].

The signs in these calculations can be tricky to get right, so we have to be very careful. Let’s see a few examples that will help us make these computations more concrete.

Examples

To build up some intuition and get some practice with the equations, we’ll review the following examples:

Example I: We’ll start by computing the relative position unit vector \widehat{P}:

\[\widehat{P}=\frac{\vec{P}_B-\vec{P}_A}{|\vec{P}_B-\vec{P}_A|}=\frac{\langle 4,0\rangle}{|\langle 4,0 \rangle|}=\langle1,0\rangle\]

Then, the closing speed is:

\[S_c=(\vec{V}_B-\vec{V}_A)\cdot\widehat{P}=\langle -3,0\rangle\cdot\langle1,0\rangle=-3\]

Based on our convention, the negative sign of S_c means that the objects are approaching each other. Due to the simple nature of the example, this result is easy to verify, as it can be immediately guessed just by looking at the diagram.

Example II: Here \widehat{P} is the same as in the previous example. The closing speed is:

\[S_c=(\vec{V}_B-\vec{V}_A)\cdot\widehat{P}=\langle 3,0\rangle\cdot\langle1,0\rangle=3\]

Same magnitude, but different sign from before, because the objects are moving farther apart.

Example III This example is to demonstrate that we get consistent results even if B is to the left of A. Here the relative position unit vector is:

\[\widehat{P}=\frac{\vec{P}_B-\vec{P}_A}{|\vec{P}_B-\vec{P}_A|}=\frac{\langle -2,0\rangle}{|\langle -2,0 \rangle|}=\langle-1,0\rangle\]

And the closing speed:

\[S_c=(\vec{V}_B-\vec{V}_A)\cdot\widehat{P}=\langle 3,0\rangle\cdot\langle -1,0\rangle=-3\]

Which is the same as in example I, as expected. The direction of \widehat{P} flipped, but so did the direction of the relative velocity vector, so the result has the same sign.

Example IV: Finally, an example showing more arbitrary positions and velocities.

\[\widehat{P}=\frac{\vec{P}_B-\vec{P}_A}{|\vec{P}_B-\vec{P}_A|}=\frac{\langle 3,4\rangle}{|\langle 3,4 \rangle|}=\langle0.6,0.8\rangle\]

Then:

\[S_c=(\vec{V}_B-\vec{V}_A)\cdot\widehat{P}=\langle -3,-6\rangle\cdot\langle 0.6,0.8\rangle=-6.6\]

This example is a good opportunity to demonstrate something important about S_c: it’s time-dependent, because positions change with time. Here, -6.6 is the closing speed at the exact moment when A’s and B’s positions and velocities are as stated in the example. In the next time step, the position of A will be \langle2,5\rangle and the position of B will be \langle2,3\rangle, while their velocities remain the same. The S_c then will be quite different. This is a good segue to the next topic - which is a more physical view of closing speed.

Closing speed as a function of time

The computation shown so far represents a static view of the world; perhaps the right word to use is instantaneous. Given the positions and velocities at a given moment, what is the closing speed between the objects at that exact moment?

But there’s no reason to not generalize this using a more standard physical interpretation of velocity.

First, let’s state the position vectors of A and B as a function of time: \vec{P}_A(t) and \vec{P}_B(t). The relative position vector between the objects is also a function of time:

\[\vec{R}(t)=\vec{P}_B(t)-\vec{P}_A(t)\]

Now we’ll define the scalar distance as the magnitude of this vector:

\[r(t)=|\vec{R}(t)|=|\vec{P}_B(t)-\vec{P}_A(t)|\]

We’re interested in \frac{dr(t)}{dt} - the change in this distance over time. Let’s start by breaking R(t) down to its constituents:

\[\begin{aligned} \vec{R}(t)&=\langle x(t),y(t)\rangle \\ r(t)&=\sqrt{x(t)^2+y(t)^2} \end{aligned}\]

By the chain rule [5]:

\[\begin{aligned} \frac{dr(t)}{dt}&=\frac{\frac{d}{dt}(x(t)^2+y(t)^2)}{2\sqrt{x(t)^2+y(t)^2}}\\ &=\frac{2x(t)x'(t)+2y(t)y'(t)}{2\sqrt{x(t)^2+y(t)^2}} =\frac{x(t)x'(t)+y(t)y'(t)}{\sqrt{x(t)^2+y(t)^2}} \end{aligned}\]

Switching back to the vector representation: since \vec{R}(t)=\langle x(t),y(t)\rangle, the numerator of the fraction above is then a dot product between \vec{R}(t) and \vec{R}'(t), we can write this as:

\[\frac{dr(t)}{dt}=\frac{\vec{R}(t)\cdot\vec{R}'(t)}{|\vec{R}(t)|}=\hat{R}(t)\cdot\vec{R}'(t)\]

But \hat{R}(t) is precisely \hat{P} from the earlier section, as a function of time. Moreover:

\[\vec{R}'(t)=\vec{P}'_B(t)-\vec{P}'_A(t)=\vec{V}_B(t)-\vec{V}_A(t)\]

Because velocity is the time derivative of position. Therefore, we end up with the same equation, just as a function of time:

\[\frac{dr(t)}{dt}=\hat{P}(t)\cdot(\vec{V}_B(t)-\vec{V}_A(t))\]

This formulation is more precise because it makes it very obvious that all the quantities we’re dealing with are time dependent.

[1]	Throughout this post, we assume our objects are sufficiently far apart that they can be treated as points (or particles) without any shape, area or volume.

[2]	The velocity components should be the same regardless of how far apart the two objects are.

[3]	In the standard projection formula we’d also divide by the magnitude of \widehat{P}, but in our case it’s a unit vector anyway.

[4]	In some online sources you may see the closing speed described as closing velocity because it has a direction. IMHO this is the wrong framing. Scalars can be signed! The closing speed is certainly a scalar, not a vector.

[5]	The notation x'(t) means \frac{dx(t)}{dt}

Notes on the Fourier Transform

2026-07-14T20:04:00-07:00

The Fourier series is a great tool for analyzing periodic functions. But what about functions that don’t repeat? We’ve seen that we can compute Fourier series for a non-periodic function defined on a finite interval, as long as we don’t care about its behavior beyond that interval.

Let’s extend this idea to functions that never repeat; that is, non-periodic functions defined on the interval (-\infty,\infty).

Visualizing Fourier series for non-repeating functions

To motivate the subject ahead, let’s look back at the example used in the earlier post about Fourier series:

\[t(x)= \begin{cases} x & 0 \leq x \leq 1 \\ 2-x & 1 < x \leq 2 \\ \end{cases}\]

With an odd extension into [-2,0]. In that post, to make the Fourier series work, we assumed t(x) keeps repeating with a period 2L=4 on the entire x axis. Here, let’s face the reality that it does not - in fact - repeat, and observe how our Fourier series work out.

Recall that the Fourier series approximating t(x) are the sine series (since it’s an odd function):

\[t(x)=\frac{8}{\pi^2}\bigg[ sin\frac{\pi x}{2}-\frac{1}{3^2} sin\frac{3\pi x}{2}+\frac{1}{5^2}sin\frac{5\pi x}{2}-\cdots\bigg]\]

The following visualization is interactive. By default, it shows t(x) (with its odd extension) and no Fourier series approximation. We’ll proceed by a series of steps and observe the outcome:

n (terms in the Fourier series)

x min

x max

Step 1: set to some non-zero number; already at 3, the approximation is very good.

The frequency spacing is \frac{\pi}{L} (this is the coefficient of x in the sines). Note that the Fourier series repeats every 2L, as expected.

Step 2: increase L to 6. This means our series are constructed assuming t(x) has a period of 12, not 4. Note how the Fourier series look now - they repeat every 12, and they don’t match t(x) as well as before. We can increase to a higher number to make the match better. As L grows, the spacing between adjacent frequencies decreases.

Step 3: increase L to 10. We no longer see the repetitions, so feel free to increase the values of x min and x max until you do. Note again that we need to add more and more coefficients to match t(x) better with this larger L, and the spacing adjacent frequencies grows smaller.

Increasing L means our function repeats at larger and larger intervals. The logical conclusion of this progression is to ask - what happens if the function never repeats, meaning L\rightarrow\infty? While not mathematically rigorous, the visual experiment here lets us make some conjectures: we’ll likely need an infinite number of coefficients for a good approximation, and moreover, the spacing between these coefficients will tend to zero.

In other words, instead of a discrete set of coefficients, we’ll end up with a continuous line, or function. The function produced by this process is the Fourier transform of t(x), and the next section shows its mathematical derivation.

Fourier series with L\rightarrow\infty leading to Fourier transform

In these notes, we’ll be using the complex exponential formulation of Fourier series:

\[f(x)=\sum_{n=-\infty}^{\infty}C_n\cdot e^{in\pi x/L}\]

With:

\[C_n=\frac{1}{2L}\int_{-L}^{L}f(x)e^{-in\pi x/L}dx\]

We’re interested in a non-periodic defined on the interval (-\infty,\infty). So we’ll be exploring the above equations for L\rightarrow\infty.

First, let’s make a slight change of notation. Instead of writing formulae in terms of the period (2L), we’ll be using the n-th harmonic angular frequency w_n:

\[w_n=\frac{n\pi}{L}\]

So we can slightly rewrite our series as:

\[f(x)=\sum_{n=-\infty}^{\infty}C_n\cdot e^{i w_n x}=\sum_{n=-\infty}^{\infty}C_n\cdot e^{i\cdot n \Delta w x}\]

Using \Delta w as the difference between two consecutive frequencies:

\[\Delta w=w_n-w_{n-1}=\frac{n\pi}{L}-\frac{(n-1)\pi}{L}=\frac{\pi}{L}\]

Using this notation, C_n is expressed as:

\[C_n=\frac{\Delta w}{2\pi}\int_{-\pi/\Delta w}^{\pi/\Delta w}f(x)e^{-i\cdot n \Delta w x}dx\]

So far there are no new insights here, just some new notation. Now we’re going to use it to facilitate the next step.

Since L\rightarrow \infty, then \Delta w\rightarrow 0. Let’s calculate the limit of the Fourier series representation of when \Delta w\rightarrow 0:

\[f(x)=\lim_{\Delta w\rightarrow 0}\sum_{n=-\infty}^{\infty}C_n\cdot e^{i\cdot n \Delta w x}\]

And substitute the latest C_n into this equation, changing its dummy integration variable from x to t to avoid confusion [1]

\[f(x)=\lim_{\Delta w\rightarrow 0}\sum_{n=-\infty}^{\infty}\left[\frac{\Delta w}{2\pi}\int_{-\pi/\Delta w}^{\pi/\Delta w}f(t)e^{-i\cdot n \Delta w t}dt\right]\cdot e^{i\cdot n \Delta w x}\]

Reordering slightly, and also replacing n\Delta w by w_n in the complex exponents:

\[f(x)=\frac{1}{2\pi}\lim_{\Delta w\rightarrow 0}\sum_{n=-\infty}^{\infty}\left[\int_{-\pi/\Delta w}^{\pi/\Delta w}f(t)e^{-i\cdot w_n t}dt\right]\cdot e^{i\cdot w_n x}\Delta w\]

Looking at the limit with the sum carefully, this is a Riemann sum (see Appendix A)! w_n is the "sampled" version of , and \Delta w\rightarrow 0. We can therefore replace it by an integral, changing w_n to and \Delta w to dw [2]:

\[f(x)=\frac{1}{2\pi}\int_{-\infty}^{\infty}\left[\int_{-\infty}^{\infty}f(t)e^{-i\cdot wt}dt\right]\cdot e^{i\cdot w x}dw\]

The inner integral is called the Fourier transform of and denoted [3]:

\[\boxed{\hat{f}(w)=\mathcal{F}\left[f(x)\right]=\int_{-\infty}^{\infty}f(x)e^{-i\cdot wx}dx}\]

And the full equation for is then the inverse Fourier transform:

\[\boxed{f(x)=\mathcal{F}^{-1}\left[\hat{f}(w)\right]=\frac{1}{2\pi}\int_{-\infty}^{\infty}\hat{f}(w)e^{i\cdot w x}dw}\]

Example calculation of Fourier transform

Let’s take our favorite odd triangular pulse example and calculate its Fourier transform. The function’s mathematical definition and plot are shown earlier in this post. Note that we’re not extending this function periodically - it’s zero beyond the range [-2,2]; this is exactly why we need the Fourier transform here - as we’ve seen, Fourier series won’t do because the function they reconstruct eventually starts repeating.

We’re looking to find:

\[\hat{t}(w)=\int_{-\infty}^{\infty}t(x)e^{-iwx}dx\]

To calculate the integral, let’s decompose the complex exponent using Euler’s formula:

\[\hat{t}(w)=\int_{-\infty}^{\infty}t(x)cos(wx)dx-i\int_{-\infty}^{\infty}t(x)sin(wx)dx\]

Since our t(x) is odd, the first integral is zero. Also t(x)sin(wx) is even, so we can write:

\[\hat{t}(w)=-2i\int_{0}^{\infty}t(x)sin(wx)dx\]

We’ve already calculated a very similar integral in the post on Fourier series, so let’s just skip to the result:

\[\hat{t}(w)=-2i\cdot\frac{2\cdot sin(w)-sin(2w)}{w^2}\]

The only remaining difficulty is its value at 0, which seems undefined at first (division by zero). However, note that as w\rightarrow 0, the numerator also tends to 0, so we can use L’Hopital’s rule (twice!) to find that:

\[\lim_{w\rightarrow 0} \hat{t}(w)=0\]

Therefore:

\[\hat{t}(w)= \begin{cases} -2i\cdot\frac{2\cdot sin(w)-sin(2w)}{w^2} & w\neq 0 \\ 0 & w=0 \\ \end{cases}\]

This function is complex-valued; in fact, it’s purely imaginary. How do we visualize it? A common way to visualize complex-valued functions is by plotting their magnitude and phase separately.

The magnitude of \hat{t}(w) is:

\[|\hat{t}(w)|=\sqrt{\hat{t}(w)\cdot\hat{t}(w)^*}=2\left|\frac{2\cdot sin(w)-sin(2w)}{w^2} \right|\]

Since \hat{t}(w) is purely imaginary, there are only two options for the phase:

When the numerator is positive, we get a negative imaginary number with phase -\pi/2, and when the numerator is negative, we get a positive imaginary number with phase \pi/2. Finally, when \hat{t}(w)=0 (which happens at w=0, by our earlier analysis, but also whenever is a whole multiple of \pi), the phase is undefined.

Here’s the magnitude and phase of \hat{t}(w) plotted against :

It is common to talk about \hat{t}(w) as the frequency domain representation of t(x).

The frequency domain representation of functions

When the functions we’re working with have time as their domain (e.g. the x in t(x) represents time), which is often the case in the study of signals and systems, the Fourier transform can be seen as computing the frequency domain representation of the function.

Here’s the Fourier transform formula again:

\[\hat{f}(w)=\mathcal{F}\left[f(x)\right]=\int_{-\infty}^{\infty}f(x)e^{-i\cdot wx}dx\]

It takes - the time domain representation of a function, and converts it to \hat{f}(w) - a frequency domain representation. For well-behaved functions, these two representations are dual - each one describes the function completely, just in a different way.

To convert back from a frequency domain representation to the time domain, we use the inverse Fourier transform:

\[\mathcal{F}^{-1}\left[\hat{f}(w)\right]=\frac{1}{2\pi}\int_{-\infty}^{\infty}\hat{f}(w)e^{i\cdot w x}dw\]

While a time-domain plot (t(x)) shows how a signal changes over time, a frequency-domain plot (\hat{t}(w)) shows how the signal is distributed across all possible frequencies. Moreover, as we’ve seen, \hat{t}(w) is complex valued. Each frequency therefore has both a magnitude and a phase: the magnitude tells us how strongly that frequency contributes, while the phase tells us how that component is shifted.

The frequency domain is extremely useful in signal analysis; for example, when designing filters.

The Fourier transform also has a number of properties that are very useful in signal analysis and processing. But first, let’s discuss what a "well-behaved function" means for the purpose of applying Fourier transforms.

Existence condition for the Fourier transform

The simplest existence condition for Fourier transforms is absolute integrability (also known as Lebesgue integrable):

\[\int_{-\infty}^{\infty}|f(x)|dx<\infty\]

With this condition, \hat{f}(w) exists on the entire domain, is continuous and vanishes (tends to 0) as |w|\rightarrow\infty [4].

While this condition is sufficient, it’s not necessary; there are less well-behaved functions that also have Fourier transforms defined with some limitations. In these notes, we’re mostly interested in well-behaved functions that are used in real-world engineering, so we won’t discuss the other cases.

Another assumption commonly made for real-world functions is that they vanish (tend to 0) as |x|\rightarrow\infty. While this is not a direct outcome of absolute integrability [5], it’s a reasonable assumption in engineering. After all, real-world signals have finite energies.

Intuitively, when we also assume is uniformly continuous, the assumption of vanishing at |x|\rightarrow\infty is a logical conclusion, because otherwise how can the total area for |f(x)| be finite?

An important outcome of this discussion is that the Fourier transform is unsuitable for periodic functions. Functions that repeat at intervals are not absolute integrable. For periodic functions, we use Fourier series.

Some useful properties of Fourier transforms

Linearity

The Fourier transform is a linear operator, because the integral is linear:

\[\begin{aligned} \mathcal{F}\left[\alpha f(x)+\beta g(x)\right]&=\int_{-\infty}^{\infty}\alpha f(x)e^{-i\cdot wx}dx+\int_{-\infty}^{\infty}\beta g(x)e^{-i\cdot wx}dx\\ &=\alpha\int_{-\infty}^{\infty}f(x)e^{-i\cdot wx}dx+\beta\int_{-\infty}^{\infty}g(x)e^{-i\cdot wx}dx\\ &=\alpha\mathcal{F}\left[f(x)\right]+\beta\mathcal{F}\left[g(x)\right] \end{aligned}\]

So is the inverse Fourier transform; it’s similarly easy to show that:

\[\mathcal{F}^{-1}\left[\alpha\hat{f}(w)+\beta\hat{g}(w)\right]= \alpha\mathcal{F}^{-1}\left[\hat{f}(w)\right]+\beta\mathcal{F}^{-1}\left[\hat{g}(w)\right]\]

Scaling

If we scale the domain of a function by a constant, its transform changes only slightly:

\[\mathcal{F}\left[f(ax)\right]=\int_{-\infty}^{\infty}f(ax)e^{-i\cdot wx}dx\]

Let’s do the variable substitution u=ax:

\[\mathcal{F}\left[f(ax)\right]=\frac{1}{a}\int_{-\infty}^{\infty}f(u)e^{-i\cdot \frac{wu}{a}}du\]

This is the Fourier transform evaluated at \frac{w}{a}, so:

\[\mathcal{F}\left[f(ax)\right]=\frac{1}{a}\hat{f}\left(\frac{w}{a}\right)\]

There’s one small caveat here; when a is negative, the integral bounds should be flipped, causing a minus sign in front of the transform. So we can write:

\[\mathcal{F}\left[f(ax)\right]=\frac{1}{|a|}\hat{f}\left(\frac{w}{a}\right)\]

Which works for any a\ne 0.

This property is intuitive when thinking about signals: suppose a>0, then f(ax) means the signal is compressed in the time domain by a factor a. The scaling property says that the frequency domain is expanded using the same factor; in other words, the higher frequencies become more prominent because we need sharper transitions to represent the compressed signal.

Time shifting

What happens to the Fourier transform if we time-shift the input signal by some constant: f(x-x_0). By definition:

\[\mathcal{F}\left[f(x-x_0)\right]=\int_{-\infty}^{\infty}f(x-x_0)e^{-i\cdot wx}dx\]

Substituting u=x-x_0, we get du=dx, so:

\[\begin{aligned} \mathcal{F}\left[f(x-x_0)\right]&=\int_{-\infty}^{\infty}f(u)e^{-i\cdot w(u+x_0)}du\\ &=e^{-iwx_0}\int_{-\infty}^{\infty}f(u)e^{-i\cdot wu}du\\ &=e^{-iwx_0}\mathcal{F}\left[f(x)\right] \end{aligned}\]

Transform of a derivative

An extremely useful property that’s often employed in the solution of partial differential equations; let’s calculate the Fourier transform of the derivative of :

\[\mathcal{F}\left[f'(x)\right]=\int_{-\infty}^{\infty}f'(x)e^{-i\cdot wx}dx\]

We’ll use integration by parts, where dv=f'(x) and u=e^{-i\cdot wx}. Therefore, v=f(x) and du=-iw\cdot e^{-i\cdot wx}:

\[\mathcal{F}\left[f'(x)\right]=\left[f(x)e^{-i\cdot wx}\right]^{\infty}_{-\infty}-\int_{-\infty}^{\infty}f(x)(-iw\cdot e^{-i\cdot wx})dx\]

Recall the assumption made in the "Existence condition..." section about vanishing at infinities. So the first part of the equation above is zero, and we’re left with:

\[\begin{aligned} \mathcal{F}\left[f'(x)\right]&=-\int_{-\infty}^{\infty}f(x)(-iw\cdot e^{-i\cdot wx})dx\\ &=iw\int_{-\infty}^{\infty}f(x)e^{-i\cdot wx}dx\\ &=iw\cdot\mathcal{F}\left[f(x)\right] \end{aligned}\]

Transform of convolution

The convolution between two continuous functions and g(x) is defined as:

\[(f\ast g)(x)=\int_{-\infty}^{\infty}f(\xi)g(x-\xi)d\xi\]

Let’s calculate the Fourier transform of this function:

\[\begin{aligned} \mathcal{F}\left[(f\ast g)(x)\right]&=\int_{-\infty}^{\infty}e^{-i\cdot wx}\left[\int_{-\infty}^{\infty}f(\xi)g(x-\xi)d\xi\right]dx\\ &=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-i\cdot wx}f(\xi)g(x-\xi)d\xi\ dx \end{aligned}\]

This step of combining the integrals into a double integral, as well as the next step (changing the order of integration) is possible due to Fubini’s theorem and our assumption that and g(x) are Lebesgue integrable.

Switch order of integration:

\[\mathcal{F}\left[(f\ast g)(x)\right]=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-i\cdot wx}f(\xi)g(x-\xi)dx\ d\xi\]

Now, f(\xi) in the inner integral doesn’t depend on x, so we can pull it out:

\[\mathcal{F}\left[(f\ast g)(x)\right]=\int_{-\infty}^{\infty}f(\xi)\int_{-\infty}^{\infty}e^{-i\cdot wx}g(x-\xi)dx\ d\xi\]

The inner integral is just the Fourier transform of a time-shifted g(x-\xi), so we can write:

\[\mathcal{F}\left[(f\ast g)(x)\right]=\int_{-\infty}^{\infty}f(\xi)e^{-i\cdot w\xi}\mathcal{F}\left[g(x)\right]d\xi=\mathcal{F}\left[g(x)\right]\int_{-\infty}^{\infty}e^{-i\cdot w\xi}f(\xi)d\xi\]

And the remaining integral is the Fourier transform of , so:

\[\mathcal{F}\left[(f\ast g)(x)\right]=\mathcal{F}\left[f\right]\cdot\mathcal{F}\left[g\right]\]

Convolution in the time domain translates to multiplication in the frequency domain! This result is so important in signal processing that it’s called the convolution theorem.

Appendix A: Riemann sum and the definite integral

Suppose we have some function and we want to know the area bounded between this function’s graph and the x axis in a certain interval [a,b]. One way to do this is to take a partition of the interval:

\[a=x_0<x_1<\cdots<x_{n-1}<x_n=b\]

And calculate the area under for every element of the partition. We can then approximate such sub-areas by rectangles, as follows:

We’ll denote the area of each rectangle as f(x^*_i)\cdot\Delta x:

\Delta x=(b-a)/n is the width of one interval (assuming a uniform partition, but the math works just as well for non-uniform ones).
x^*_i is some value in the interval [x_{i-1},x_i].

There are many ways to choose which point of the interval [x_{i-1},x_i] to denote as x^*_i: left point (x_{i-1}), right point (), mid-point between the two (which is what our plot shows) or anything in between. The distinction doesn’t really matter for our purpose, as we will soon see.

We can approximate the area under the curve of in the interval [a,b] with the Riemann sum, using a uniform partition:

\[S=\sum_{i=1}^{n}f(x^*_i)\Delta x\]

If is continuous on [a,b], then as n\rightarrow \infty:

\[S=\lim_{n\rightarrow \infty}\sum_{i=1}^{n}f(x^*_i)\Delta x=\int_{a}^{b}f(x)dx\]

This is known as the Riemann integral, or just the definite integral. The limit is why the exact choice of x^*_i doesn’t matter: as n\rightarrow\infty we have \Delta x\rightarrow 0, and all points within [x_{i-1}, x_i] are equally good.

[1]	Note that C_n is not a function of x; in its definition, x only serves as a dummy integration variable and can be called anything we choose. When we substitute C_n into the equation for , which is a function of x, we have to be careful. Thus the renaming.

[2]	Note we apply the limit; therefore, the bounds of the inner integral (in the square brackets) are now also between -\infty and \infty

[3]	We change the dummy integration variable back to x here, for consistency. Once again, since x is just the integration variable and the integral is definite, the final result doesn’t depend on x. It’s a function of .

[4]	The vanishing at infinity part is the Riemann-Lebesgue lemma; you can find a proof on Wikipedia

[5]	A pathological absolute-integrable function can have spikes at infinity but still have a finite total area.