To evaluate such a polynomial using a computer program, several approaches can be employed.

The simplest, naive method is to compute each term of the polynomial separately and then add them up. Here’s the Python code for it:

def poly_naive(A, x):
    p = 0
    for i, a in enumerate(A):
        p += (x ** i) * a
    return p

A is an array of coefficients, lowest first, until .

This method is quite inefficient. It requires n additions (since there are n+1 terms to be added) and multiplications.

def poly_iter(A, x):
    p = 0
    xn = 1
    for a in A:
        p += xn * a
        xn *= x
    return p

def poly_horner(A, x):
    p = A[-1]
    i = len(A) - 2
    while i >= 0:
        p = p * x + A[i]
        i -= 1
    return p

Related posts:

1. Efficient integer exponentiation algorithms Did you ever think about the most efficient method to...
2. Efficient modular exponentiation algorithms Earlier this week I’ve discussed efficient algorithms for exponentiation. However,...
3. the rule of 72 Say you invest some amount of money, and you get...

Where is Euler’s totient function – the amount of numbers smaller than n that are coprime to it.

Here I want to present a nice proof of this theorem, based on group theory. I begin with some preliminary definitions and gradually move towards the final goal.

(I) Congruence class: Let a and n > 0 be integers. The set of all integers that have the same remainder as a when divided by n is called the congruence class of a modulo n and is denoted by , where:

is the set of all congruence classes modulo n.

(II) Units of : If for we find some such that , we call a unit of . The set of units of is denoted by

For example, is a unit of , because .

(III) The congruence class is a unit of if and only if (the GCD of a and n is 1, in other words they’re co-prime).

Proof: By definition of units, there exists some such that . Therefore , which implies that for some q, . Thus . So 1 is a linear combination of a and n. Therefore .
On the other hand, if , there exist q and b such that , or , so .

(IV) By definition, since every unit of is coprime to n, the amount of units of (or, the number of elements of ) is .

Let’s keep this result in mind and prepare some more theorems in order to attack the proof.

(V) Lagrange’s theorem: If H is a subgroup of the finite group G, then the order of H is a divisor of the order of G.

Proof: Let’s first define and . Also, let be the equivalence relation defined in example (III) of the previous post. Since it’s an equivalence relation, it partitions G into equivalence classes. Define as the equivalence class of a with , for any .

To prove Lagrange’s theorem, we’re going to show that has the same amount of elements as H. For this purpose, let’s define a function by for all and prove that it’s an isomorphism. To do that, we’ll have to separately prove that it’s onto and one-to-one.

But first, let’s verify that the stated codomain of is correct. If then because , so by definition of we have . So indeed the codomain of is .

1. Let’s pick some y in G such that . By definition of our it means that for some . So has a solution (since ). Therefore is onto.
2. Suppose that with . Then and by cancellation in groups we have , which proves that is one-to-one.

So we’ve proved that is an isomorphism, which means that (we can map each element of to one and only one element of ).

We’ve previously shown that the equivalence classes of partition G. But now we see that the size of each equivalence class is equal to . Therefore, all the equivalence classes are of the same size, and where t is the amount of equivalence classes. This proves Lagrange’s theorem.

We’re almost there. To see how all of this relates to Euler’s theorem, let’s first define the order of an element of a group.

(VI) Order of group element: Let . If there exists a positive integer n such that , then a is said to have finite order and the smallest such positive integer is called the order of a, denoted by .

We’ll also define the subgroup generated by an element:

(VII) Cyclic subgroup: is a cyclic subgroup of G generated by . For a finite G this subgroup is also finite, and its size is: .

Armed with these definitions, we’re now ready for the following corollary of Lagrange’s theorem:

(VIII) Lagrange theorem corollary: Let G be a finite group of order n. Then:

1. For any , divides n
2. For any ,

Proof: As we’ve seen, and by Lagrange’s theorem divides n (since is a subgroup of G). Therefore (1) is proven.
For (2), note that if a has order m, then by (1) we have for some integer q. Thus . But a has order m, so and therefore . Q.E.D.

We now finally have all the tools required to prove Euler’s theorem.

Proof of Euler’s theorem: Let the group of units modulo n. The order of G is (by (IV)). Now, by (VIII) part (2), raising any congruence class to the power must give the identity element. The statement is equivalent to

Related posts:

1. Equivalence classes and group partitions In this post I want to show some interesting definitions...
2. The GCD and linear combinations A linear combination of a and b is some integer...
3. Pythagorean – the theorem with most proofs ? This page shows 76 different proofs of the Pythagorean theorem....

Relations are an important topic in algebra. Conceptually, a relation is a statement aRb about two elements of a set. If the elements are integers, then is a relation, and so is .

Here’s a formal set-theoretic definition:

(I) Binary relation: A binary relation on a set A is a collection of ordered pairs of elements of A. In other words, it is the subset of . More generally, a binary relation between two sets A and B is a subset of .

Note it says ordered pairs. What this means is that the order of elements in a relation is important. Intuitively, given the relation and the set of integers, it’s clear that does not generally imply .

An important sub-class of relations we’ll be most interested with is the equivalence relations:

(II) Equivalence relation: A relation on a set is called an equivalence relation if it’s reflexive, symmetric and transitive:

• Reflexive: for all a in S it holds that .
• Symmetric: for all a and b in S it holds that if then .
• Transitive: for all a, b and c in S it holds that if and then .

Examples: equality is an equivalence relation, but greater-or-equal is not, as doesn’t imply – the symmetric condition doesn’t hold.

Another important equivalence relation is the congruence modulo an integer, .

(III) Example: Let G be a group and H a subgroup of G. For , define if . Then is an equivalence relation on G.

Proof: To prove that some relation is an equivalence relation, we have to prove the three properties of equivalence relations.

Reflexive: is the identity element e. However, since H is a subgroup of G, it means that . Therefore , so .

Symmetric: Assume that . Since H is a subgroup, then this element has an inverse in H: . Using the associative law of groups several times it’s possible to show that
So, , hence .

Transitive: Given and .
So, , hence .

Thus, we’ve proved that this is an equivalence relation. Let’s see a concrete application of the result we’ve just proved:

Consider that if with the operation +, and H is the subgroup consisting of all multiples of some , then actually means that for some . In other words . This proves that congruence modulo n is an equivalence relation, since it’s a special case of (III).

(IV) Equivalence class: If is an equivalence relation on S, then [a], the equivalence class of a is defined by

For example, let’s take the integers and define an equivalence relation "congruent modulo 5". For instance, . The congruence class of 1 modulo 5 (denoted ) is .

(V) Group partition: If is an equivalence relation on S, then for all , and implies that . In other words, partitions S into disjoint equivalence classes.

Proof: the first part is easy. Since always , then . To prove the second part, we’ll show that if then .

Suppose that , and let . Therefore and . But is an equivalence relation and thus is transitive and symmetric. So . But this means that a and b are in the same equivalence class: . Q.E.D.

Related posts:

1. A group-theoretic proof of Euler’s theorem A very important and useful theorem in number theory is...
2. The GCD and linear combinations A linear combination of a and b is some integer...
3. The well-ordering principle The well-ordering principle states: The well-ordering principle: Any nonempty set...

Well, no more! Detexify is a great new service that allows you to “draw” the symbol you’re looking for:

… and it will suggest the Latex code.

Detexify is a learning OCR classifier, and can be “trained” by users to improve it’s performance.

Kudos to the creator of Detexify for a great project. It will definitely be useful…

No related posts.

Here I want to show a generalization, that uses n-nary arithmetic. But first, let’s define the problem:

Suppose you have a set of elements and you want to select multi-subsets from it. By multi-subset in this context I mean that an element can appear more than once in it. For example, given the set {0, 1, 2, 3, 4, 5}, then {1, 1, 2} is a multi-subset. So are {5, 5, 5, 5} and {0, 1, 2, 3, 4, 5}. Suppose you want to go over all multi-subsets of a set. How can this be done?

Note that generating a superset is a private case of this problem, restricting each element to appear either 0 or 1 times in the resulting subset.

So the solution is a generalization of the binary-arithmetic solution for the powerset problem.

Intuitive motivation: consider the decimal numbers, for example 25. If we use the position of each digit (starting with the units) to convey information, this leads to an interesting observation. If we have two elements to choose from, 25 may mean 5 times the 1st element, 2 times the second element. Now, going over all numbers from 0 to 99, we are actually generating all multi-subsets of two elements where each can be picked from 0 to 9 times.

Once this is clear, the algorithm is simple. Let’s generalize to a n-ary base system, using position to point to an element and the ‘digit’ at this position to say how many times it appears in a given multi-subset. And the best part – the simple rules of addition with carry can now be used to efficiently generate all multi-subsets, given the amount of elements we have (length) and the maximal amount of times each can be picked (upto), the minimum being assumed 0.

Here’s the code:

def multiselects(upto, length):
  # Arithmetically, we create an array of digits
  # (each in the range 0..upto).
  # It's initialized with '1'
  #
  ar = [1] + [0] * (length - 1)

  while True:
      yield ar

      # The index we're currently trying to
      # advance
      #
      idx = 0

      # Advance the current index. If it reaches
      # the limit (upto), pefrorm a carry to the
      # next index (digits)
      #
      while idx < length:
          ar[idx] += 1
          if ar[idx] <= upto:
              break
          else:
              ar[idx] = 0
              idx += 1

      # We've reached the last number...
      #
      if idx == length:
          break

An an example run of:

for s in multiselects(2, 3):
    print s

Produces:

[1, 0, 0]
[2, 0, 0]
[0, 1, 0]
[1, 1, 0]
[2, 1, 0]
[0, 2, 0]
[1, 2, 0]
[2, 2, 0]
[0, 0, 1]
[1, 0, 1]
[2, 0, 1]
[0, 1, 1]
[1, 1, 1]
[2, 1, 1]
[0, 2, 1]
[1, 2, 1]
[2, 2, 1]
[0, 0, 2]
[1, 0, 2]
[2, 0, 2]
[0, 1, 2]
[1, 1, 2]
[2, 1, 2]
[0, 2, 2]
[1, 2, 2]
[2, 2, 2]

Note that the solution is general, as the lists it returns are lists of indices. These can be employed with any set to generate multi-subsets.

Background and links

I came up with this function while working on Project Euler’s problem 77. I ended up using a different method, but visualizing the possible partitions of primes was very useful.

Here are some interesting mathematical links related to this problem:

• Euler transform
• Partition function P
• Prime partition

Related posts:

1. application of combinations The text buffewring widget I’m building allows the user to...
2. Allocating multi-dimensional arrays in C++ Updated on 04.06.2010 Allocating multi-dimensional arrays in C++ (and C)...
3. Generating random sentences from a context free grammar Sometimes it’s interesting to randomly generate a large amount of...

There’s a very interesting theorem that gives a useful connection between linear combinations and the GCD of a and b, called Bézout’s identity:

Bézout’s identity: (the GCD of a and b) is the smallest positive linear combination of non-zero a and b.

Both Bézout’s identity and its corollary I show below are very useful tools in elementary number theory, being used for the proofs of many of the most fundamental theorems. Let’s see why it’s true.

(I) Intuition: First I’d like to explain this (surprising at first sight) theorem intuitively. By defintion, any common divisor of a and b will divide for all . In particular, also divides any .

Now, assume we’ve found some small which isn’t the GCD. But we’ve just said that divides all linear combinations, so it also divides x. Therefore, x can not be smaller than the GCD. In other words, the smallest positive linear combination can only be itself.

Corollary: An integer is a linear combination of a and b IFF it is a multiple of their GCD.

To prove Bézout’s identity more formally, and along the way to see why the corollary is also true, let’s first prove the following:

(III) Let I be a nonempty set of integers that is closed under addition and subtraction, and contains at least one non-zero integer. Then there exists a smallest positive element , and I consists of all multiples of b ().

Proof: I contains at least one non-zero integer. Then it definitely contains at least one positive integer, because it is closed under addition and subtraction. Assume we have for some . Therefore and then also . Thus we have positive integers in I. According to the well-ordering principle, I has a smallest positive element which we’ll call b.

Now we’ll want to show that . As usual, to prove equalities of sets, it will be shown that they contain one another.

is obvious – since I contains b and is closed under addition and subtraction, it contains all the multiples of b.

To prove we’ll demonstrate that any element is a multiple of b. Using the division algorithm we write for some integers q and 0 <= r < b. But this means that (because I contains bq and c and is closed under subtraction and addition). However, recall that b was chosen to be the smallest positive element of I, so r must be equal to 0. Therefore c is a multiple of b, and we have shown that . Q.E.D.

Now back to Bézout’s identity. We’ll define:

This I is obviously non-empty and is closed under addition and subtraction (by its definition as a linear combination). Note, in particular, that it also contains a and b. By (III), I consists of all multiples of its smallest positive element, which we’ll call d here.

To show that we have to show that d|a, d|b and if c|a and c|b then c|d. First, by definition d is a divisor of any element in I, so it also divides a and b. If c|a and c|b, say a=cq and b=cp, then:

So d|c, which completes our proof that d=(a,b). Q.E.D.

Regarding the corollary, it stems trivially from the definition of I and the proof above.

Related posts:

1. A group-theoretic proof of Euler’s theorem A very important and useful theorem in number theory is...
2. mathematical musing This morning while washing the dishes I was contemplating a...
3. application of combinations The text buffewring widget I’m building allows the user to...

The well-ordering principle: Any nonempty set of nonnegative integers has a smallest element.

DUH, you don’t say! – seems obvious, doesn’t it? This principle is, nevertheless, a very important and fundamental tool for proving other basic principles of number theory.

Consider, for instance, the Division Algorithm:

The Division Algorithm: If m and n are integers with n > 0, then there exist integers q and r, with 0 <= r < n, such that .

Again, this is so basic that one may doubt whether it should even be proved. But the well-ordering principle allows us, in fact, to prove the division algorithm in a rigorous manner:

Let . It is obvious that W contains nonnegative integers. Let . By the well-ordering principle, V has a smallest element, which we’ll call r. , so for some q and r >= 0 (by the definition of sets W and V, correspondingly).

Now, what’s left to prove is that r < n. Let’s assume the opposite, namely that . Rearranging: . By the definition of V, (since it has the form for some integer t and is nonnegative). But recall that we called r the smallest element of V, and , so we have a contradiction.

Therefore, we see that r < n. This completes the proof. Q.E.D.

Related posts:

1. The GCD and linear combinations A linear combination of a and b is some integer...
2. Equivalence classes and group partitions In this post I want to show some interesting definitions...
3. A group-theoretic proof of Euler’s theorem A very important and useful theorem in number theory is...

I will not post the solution or the full code for the problem here, just a couple of hints.

After a very short bout of Googling, you’ll discover that the Diophantine equation:

Is quite famous and is called Pell’s equation. From here, further web searches and Wikipedia-reading will bring you to at least two methods for finding the fundamental solution, which is the pair of x and y with minimal x solving it.

One of the methods involves computing the continued-fraction representation of the square root of D. This page is a must read on this topic, and will help you with other Euler problems as well.

I want to post here a code snippet that implements the continued-fraction computation described in that link. Its steps follow the Algebraic algoritm given there:

def CF_of_sqrt(n):
    """ Compute the continued fraction representation of the
        square root of N.

        The first element in the returned array is the whole
        part of the fraction. The others are the denominators
        up to (and not including) the point where it starts
        repeating.

        Uses the algorithm explained here:
        http://www.mcs.surrey.ac.uk/Personal/R.Knott/Fibonacci/cfINTRO.html
        In the section named: "Methods of finding continued
        fractions for square roots"
    """
    if is_square(n):
        return [int(math.sqrt(n))]

    ans = []

    step1_num = 0
    step1_denom = 1

    while True:
        nextn = int((math.floor(math.sqrt(n)) + step1_num) / step1_denom)
        ans.append(int(nextn))

        step2_num = step1_denom
        step2_denom = step1_num - step1_denom * nextn

        step3_denom = (n - step2_denom ** 2) / step2_num
        step3_num = -step2_denom

        if step3_denom == 1:
            ans.append(ans[0] * 2)
            break

        step1_num, step1_denom = step3_num, step3_denom

    return ans

As I said, this still isn’t enough to solve the problem, but with this code in hand, the solution isn’t too far. Read some more about Pell’s equation and you’ll discover how to use this code to reach a solution.

It took my program ~30 milliseconds to find an answer to the problem, by the way. It took less than a second to solve a 10-times larger problem (for D <= 10000), so I believe it to be a pretty good implementation.

Related posts:

1. Project Euler problem 69 Problem 69 is a really nice one. I looked at...
2. Project Euler problem 12 I’ve got hooked to Project Euler this week, after hearing...
3. Project Euler problem 83 – how creeps can help Problems 18, 67, 81, 82 and 83 in Project Euler...

However, for real-life needs of number theoretic computations, just raising numbers to large exponents isn’t very useful, because extremely huge numbers start appearing very quickly [1], and these don’t have much use. What’s much more useful is modular exponentiation, raising integers to high powers [2]

Luckily, we can reuse the efficient algorithms developed in the previous article, with very few modifications to perform modular exponentiation as well. This is possible because of some convenient properties of modular arithmetic.

def modexp_mul(a, b, n):
    r = 1
    for i in xrange(b):
        r = r * a % n
    return r

def modexp_rl(a, b, n):
    r = 1
    while 1:
        if b % 2 == 1:
            r = r * a % n
        b /= 2
        if b == 0:
            break
        a = a * a % n

    return r

def modexp_lr(a, b, n):
    r = 1
    for bit in reversed(_bits_of_n(b)):
        r = r * r % n
        if bit == 1:
            r = r * a % n
    return r

def _bits_of_n(n):
    """ Return the list of the bits in the binary
        representation of n, from LSB to MSB
    """
    bits = []

    while n:
        bits.append(n % 2)
        n /= 2

    return bits

def modexp_lr_k_ary(a, b, n, k=5):
    """ Compute a ** b (mod n)

        K-ary LR method, with a customizable 'k'.
    """
    base = 2 << (k - 1)

    # Precompute the table of exponents
    table = [1] * base
    for i in xrange(1, base):
        table[i] = table[i - 1] * a % n

    # Just like the binary LR method, just with a
    # different base
    #
    r = 1
    for digit in reversed(_digits_of_n(b, base)):
        for i in xrange(k):
            r = r * r % n
        if digit:
            r = r * table[digit] % n

    return r

def _digits_of_n(n, b):
    """ Return the list of the digits in the base 'b'
        representation of n, from LSB to MSB
    """
    digits = []

    while n:
        digits.append(n % b)
        n /= b

    return digits

Summary

As we’ve seen, exponentiation and modular exponentiation are one of those applications in which an efficient algorithm is required for feasibility. Using the trivial/naive algorithms is possible only for small cases which aren’t very interesting. To process realistically large numbers (such as the ones required for cryptographic algorithms), one needs powerful methods in his toolbox.

[1]	For instance, is a 4772-digit number.

[2]	Modular exponentiation is essential for the RSA algorithm, for example.

[3]	To be a bit more rigorous, we start with . This means that , so also . Similarly , so also . Adding these two we get ), which means that .

[4]	Using the 3-argument form of pow, you can perform modular exponentiation.

[5]	FIVEARY_CUTOFF in the code of pow is set to 8. This means that for exponents with more than 8 digits, a special 5-ary algorithm is used. For smaller exponents, the regular LR binary method is used – just like the one I presented, just coded in C.

[6]	Note that for m = 2 this is the familiar binary LR method.

Related posts:

1. Efficient integer exponentiation algorithms Did you ever think about the most efficient method to...
2. Computing modular square roots in Python Consider the congruence of the form: n is a quadradic...
3. Horner’s rule: efficient evaluation of polynomials Here’s a general degree-n polynomial: To evaluate such a polynomial...

def expt_mul(a, b):
    r = 1
    for i in xrange(b):
        r *= a
    return r

Exponentiation by squaring

The efficient exponentiation algorithm is based on the simple observation that for an even b, . This may not look very brilliant, but now consider the following recursive definition:

The case of odd b is trivial, as it’s obvious that . So now we can compute by doing only log(b) squarings and no more than log(b) multiplications, instead of b multiplications – and this is a vast improvement for a large b.

This algorithm can be coded in a straightforward way:

def expt_rec(a, b):
    if b == 0:
        return 1
    elif b % 2 == 1:
        return a * expt_rec(a, b - 1)
    else:
        p = expt_rec(a, b / 2)
        return p * p

Indeed, this algorithm is about 10 times faster than the naive one for exponents in the order of a few thousands. When the exponent is about 100K, it is more than 100 times faster, and the difference keeps growing for larger exponents.

def expt_bin_rl(a, b):
    r = 1
    while 1:
        if b % 2 == 1:
            r *= a
        b /= 2
        if b == 0:
            break
        a *= a

    return r

def expt_bin_lr(a, b):
    r = 1
    for bit in reversed(_bits_of_n(b)):
        r *= r
        if bit == 1:
            r *= a
    return r

def _bits_of_n(n):
    """ Return the list of the bits in the binary
        representation of n, from LSB to MSB
    """
    bits = []

    while n:
        bits.append(n % 2)
        n /= 2

    return bits

Performance

As I’ve mentioned, the squaring method of exponentiation is far more efficient than the naive method of repeated multiplication. In the tests I ran, the iterative left-to-right method is about the same speed as the recursive one, while the iterative right-to-left method is somewhat slower. In fact, both the recursive and the iterative left-to-right methods are so efficient they’re completely on par with Python’s built-in pow method [3].

This is surprising, as I’d actually expect the right-to-left method to be faster, because it skips the reversing of bits when computing the binary representation of the exponent. I’d also expect the built-in pow to be faster.

However, thinking harder for a moment, I think I can see why this happens. The RL (right-to-left) version has to multiply larger numbers at all stages, because LR sometimes multiplies by a itself, which is relatively small. Python’s bignum implementation can multiply by a small number faster, and this compensates for the need to reverse the bit list. I’ll come back to this issue when I’ll discuss modular exponentiation. But this is a topic for another article…

[1]

From the looks of it (featuring the binary representation) you’d think this is a modern algorithm. Not at all! According to Knuth, it was first mentioned by the Persian mathematician Jamshīd al-Kāshī in 1427. The left-to-right method presented later in the article is even more ancient – it appeared in a Hindu book in about 200 BC.

[2]	This holds for any base, by the way. You can similarly build a number from its decimal digits by multiplying by 10 for each digit you see and adding the digit, at each step.

[3]	pow is coded in C and uses the iterative left-to-right method I described with some optimizations and complicated tricks.

Related posts:

1. Efficient modular exponentiation algorithms Earlier this week I’ve discussed efficient algorithms for exponentiation. However,...
2. Horner’s rule: efficient evaluation of polynomials Here’s a general degree-n polynomial: To evaluate such a polynomial...
3. Rabin-Miller primality test implementation Here’s a fairly efficient Python (2.5) and well-documented implementation of...