A problem I keep encountering and its solution.

Vectors are lists of numbers. Matrices are squares of numbers. These two show up in many applications, and many computational libraries exist for solving them.

But what tensors: cubes and hypercubes of numbers? It seems as though I am continually re-encountering these. Multi-dimensional polynomials, accelerations in multidimensional time, data collected from planar laser-induced florescence—it’s not like these are every-day kinds of tasks; they just are tasks that my research runs into. And since they are not common, there aren’t many libraries for them out there.

Multi-Index Operations

Suppose you have a m-dimensional tensor. You can express it’s size with a list of m sizes: N = [n₁, n₂, …, n_m]. A multi-index is a list of indices, one for every dimension of a tensor: I = [i₁, i₂, …, i_m]. Per computing tradition (and because it makes moduli more convenient) I assume that 0 ≤ i_j < n_j. I could have done 1 ≤ i_j ≤ n_j instead but that makes the math more complicated. Each multi-index names one cell in the tensor, and each cell in the tensor has exactly one multi-index referring to it.

The most basic operation of a multi-index in a computer is to treat a big vector or array as if it were a tensor. For this, as well as many operations that follow, it is useful to define the product of earlier sizes, p_j ≜ ∏^j_k=1 n_k. With this, converting a multi-index into a flat index is as simple as ∑^m_k=1 i_k p_k–1.

Two more operations make multi-indices useful tools in practice: partial enumeration and subindex replacement. I’ve never seen these discussed elsewhere, so the name, notation, etc., are original; if someone knows other names for them I welcome feedback in the comments.

Subindex replacement is very simple. Given a multi-index I = [i₁, i₂, … i_m], the subindex replacement I_k=x = [i₁, …, i_k–1, x, i_k+1, … i_m].

Partial enumeration lists all valid multi-indices with a particular subset of the sub-indices fixed at 0. The simplest implementation I know is via partial increments:

Partial enumeration

for each fixed index j { temporarily set n_j to 1 }; let I = [0, 0, …, 0]; repeatedly { for each j from 1 to m { let x = i_j + 1; if x ≥ n_j { update I to be I_j=0 } otherwise { update I = I_j=x; emit I; exit this “‍for each‍” loop } }; if I = [0, 0, …, 0], stop enumerating }

It is possible to efficiently implement multi-indices using only a single integer instead of a list of integers, but that is a subject too technical for this post.

Why this post?

Usually I make posts because I think some one of my usual readers will be interested. This post is unusual in that I don’t expect any of the readership I know about to find anything within it of value. As I said, tensors aren’t really all that common. But I didn’t find this information when I searched for it and thought it might be nice to share it in case someone is searching for it.