Can a symmetric rank-1 symmetric tensor be written as a linear combination of other symmetric rank-1 symmetric tensors?

Question

I know that a symmetric tensor of symmetric rank $1$ can be viewed as a point on the so-called Veronese variety. A symmetric tensor of rank $r$ is then in the linear space spanned by $r$ points of the Veronese variety. My question is the following: can any given symmetric tensor of rank $1$ ever reside in the linear space spanned by $r$ other points of the Veronese variety, i.e. be written as a linear combination of $r$ other symmetric rank-$1$ symmetric tensors?

I am an engineer, currently working on tensor decompositions for signal processing applications. I'm not very familiar with algebraic geometry, but it seems that I need the answer to the question above, to ensure uniqueness of one such decomposition. I looked for (a hint toward) an answer in the literature on Veronese varieties, but it is rather hard to dig into.

Answer 1

First let me make a point about the terminology. I believe what you mean by "rank-$1$ tensor" is what most people call a "pure" tensor: it is a tensor $t$ that can be written $v_1\otimes\cdots\otimes v_n$ for some possibly different vectors $v_i$. I believe what you mean by "pure symmetric" is that all the $v_i$ are the same; you can prove that this is equivalent to being pure and symmetric separately, so the language "symmetric pure symmetric" is redundant.

Let $t=v\otimes\cdots\otimes v$ be a pure symmetric tensor. Choose a basis $E_i$ in which $v=E_1$, and suppose $v$ can be written as a linear combination of pure tensors. Then

$$E_1\otimes\dots\otimes E_1 = v\otimes\dots\otimes v = \sum_k w^k_1\otimes...\otimes w_1^k$$

for some vectors $w_k=\sum_i c_k^i E_i$. Thus

$$E_1\otimes\dots\otimes E_1=\sum_{k,J}c_k^{i_1}\cdots c_k^{i_j}E_{i_1}\otimes\cdots\otimes E_{i_j}$$

A basis representation of a vector is unique, so equating the coefficients on both sides, we see that the coefficient of each $E_{i_1}\otimes\cdots\otimes E_{i_k}$ on the right is a polynomial in the $c_i^k$ that must be equal to $0$ or, in the case $i_1=\cdots=i_j=1$, equal to $1$; the solution is obtained by checking the common solutions of these equations. In particular for $n\ne 1$,

$$\sum_k c_k^n\cdots c_k^n = 0$$

If your vector space is even dimensional and real, then each term in this sum is an even power of a real number, thus positive, and thus each term must be zero. In other words, $c_k^n=0$ for $n\ne 1$. This means each $w_i$ is a scalar multiple of $E_1$, so the answer to your question is no.

For odd dimensional real vector spaces and complex vector spaces it seems like the solution would be more complicated.

Answer 2

Your question is: Can a given symmetric tensor of rank $1$ be written as a linear combination of other symmetric tensors of rank $1$? The answer is yes. Yes, a symmetric tensor of rank $1$ can be written as a linear combination of other symmetric tensors of rank $1$. This is true over the complex numbers, the real numbers, and other fields. For simplicity I'll write this answer over a field of characteristic zero; everything I say will be valid over real numbers and over complex numbers. (It can also be valid over other fields with mild conditions but I will ignore that for now.)

Before proceeding with generalities, perhaps a concrete example might be helpful? Let $V$ be a two-dimensional vector space with basis $x,y$. (You are welcome to think of this as $V = \mathbb{R}^2$ and $x = (1,0)$, $y=(0,1)$.) One of the symmetric tensors of rank $1$, and order $3$, in $V \otimes V \otimes V$, is $x \otimes x \otimes x$, which for short we might denote $x^3$. Can we write this $x^3$ as a linear combination of other rank $1$ symmetric tensors? Yes, in various ways; here is one. $$ \begin{split} (x+y) \otimes (x+y) \otimes (x+y) &= x \otimes x \otimes x + x \otimes x \otimes y \\ &\quad + x \otimes y \otimes x + y \otimes x \otimes x \\ &\quad + x \otimes y \otimes y + y \otimes x \otimes y \\ &\quad + y \otimes y \otimes x + y \otimes y \otimes y \\ &= x^3 + 3x^2y + 3xy^2 + y^3, \end{split} $$ where $x^3 = x \otimes x \otimes x$, $x^2y = \frac{1}{3}(x \otimes x \otimes y + x \otimes y \otimes x + y \otimes x \otimes x)$, and in general $xyz = \frac{1}{6}(x \otimes y \otimes z + \text{all permutations})$.

Well, $$ (x+y)^3 + (x-y)^3 = x^3 + 6xy^2, $$ and $$ (x+2y)^3 + (x-2y)^3 = x^3 + 24xy^2 . $$ So therefore $$ 4(x+y)^3 + 4(x-y)^3 - (x+2y)^3 - (x-2y)^3 = 3x^3. $$ So the rank $1$ symmetric tensor $x^3$ is in the span of the $4$ rank $1$ symmetric tensors $(x \pm y)^3$ and $(x \pm 2y)^3$.

Now, this example begins to point the way to a more general answer. Because when we consider the space of symmetric tensors in $V \otimes V \otimes V$—it is a subspace that can be denoted $S^3 V$ ($S$ for Symmetric), or $S_3 V$, depending on the author—we see that it has dimension $4$. A basis is given by $x^3, x^2y, xy^2, y^3$. And another different basis is given by $(x\pm y)^3$, $(x \pm 2y)^3$. At that point it becomes clear that every symmetric tensor is a linear combination of those $4$ symmetric rank $1$ tensors, including all the symmetric rank $1$ tensors such as $x^3$.

So we have the first answer to your question: Let $V$ be any finite dimensional vector space. Let $d \geq 1$. Let $N$ be the dimension of $S^d V$. Now $S^d V$ is spanned by the symmetric rank $1$ tensors, so we can take some $N$ of them to form a basis. And now all the symmetric rank $1$ tensors can be written as linear combinations of those.

Now there is a reasonable follow-up question, which might perhaps be a natural question from a standpoint of an engineer who is interested in identifiability issues: Okay, if some symmetric rank $1$ tensor is equal to a linear combination of some $r$ other symmetric rank $1$ tensors, then what is $r$?

We can certainly have such linear combinations when $r=N$ (the dimension of $S^d V$) but unfortunately it can happen sooner than that. It happens as soon as $r=d+1$. Because if $V$ has a basis $x_1,\dotsc,x_n$, then $x_1^d$ can be written as a linear combination of $(x_1 + t_i x_2)^d$ for any (!) $d+1$ nonzero values of $t_i$. (Nutshell: with Vandermonde matrices you can show these are linearly independent, and there are enough of them to span all the degree $d$ polynomials in $x_1,x_2$, including $x_1^d$.)

I believe the converse may be true: if $L^d$ is a linear combination of some $r$ other symmetric rank $1$ tensors, then $r \geq d+1$. But at the moment I'm not sure where to find a result along those lines. (I looked in some papers of Reznick and the book of Iarrobino-Kanev; it might be in there somewhere but so far I have not been able to find it.) If you are interested, please let me know and I will be happy to resume the search.