27
$\begingroup$

Let $V$ and $W$ be two algebraic structures, $v\in V$, $w\in W$ be two arbitrary elements.

Then, what is the geometric intuition of $v\otimes w$, and more complex $V\otimes W$ ? Please explain for me in the most concrete way (for example, $v, w$ are two vectors in 2 dimensional vector spaces $V, W$)

Thanks

  • 0
    @eduardo: Related: http://math.stackexchange.com/questions/309838/the-physical-meaning-of-the-tensor-product#comment674782_3098382013-02-23

4 Answers 4

5

You want to stay concrete, so let's let $V$ be a two-dimensional real vector space and $W = \operatorname{Hom}(V,\mathbb{R})$. Then $V = T^1_0(V)$ and $W = T^0_1(V)$, so for any $v\in V$ and $w\in W$, $v\otimes w\in T^1_1(V)$.

Each of $v,w$ has two components; $v = v^1e_1 + v^2e_2$ and $w = w_1e^1 + w_2e^2$. Here $e_1,e_2$ is a basis for $V$ and $e^1,e^2$ is the dual basis in $V^*$.

The components of $v\otimes w$ are all the components of $v$ times all the components of $w$: $(v\otimes w)^i_j = v^iw_j.$

To see this, observe that $(v\otimes w)(\theta, x)=v(\theta)w(x),$ so that $(v\otimes w)(e^i, e_j) = v(e^i)w(e_j).$

More generally, if $A = A^{i_1\cdots i_p}_{j_1\cdots j_q}\in T^p_q$ and $B = B^{k_1\cdots k_r}_{l_1\cdots l_s}\in T^r_s$, then

$(A\otimes B)^{i_1\cdots i_pk_1\cdots k_r}_{j_1\cdots j_q l_1\cdots l_s} = A^{i_1\cdots i_p}_{j_1\cdots j_q}B^{k_1\cdots k_r}_{l_1\cdots l_s}.$

Note that our example is from the derived tensor algebra over a two-dimensional vector space, $T^p_q(V) = (V^*)^{\otimes p}V^{\otimes q}.$ Hopefully this helps build your intuition about the case where $V$ and $W$ are two vector spaces of potentially different dimension.

  • 0
    $T^p_q(V) = (V^*)^{\otimes p}V^{\otimes q}$” – shouldn’t that be $T^p_q(V) = V^{\otimes p} \otimes (V^*)^{\otimes q}$?2018-05-03
4

The difference between the ordered pair $(v,w)$ of vectors and the tensor product $v\otimes w$ of vectors is that for a scalar $c\not\in\{0,1\}$, the pair $(cv,\;w/c)$ is different from the pair $(v,w)$, but the tensor product $(cv)\otimes(w/c)$ is the same as the tensor product $v\otimes w$.

4

The way I like to think about this is that if we have two vector spaces $V$ and $W$, their tensor product $V \otimes W$ intuitively is the idea "for each vector $v\in V$, attach to it the entire vector space $W$." Notice however that since we have $V\otimes W \cong W \otimes V$ we can rephrase this as the idea "for every $w \in W$ attach to it the entire vector space $V$." What distinguishes $V\times W$ from $V\otimes W$ is that the tensor space is essentially a linearized version of $V\times W$. To obtain $V\otimes W$ we start with the product space $V\times W$ and consider the free abelian group over it, denoted by $\mathcal{F}(V\times W)$. Given this larger space, that admits scalar products of ordered pairs, we set up equivalence relations of the form $(av,w) \sim a(v,w) \sim (v, aw)$ with $v\in V, w \in W$ and $a \in \mathbb{F}$ (a field), so that we identify points in the product space that yield multilinear relationships in the quotient space generated by these relationships. If $\mathcal{S}$ is the subspace that is spanned by these equivalence relations, we define $V\otimes W = \mathcal{F}(V\times W)/\mathcal{S}$.

Personally, I like to understand the tensor product in terms of multilinear maps and differential forms since this further makes the notion of tensor product more intuitive for me (and this is typically why tensor products are used in physics/applied math). For instance if we take an $n$-dimensional real vector space $V$, we can consider the collection of multilinear maps of the form

$ F: \underbrace{V \otimes \ldots \otimes V}_{k \; times} \to \mathbb{R} $

where $F \in V^* \otimes \ldots \otimes V^*$, and this tensor space has basis vectors $dx^{i_1} \otimes \ldots \otimes dx^{i_k}$ and $\{i_1, \ldots, i_k\} \subseteq \{1, \ldots, n\}$. Notice now that given $k$ vectors $v_1, \ldots, v_k \in V$ we have that multilinear tensor functional $dx^{i_1} \otimes \ldots \otimes dx^{i_k}$ acts on the ordered pair $(v_1, \ldots, v_k)$ as

$ dx^{i_1} \otimes \ldots \otimes dx^{i_k}(v_1, \ldots, v_k) \;\; =\;\; dx^{i_1}(v_1) \ldots dx^{i_k}(v_k). $

Where we have that $dx^{i_j}(v_j)$ takes the vector $v_j$ and picks out its $i_j$-th component. Extending this notion to differential forms where instead we would have basis elements $dx^{i_1} \wedge \ldots \wedge dx^{i_k}$ (where antisymmetrization is taken into account), we would have that this basis form would take a set of $k$ vectors and in some sense "measure" how much these vectors overlap with the subspace generated by the basis $\{x_{i_1}, \ldots, x_{i_k}\}$.

  • 1
    I see. I did not think you had shortened the 'definition' of tensor product deliberately but assumed you had forgotten that this part existed, so I wanted to point it out to you to improve the quality of the answer. I understand I was wrong with my assumption.2016-12-15
1

geometric intuition of $v\otimes w$ [...] in the most concrete way

Let me try to throw in an answer on freshmen level. I deliberately try to leave out any information that is not essential to the problem, e.g. don't bother about the specific dimensions of the quantities.

Say we have a matrix $V$ and a vector $w$ resulting in $Vw=v$ (implying consistent dimensions of the matrix $V$ and of the column vectors $v$ and $w$ over, e.g., $\mathbb{R}$). Now, let's forget about $V$ and try to reconstruct it given only $v$ and $w$.

It is generally not possible to fully reconstruct it from this little information, but we can do our best by trying $\tilde{V}:=v\otimes w \frac{1}{\lVert w\rVert^2}$ In this case the tensor product can be rewritten to $v\otimes w := v\,w^{\sf T}$ Then, checking the ability of $\tilde{V}$ to approximate the action of $V$ we see $\tilde{V}w=v\,w^{\sf T}w\frac{1}{\lVert w\rVert^2}=v$ which is the best we can do.

To make it more concrete, let's throw in some numbers in the easiest non-trivial case: $Vw=\left(\begin{matrix}1&0\\0&2\end{matrix}\right)\left(\begin{matrix}1\\0\end{matrix}\right)=\left(\begin{matrix}1\\0\end{matrix}\right)=v$ Then $\tilde{V}=\left(\begin{matrix}1\\0\end{matrix}\right)\left(\begin{matrix}1&0\end{matrix}\right)=\left(\begin{matrix}1&0\\0&0\end{matrix}\right)\approx V=\left(\begin{matrix}1&0\\0&2\end{matrix}\right)$ The subspace spanned by $w=\left(\begin{matrix}1\\0\end{matrix}\right)$ is accurately mapped by $\tilde{V}$.

To give more (very elementary) geometric insight, let's make the matrix slightly more interesting: $V=\left(\begin{matrix}1&1\\0&2\end{matrix}\right)$ This leaves the $\left(\begin{matrix}1\\0\end{matrix}\right)$-space invariant and scales and shears the $\left(\begin{matrix}0\\1\end{matrix}\right)$-space. We now "test" the matrix by the "arbitrary" vector $w=\left(\begin{matrix}1\\1\end{matrix}\right)$ and get $Vw=\left(\begin{matrix}1&1\\0&2\end{matrix}\right)\left(\begin{matrix}1\\1\end{matrix}\right)=\left(\begin{matrix}2\\2\end{matrix}\right)=v$ and the approximate reconstruction $\tilde{V}=\left(\begin{matrix}2\\2\end{matrix}\right)\left(\begin{matrix}1&1\end{matrix}\right)\frac{1}{2}=\left(\begin{matrix}1&1\\1&1\end{matrix}\right)\approx V=\left(\begin{matrix}1&1\\0&2\end{matrix}\right)$ We have lost the information about the invariance of the $\left(\begin{matrix}1\\0\end{matrix}\right)$-axis and the scaling & shearing behavior of the $\left(\begin{matrix}0\\1\end{matrix}\right)$-axis, but the diagonal $\left(\begin{matrix}1\\1\end{matrix}\right)$-space's behavior is captured by $\tilde{V}$.

To increase the level of this answer by an $\varepsilon>0$, the spectral decomposition of a diagonalizable matrix $V$ should be mentioned. If $v_1,\dots,v_n$ are normalized eigenvectors with corresponding eigenvalues $\lambda_1,\dots,\lambda_n$, then $V=\sum_{i=1}^n v_i\otimes v_i \lambda_i$

More intuitively, we can say - very roughly speaking - that the tensor product $v\otimes w$ is the pseudo-inverse operation of the matrix vector multiplication $Vw=v$.


To finish this, I'd like to give one concrete (and special) application of the tensor product in partial differential equations. It links this question to the divergence theorem, which is itself very intuitive. To keep it descriptive, let's stick to the case of solid mechanics (because everyone has inherent understanding of bodies and forces), although this holds for all elliptic PDEs in general.

Say we have an elastic body (e.g. a rubber band) occupying a domain $\Omega\subset\mathbb{R}^3$. Some part of it is subject to Dirichlet boundary conditions (e.g. you're statically stretching it). This results internal stresses, which are typically represented by matrices $\sigma\in\mathbb{R}^{3\times3}$. The system of PDEs describing this state is ${\rm div}(\sigma)=0$ (considering the boundary conditions and the governing material law which closes these equations).

In some applications you are interested in the average stress throughout the body $\Omega$ $\bar{\sigma}=\frac{1}{\mu(\Omega)}\int_\Omega \sigma \, {\rm d}\mu$ The divergence theorem implies that the average stress can be represented by the surface integral $\bar{\sigma}=\frac{1}{\mu(\Omega)}\int_{\partial\Omega} \left(\sigma n\right) \otimes x \, {\rm d}A$ where $\partial\Omega$ is the boundary of $\Omega$, $n$ is the surface normal, and $x$ the coordinate vector.

This means that the volume average of a matrix can be represented integrating the pseudo-inverse of its application over the surface.