1
$\begingroup$

Given a linear objective function $f(\vec{x})=\sum_ia_ix_i$, the direction in which $f$ varies the greatest is known to be $\vec{\nabla}{f}$.

Now given a non-zero vector $\vec{v}$, I am interested in finding the direction $\vec{d}$ in which $f$ varies the greatest, subject to the constraint that $\vec{v}^{T}\vec{d} = 0$. How do I go about finding $\vec{d}$ subject to such an orthogonal constraint?

2 Answers 2

1

Project $\nabla \mathbf{f}$ onto the orthogonal space of $\mathbf{v}$, that's the direction $\mathbf{d}$, which can be computed by $\mathbf{d}=\nabla\mathbf{f}-\mathbf{v}\mathbf{v}^T\nabla\mathbf{f}$ ($\mathbf{v}$ should be normalized first).

0

Suppose we are working in $\mathbb R^n$ and $A$ is a full-rank $m$-by-$n$ matrix whose columns span a subspace $U \subseteq \mathbb R^n$ of dimension $m$. Each vector in $U$ can be expressed as $\vec x = A\vec y$ where $\vec y \in \mathbb R^m$. The change of $f$ with respect to the direction indicated by $\vec y \in \mathbb R^m$ is $\nabla f^T A\vec y$. Therefore, $\nabla f^T A$ is the direction of greatest change with respect to the coordinate system $A$.

In your case, $U = \{\vec v\}^\perp$. The matrix $A$ can be found by a Gram-Schmidt process.