4
$\begingroup$

Let $\| \cdot \|$ be a Euclidean distance.
How to optimize \begin{align} \max_{ \|a \| \le r } E[ \|a+Z\|^k], \end{align} where $Z \in \mathbb{R}^n$ is i.i.d. Gaussian standard normal and $k>0$?

Solution for the case of $n=1$.

For $n=1$ I found that \begin{align} E[|a+Z|^k]= \frac{2^{k/2} \Gamma \left(\frac{k+1}{2} \right)}{\sqrt{\pi}}F_{1,1} \left(-\frac{k}{2},\frac{1}{2}; -\frac{a^2}{2} \right), \end{align} where $F_{1,1}$ is the Kummer’s confluent hypergeometric function. Since, $f(x)=F_{1,1} \left(-\frac{k}{2},\frac{1}{2}; -\frac{x^2}{2} \right)$ is increasing $x$ it follows that

\begin{align} \max_{|a| \le r }E[|a+Z|^k]= E[|r+Z|^k]. \end{align}

My Question: How can we find the maximum for any dimension $n\ge 1$?

My conjecture is that the optimal value of $a$ is such that $\|a \|=r$. That is $a$ is a point on a ball of radius $r$.

Thanks.

  • 0
    Clearly the $a$ in your conjecture is a KKT point. Do you see this? – 2017-02-18
  • 0
    @RobertHannah89 No. I would really like some explanation. – 2017-02-18
  • 0
    Convex functions are maximized at extreme points. Can you show $f(a) = E[||a+Z||^k]$ is convex in $a$? – 2017-02-20
  • 0
    Alternatively, you can reduce to the 1-d case by arguing that $f(a)$ depends only on the magnitude of $a$, not its orientation. – 2017-02-20
  • 0
    @Michael I can see how $f(a)$ only depends on the magnitude of $a$ but not sure how to proceed next. Also not very sure how to show that $f(a)$ is convex due to the norm. Be great if you can put in some details. – 2017-02-20
  • 0
    It would be convex for $k\geq 1$, you can think of it as a composition of the convex norm function and the convex $x^k$ function defined for $x\geq 0$. For $k<1$ it may not be convex. – 2017-02-20
  • 0
    What is the context of this question? Is it a homework problem? – 2017-02-20
  • 0
    I was working on the $n=1$ case and had a question of how this can be extended to higher dimensions. I know understand the case of $k \ge 1$. Do you have any idea of how to show it for $k<1$? – 2017-02-20
  • 1
    See my edits below. – 2017-02-22
  • 0
    @RobertHannah89 Thanks. – 2017-02-22

1 Answers 1

1

This is a simple Lagrange multiplier problem. Let's look at the gradient (where it is defined: i.e. not at the origin)

\begin{align} \nabla_a f(a) = \mathbb{E}[k\|a+X\|^{k-2}a] \end{align}

This (sub)gradient is never $0$ except at the origin. The solution must either be at the origin (which you can obviously eliminate) or on the boundary at a KKT point. The constraint can we phrased as $g(a) = \|a\|^2-r^2 \leq 0$. KKT conditions:

\begin{align} \nabla_a f(a) &= \lambda \nabla g(a)\\ \mathbb{E}[k\|a+X\|^{k-2}]a &= 2\lambda a \end{align}

Clearly this is satisfied for any $\|a\|=r$, using

\begin{align} \lambda = \frac{1}{2}\mathbb{E}[k\|a+X\|^{k-2}] \end{align}

Hence any point on the boundary is a solution to the maximization problem by radial symmetry, and the fact that the origin is not a solution.

Edit: In response to comments. You don't really need to prove convexity to do this problem. $f$ is not differentiable, but we can use a version of KKT for nondifferentiable functions.

It can be shown that for $k\geq1$ that this function is convex. From Proposition 8.18 of Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Bauschke and Combettes, it can be shown that so long as the 1D version of a radially symmetric function (i.e. only depends on the norm) is convex, the ND version of this function is also convex. The 1D version of $f$ is convex for $k\geq 1$, hence $f$ is convex for $k\geq 1$.

For $0< k < 1$, the function is what's called quasiconvex. See Bauschke and Combettes, Example 10.28 for this result. This is similar, in that it will be maximized at extreme points (points on the boundary). See Maximizing and minimizing quasiconvex functions: related properties, existence and optimality conditions via radial epiderivatives, Fabian Flores-Bazan Fernando Flores-Bazan Cristian Vera.

  • 0
    Thank you. Can you tell me how you got the the gradient? Why is it $k-2$ not $k-1$? – 2017-02-20
  • 0
    Just simple partial integration on $a$ with the chain rule. It's $k-2$ and $1$: Notice there is an $a$ on the end of the expression. – 2017-02-20
  • 0
    Does the result change if we introduce an extra matrix $K$ and try to optimize $\max_{ \|a\| \le r} E{ \| Ka+Z\|}$?? Is the maximum still the same? – 2017-02-25
  • 0
    Well yeah. So your original $f$ is radial, and is strictly increasing radially. Therefore the maximum will occur at points of largest norm. So finding the maximizers is equivalent to finding $\arg\max_{\|a\|\leq r} \|Ka\|$ (i.e. just totally ignore the Gaussian and the value of $k>0$). The solution set can also be written as $a$ such that $\|a\|=r$ and $\|Ka\|=\|K\|\times\|a\|$. – 2017-02-26
  • 0
    How did you get $\| Ka \|=\| K\| \|a\|$, since $K$ is matrix, can you do it? – 2017-02-26
  • 0
    $\|K\|$ refers to the [matrix norm](https://en.wikipedia.org/wiki/Matrix_norm). – 2017-02-27
  • 0
    But $a$ is a vector. Which matrix norm satisfies this property? – 2017-02-27
  • 0
    Just ignore the last sentence, you can understand without it. This is just the $L^2$ matrix norm which is defined as: $\|K\|=\sup_{\|v\|=1}\|Kv\|$. – 2017-02-27