2
$\begingroup$

For given function $g(W)$, where $W \in R^{M \times T}$. I have seen two different definition of proximal operator of it, but I don't know which one is correct ? One with $L_2$-norm, and second one with Frobenius norm

$$prox_g(W)={arg\,min}_u(g(u) + \frac{1}{2} |||u-W||_2^2 )$$

$$prox_g(W)={arg\,min}_u(g(u) + \frac{1}{2} |||u-W||_F^2 )$$

  • 1
    Make sure to read the accepted answer in the linked question. Interestingly enough, it is the same as the answer I already gave you. – 2017-01-05
  • 0
    The best reason of all to choose the squared Frobenius norm over all others is that it makes the derivative of that term trivial: $u-W$... – 2017-01-06

1 Answers 1

1

It depends on the Hilbert space structure you put on the domain of $g$.

Put differently: You may choose different scalar products for the space on which $g$ is defined and each gives rise to a valid definition of a proximal operator.

  • 0
    what do you mean exactly with this ? the domain of funtion is $R^{M \times T}$ – 2017-01-05
  • 0
    I suspected that. There are different ways to define scalar products on the space of matrices, one of them being $tr(B^T A) $. – 2017-01-05
  • 0
    What would be the scalar product that leads you to the spectral norm? I'm not seeing it. Regardless, proximal operators can be built using the square of any norm, or even a Bregman distance. – 2017-01-05
  • 0
    I do agree, however, that the natural choice is $\langle x, x \rangle^{1/2}$. – 2017-01-05
  • 0
    @MichaelGrant Can you recommend a reference that discusses proximal operators built using Bergman distance? – 2017-01-08
  • 1
    @littleO there was a great paper I read awhile back that did a great job of showing that the Bregman distances provided all of the necessary computational criteria to make an effective prox function. I can't recall it right now. I note that Googling "Bregman distance proximal" provides some good hits, but I'll see if I can find the one I'm thinking of. – 2017-01-08
  • 1
    From what I'm seeing, this may be the genesis of the concept: http://link.springer.com/article/10.1007/BF00940051 – 2017-01-08
  • 0
    Here's a good one: http://epubs.siam.org/doi/abs/10.1137/S0363012902407120 PDF here: http://www4.ncsu.edu/~pcombet/sicon2.pdf – 2017-01-08
  • 0
    @littleO One more for now: here are some lecture notes by Dimitri Bertsekas where he refers to the use of Bregman distance functions as a "useful extension of the proximal approach". It's just on the advanced topic slide, where he mentions "entropy-like regularization". Indeed, that's the one specific example I distinctly recall being useful. Perhaps using an entropy function when "projecting" onto the simplex? – 2017-01-08
  • 0
    Ah, Emmanuel's slides here: https://statweb.stanford.edu/~candes/math301/Lectures/Bregman.pdf – 2017-01-08
  • 0
    @MichaelGrant Thanks, this is great. – 2017-01-08