1
$\begingroup$

The standard logistic function is [1]:

$$ f(x) = \frac{1}{1+e^{-x}} $$

But the logistic loss function is typically defined as [2]:

$$ l(w^{\top} \cdot x) = \ln(1 + e^{-y(w^{\top} \cdot x)}) $$

I don't see a clear relationship, but when I tried to take the derivative of the logistic loss function, I got:

$$ \frac{\partial l}{\partial z} \frac{\partial g}{\partial w} = (-\frac{e^{-yz}}{1+e^{-yz}})(x) = -\frac{xe^{-y(w^{\top} \cdot x)}}{1+e^{-y(w^{\top} \cdot x)}} $$

Which is really close to the derivative of the logistic function:

$$ f^{\prime}(x) = \frac{-e^{-x}}{(1+e^{-x})^2} $$

What's the relationship?

1 Answers 1

2

\begin{align}l(w^Tx)&=\ln(1+\exp(-y(w^T.x))\\&=-\ln\left( \frac{1}{1+\exp(-y(w^T.x)} \right)\\&=-\ln(f(-y(w^T.x))\end{align}

  • 0
    Thanks. Just to confirm, is my derivative of the logistic loss function correct?2017-01-18
  • 0
    I think you miss a $y$ in your numerator.2017-01-18