4
$\begingroup$

I would like to code this neural net activation function, using the C language:

$f(x) = 1.7159 \tanh( \frac{2}{3} x )$

and I will also need to code its derivative.

I've read that the derivative of $\tanh(x)$ is $\operatorname{sech}^2(x)$, but since C doesn't have a hyperbolic secant function I will need to use $\cosh$, i.e. the derivative of $\tanh(x)$ is $1\over \cosh^2(x)$, I think.

Since my knowledge of calculus is very rusty, my best attempt for the derivative of the above function is:

$1 \over \cosh^2(\frac{2}{3}x)$

Is this correct?

1 Answers 1

4

The derivative is $\frac{2*1.7159}{3} / \cosh^2 \frac{2x}{3}$ by chain rule. Note that it can be also written as $\frac{2*1.7159}{3} (1- \tanh^2 \frac{2x}{3})$ which might make slightly more sense as you only need to find $\tanh$ then (if you want to find $f(x)$ and $f'(x)$ at the same time). All the useful relations can be found in wikipedia.