3
$\begingroup$

How to prove the Energy function of a Hopfield net is monotonically decreasing?

$E = -1/2 \sum_{i,j} {w_{ij}}{s_i}{s_j} + \sum_{i}^{}s_{i} \theta_i$

I'll assume a proof involves the standard definition(s) of the activation function:

(1) $a_i \leftarrow \left\{ \begin{matrix} 1 & \mbox {if }\sum_{j}{w_{ij}s_j}>\theta_i, \\ -1 & \mbox {otherwise.}\end{matrix} \right\}$

(2) $a_i \leftarrow \left\{ \begin{matrix} 1 & \mbox {if }\sum_{j}{w_{ij}s_j}>\theta_i, \\ 0 & \mbox {otherwise.}\end{matrix} \right\}$

And the normal rules for weights:

$w_{ij}=0, \forall i=j$ (not reflexive)

$w_{ij} = w_{ji}, \forall i,j$ (symmetric)

If someone could set me on the right path, it would be appreciated.

  • 0
    I think this question could use some context. If the following is correct, I suggest you edit it into the question: The Hopfield net undergoes state changes in which the new value of an arbitrary node $i$ is determined as the current value of $a_i$. Definition (1) of $a_i$ pertains to a net with possible values $s_i=\pm1$, and definition (2) of $a_i$ pertains to a net with possible values $s_i=0,1$. The question is how to prove that the given energy function monotonically decreases under these state changes.2011-04-07

1 Answers 1

2

This answer assumes that the question is intended as clarified in my comment.

The energy is linear in each $s_i$, that is, it's a term independent of $s_i$ plus a term proportional to $s_i$. The coefficient of the proportional term is $\theta_i-\sum_{j}{w_{ij}s_j}$. (No factor of $1/2$ since each term appears twice in the sum, with the same weight because the weights are symmetric.) So $a_i$ is defined such that it chooses the higher value if the coefficient of the linear term is negative and the lower value if it's positive, and thus it always chooses the lower of the two possible values of the energy in each step.