3
$\begingroup$

How to prove the Energy function of a Hopfield net is monotonically decreasing?

$E = -1/2 \sum_{i,j} {w_{ij}}{s_i}{s_j} + \sum_{i}^{}s_{i} \theta_i$

I'll assume a proof involves the standard definition(s) of the activation function:

(1) $a_i \leftarrow \left\{ \begin{matrix} 1 & \mbox {if }\sum_{j}{w_{ij}s_j}>\theta_i, \\ -1 & \mbox {otherwise.}\end{matrix} \right\}$

(2) $a_i \leftarrow \left\{ \begin{matrix} 1 & \mbox {if }\sum_{j}{w_{ij}s_j}>\theta_i, \\ 0 & \mbox {otherwise.}\end{matrix} \right\}$

And the normal rules for weights:

$w_{ij}=0, \forall i=j$ (not reflexive)

$w_{ij} = w_{ji}, \forall i,j$ (symmetric)

If someone could set me on the right path, it would be appreciated.

  • 0
    Check the definition of “monotonically decreasing.” I am afraid that either you have some understanding about the definition or your function is not the the one you are talking. If the stated function is correct and each variable is restricted to nonnegative reals, then this is not a research-level question. Voted to close as off topic.2011-02-24
  • 0
    You had posted this quesiton on cstheory.stackexchage.com, which is a website for research-level questions in theoretical computer science.2011-02-24
  • 2
    When you post something on a website you are not familiar with, try to check the scope of the website first. http://cstheory.stackexchange.com/faq2011-02-25
  • 0
    I think this question could use some context. If the following is correct, I suggest you edit it into the question: The Hopfield net undergoes state changes in which the new value of an arbitrary node $i$ is determined as the current value of $a_i$. Definition (1) of $a_i$ pertains to a net with possible values $s_i=\pm1$, and definition (2) of $a_i$ pertains to a net with possible values $s_i=0,1$. The question is how to prove that the given energy function monotonically decreases under these state changes.2011-04-07

1 Answers 1

2

This answer assumes that the question is intended as clarified in my comment.

The energy is linear in each $s_i$, that is, it's a term independent of $s_i$ plus a term proportional to $s_i$. The coefficient of the proportional term is $\theta_i-\sum_{j}{w_{ij}s_j}$. (No factor of $1/2$ since each term appears twice in the sum, with the same weight because the weights are symmetric.) So $a_i$ is defined such that it chooses the higher value if the coefficient of the linear term is negative and the lower value if it's positive, and thus it always chooses the lower of the two possible values of the energy in each step.