1
$\begingroup$

My problem setup is the following: states in MCTS are described with a sequence of features that take values from set {0, 1}. For every state I get a certain decimal value. I want to 'learn' a function that would give me an estimation for any given state.

I wanted to use Gradient Descent for that purpose but I came to realize that it is not right way to do it because I'm actually working with a discrete function. $$f(s) \in R, s \in \{0,1\}^n$$

Since the function is not continuous but discrete, I cannot get a derivative of function f.

So my question would be: am I right? If not, why? If yes, could you point me to some references that would back up what I stated here while those are not wiki links since I need to present this to my mentor.

Thanks you!

  • 0
    Although this is not an answer to my question, I overcame it. In the setup I described, clearly the conditions for calculating a derivative are not satisfied (especially 2nd derivative) BUT the idea of using derivatives in Gradient Descent approach is just to take partial ones. And that means that basically every weight that is assigned to a feature will be the value of partial derivative - the step in moving the function. – 2017-03-15

1 Answers 1

0

The general approach used in these circumstances is, rather than directly trying to estimate the response function, to estimate a continuous function that represents the probability of the response function taking the value 1. Obviously a good choice of function is one that is monotonically increasing with range between 0 and 1, and the most common choice is the logistic function, $f(x) = \frac{1}{1+e^{-x}}$.

To then get results for your response function, you take your output, and if it's above some threshold value you return 1 and below it you return 0 (the simplest threshold is 0.5, but you could also tune it based on your training data to get a suitable balance of Type I and Type II errors).

  • 0
    While I was searching for solution to my question, I ran into this equation you mentioned which is used in neural networks and I guess you wanted to direct me more in that direction. This is not quite what I was looking for. – 2017-03-15