0
$\begingroup$

I'm playing Overwatch in ranked, and want to know whether my matchups are "fair". That is I'm not being placed with significantly higher or lower skilled players.

So I can play lots of games (say 30), and record "win" or "lose" for each of them. At the end, I want to know whether or not my wins and losses were random based on a certain level of confidence.

If I'm being matched with players of similar skill, then the outcome of my games should be truly random.

Do you have any suggestions for how I would go about this?

  • 0
    It is nearly impossible to tell if a set of data was generated by a good random generator. There are some tests of a bad generator. However, I wouldn't expect they data to appear to be it to be quite random for a few reasons: Most scenarios are biased to favor the defenders, and some are more skewed than others. This will bias the output. Next, streaks do happen. They should happen. But if the software is adapting your skill score on your previous results, you will ultimately see a less streaky pattern then would be expected. – 2017-01-27
  • 0
    @DougM A typical ranked match switches sides so the defenders become the attackers and vice versa. That takes care of the bias. Also, streaks occur even in randomly generated data don't they? It's not as if a random coin toss will be 11001001. It could well be 1000011001111. – 2017-01-27
  • 0
    It is going to result in shorter streaks of consecutive wins and consecutive losses then you would get from say a flipping coin. – 2017-01-27

1 Answers 1

0

I assume you don't really care about random so much as fair - that is, you want to know whether you're winning roughly as often as you're losing, not whether the sequence of wins and losses is predictable.

$30$ games isn't really enough for a good picture, but we can work with it anyway. Score each game as a $0$ for a loss and a $1$ for a win. Take the average of these scores - presumably, about $0.5$. Now, take the standard deviation: $\sigma = \frac{\sqrt{\sum (x_i - x)^2}}{n - 1}$. That is, the sum of the squares of the differences from the mean, divided by one less than the number of samples. For example, say we have four samples, three wins and a loss. The mean is $(1 + 1 + 1 + 0) / 4 = 3/4$. The standard deviation is $\sqrt{0.25^2 + 0.25^2 + 0.25^2 + 0.75^2}/3$, roughly $0.289$. Finally, take $1.96$ standard deviations on either side of the mean - in this case, that gives an interval $[0.461, 1.039]$. That's pretty close to a $95\%$ confidence interval - the mean is probably between $0.461$ and $1.039$. Since $0.5$ is in that interval, it's reasonable to suppose that the games in this example were fair.

Now, this isn't quite right. There's lots of complicated stuff that goes into confidence intervals; but this approach will work for a first approximation if you're unfamiliar with statistics.

  • 0
    Thanks! I'll use this and see what comes up :) – 2017-01-27
  • 0
    I just thought of something. This method only relies on the average. So for example, if I have a win/lose streak like this: 11111111110000000000, the average will come to 0.5 and well within the standard deviation. However, this sequence of events is highly unlikely to be random. Of course it's POSSIBLE. But unlikely. Basically what I'm saying is that the method you're suggesting says nothing about the order of the wins/loses. Only the aggregate. And that's problematic. What I'm looking for is an algorithm that will take the above sequence and say "This is only 2% chance of randomness". – 2017-01-28
  • 0
    @BhagwadJalPark Yes, as I mentioned, my answer assumes that you just want to know whether wins and losses occur in equal proportion. This method will not detect lack of randomness in ordering. It is *considerably* more difficult to detect randomness in a sequence. – 2017-01-28