3
$\begingroup$

The decoding error occurs where there are more than 1 bits that are flipped. Hence, if the probability that a bit is flip is $10\%$, then shouldn't the decoding error percentage be $1-0.1\cdot0.9^6\cdot7-0.9^7 = 0.149$? Why my textbook (by MacKay) claims that it is $7\%$?

  • 0
    According to Wikipedia, an $(m,n)$ Hamming code means $m$ bits total of which $n$ are data (the others being used for error correction). Do you mean a $(7,4)$ Hamming code? – 2017-01-14
  • 1
    What you have calculated is the _word_ error rate: $14.9\%$ of the time, the decoder output will be the wrong codeword. But what is of interest is the _bit_ error rate of the decoded data bits. Even when the output of the decoder is the wrong codeword, _some_ of 4 data bits might still be correct. – 2017-01-14
  • 0
    @DilipSarwate how is the bit error rate 7% then? i can't get it. – 2017-01-14

1 Answers 1

2

Let $E$ denote the event that the output of the decoder is a codeword that is not the same as the transmitted codeword, and let $E_i$ denote the event that the $i$-th data bit in the output codeword differs from the $i$-th data bit of the transmitted codeword. Clearly we have that $$E = E_1 \cup E_2 \cup \cdots \cup E_k.\tag{1}$$ since if none of the $E_i$ occur, then the data bits in the output codeword are all correct and so the output codeword must be the same as the transmitted codeword.

Now, in general, the events $E_i$ are neither independent events nor equiprobable events. The Bit Error Rate (BER) for the decoded data bits, denoted by $P_b$ here, is defined as the arithmetic average of the $k$ $P(E_i)$ values. It is generally quite time-consuming to determine the values of the $P(E_i)$ or their average $P_b$ except for codes of very short blocklengths, whereas $P(E)$, the probability that the decoder output is not the same as the transmitted codeword, is easier to determine (or overbound for bounded-distance decoders that are prone to decoder failure). It is straightforward to show that $$\frac 1k P(E) \leq P_b \leq P(E)\tag{2}$$ which in the instance that is puzzling the OP tells us that the BER must be between $3.725\%$ and $14.9\%$. More can be said in this particular case, however, the codewords of a $[7,4]$ Hamming code are very closely related to what is called a biorthogonal signal set in the communications literature (the codewords of the $[8,4]$ extended Hamming code are exactly a biorthogonal signal set). For a biorthogonal signal set, the BER is very nearly $\frac 12P(E)$, and so the $7\%$ BER claimed in the Mackay text can be explained as being consistent with this result, with some differences because the $[7,4]$ Hamming code is not quite a biorthogonal signal set, and because $P_b$ is not exactly $\frac 12 P(E)$ for a biorthogonal signal set.

Masochists who really want to delve into the details can find information about orthogonal and biorthogonal signal sets and their relationship to first-order Reed-Muller codes (of which class the $[8,4]$ extended Hamming code is an example) can find some basic information on pp. 161-180 of this ancient Lecture Note of mine.

  • 0
    i don't quite get the first inequality above. Do you mind explaining more? Shouldn't the probability of error of each bit be the same such that you don't need to take average? – 2017-01-21
  • 0
    also why is k = 4 and not 7 – 2017-01-21
  • 0
    $k=4$ because there are 4 data bits. The decoder output is always a valid _codeword_. This codeword _cannot_ match the transmitted codeword in the 4 data bits and differ from the transmitted codeword only in the parity bits. _Think_ a little bit about why this must be so before demanding an explanation and read the last part of the first paragraph to see how $(1)$ follows from all this. Now, $$P(E) \leq \sum_{I=1}^k P(E_i)$$ (again, think a bit about how this follows from $(1)$) and dividing both sides by $k$ gives the first inequality in $(2)$. .... (continued) – 2017-01-21
  • 0
    .. (continuation)... It doesn't matter in the least whether the $P(E_i)$ are all the same (they aren't in general) or different, If the $P(E_i)$ are all the same, then $P_b$ also has the same value and the stated bounds on $P_b$ apply to each $P(E_i)$ also. If the $P(E_i)$ are different, then the bounds on $P_b$ don't necessarily hold for all the $P(E_i)$. – 2017-01-21