1
$\begingroup$

Consider channel $\Gamma_n$ which transforms elements from $\{0,1\}^n$ into elements of $\{0,1\}^{n+1}$ by simply adding a random bit in a random position. Formally, for $c\in\{0,1\},\ k\in\{1,...,n\}$ the sequence $a_1...a_n$ is transformed into $a_1...a_{k-1}ca_k...a_n$ with probability $\frac{1}{2(n+1)}$.

The question is to prove the capacity $C_{\Gamma}\geq n+1-\log (n+2).$

How would you try to solve this problem?

Obviously, we may try to use $H(A)-H(A|B) = H(B) - H(B|A)$ formulas. The first one gives an approximation which is less than wanted one. More precisely, if we know some sequence $b$ then, in the worst case, we have $n+1$ possibilities for sequence $a$ (by removing each bit) and may choose among them uniformly. Hence, $H(A|B)\leq\log(n+1)$ and by making $A$ distributed uniformly we obtain $C_{\Gamma}\geq n-\log(n+1)$.

Yet what about $H(B|A)$? The formula in the statement of the problem indicates it should be less-than-or-equal to $\log(n+2)$ but I don't really see how...

I'd appreciate you help.

1 Answers 1

1

Have you made any progress on that? I compute $$ \begin{align} H(B|A)&=H(k,c|A)\\ &= H(k)+H(c) \text{ ($k$, $c$ are independent})\\ &=\log_2(n+1)+1, \end{align} $$ which results in $C_\Gamma = H(B)-H(B|A) \leq n-\log_2(n+1)$