1
$\begingroup$

Suppose I know the capacity of channel C1 and the capacity of another channel C2. Both are achieved by a uniform distribution. Both have a binary input.

Now I have a random variable Z which takes values {0,1} uniformly. I build a system that has a binary input, which is multiplexed to C1 if Z=0 and to C2 if Z=1. Outputs can be combined together (i.e. if C1 had outputs {0,1,2} and C2 has outputs {0,2}, then the new system can have outputs {0,1,2}).

What would be the capacity of the combined channel?

1 Answers 1

1

Let the new channel obtained by multiplexing channels $C_1$ and $C_2$ be called $C_\frac{1}{2}$. With some abuse of notation we'll refer to the capacities of these channels as $C_1$, $C_2$ and $C_\frac{1}{2}$ as well. Let us call the binary input as $X$ and the output as $Y$.

Note that $p_\frac{1}{2}(y|x) = \frac{1}{2}p_1(y|x) + \frac{1}{2}p_2(y|x)$ Suppose now that $C_\frac{1}{2}$ achieves its capacity at a certain distribution of $X$, say $p^*(x)$. We know that $I(X;Y)$ is a convex function of $p(y|x)$ for a fixed $p(x)$. Fixing the distribution of$X$ to be $p^*(x)$, we get the inequality $ C_\frac{1}{2} = I_\frac{1}{2}(X;Y) \leq \frac{1}{2}I_1(X;Y) + \frac{1}{2}I_2(X;Y) \leq \frac{1}{2}C_1 + \frac{1}{2}C_2$ This upper bound is tight (to see this consider two channels whose outputs don't overlap).

Your question asks the for the exact value of $C_\frac{1}{2}$. The best you can do without knowing the specifics of the channels is the above bound. One can ask the question

Is $p^*(x)$ uniform?

Though it may seem intuitively so, its not true.

Say $C_1$ looks like: $0-->a,b$ and $1 -->c,d$ with some probabilites and for $C_2$ : $0-->c,d$ and $1-->a,b$. Both these channels achieve capacity for a uniform distribution on $X$, however mixing these channels you can create all sorts of ugly looking channels which won't achieve capacity at uniform input distribution.