3
$\begingroup$

This is regarding the following exercise, especially the second inequality:

enter image description here

Our definition of Rademacher complexity:

enter image description here

Let's say my sample $S$ consists of only one point $Z_1$, my function class $F$ only contains one function $f$ with $f(Z_1)=1$. Then the Rademacher complexity of $F$ would be $0$, because $\sigma f$ is $1$ or $-1$ with probability $\frac{1}{2}$. Now let's say I have another function class $G$ which only consists of the function $g$ with $g(Z_1)=-1$. The Rademacher complexity of $G$ is also $0$.

But when we take $F \cup G$, isn't the Rademacher complexity $1$? Because no matter what value $\sigma$ takes, there is always a function $h$ in $F \cup G$ with $\sigma h(Z_1)=1$. What am I missing here?

0 Answers 0