8
$\begingroup$

Let $X_1,\dots,X_n$ be independent random variables in $\{0,1\}$, and $X=X_1+\dots+X_n$. Suppose that $\mathbb{E}[X]=1$. What is the best possible upper bound on $\text{Pr}(X>\log n)$?

Using the multiplicative form of Chernoff's bound, we have that $\text{Pr}(X>1+\delta)<\dfrac{e^\delta}{(1+\delta)^{1+\delta}}$ for any $\delta>0$. When $\delta$ is $\log n-1$, then this becomes $\dfrac{e^{\log n-1}}{\log n^{\log n}}$. This is approximately $n^{1-\log\log n}$. Are there examples of random variables $X_1,\dots,X_n$ that shows that this bound resulting from Chernoff is (approximately) tight?

  • 0
    Do you need asymptotic analysis i.e. as $n \rightarrow \infty$? – 2017-02-20
  • 0
    No, I'd like a bound that holds for all $n$. – 2017-02-20

2 Answers 2

2

Let $\ell(n)=\lfloor \ln(n)\rfloor+1$, and let all $E[X_i]=\frac{1}{\ell(n)}$ for $i\leq \ell(n)$ and $E[X_i]=0$ otherwise. Essentially, you are concentrating all the "mass" into the first $\ell(n)\approx\ln(n)$ variables (the minimum number whose sum can exceed $\ln(n)$), divided evenly.

The probability that $X_i=1$ for $i\leq \ell(n)$ is $\left(\frac{1}{\ell(n)}\right)^{\ell(n)}\approx\left(\frac{1}{\ln(n)}\right)^{\ln(n)}=n^{-\ln\ln(n)}\approx n^{-\ln\ln(n)+1}$, where the last term is the Chernoff Bound you obtained.

Note that the (multiplicative) error "hidden" in the first of the two $\approx$ (due to having to use $\lfloor \ln(n)\rfloor+1$ instead of $\ln(n)$ because the $X_i$ are discrete) is of the order of $\ln(n)$, so smaller than that "hidden" in the second $\approx$ which is $n$. The latter (which is, after all, just a $+1$ added to a $-\ln\ln(n)$ exponent) is mostly a consequence of the approximations required to produce a manageable formula like the $\frac{e^\delta}{(1+\delta)^{1+\delta}}$ one you used, from the "full-power" Chernoff-Hoeffding bound written in terms of relative entropy for $n$ independent $X_i$ with values in $[0,1]$ and expected sum $\mu$:

$\Pr\left[\sum_{i=1}^n X_i \geq \mu+\lambda)\right]\leq e^{−nH_{\mu/n}(\mu/n+ \lambda)}$, where $H_p(x)=x\ln(\frac{x}{p})+(1−x)\ln(\frac{1−x}{1-p})$ is the relative entropy of $x$ with respect to $p$.

  • 0
    Should $n^{-\ln(n)\ln(n)+1}$ be $n^{-\ln\ln(n)+1}$? – 2017-02-28
  • 0
    @pi66 yes, it should, and now it is :) – 2017-02-28
0

This doesn't hold for all $n$, but asymptotically I can get a bound better then $n^{1-\log\log n}$, although it's not as good as Chernoff. Let $X_i=1$ with probability $\frac{1}{n}$. Then $X=\text{Bin}(n,\frac{1}{n})$ which is approximately $\text{Poisson}(1)$ for large $n$. Therefore $\mathbb{P}[X>\log n]\approx\sum_{i=\log n}^{\infty}\frac{e^{-1}}{i!}\approx \int_{\log n}^{\infty}\frac{e^{-1}}{\Gamma(x)}dx$. See that by L'Hopital's Rule $\lim_{n\rightarrow\infty}\frac{\int_{\log n}^{\infty}\frac{e^{-1}}{\Gamma(x)}dx}{n^{1-\log log n}}=\frac{\frac{1}{n}\frac{1}{\Gamma(\log n)}}{e(\log\log n)n^{-\log log n}}\sim \frac{n^{log log n}}{en\log log n\sqrt{2\pi(\log n-1)}\left(\frac{\log (n-1)}{e}\right)^{\log (n-1)}}\sim\frac{n-1}{n\log\log n\sqrt{2\pi(\log n-1)}}\rightarrow 0$ as $n\rightarrow\infty$, where by $\sim$ I mean I removed a term whose limit was equal to 1 as $n\rightarrow\infty$.