4
$\begingroup$

Say we have two functions: $f(n) = n$ and $g(n) = 2n$.

$$\lim_{n\to\infty} \frac{f(n)}{g(n)} = \lim_{n\to\infty} \frac{n}{2n} = \lim_{n\to\infty} \frac{1}{2} = \frac{1}{2}$$

Therefore, according to the answer here, function $g(n)$ grows faster than $f(n)$ because: $$0 \leq \lim_{n\to\infty} \frac{f(n)}{g(n)} < 1$$

But with big O notation: $f(n) = O(n)$ and $g(n) = O(n)$

Which means that both functions grow at the same rate (in an apparently different magnitude).

I realise that this is only happening because the coefficient of the largest term is ignored with big O notation but I want to know if you can use limits to give information about the big O notation of a function and vice versa because I find some of this terminology and these concepts hard to differentiate.

Thanks.

  • 1
    Oops, I found an answer here: http://math.stackexchange.com/questions/925053/using-limits-to-determine-big-o-big-omega-and-big-theta – 2017-01-02

1 Answers 1

4

Definition: $f=O(g)$ if and only if $\exists c, N>0$ such that $\forall x > N, f(x) < cg(x)$.

Big-O notation explicitly ignores constant factors (this is what the $c$ does in the definition), so any statement in big-O notation about $f(x)=x$ will also hold for $f(x)=2x,f(x)=1000x,$ and $f(x)=\frac{1}{2^{100}}x$.

What big-O notation tells you is that, asymptotically, one function is bounded by another. So if $f(x)=x$ and $g(x)=x^2$ then $f=O(g)$. This remains true when we replace $g$ with $g'(x)=x^2-2^{100}$ or with $g''(x)=210x^2-10$. The $N$ in the definition is what establishes this is an asymptotic property. The notation doesn't tell you anything about how good the bound is though. $f=O(f)$ is always true, as is $f=O(2^f)$. Concretely, if $f(x)=x$ then $f=O(g)$ for any polynomial $g$, as well as pretty much anything that looks like $2^x$.

There is a parallel notation for lower bounds (the "big-Omega" notation), which is $f=\Omega(g)$ which has precisely the same definition but the inequality is reversed. When $f=\Omega(g)$ AND $f=O(g)$ then we say that $f=\Theta(g)$ (the "big-theta" notation). There is also little-omega and little-theta with the same basic idea, but they are similar to the little-o notation.

These symbols can also be defined explicitly in terms of limits, as seen here. However that way of thinking about the notation is more limited and is not always defined.

  • 1
    The last three statements of your answer are false. An $O$/$\Omega$/$\Theta$ relationship does not tell you that the limits exist. What if $f(x) = 1$ and $g(x) = 1 + \sin(x)/2$? Then $f = \Theta(g)$ but neither $\lim f/g$ nor $\lim g/f$ exist. – 2017-01-03
  • 0
    @AntonioVargas interesting point! I appear to have been to hastey, though I think it's true when the limits exist? Intuitively, it should be related to the sup/inf of the set of values of $c$ that satisfy the definition. – 2017-01-03
  • 0
    The last three statements are definitely false. – 2017-03-14
  • 0
    @Lembik Huh, I thought I had fixed this a while ago. They are now correct. – 2017-03-14
  • 0
    The limit $\lim{x \to \infty} f(x)/g(x)$ does not have to exist . – 2017-03-14
  • 0
    @Lembik I didn't say that it did. I said that when it exists we can draw certain conclusions. – 2017-03-14
  • 0
    You have written "When $f=O(g)$, this means that $\lim_{x\to\infty}f(x)/g(x)$ is constant". That's not true. We can have that $f=O(g)$ but $\lim_{x\to\infty}f(x)/g(x)$ doesn't exist. – 2017-03-14
  • 0
    @Lembik And on the immediately preceding line I wrote "(assuming that they are all defined)" – 2017-03-14
  • 0
    Maybe it's the word "they" that is not clear. You could write, assuming $\lim_{x \to \infty} f(x)/g(x)$ exists then, when $f = O(g)$.... You never say what has to be defined for your inferences to be valid. – 2017-03-14
  • 0
    @Lembik how about that – 2017-03-14
  • 0
    I am afraid it's just plain wrong. Look at "When $\lim_{x\to\infty}f(x)/g(x)$ is non-zero (including infinite), we have that $f=\Omega(g)$" . Is this case meant to cover the case where the limit doesn't exist? The big O line doesn't make sense and you contradict yourself by having infinite limit in one case and non-existing limit in another. I would delete everything after "In terms of limits" to be honest. – 2017-03-14
  • 0
    @Lembik I guess. I would rather t be less complete than wrong, and this is turning into a lot of discussion an answered question :P – 2017-03-14