Hykilpikonna
220 lines
25 KiB
HTML
220 lines
25 KiB
HTML
<!DOCTYPE html>
|
||
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="generator" content="pandoc" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
|
||
<title>8.3 Big-O, Omega, and Theta</title>
|
||
<style>
|
||
code{white-space: pre-wrap;}
|
||
span.smallcaps{font-variant: small-caps;}
|
||
span.underline{text-decoration: underline;}
|
||
div.column{display: inline-block; vertical-align: top; width: 50%;}
|
||
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
|
||
ul.task-list{list-style: none;}
|
||
</style>
|
||
<link rel="stylesheet" href="../tufte.css" />
|
||
<script src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js" type="text/javascript"></script>
|
||
<!--[if lt IE 9]>
|
||
<script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
|
||
<![endif]-->
|
||
</head>
|
||
<body>
|
||
<div style="display:none">
|
||
\(
|
||
\newcommand{\NOT}{\neg}
|
||
\newcommand{\AND}{\wedge}
|
||
\newcommand{\OR}{\vee}
|
||
\newcommand{\XOR}{\oplus}
|
||
\newcommand{\IMP}{\Rightarrow}
|
||
\newcommand{\IFF}{\Leftrightarrow}
|
||
\newcommand{\TRUE}{\text{True}\xspace}
|
||
\newcommand{\FALSE}{\text{False}\xspace}
|
||
\newcommand{\IN}{\,{\in}\,}
|
||
\newcommand{\NOTIN}{\,{\notin}\,}
|
||
\newcommand{\TO}{\rightarrow}
|
||
\newcommand{\DIV}{\mid}
|
||
\newcommand{\NDIV}{\nmid}
|
||
\newcommand{\MOD}[1]{\pmod{#1}}
|
||
\newcommand{\MODS}[1]{\ (\text{mod}\ #1)}
|
||
\newcommand{\N}{\mathbb N}
|
||
\newcommand{\Z}{\mathbb Z}
|
||
\newcommand{\Q}{\mathbb Q}
|
||
\newcommand{\R}{\mathbb R}
|
||
\newcommand{\C}{\mathbb C}
|
||
\newcommand{\cA}{\mathcal A}
|
||
\newcommand{\cB}{\mathcal B}
|
||
\newcommand{\cC}{\mathcal C}
|
||
\newcommand{\cD}{\mathcal D}
|
||
\newcommand{\cE}{\mathcal E}
|
||
\newcommand{\cF}{\mathcal F}
|
||
\newcommand{\cG}{\mathcal G}
|
||
\newcommand{\cH}{\mathcal H}
|
||
\newcommand{\cI}{\mathcal I}
|
||
\newcommand{\cJ}{\mathcal J}
|
||
\newcommand{\cL}{\mathcal L}
|
||
\newcommand{\cK}{\mathcal K}
|
||
\newcommand{\cN}{\mathcal N}
|
||
\newcommand{\cO}{\mathcal O}
|
||
\newcommand{\cP}{\mathcal P}
|
||
\newcommand{\cQ}{\mathcal Q}
|
||
\newcommand{\cS}{\mathcal S}
|
||
\newcommand{\cT}{\mathcal T}
|
||
\newcommand{\cV}{\mathcal V}
|
||
\newcommand{\cW}{\mathcal W}
|
||
\newcommand{\cZ}{\mathcal Z}
|
||
\newcommand{\emp}{\emptyset}
|
||
\newcommand{\bs}{\backslash}
|
||
\newcommand{\floor}[1]{\left \lfloor #1 \right \rfloor}
|
||
\newcommand{\ceil}[1]{\left \lceil #1 \right \rceil}
|
||
\newcommand{\abs}[1]{\left | #1 \right |}
|
||
\newcommand{\xspace}{}
|
||
\newcommand{\proofheader}[1]{\underline{\textbf{#1}}}
|
||
\)
|
||
</div>
|
||
<header id="title-block-header">
|
||
<h1 class="title">8.3 Big-O, Omega, and Theta</h1>
|
||
</header>
|
||
<section>
|
||
<p>Big-O is a useful way of describing the long-term growth behaviour of functions, but its definition is limited in that it is not required to be an exact description of growth. After all, the key inequality <span class="math inline">\(g(n) \leq c f(n)\)</span> can be satisfied even if <span class="math inline">\(f\)</span> grows much, <em>much</em> faster than <span class="math inline">\(g\)</span>. For example, we could say that <span class="math inline">\(n + 10 \in \cO(n^{100})\)</span> according to our definition, but this is not necessarily informative.</p>
|
||
<p>In other words, the definition of Big-O allows us to express <em>upper bounds</em> on the growth of a function, but does not allow us to distinguish between an upper bound that is tight and one that vastly overestimates the rate of growth.</p>
|
||
<p>In this section, we will introduce the final new pieces of notation for this chapter, which allow us to express tight bounds on the growth of a function.</p>
|
||
<h2 id="omega-and-theta">Omega and Theta</h2>
|
||
<div class="definition">
|
||
<p>Let <span class="math inline">\(f, g : \N \TO \R^{\ge 0}\)</span>. We say that <strong><span class="math inline">\(g\)</span> is Omega of <span class="math inline">\(f\)</span></strong> when there exist constants <span class="math inline">\(c, n_0 \in \R^+\)</span> such that for all <span class="math inline">\(n \in \N\)</span>, if <span class="math inline">\(n \geq n_0\)</span>, then <span class="math inline">\(g(n) \geq c \cdot f(n)\)</span>. In this case, we can also write <span class="math inline">\(g \in \Omega(f)\)</span>.</p>
|
||
</div>
|
||
<p>You can think of Omega as the dual of Big-O: when <span class="math inline">\(g \in \Omega(f)\)</span>, then <span class="math inline">\(f\)</span> is a <em>lower</em> bound on the growth rate of <span class="math inline">\(g\)</span>. For example, we can use the definition to prove that <span class="math inline">\(n^2 - n \in \Omega(n)\)</span>.</p>
|
||
<p>We can now express a bound that is tight for a function’s growth rate quite elegantly by combining Big-O and Omega: if <span class="math inline">\(f\)</span> is asymptotically both a lower and upper bound for <span class="math inline">\(g\)</span>, then <span class="math inline">\(g\)</span> must grow at the same rate as <span class="math inline">\(f\)</span>.</p>
|
||
<div class="definition" data-terms="Omega, Theta">
|
||
<p>Let <span class="math inline">\(f, g : \N \TO \R^{\ge 0}\)</span>. We say that <strong><span class="math inline">\(g\)</span> is Theta of <span class="math inline">\(f\)</span></strong> when <span class="math inline">\(g\)</span> is both Big-O of <span class="math inline">\(f\)</span> and Omega of <span class="math inline">\(f\)</span>. In this case, we can write <span class="math inline">\(g \in \Theta(f)\)</span>, and say that <span class="math inline">\(f\)</span> is a <strong>tight bound</strong> on <span class="math inline">\(g\)</span>.<label for="sn-0" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-0" class="margin-toggle"/><span class="sidenote">Most of the time, when people say “Big-O” they actually mean Theta, i.e., a Big-O upper bound is meant to be the tight one, because we rarely say upper bounds that overestimate the rate of growth. However, in this course we will always use <span class="math inline">\(\Theta\)</span> when we mean tight bounds, because we will see some cases where coming up with tight bounds isn’t easy.</span></p>
|
||
<p>Equivalently, <span class="math inline">\(g\)</span> is Theta of <span class="math inline">\(f\)</span> when there exist constants <span class="math inline">\(c_1, c_2, n_0 \in \R^+\)</span> such that for all <span class="math inline">\(n \in \N\)</span>, if <span class="math inline">\(n \geq n_0\)</span> then <span class="math inline">\(c_1 f(n) \leq g(n) \leq c_2 f(n)\)</span>.</p>
|
||
</div>
|
||
<p>When we are comparing function growth rates, we typically look for a “Theta bound”, as this means that the two functions have the same approximate rate of growth, not just that one is larger than the other. For example, it is possible to prove that <span class="math inline">\(10n + 5 \in \Theta(n)\)</span>, but <span class="math inline">\(10n + 5 \notin \Theta(n^2)\)</span>.<label for="sn-1" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-1" class="margin-toggle"/><span class="sidenote"> Both of these are good exercises to prove, using the above definitions!</span></p>
|
||
<!-- <div class="example">
|
||
Let $f(n) = n^2$ and $g(n) = n + 10$. Then $g \in \cO(f)$, but
|
||
$g \notin \Theta(f)$. That is, $f$ is an upper bound for the growth
|
||
rate of $g$, but it is not a tight upper bound.
|
||
</div>
|
||
|
||
<div exercise>
|
||
<div questions data-series="chapter5">
|
||
\item
|
||
Prove the statement in the previous example.
|
||
Note that the correct translation uses an **AND**, so you'll actually need to prove two different statements here.
|
||
</div>
|
||
</div> -->
|
||
<h2 id="a-special-case-co1-omega1-and-theta1">A special case: <span class="math inline">\(\cO(1)\)</span>, <span class="math inline">\(\Omega(1)\)</span>, and <span class="math inline">\(\Theta(1)\)</span></h2>
|
||
<p>So far, we have seen Big-O expressions like <span class="math inline">\(\cO(n)\)</span> and <span class="math inline">\(\cO(n^2)\)</span>, where the function in parentheses has grown to infinity. However, not every function takes on larger and larger values as its input grows. Some functions are <em>bounded</em>, meaning they never take on a value larger than some fixed constant.</p>
|
||
<p>For example, consider the constant function <span class="math inline">\(f(n) = 1\)</span>, which always outputs the value <span class="math inline">\(1\)</span>, regardless of the value of <span class="math inline">\(n\)</span>. What would it mean to say that a function <span class="math inline">\(g\)</span> is Big-O of this <span class="math inline">\(f\)</span>? Let’s unpack the definition of Big-O to find out.</p>
|
||
<p><span class="math display">\[\begin{align*}
|
||
& g \in \cO(f) \\
|
||
& \exists c, n_0 \in \R^+,~ \forall n \in \N,~ n \geq n_0 \IMP g(n) \leq c \cdot f(n) \\
|
||
& \exists c, n_0 \in \R^+,~ \forall n \in \N,~ n \geq n_0 \IMP g(n) \leq c \tag{since $f(n) = 1$}
|
||
\end{align*}\]</span></p>
|
||
<p>In other words, there exists a constant <span class="math inline">\(c\)</span> such that <span class="math inline">\(g(n)\)</span> is eventually always less than or equal to <span class="math inline">\(c\)</span>. We say that such functions <span class="math inline">\(g\)</span> are <strong>asymptotically bounded</strong> with respect to their input, and write <span class="math inline">\(g \in \cO(1)\)</span> to represent this.</p>
|
||
<p>Similarly, we use <span class="math inline">\(g \in \Omega(1)\)</span> to express that functions are greater than or equal to some constant <span class="math inline">\(c\)</span>. You might wonder why we would ever say this—don’t all functions satisfy this property? While the functions we’ll be studying in later chapters in this section are generally going to be <span class="math inline">\(\geq 1\)</span>, this is not true for all mathematical functions. For example, the function <span class="math inline">\(g(n) = \frac{1}{n + 1}\)</span> is <span class="math inline">\(\cO(1)\)</span>, but <em>not</em> <span class="math inline">\(\Omega(1)\)</span>.<label for="sn-2" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-2" class="margin-toggle"/><span class="sidenote"> More generally, any function <span class="math inline">\(g\)</span> where <span class="math inline">\(\lim_{n \to \infty} g(n) = 0\)</span> is not <span class="math inline">\(\Omega(1)\)</span>.</span></p>
|
||
<p>On the other hand, the function <span class="math inline">\(g(n) = n^2\)</span> is <span class="math inline">\(\Omega(1)\)</span> but not <span class="math inline">\(\cO(1)\)</span>. So we reserve <span class="math inline">\(\Theta(1)\)</span> to refer to the functions that are both <span class="math inline">\(\cO(1)\)</span> and <span class="math inline">\(\Omega(1)\)</span>.</p>
|
||
<h2 id="section:asymptotic-properties">Properties of Big-O, Omega, and Theta</h2>
|
||
<p>If we had you always write chains of inequalities to prove that one function is Big-O/Omega/Theta of another, that would get quite tedious rather quickly. Instead, in this section we will prove some properties of this definition which are extremely useful for combining functions together under this definition. These properties can save you quite a lot of work in the long run. We’ll illustrate the proof of one of these properties here; most of the others can be proved in a similar manner, while a few are most easily proved using some techniques from calculus.<label for="sn-3" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-3" class="margin-toggle"/><span class="sidenote">We discuss the connection between calculus and asymptotic notation in the following section, but this is <em>not</em> a required part of CSC110.</span></p>
|
||
<h3 id="elementary-functions">Elementary functions</h3>
|
||
<p>The following theorem tells us how to compare four different types of “elementary” functions: constant functions, logarithms, powers of <span class="math inline">\(n\)</span>, and exponential functions.</p>
|
||
<div id="elementary-function-hierarchy" class="theorem" data-label="Elementary function hierarchy">
|
||
<p>(<em>Elementary function growth hierarchy</em>)</p>
|
||
<p>For all <span class="math inline">\(a, b \in \R^+\)</span>, the following statements are true:</p>
|
||
<ol type="1">
|
||
<li>If <span class="math inline">\(a > 1\)</span> and <span class="math inline">\(b > 1\)</span>, then <span class="math inline">\(\log_a n \in \Theta(\log_b n)\)</span>.</li>
|
||
<li>If <span class="math inline">\(a < b\)</span>, then <span class="math inline">\(n^a \in \cO(n^b)\)</span> and <span class="math inline">\(n^a \notin \Omega(n^b)\)</span>.</li>
|
||
<li>If <span class="math inline">\(a < b\)</span>, then <span class="math inline">\(a^n \in \cO(b^n)\)</span> and <span class="math inline">\(a^n \notin \Omega(b^n)\)</span>.</li>
|
||
<li>If <span class="math inline">\(a > 1\)</span>, then <span class="math inline">\(1 \in \cO(\log_a n)\)</span> and <span class="math inline">\(1 \notin \Omega(\log_a n)\)</span>.</li>
|
||
<li><span class="math inline">\(\log_a n \in \cO(n^b)\)</span> and <span class="math inline">\(\log_a n \notin \Omega(n^b)\)</span>.</li>
|
||
<li>If <span class="math inline">\(b > 1\)</span>, then <span class="math inline">\(n^a \in \cO(b^n)\)</span> and <span class="math inline">\(n^a \notin \Omega(b^n)\)</span>.</li>
|
||
</ol>
|
||
</div>
|
||
<p>And here is a handy figure to show the progression of functions toward longer running times:</p>
|
||
<p><img src="images/elementary_function_hierarchy.png" /></p>
|
||
<h3 id="basic-properties">Basic properties</h3>
|
||
<div class="theorem" data-label="Reflexivity">
|
||
<p>For all <span class="math inline">\(f : \N \to \R^{\geq 0}\)</span>, <span class="math inline">\(f \in \Theta(f)\)</span>.</p>
|
||
</div>
|
||
<div class="theorem" data-label="Quasi-symmetry">
|
||
<p>For all <span class="math inline">\(f, g : \N \to \R^{\geq 0}\)</span>, <span class="math inline">\(g \in \cO(f)\)</span> if and only if <span class="math inline">\(f \in \Omega(g)\)</span>.<label for="sn-4" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-4" class="margin-toggle"/><span class="sidenote">As a consequence of this, <span class="math inline">\(g \in \Theta(f)\)</span> if and only if <span class="math inline">\(f \in \Theta(g)\)</span>.</span></p>
|
||
</div>
|
||
<div class="theorem" data-label="Transitivity">
|
||
<p>For all <span class="math inline">\(f, g, h : \N \to \R^{\geq 0}\)</span>:</p>
|
||
<ul>
|
||
<li>If <span class="math inline">\(f \in \cO(g)\)</span> and <span class="math inline">\(g \in \cO(h)\)</span>, then <span class="math inline">\(f \in \cO(h)\)</span>.</li>
|
||
<li>If <span class="math inline">\(f \in \Omega(g)\)</span> and <span class="math inline">\(g \in \Omega(h)\)</span>, then <span class="math inline">\(f \in \Omega(h)\)</span>.</li>
|
||
<li>If <span class="math inline">\(f \in \Theta(g)\)</span> and <span class="math inline">\(g \in \Theta(h)\)</span>, then <span class="math inline">\(f \in \Theta(h)\)</span>.<label for="sn-5" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-5" class="margin-toggle"/><span class="sidenote"> Exercise: prove this using the first two.</span></li>
|
||
</ul>
|
||
</div>
|
||
<h3 id="operations-on-functions">Operations on functions</h3>
|
||
<div class="definition" data-terms="sum of functions">
|
||
<p>Let <span class="math inline">\(f, g : \N \TO \R^{\ge 0}\)</span>. We can define the <strong>sum of <span class="math inline">\(f\)</span> and <span class="math inline">\(g\)</span></strong> as the function <span class="math inline">\(f + g : \N \TO \R^{\ge 0}\)</span> such that <span class="math display">\[\forall n \in \N,~ (f + g)(n) = f(n) + g(n).\]</span></p>
|
||
</div>
|
||
<div class="theorem" data-label="Sum of functions">
|
||
<p> For all <span class="math inline">\(f, g, h : \N \to \R^{\geq 0}\)</span>, the following hold:</p>
|
||
<ol type="1">
|
||
<li>If <span class="math inline">\(f \in \cO(h)\)</span> and <span class="math inline">\(g \in \cO(h)\)</span>, then <span class="math inline">\(f + g \in \cO(h)\)</span>.</li>
|
||
<li>If <span class="math inline">\(f \in \Omega(h)\)</span>, then <span class="math inline">\(f + g \in \Omega(h)\)</span>.</li>
|
||
<li>If <span class="math inline">\(f \in \Theta(h)\)</span> and <span class="math inline">\(g \in \cO(h)\)</span>, then <span class="math inline">\(f + g \in \Theta(h)\)</span>.<label for="sn-6" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-6" class="margin-toggle"/><span class="sidenote"> Exercise: prove this using the first two.</span></li>
|
||
</ol>
|
||
<p>We’ll prove the first of these statements.</p>
|
||
<div class="translation">
|
||
<p><span class="math display">\[\forall f, g, h : \N \TO \R^{\ge 0},~ \big(f \in \cO(h) \AND g \in \cO(h)\big) \IMP f + g \in \cO(h).\]</span></p>
|
||
</div>
|
||
<div class="discussion">
|
||
<p>This is similar in spirit to the divisibility proofs we did in <a href="../06-proofs/02-number-theory-proofs.html">Section 6.2</a>, which used a term (divisibility) that contained a quantifier.<label for="sn-7" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-7" class="margin-toggle"/><span class="sidenote">The definition of Big-O here has <em>three</em> quantifiers, but the idea is the same.</span> Here, we need to assume that <span class="math inline">\(f\)</span> and <span class="math inline">\(g\)</span> are both Big-O of <span class="math inline">\(h\)</span>, and prove that <span class="math inline">\(f + g\)</span> is also Big-O of <span class="math inline">\(h\)</span>.</p>
|
||
<p>Assuming <span class="math inline">\(f \in \cO(h)\)</span> tells us there exist positive real numbers <span class="math inline">\(c_1\)</span> and <span class="math inline">\(n_1\)</span> such that for all <span class="math inline">\(n \in \N\)</span>, if <span class="math inline">\(n \geq n_1\)</span> then <span class="math inline">\(f(n) \leq c_1 \cdot h(n)\)</span>. There similarly exist <span class="math inline">\(c_2\)</span> and <span class="math inline">\(n_2\)</span> such that <span class="math inline">\(g(n) \leq c_2 \cdot h(n)\)</span> whenever <span class="math inline">\(n \geq n_2\)</span>. <em>Warning:</em> we can’t assume that <span class="math inline">\(c_1 = c_2\)</span> or <span class="math inline">\(n_1 = n_2\)</span>, or any other relationship between these two sets of variables.</p>
|
||
<p>We want to prove that there exist <span class="math inline">\(c, n_0 \in \R^+\)</span> such that for all <span class="math inline">\(n \in \N\)</span>, if <span class="math inline">\(n \geq n_0\)</span> then <span class="math inline">\(f(n) + g(n) \leq c \cdot h(n)\)</span>.</p>
|
||
<p>The forms of the inequalities we can assume—<span class="math inline">\(f(n) \leq c_1 h(n)\)</span>, <span class="math inline">\(g(n) \leq c_2 h(n)\)</span>—and the final inequality are identical, and in particular the left-hand side suggests that we just need to add the two given inequalities together to get the third. We just need to make sure that both given inequalities hold by choosing <span class="math inline">\(n_0\)</span> to be large enough, and let <span class="math inline">\(c\)</span> be large enough to take into account both <span class="math inline">\(c_1\)</span> and <span class="math inline">\(c_2\)</span>.</p>
|
||
</div>
|
||
<div class="proof">
|
||
<p>Let <span class="math inline">\(f, g, h : \N \TO \R^{\ge 0}\)</span>, and assume <span class="math inline">\(f \in \cO(h)\)</span> and <span class="math inline">\(g \in \cO(h)\)</span>. By these assumptions, there exist <span class="math inline">\(c_1, c_2, n_1, n_2 \in \R^+\)</span> such that for all <span class="math inline">\(n \in \N\)</span>,</p>
|
||
<ul>
|
||
<li>if <span class="math inline">\(n \geq n_1\)</span>, then <span class="math inline">\(f(n) \leq c_1 \cdot h(n)\)</span>, and</li>
|
||
<li>if <span class="math inline">\(n \geq n_2\)</span>, then <span class="math inline">\(g(n) \leq c_2 \cdot h(n)\)</span>.</li>
|
||
</ul>
|
||
<p>We want to prove that <span class="math inline">\(f + g \in \cO(h)\)</span>, i.e., that there exist <span class="math inline">\(c, n_0 \in \R^+\)</span> such that for all <span class="math inline">\(n \in \N\)</span>, if <span class="math inline">\(n \geq n_0\)</span> then <span class="math inline">\(f(n) + g(n) \leq c \cdot h(n)\)</span>.</p>
|
||
<p>Let <span class="math inline">\(n_0 = \max \{n_1, n_2\}\)</span> and <span class="math inline">\(c = c_1 + c_2\)</span>. Let <span class="math inline">\(n \in \N\)</span>, and assume that <span class="math inline">\(n \geq n_0\)</span>. We now want to prove that <span class="math inline">\(f(n) + g(n) \leq c \cdot h(n)\)</span>.</p>
|
||
<p>Since <span class="math inline">\(n_0 \geq n_1\)</span> and <span class="math inline">\(n_0 \geq n_2\)</span>, we know that <span class="math inline">\(n\)</span> is greater than or equal to <span class="math inline">\(n_1\)</span> and <span class="math inline">\(n_2\)</span> as well. Then using the Big-O assumptions, <span class="math display">\[\begin{align*}
|
||
f(n) &\leq c_1 \cdot h(n) \\
|
||
g(n) &\leq c_2 \cdot h(n)
|
||
\end{align*}\]</span></p>
|
||
<p>Adding these two inequalities together yields</p>
|
||
<p><span class="math display">\[f(n) + g(n) \leq c_1 h(n) + c_2 h(n) = (c_1 + c_2) h(n) = c \cdot h(n).\]</span></p>
|
||
</div>
|
||
</div>
|
||
<div class="theorem" data-label="Multiplication by a constant">
|
||
<p>For all <span class="math inline">\(f : \N \to \R^{\geq 0}\)</span> and all <span class="math inline">\(a \in \R^+\)</span>, <span class="math inline">\(a \cdot f \in \Theta(f)\)</span>.</p>
|
||
</div>
|
||
<div class="theorem" data-label="Product of functions">
|
||
<p>For all <span class="math inline">\(f_1, f_2, g_1, g_2 : \N \to \R^{\geq 0}\)</span>, if <span class="math inline">\(g_1 \in \cO(f_1)\)</span> and <span class="math inline">\(g_2 \in \cO(f_2)\)</span>, then <span class="math inline">\(g_1 \cdot g_2 \in \cO(f_1 \cdot f_2)\)</span>. <em>Moreover</em>, the statement is still true if you replace Big-O with Omega, or if you replace Big-O with Theta.</p>
|
||
</div>
|
||
<div class="theorem" data-label="Floor and ceiling">
|
||
<p>For all <span class="math inline">\(f : \N \to \R^{\geq 0}\)</span>, if <span class="math inline">\(f(n)\)</span> is eventually greater than or equal to <span class="math inline">\(1\)</span>, then <span class="math inline">\(\floor{f} \in \Theta(f)\)</span> and <span class="math inline">\(\ceil{f} \in \Theta(f)\)</span>.</p>
|
||
</div>
|
||
<h3 id="properties-from-calculus">Properties from calculus</h3>
|
||
<p>[<em>Note: this subsection is <strong>not</strong> part of the require course material for CSC110. It is presented mainly for the nice connection between Big-O notation and calculus.</em>]</p>
|
||
<p>Our asymptotic notation of <span class="math inline">\(\cO\)</span>, <span class="math inline">\(\Omega\)</span>, and <span class="math inline">\(\Theta\)</span> are concerned with the comparing the <em>long-term behaviour</em> of two functions. It turns out that the concept of “long-term behaviour” is captured in another object of mathematical study, familiar to us from calculus: the <em>limit</em> of the function as its input approaches infinity.</p>
|
||
<p>Let <span class="math inline">\(f: \N \to \R\)</span> and <span class="math inline">\(L \in \R\)</span>. We have the following two definitions:<label for="sn-8" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-8" class="margin-toggle"/><span class="sidenote">We’re restricting our attention here to functions with domain <span class="math inline">\(\N\)</span> because that’s our focus in computer science.</span> <span class="math display">\[
|
||
\lim_{n \to \infty} f(n) = L:~ \forall \epsilon \in \R^+,~ \exists n_0 \in \N,~ \forall n \in \N,~ n \geq n_0 \IMP |f(n) - L| < \epsilon
|
||
\]</span> <span class="math display">\[
|
||
\lim_{n \to \infty} f(n) = \infty:~ \forall M \in \R^+,~ \exists n_0 \in \N,~ \forall n \in \N,~ n \geq n_0 \IMP f(n) > M
|
||
\]</span></p>
|
||
<p>Using just these definitions and the definitions of our asymptotic symbols <span class="math inline">\(\cO\)</span>, <span class="math inline">\(\Omega\)</span>, and <span class="math inline">\(\Theta\)</span>, we can prove the following pretty remarkable results:</p>
|
||
<div class="theorem" data-label="Limits and Big-O">
|
||
<p>For all <span class="math inline">\(f, g: \N \to \R^{\geq 0}\)</span>, if <span class="math inline">\(g(n) \neq 0\)</span> for all <span class="math inline">\(n \in \N\)</span>, then the following statements hold:</p>
|
||
<ol type="i">
|
||
<li>If there exists <span class="math inline">\(L \in \R^+\)</span> such that <span class="math inline">\(\lim_{n \to \infty} f(n)/g(n) = L\)</span>, then <span class="math inline">\(g \in \Omega(f)\)</span> and <span class="math inline">\(g \in \cO(f)\)</span>. (In other words, <span class="math inline">\(g \in \Theta(f)\)</span>.)</li>
|
||
<li>If <span class="math inline">\(\lim_{n \to \infty} f(n)/g(n) = 0\)</span>, then <span class="math inline">\(f \in \cO(g)\)</span> and <span class="math inline">\(g \notin \cO(f)\)</span>.</li>
|
||
<li>If <span class="math inline">\(\lim_{n \to \infty} f(n)/g(n) = \infty\)</span>, then <span class="math inline">\(g \in \cO(f)\)</span> and <span class="math inline">\(f \notin \cO(g)\)</span>.</li>
|
||
</ol>
|
||
</div>
|
||
<p>Proving this theorem is actually a very good (lengthy) exercise for a CSC110 student; they involve keeping track of variables and manipulating inequalities, two key skills you’re developing in this course! And they do tend to be useful in practice (although again, not for this course) to proving asymptotic bounds like <span class="math inline">\(n^2 \in \cO(1.01^n)\)</span>. But note that the converse of these statements is not true; for example, it is possible (and another nice exercise) to find functions <span class="math inline">\(f\)</span> and <span class="math inline">\(g\)</span> such that <span class="math inline">\(g \in \Theta(f)\)</span>, but <span class="math inline">\(\lim_{n \to \infty} f(n)/g(n)\)</span> is undefined.</p>
|
||
</section>
|
||
<footer>
|
||
<a href="https://www.teach.cs.toronto.edu/~csc110y/fall/notes/">CSC110 Course Notes Home</a>
|
||
</footer>
|
||
</body>
|
||
</html>
|