slides/04-arrays-bigoh.html

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>CS 2150: 04-arrays-bigoh slide set</title>
    <meta name="description" content="A set of slides for a course on Program and Data Representation">
    
    <meta name="apple-mobile-web-app-capable" content="yes" />
    <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui">
    <link rel="stylesheet" href="../slides/reveal.js/dist/reset.css">
    <link rel="stylesheet" href="../slides/reveal.js/dist/reveal.css">
    <link rel="stylesheet" href="../slides/reveal.js/dist/theme/black.css" id="theme">
    <link rel="stylesheet" href="../slides/css/pdr.css">
    <!-- Code syntax highlighting -->
    <link rel="stylesheet" href="../slides/reveal.js/plugin/highlight/zenburn.css">
    <!-- Printing and PDF exports -->
    <script>
      var link = document.createElement( 'link' );
      link.rel = 'stylesheet';
      link.type = 'text/css';
      link.href = window.location.search.match( /print-pdf/gi ) ? '../slides/reveal.js/css/print/pdf.scss' : '../slides/reveal.js/css/print/paper.scss';
      document.getElementsByTagName( 'head' )[0].appendChild( link );
    </script>
    <!--[if lt IE 9]>
	<script src="../slides/reveal.js/lib/js/html5shiv.js"></script>
	<![endif]-->
    <style>.reveal li { font-size:93%; line-height:120%; }</style>
  </head>

  <body>

    <div class="reveal">

      <!-- Any section element inside of this container is displayed as a slide -->
      <div class="slides">

	<section data-markdown><script type="text/template">
# CS 2150
&nbsp;
### Program and Data Representation

<p class='titlep'>&nbsp;</p>
<div class="titlesmall"><p>
<a href="http://www.cs.virginia.edu/~mrf8t">Mark Floryan</a> (mrf8t@virginia.edu)<br>
<a href="http://www.cs.virginia.edu/~asb">Aaron Bloomfield</a> (aaron@virginia.edu)<br>
<a href="http://github.com/uva-cs/pdr">@github</a> | <a href="index.html">&uarr;</a> | <a href="./04-arrays-bigoh.html?print-pdf"><img class="print" width="20" src="../slides/images/print-icon.png" style="top:0px;vertical-align:middle"></a>
</p></div>
<p class='titlep'>&nbsp;</p>

## Arrays & Big-Oh
	</script></section>
    <section>
<h2>CS 2150 Roadmap</h2>
<table class="wide">
  <tr><td colspan="3"><p class="center">Data Representation</p></td><td></td><td colspan="3"><p class="center">Program Representation</p></td></tr>
  <tr>
    <td class="top"><small>&nbsp;<br>&nbsp;<br>string<br>&nbsp;<br>&nbsp;<br>&nbsp;<br>int x[3]<br>&nbsp;<br>&nbsp;<br>&nbsp;<br>char x<br>&nbsp;<br>&nbsp;<br>&nbsp;<br>0x9cd0f0ad<br>&nbsp;<br>&nbsp;<br>&nbsp;<br>01101011</small></td>
    <!-- image adapted from http://openclipart.org/detail/3677/arrow-left-right-by-torfnase -->
    <td><img class="noborder" src="images/red-double-arrow.png" height="500" alt="vertical red double arrow"></td>
    <td class="top">&nbsp;<br>Objects<br>&nbsp;<br>Arrays<br>&nbsp;<br>Primitive types<br>&nbsp;<br>Addresses<br>&nbsp;<br>bits</td>
    <td>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</td>
    <td class="top"><small>&nbsp;<br>&nbsp;<br>Java code<br>&nbsp;<br>&nbsp;<br>C++ code<br>&nbsp;<br>&nbsp;<br>C code<br>&nbsp;<br>&nbsp;<br>x86 code<br>&nbsp;<br>&nbsp;<br>IBCM<br>&nbsp;<br>&nbsp;<br>hexadecimal</small></td>
    <!-- image adapted from http://openclipart.org/detail/3677/arrow-left-right-by-torfnase -->
    <td><img class="noborder" src="images/green-double-arrow.png" height="500" alt="vertical green double arrow"></td>
    <td class="top">&nbsp;<br>High-level language<br>&nbsp;<br>Low-level language<br>&nbsp;<br>Assembly language<br>&nbsp;<br>Machine code</td>
  </tr>
</table>
    </section>
	<section data-markdown><script type="text/template">
# Contents
&nbsp;  
[Arrays in C/C++](#/arrays)  
[Orders of Growth](#/growth)  
[Determining Running Times](#/runningtimes)  
	</script></section>


	<section>

	  <section id="arrays" data-markdown class="center"><script type="text/template">
# Arrays in C/C++
	  </script></section>

	  <section data-markdown><script type="text/template">
## Primitive Arrays in C++
- Two ways to declare arrays:
  - Specify name, type, and size
```
int someInts[3];
```
  - Specify name, type, and initialization list
```
int someInts[] = {4, 37, 18};
```
    - Size not needed in this case
- Either way, this creates 3 ints:
```
someInts[0], someInts[1], someInts[2]
```
	  </script></section>

	  <section data-markdown><script type="text/template">
## Things to remember about arrays in C++
1. No index checking!
   - Compiler and runtime system will not catch accesses out of array index bounds
   - Out of bounds access will cause unpredictable behavior
2. When passed as an argument to a function
   - Function does not know the size of the array
   - Need to pass another argument containing the size
3. Cannot be copied with =
   - Must be copied element by element
4. Cannot be compared with ==
   - It won't do what you want it to do
	  </script></section>

	  <section>
<h2>C++ Implementation of Arrays</h2>
<ul>
<li>Array names are pointers to beginning of array
<pre><code>int someInts[3];</code></pre><ul>
  <li>The size is only used by compiler to set aside memory for the array; it is not stored anywhere</li>
  <li><code>someInts</code> is set to <code>&amp;someInts[0]</code><ul>
      <li>Also called the base address</li></ul></li>
  <li>Local constant that cannot be changed</li></ul></li>
<li>Note that the previous declaration is almost the same as:
<pre><code>int* someInts = new int[3];</code></pre><ul>
  <li>But since this second declaration is declared with <code>new</code>, the programmer must delete it: <code>delete []</code></li></ul></li>
</ul>
	  </section>

	  <section>
<h2>int someInts[3] = {2, 4, 6};</h2>

<table class="transparent">
<tr><th>Array part</th><th>Address</th><th>Memory</th></tr>
<tr><td><code>&amp;someInts[0]</code></td><td>1000</td><td class="border">2</td></tr>
<tr><td><code>&amp;someInts[1]</code></td><td>1004</td><td class="border">4</td></tr>
<tr><td><code>&amp;someInts[2]</code></td><td>1008</td><td class="border">6</td></tr>
<tr><td></td><td></td><td class="border">...</td></tr>
<tr><td><code>someInts</code></td><td>???</td><td class="border">1000</td></tr>
<tr><td></td><td></td><td class="border" style="border-bottom:medium solid;">...</td></tr>
</table>
<p>This assumes that ints are 4 bytes in size</p>
	  </section>

	  <section data-markdown><script type="text/template">
## Where is someInts[2] located in memory?
- Remember
  - `someInts` is a pointer to address of `someInts[0]` (i.e., `&someInts[0]`)
  - Array elements are laid out sequentially in contiguous memory addresses

```
&someInts[i] = {addr of someInts} + (sizeof(int) * i)
```
	  </script></section>

	  <section data-markdown><script type="text/template">
## Operations on Arrays
- `someInts = someOtherArray;` is  illegal
  - Array base addresses are considered constant (cannot be reassigned)
  - This is one example of when pointers and arrays are NOT the same
- Is `someInts == someOtherArray` allowed?
  - Yes, but will only evaluate to true if `someInts` and `someOtherArray` point at the same memory location
	  </script></section>

	  <section data-markdown><script type="text/template">
## Function Calls and Arrays
(no external source code)
```
void someFunc(int arrayOfInts[]) { /* ... */ }

// ...

int main() {
    int someInts[3];
    someFunc(someInts); // no brackets
    return 0;
}
```
- `someInts` is a memory address
- Size of `someInts[]` unknown
- Is this pass by reference?  Pass by value?
	  </script></section>

	  <section data-markdown><script type="text/template">
## Multidimensional Arrays
(no external source code)
```
int m[2][3]; // 2 rows, 3 columns
int n[2][3] = // dimensions optional here
    { {1,2,3}, {4,5,6} };
    // initialize row 0: 1,2,3
    // initialize row 1: 4,5,6
```
	  </script></section>

	  <section data-markdown><script type="text/template">
## C++ Implementation
Multi-dimensional arrays in C++ are stored in ***row major*** order
![2-d array diagram](images/04-arrays-bigoh/2d-array-layout.png)
	  </script></section>

	  <section data-markdown><script type="text/template">
## Command-line parameters
- You can define the `main()` function as either:
  - `int main()`
  - `int main (int argc, char **argv)`
    - Or: `int main (int argc, char* argv[])`
- The second version gives us two parameters:
  - The number of command line parameters
    - Note that the program executable is one of them (`argv[0]`)
  - An array of C-style strings (`char*`) containing those parameters
	  </script></section>

	  <section data-markdown id="cmdlineparams"><script type="text/template">
## Command-line parameters
Source code: [cmdlineparams.cpp](code/04-arrays-bigoh/cmdlineparams.cpp.html) ([src](code/04-arrays-bigoh/cmdlineparams.cpp))
```
int main (int argc, char* argv[]) {
    // the command-line parameters are stored as C-style 
    // strings in the argv[] array. argc contains 1 more 
    // than the number of command line parms specified.
    // The 0th command line parameter is the program 
    // name. We can print a C-style string via cout:
    cout << "This program is called '" << argv[0] 
         << "'" << endl;
    cout << "The following are the command " 
         << "line parameters you specified: " << endl;
    // this for loop starts at 1 to avoid printing the 
    // name of the program
    for ( int i = 1; i < argc; i++ ) {
        // we can convert the C-style strings into 
        // C++-style strings, and then print them:
        string s(argv[i]);
        cout << "\t" << s << endl;
    }
    return 0;
}
```
	  </script></section>

	</section>


	<section>

	  <section id="growth" data-markdown class="center"><script type="text/template">
# Orders of Growth
	  </script></section>

	  <section data-markdown><script type="text/template">
## Classifying functions by their<br>Asymptotic Growth Rates
- Asymptotic growth rate or asymptotic order
  - Comparing and classifying functions that ignores constant factors and small inputs. 
- The sets are <span class='red'>big-omega</span>, big-theta, and <span class='skyblue'>big-oh</span>:
  - <span class='red'>&Omega;</span>(*g*): functions that grow ***at least as fast*** as g
  - &Theta;(*g*): functions that grow ***at the same rate*** as g
  - <span class='skyblue'>O</span>(*g*): functions that grow ***no faster than*** g

![venn diagram](images/04-arrays-bigoh/venn-diagram.png)
	  </script></section>

	  <section data-markdown><script type="text/template">
## Why do we care?
- Some data structures are faster than others
  - Each data structure has some operations that are fast, and some operations that are slow
- We need a way to compare them
- This allows us to:
  - Better choose the data structures that we will use
  - Better design additional data structures
	  </script></section>

	  <section data-markdown><script type="text/template">
## Input sizes
- Your algorithm does not matter if you have 10 elements
  - A [bogosort](http://en.wikipedia.org/wiki/Bogosort) will work just fine
- Consider big input sizes:
  - UVa's e-mail probably has about 100,000 e-mail addresses
  - [OpenStreetMap](http://www.openstreetmap.org), for driving routes, has over 4.7 billion nodes and 6.4 million GPS points ([ref](http://www.openstreetmap.org/stats/data_stats.html)) (as of Sep 2018)
    - A 'node' is a point on the Earth's surface, and not necessarily an intersection
  - Google's index has almost 47 billion items ([ref](http://www.worldwidewebsize.com/)) (as of Sep 2018)
	  </script></section>

	  <section data-markdown><script type="text/template">
## Even for smaller input sizes...
- This is from the first lecture set
- All times are in ms (1/1000<sup>th</sup> of a second)
- Source code: [DataStructureComparison.java](../uva/code/introduction//DataStructureComparison.java.html) ([src](../uva/code/introduction//DataStructureComparison.java))
&nbsp;  
&nbsp;  

| Data Structure | Total time | Insert time | Search time | Delete time |
|----------------|------------|-------------|-------------|-------------|
| Vector         | 17,311     | 30          | 12,620      | 4,661       |
| ArrayList      | 17,281     | 28          | 12,609      | 4,644       |
| LinkedList     | 24,255     | 54          | 17,934      | 6,267       |
| HashSet        | 122        | 103         | 9           | 10          |
| TreeSet        | 119        | 93          | 14          | 12          |
| Vector, sorted | 294        | 36          | 19          | 239         |

&nbsp;  
	  </script></section>

	  <section data-markdown><script type="text/template">
## Assumptions
- We have measured the running time of our program with different input sizes, and that result is encapsulated in some function *f*(*n*)
  - *n* is the input size, and is always a positive integer
  - We may not know *f*(*n*) exactly, but we still will want to classify it
- We have some other function *g* that we want to compare our program to
- So we will compare *f*(*n*) to *g*(*n*), such as:
  - *f*(*n*) &isin; O(*g*(*n*))
    - We could shorten this: *f* &isin; O(*g*)
  - *f*(*n*) &notin; &Omega;(*g*(*n*))
    - We could shorten this: *f* &notin; &Omega;(*g*)
	  </script></section>

	  <section data-markdown><script type="text/template">
## Worst-case Scenario
- We always analyze the ***worst case*** run-time
  - It makes no sense to analyze the best case, as that is rarely likely to happen
  - And the *average* case (if you could even define what that is) may not be representative and is not used in these analyses either
- A more formal definition of *worst case*, should you be interested, can be found [here](http://en.wikipedia.org/wiki/Worst_case)
	  </script></section>

	  <section>
<h2>The Sets O(<i>g</i>), &Theta;(<i>g</i>), &Omega;(<i>g</i>)</h2>
<ul>
<li>Let <i>f</i> and <i>g</i> be a functions from the non-negative integers into the positive real numbers</li>
<li>For some real constant <i>c</i> > 0 and some non-negative integer constant <i>n</i><sub>0</sub><ul>
  <li>O(<i>g</i>) is the set of functions <i>f</i>, such that:<ul>
    <li><i>f</i>(<i>n</i>) <span class='red'>&le;</span> c * <i>g</i>(<i>n</i>) for all <i>n</i> &ge; <i>n</i><sub>0</sub></li></ul></li>
  <li>&Omega;(<i>g</i>) is the set of functions <i>f</i>, such that:<ul>
    <li><i>f</i>(<i>n</i>) <span class='red'>&ge;</span> c * <i>g</i>(<i>n</i>) for all <i>n</i> &ge; <i>n</i><sub>0</sub></li></ul></li>
  <li>&Theta;(<i>g</i>) = O(<i>g</i>) &cap; &Omega;(<i>g</i>)<ul>
    <li>&Theta;(<i>g</i>) is the asymptotic order of g or the <i>order</i> of g</li>
    <li><i>f</i> &isin; &Theta;(<i>g</i>) read as &quot;<i>f</i> is asymptotic order <i>g</i>&quot; or &quot;<i>f</i> is order <i>g</i>&quot;</li></ul></li>
</ul></li>
</ul>
<!-- <script type="text/javascript">insertCanvas();</script> -->
	  </section>

	  <section data-markdown><script type="text/template">
## Asymptotic Bounds
- For the sets big-oh O(*g*), big-theta &Theta;(*g*), and big-omega &Omega;(*g*), remember these meanings:
  - O(*g*): functions that grow ***no faster than*** *g*, or ***asymptotic upper bound***
  - &Omega;(*g*): functions that grow ***at least as fast as*** *g*, or ***asymptotic lower bound***
  - &Theta;(*g*): functions that grow ***at the same rate as*** *g*, or ***asymptotic tight bound***
	  </script></section>

	  <section>
<h2>Big-Oh Examples</h2>
<p><i>f</i>(<i>n</i>) &isin; O(<i>g</i>(<i>n</i>)) means that there are positive constants <i>c</i> and <i>n</i><sub>0</sub> such that <i>f</i>(<i>n</i>) &le; <i>c</i>*<i>g</i>(<i>n</i>) for all values <i>n</i> &ge; <i>n</i><sub>0</sub></p>
&nbsp;
<ul>
<li>Is <i>n</i> &isin; O(<i>n</i><sup>2</sup>)?
  <ul><li class="fragment">Yes, <i>c</i> = 1, <i>n</i><sub>0</sub> = 2 works fine</li></ul></li>
<li>Is 10<i>n</i> &isin; O(<i>n</i>)?
  <ul><li class="fragment">Yes, <i>c</i> = 11, <i>n</i><sub>0</sub> = 2 works fine</li></ul></li>
<li>Is <i>n</i><sup>2</sup> &isin; O(<i>n</i>)?
  <ul><li class="fragment">No; no matter what values for <i>c</i> and <i>n</i><sub>0</sub> we pick, <i>n</i><sup>2</sup> &gt; <i>c</i>*<i>n</i> for big enough <i>n</i></li></ul></li>
</ul>
<script type="text/javascript">insertCanvas();</script>
	  </section>

	  <section>
<h2>Given <i>f</i> &isin; O(<i>h</i>) and <i>g</i> &notin; O(<i>h</i>),<br>which of these are true?</h2>
<ol>
<li>For <b><i>all</i></b> positive integers <i>m</i>, <i>f</i>(<i>m</i>) &lt; <i>g</i>(<i>m</i>).</li>
<li>For <b><i>some</i></b> positive integer <i>m</i>, <i>f</i>(<i>m</i>) &lt; <i>g</i>(<i>m</i>).</li>
<li>For <b><i>some</i></b> positive integer <i>m</i><sub>0</sub>, and <b><i>all positive</i></b> integers <i>m</i> > <i>m</i><sub>0</sub>,  <i>f</i>(<i>m</i>) &lt; <i>g</i>(<i>m</i>).</li>
<li>1 and 2</li>
<li>2 and 3</li>
<li>1 and 3</li>
</ol>
<script type="text/javascript">insertCanvas();</script>
	  </section>

	  <section data-markdown><script type="text/template">
## 1 is false: Prove by Counter-Example
- Recall that *f* &isin; O(*h*) and *g* &notin; O(*h*)
- We are disproving: for ***all*** positive integers m, *f*(*m*) < *g*(*m*):
  - To disprove by counter-example, you have to find just ***one*** example that does not fulfill the stated claim
  - Pick *f*(*n*) = 5\**n*, *g*(*n*) = *n*<sup>3</sup>, *h*(*n*) = *n*<sup>2</sup>
    - *f* &isin; O(*h*), as 5\**n* &le; *c* \* *n*<sup>2</sup> &nbsp;&nbsp;&forall; *n* > *n*<sub>0</sub> when *c* = 6 and *n*<sub>0</sub> = 2
    - *g* &notin; O(*h*), as *n*<sup>3</sup> &le; *c* \* *n*<sup>2</sup> &nbsp;&nbsp;&forall; *n* > *n*<sub>0</sub> cannot be satisifed, for large *n*, with any values of *c* and *n*<sub>0</sub>
  - For *m*=2, *f*(*m*) = 10 > 8 = *g*(*m*)
  - Therefore, this is false
- *f*(*n*) &isin; O(*g*(*n*)) means there are positive constants *c* and *n*<sub>0</sub> such that *f*(*n*) &le; *c*\**g*(*n*) for all values *n* &ge; *n*<sub>0</sub> (but not ***all*** values of *n*, which is waht this was trying to show)
	  </script></section>

	  <section data-markdown><script type="text/template">
## 2 is true: Intuition
- Recall that *f* &isin; O(*h*) and *g* &notin; O(*h*)
- Claim: for ***some*** positive integer *m*, *f*(*m*) < *g*(*m*)
- We know:
  - *g* must grow faster than *h*, otherwise *g* would be in O(*h*)
  - *f* must grow no faster than *h*, since *f* &isin; O(*h*)
    - Rephrased: *f* grows as slow as, or slower than, *h*
- So, if *g* grows faster than *h*, but *f* grows as slow or slower than *h*, eventually, then:
  - *g* will grow faster than *f*
  - Or *g*(*n*) > *f*(*n*) for some large *n*
  - Thus, for some *m*, *f*(*m*) < *g*(*m*)
	  </script></section>

	  <section data-markdown><script type="text/template">
## 3 is true: Deduction
- Recall that *f* &isin; O(*h*) and *g* &notin; O(*h*)
- Claim: for ***some*** positive integer *m*<sub>0</sub>, and ***all positive*** integers *m* > *m*<sub>0</sub>,  *f*(*m*) < *g*(*m*)
- We know, from the last slide, that *g* will grow faster than *f*
  - This is essentially the definition of big-oh, which is what the claim is basically stating: that *f* &isin; O(*g*)
	  </script></section>

	  <section data-markdown><script type="text/template">
## Given *f* &isin; O(*h*) and *g* &notin; O(*h*),<br>which of these are true?
1. For ***all*** positive integers *m*, *f*(*m*) < *g*(*m*).
2. For ***some*** positive integer *m*, *f*(*m*) < *g*(*m*).
3. For ***some*** positive integer *m*<sub>0</sub>, and ***all positive*** integers *m* > *m*<sub>0</sub>,  *f*(*m*) < *g*(*m*).
4. 1 and 2
5. 2 and 3
6. 1 and 3
	  </script></section>

	  <section data-markdown><script type="text/template">
## Lower Bound: &Omega; (Omega)
- *f*(*n*) &isin; &Omega; (*g*(*n*)) means:
  - There are positive constants *c* and *n*<sub>0</sub> such that *f*(*n*) <span class='red'>&ge;</span> *c*\*g*(*n*) for all *n* &ge; *n*<sub>0</sub> 
  - The difference from big-oh is the &ge; in big-omega versus &le; in big-oh
- This is a *lower* bound
	  </script></section>

	  <section>
<img class="stretch" alt="big-omega" src="images/04-arrays-bigoh/omega-diagram-1.png">
	  </section>

	  <section data-transition="fade" data-transition-speed="fast">
<img class="stretch" alt="big-omega" src="images/04-arrays-bigoh/omega-diagram-2.png">
	  </section>

	  <section data-markdown><script type="text/template">
## Theta ("Order of")
- Intuition: the set &Theta;(*f*) is the set of functions that grow ***as fast as*** *f*
- Definition: *f*(*n*) &isin; &Theta;(*g*(*n*)) if and only if both: 
  1. *f*(*n*) &isin; O (*g*(*n*))
  2. *f*(*n*) &isin; &Omega; (*g*(*n*))
- Note that we do not have to pick the same *c* and *n*<sub>0</sub> values for cases 1 and 2
- When we say, "*f* is order *g*" that means *f*(*n*) &isin; &Theta; (*g*(*n*))
	  </script></section>

	  <section>
<img class="stretch" alt="big-theta" src="images/04-arrays-bigoh/theta-diagram.png">
	  </section>

	  <section data-markdown><script type="text/template">
## Little-oh
- Let *f*(*n*) &isin; <span class='red'>o</span>(*g*(*n*))
  - Any function that is o(*g*) is also O(*g*)
    - So both act as an upper bound
  - But a O(*g*) bound can also be &Omega;(*g*) and thus &Theta;(*g*)
    - Meaning the big-oh bound can be tight
  - Little-oh means the bound can not be tight
    - Thus, it's O(*g*) but not &Theta;(*g*)
    - Or, that *f*(*n*) will ***always*** be ***strictly*** less than *g*(*n*):
      - *f*(*n*) <span class='red'>&lt;</span> *c* \* *g*(*n*) for all *n* &ge; *n*<sub>0</sub>, for some constants *c* and *n*<sub>0</sub>
  - Rarely used in computer science...
	  </script></section>

	  <section data-markdown><script type="text/template">
## Little-omega
- Analogous to little-oh
  - It's a lower bound that cannot be a tight lower bound
  - If something is &omega;(*g*), then it's &Omega;(*g*) but not &Theta;(*g*)
- Rarely used in computer science...
- What does this mean for little-theta?
	  </script></section>

	  <section>
<h2>Another Way to Define Order Classes</h2>
<ul>
<li>Comparing <i>f</i>(<i>n</i>) and <i>g</i>(<i>n</i>) as <i>n</i> approaches infinity...</li>
<li>If \( \lim{n \to \infty}\frac{f(n)}{g(n)} \) is:
<ul>
  <li>&lt; &infin;, including the case in which the limit is 0, then <i>f</i> &isin; O(<i>g</i>)</li>
  <li>&gt; 0, including the case in which the limit is &infin;, then <i>f</i> &isin; &Omega;(<i>g</i>)</li>
  <li>= <i>c</i> and 0 &lt; <i>c</i> &lt; &infin; then <i>f</i> &isin; &Theta;(<i>g</i>)</li>
  <li>= 0  then <i>f</i> &isin; o(<i>g</i>)<ul><li>read as &quot;little-oh of <i>g</i>&quot;</li></ul></li>
  <li>= &infin;  then <i>f</i> &isin; &omega;(<i>g</i>)<ul><li>read as &quot;little-omega of <i>g</i>&quot;</li></ul></li>
</ul></li></ul>
	  </section>

	  <section data-markdown><script type="text/template">
## A note about notation
- Stating *f*(*n*) = O(*g*(*n*)) implies an equality
  - This is a bit of an abuse of the notation, as O(*g*(*n*)) is a ***set***
  - It should be *f*(*n*) &isin; O(*g*(*n*))
	  </script></section>

	  <section data-markdown><script type="text/template">
## Some Properties of O(*g*), &Theta;(*g*), &Omega;(*g*)
- Reversing big-oh: *f* &isin; O(*g*) &hArr; *g* &isin; &Omega;(*f*)
- Adding big-oh: O(*f* + *g*) = O(max(*f*, *g*)) 
  - Similar equations hold for &Omega; and &Theta;
- Transitive: If *f* &isin; O(*g*) and *g* &isin; O(*h*), then *f* &isin; O(*h*)
  - Big-Oh is transitive, as are all the others (big-theta, big-omega, little-oh, and little-omega)
- Reflexive: *f* &isin; &Theta;(*f*)
  - As are big-oh and big-omega
- Symmetric: If *f* &isin; &Theta;(*g*), then *g* &isin; &Theta;(*f*)
  - Big-Oh is not symmetric!  (neither are &Omega;, o, or &omega;)
- &Theta; defines an ***equivalence relation*** on the functions
  - Each set &Theta;(*f*) is an equivalence class (complexity class)
	  </script></section>

	  <section data-markdown><script type="text/template">
## Classification of functions
- &Theta;(1) denotes the set of functions bounded by a constant (for large *n*)
  - *f* &isin; &Theta;(log *n*) means *f* is logarithmic
  - *f* &isin; &Theta;(*n*) means *f* is linear
  - *f* &isin; &Theta;(*n* log *n*) means *f* is log-linear
  - *f* &isin; &Theta;(*n*<sup>2</sup>) means *f* is quadratic
  - *f* &isin; &Theta;(*n*<sup>3</sup>) means *f* is cubic
  - *f* &isin; &Theta;(2<sup>*n*</sup>) means *f* is exponential
- *n*<sup>*k*</sup> &isin; <span class='red'>o</span>(*c*<sup>*n*</sup>) for any *k* > 0 and any *c* > 1
  - Powers of *n* grow slower than any exponential function *c*<sup>*n*</sup>
    - As long as *n* &ge; 1 and *c* &ge; 1, which is always the case with input sizes
	  </script></section>

	  <section data-markdown><script type="text/template">
## A note about logs
- The difference between log<sub>10</sub>*n* and log<sub>2</sub>*n* is always a constant
  - Specifically, about 3.322
  - Since we don't care about constants in these analyses, we'll ignore the log base
- Most things in computer science are log base 2 anyway...
	  </script></section>

	  <section data-markdown><script type="text/template">
## Typical Growth Rates

| Function | Name |
|----------|------|
| *c* | constant |
| log *n* | logarithmic |
| *n* | linear |
| *n* log *n* | log-linear |
| *n*<sup>2</sup> | quadratic |
| *n*<sup>3</sup> | cubic |
| 2<sup>*n*</sup> | exponential |
	  </script></section>

	  <section data-markdown><script type="text/template">
## Does Order Class Matter?
- No, not for small inputs
- Yes, for many real problems
	  </script></section>

	  <section data-markdown><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-1.png)
	  </script></section>

	  <section data-markdown data-transition="fade" data-transition-speed="fast">><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-2.png)
	  </script></section>

	  <section data-markdown data-transition="fade" data-transition-speed="fast">><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-3.png)
	  </script></section>

	  <section data-markdown data-transition="fade" data-transition-speed="fast">><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-4.png)
	  </script></section>

	  <section data-markdown data-transition="fade" data-transition-speed="fast">><script type="text/template">
## Practical Complexity
![graph](images/04-arrays-bigoh/graph-5.png)
	  </script></section>

	  <section data-markdown><script type="text/template">
## General Rules for Running<br>Time Calculations (Big-Oh)
- For loops
  - The number of times the for loop runs times the total running time of the statements inside the for loop
- Nested loops
  - Analyze from inside to out
    - Runtime of the statement \* product of the sizes of the loops
- Consecutive statements
  - Take the max runtime of the individual statements
- if/else
  - Time for the test plus the longer of the runtimes
	  </script></section>

	</section>


	<section>

	  <section id="runningtimes" data-markdown class="center"><script type="text/template">
# Determining Running Times
	  </script></section>

	  <section data-markdown><script type="text/template">
## How to tell the running time...
- You need to imagine how long the algorithm would run given varying input sizes
  - We will assume our input size is *n*
- You should ask yourself...
  - How long will this algorithm take when *n* = 1?
  - When *n* = 100?
  - When *n* = 1,000,000,000?
- Recall that we always analyze the worst case!
	  </script></section>

	  <section data-markdown><script type="text/template">
## Running time: constant
- Constant (&Theta;(1)) time means that the running time does NOT depend on the size of the input
- Examples:
  - Getting a vector's size
  - Linked list insert or delete (at the known end)
  - Finding the k<sup>th</sup> element in an array or vector
	  </script></section>

	  <section data-markdown><script type="text/template">
## Running time: logarithmic
- Logarithmic (&Theta;(log *n*)) running time is when the search space is cut in half in each iteration
  - Or when you are iterating up or down a data structure, such as a tree, that has &Theta;(log *n*) height
- Examples:
  - Binary search in a sorted array or vector
  - Search (or insert or delete) with a balanced tree
    - Balanced tree means an AVL tree or Red-black tree
	  </script></section>

	  <section data-markdown><script type="text/template">
## Running time: linear
- Linear (&Theta;(*n*)) running time means that we have to process each element, and there is one step (or a constant number of steps) for each one
- Examples:
  - Printing (or otherwise iterating through) a list
  - Finding an element in an unsorted array or vector
  - Finding an element in a sorted or unsorted linked list
  - Doubling the size of a vector's underlying array
	  </script></section>

	  <section data-markdown><script type="text/template">
## Running time: Log-linear
- Log-linear (&Theta;(*n* log *n*)) typically occurs when you are going to take a linear number of steps, but each one takes log *n* time
  - Or log *n* steps, each one of which takes *n* time
- Examples:
  - Fast sorts: merge sort and heap sort
  - Quicksort, on a good day (but this is not guaranteed!)
  - Inserting *n* elements into a data structure where each insert takes log *n* time
	  </script></section>

	  <section data-markdown><script type="text/template">
## Running time: quadratic
- A quadratic (&Theta;(*n*<sup>2</sup>)) running time occurs when, for each input, you have to search through the entire input again
  - (among other ways it can occur)
- Examples:
  - Slow sorts: insertion sort, bubble sort, selection sort
  - Quicksort, on a bad day
  - Some graph algorithms we'll see later this semester
  - Doubly nested for loops (where each loop goes 1 to *n*)
	  </script></section>

	  <section data-markdown><script type="text/template">
## Running time: exponential
- An exponential (&Theta;(2<sup>*n*</sup>)) running time typically means you are going to try every possible solution, and there are 2<sup>*n*</sup> of them
- Examples (we have not seen any yet)
  - Trying to crack a binary combination lock by trying every single possibility
  - Traveling salesperson problem (we'll see that later this semester)
  - Satisfiability of a Boolean expression
	  </script></section>

	</section>


      </div>

    </div>

    <script src='../slides/reveal.js/dist/reveal.js'></script><script src='../slides/reveal.js/plugin/zoom/zoom.js'></script><script src='../slides/reveal.js/plugin/notes/notes.js'></script><script src='../slides/reveal.js/plugin/search/search.js'></script><script src='../slides/reveal.js/plugin/markdown/markdown.js'></script><script src='../slides/reveal.js/plugin/highlight/highlight.js'></script><script src='../slides/reveal.js/plugin/math/math.js'></script>
    <script src="js/settings.js"></script>

  </body>
</html>