Skip to content

Commit a279f97

Browse files
davpintodavpinto
committed
Create simulation for gradient boosting iterations
1 parent 0c02842 commit a279f97

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

74 files changed

+633
-8
lines changed

‎README.Rmd

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,22 +11,30 @@ knitr::opts_chunk$set(echo = TRUE, fig.align = 'center')
1111
1212
This repository contains the source code for **animated visualizations** of some famous **machine learning** algorithms. They were created using the `R` package `animation`, and ilustrate algorithm convergence and the effect of hyper-parameter tuning. The simulations available so far are:
1313

14+
- **XGBoost** decision boundary as **iterations** proceed.
15+
16+
```{r, out.width="75%", fig.align='center', echo=FALSE}
17+
knitr::include_graphics("./img/xgb-iterations.jpg")
18+
```
19+
20+
- **KNN** decision boundary varying the number of nearest neighbors $k$.
21+
1422
- Multivariate **Gaussian Mixture Models** (GMMs) fitting by **EM algorithm**.
1523

1624
- Multimodal **Density Estimation** using GMMs.
1725

1826
- **Tikhonov Regularization** effect in **Extreme Learning Machines** (ELMs).
1927

2028
```{r, out.width="75%", fig.align='center', echo=FALSE}
21-
knitr::include_graphics("./img/elm-regularization.jpeg")
29+
knitr::include_graphics("./img/elm-regularization.jpg")
2230
```
2331

2432
- **Image Segmentation** using **K-means**.
2533

2634
- Image **Reconstruction** using **Principal Components Analysis** (PCA).
2735

2836
```{r, out.width="75%", fig.align='center', echo=FALSE}
29-
knitr::include_graphics("./img/pca-reconstruction.jpeg")
37+
knitr::include_graphics("./img/pca-reconstruction.jpg")
3038
```
3139

3240
****

‎README.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,19 +5,25 @@ Machine Learning Simulations
55
66
This repository contains the source code for **animated visualizations** of some famous **machine learning** algorithms. They were created using the `R` package `animation`, and ilustrate algorithm convergence and the effect of hyper-parameter tuning. The simulations available so far are:
77

8+
- **XGBoost** decision boundary as **iterations** proceed.
9+
10+
<img src="./img/xgb-iterations.jpg" width="75%" style="display: block; margin: auto;" />
11+
12+
- **KNN** decision boundary varying the number of nearest neighbors *k*.
13+
814
- Multivariate **Gaussian Mixture Models** (GMMs) fitting by **EM algorithm**.
915

1016
- Multimodal **Density Estimation** using GMMs.
1117

1218
- **Tikhonov Regularization** effect in **Extreme Learning Machines** (ELMs).
1319

14-
<img src="./img/elm-regularization.jpeg" width="75%" style="display: block; margin: auto;" />
20+
<img src="./img/elm-regularization.jpg" width="75%" style="display: block; margin: auto;" />
1521

1622
- **Image Segmentation** using **K-means**.
1723

1824
- Image **Reconstruction** using **Principal Components Analysis** (PCA).
1925

20-
<img src="./img/pca-reconstruction.jpeg" width="75%" style="display: block; margin: auto;" />
26+
<img src="./img/pca-reconstruction.jpg" width="75%" style="display: block; margin: auto;" />
2127

2228
------------------------------------------------------------------------
2329

‎docs/index.Rmd

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,20 @@ knitr::opts_chunk$set(echo = TRUE)
1717
1818
Here we present beautiful animated visualizations for some famous machine learning algorithms, built with the `R` package `animation`. The source code is available on [GitHub](https://github.com/davpinto/ml-simulations).
1919

20+
## Gradient Boosting Machine Iterations
21+
22+
Classification **Decision Boundary** of the **Gradient Boosting Machine** (GBM) from the `R` package `xgboost` as boosting iterations proceed.
23+
24+
<iframe src="./simul/xgb-iterations/index.html" frameborder="0" height="600" width="100%"></iframe>
25+
26+
*__[+ Source Code](https://github.com/davpinto/ml-simulations/tree/master/simulations/xgb-iterations)__*
27+
28+
----
29+
30+
## K-Nearest Neighbors Classifier
31+
32+
Analysing how the the number of nearest neighbors $k$ affects the classification **Decision Boundary** of the **KNN** algorithm from the `R` package [fastknn](https://davpinto.github.io/fastknn).
33+
2034
## Expectation-Maximization Algorithm
2135

2236
**Gaussian Mixture Model** (GMM) fitted by **Expectation-Maximization** (EM) algorithm with random initialization.

‎docs/index.html

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,8 @@ <h1 class="title toc-ignore">Animated Visualizations for Machine Learning Algori
106106

107107
<div id="TOC">
108108
<ul>
109+
<li><a href="#gradient-boosting-machine-iterations">Gradient Boosting Machine Iterations</a></li>
110+
<li><a href="#k-nearest-neighbors-classifier">K-Nearest Neighbors Classifier</a></li>
109111
<li><a href="#expectation-maximization-algorithm">Expectation-Maximization Algorithm</a></li>
110112
<li><a href="#gaussian-mixture-density">Gaussian Mixture Density</a></li>
111113
<li><a href="#regularized-extreme-learning-machine">Regularized Extreme Learning Machine</a></li>
@@ -119,6 +121,18 @@ <h1 class="title toc-ignore">Animated Visualizations for Machine Learning Algori
119121
<p>Simulation of famous machine learning algorithms</p>
120122
</blockquote>
121123
<p>Here we present beautiful animated visualizations for some famous machine learning algorithms, built with the <code>R</code> package <code>animation</code>. The source code is available on <a href="https://github.com/davpinto/ml-simulations">GitHub</a>.</p>
124+
<div id="gradient-boosting-machine-iterations" class="section level2">
125+
<h2>Gradient Boosting Machine Iterations</h2>
126+
<p>Classification <strong>Decision Boundary</strong> of the <strong>Gradient Boosting Machine</strong> (GBM) from the <code>R</code> package <code>xgboost</code> as boosting iterations proceed.</p>
127+
<iframe src="./simul/xgb-iterations/index.html" frameborder="0" height="600" width="100%">
128+
</iframe>
129+
<p><em><strong><a href="https://github.com/davpinto/ml-simulations/tree/master/simulations/xgb-iterations">+ Source Code</a></strong></em></p>
130+
<hr />
131+
</div>
132+
<div id="k-nearest-neighbors-classifier" class="section level2">
133+
<h2>K-Nearest Neighbors Classifier</h2>
134+
<p>Analysing how the the number of nearest neighbors <span class="math inline">\(k\)</span> affects the classification <strong>Decision Boundary</strong> of the <strong>KNN</strong> algorithm from the <code>R</code> package <a href="https://davpinto.github.io/fastknn">fastknn</a>.</p>
135+
</div>
122136
<div id="expectation-maximization-algorithm" class="section level2">
123137
<h2>Expectation-Maximization Algorithm</h2>
124138
<p><strong>Gaussian Mixture Model</strong> (GMM) fitted by <strong>Expectation-Maximization</strong> (EM) algorithm with random initialization.</p>

‎docs/simul/xgb-iterations/css/reset.css

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
/* http://meyerweb.com/eric/tools/css/reset/ */
2+
/* v1.0 | 20080212 */
3+
4+
html, body, div, span, applet, object, iframe,
5+
h1, h2, h3, h4, h5, h6, p, blockquote, pre,
6+
a, abbr, acronym, address, big, cite, code,
7+
del, dfn, em, font, img, ins, kbd, q, s, samp,
8+
small, strike, strong, sub, sup, tt, var,
9+
b, u, i, center,
10+
dl, dt, dd, ol, ul, li,
11+
fieldset, form, label, legend,
12+
table, caption, tbody, tfoot, thead, tr, th, td {
13+
margin: 0;
14+
padding: 0;
15+
border: 0;
16+
outline: 0;
17+
font-size: 100%;
18+
vertical-align: baseline;
19+
background: transparent;
20+
}
21+
body {
22+
line-height: 1;
23+
}
24+
ol, ul {
25+
list-style: none;
26+
}
27+
blockquote, q {
28+
quotes: none;
29+
}
30+
blockquote:before, blockquote:after,
31+
q:before, q:after {
32+
content: '';
33+
content: none;
34+
}
35+
36+
/* remember to define focus styles! */
37+
:focus {
38+
outline: 0;
39+
}
40+
41+
/* remember to highlight inserts somehow! */
42+
ins {
43+
text-decoration: none;
44+
}
45+
del {
46+
text-decoration: line-through;
47+
}
48+
49+
/* tables still need 'cellspacing="0"' in the markup */
50+
table {
51+
border-collapse: collapse;
52+
border-spacing: 0;
53+
}

‎docs/simul/xgb-iterations/css/scianimator.blue.css

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
/**
2+
* SciAnimator - Scientific Image Animator Plugin for jQuery
3+
*
4+
* Copyright (c) 2010 Brent Ertz
5+
* Released under the MIT license.
6+
* http://github.com/brentertz/scianimator
7+
*/
8+
9+
/* Blue theme */
10+
.scianimator.blue,
11+
.scianimator.blue a,
12+
.scianimator.blue a:visited {
13+
color: #3985a8;
14+
}
15+
.scianimator.blue {
16+
background: #ddeef6;
17+
}
18+
.scianimator.blue .controls,
19+
.scianimator.blue .control {
20+
background-color: #ddeef6;
21+
border: 1px solid #89bad2;
22+
color: #3985a8;
23+
text-shadow: 0 1px #fff;
24+
}
25+
.scianimator.blue .controls {
26+
box-shadow: inset 0 1px 3px #fff, inset 0 -17px #cbe6f2, 0 0 3px #89bad2;
27+
-o-box-shadow: inset 0 1px 3px #fff, inset 0 -17px #cbe6f2, 0 0 3px #89bad2;
28+
-webkit-box-shadow: inset 0 1px 3px #fff, inset 0 -17px #cbe6f2, 0 0 3px #89bad2;
29+
-moz-box-shadow: inset 0 1px 3px #fff, inset 0 -17px #cbe6f2, 0 0 3px #89bad2;
30+
}
31+
.scianimator.blue .control {
32+
box-shadow: inset 0 1px 3px #fff, inset 0 -10px #cbe6f2, 0 0 3px #89bad2;
33+
-o-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #cbe6f2, 0 0 3px #89bad2;
34+
-webkit-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #cbe6f2, 0 0 3px #89bad2;
35+
-moz-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #cbe6f2, 0 0 3px #89bad2;
36+
}
37+
.scianimator.blue a:hover {
38+
box-shadow: inset 0 1px 3px #fff, inset 0 -10px #cbe6f2, 0 0 3px #3985a8;
39+
-o-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #cbe6f2, 0 0 3px #3985a8;
40+
-webkit-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #cbe6f2, 0 0 3px #3985a8;
41+
-moz-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #cbe6f2, 0 0 3px #3985a8;
42+
color: #fff;
43+
text-shadow: 0px 1px #3985a8;
44+
}

‎docs/simul/xgb-iterations/css/scianimator.css

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
/**
2+
* SciAnimator - Scientific Image Animator Plugin for jQuery
3+
*
4+
* Copyright (c) 2010 Brent Ertz
5+
* Released under the MIT license.
6+
* http://github.com/brentertz/scianimator
7+
*/
8+
9+
/* Comment out @imports for themes not in use */
10+
@import url('scianimator.light.css');
11+
@import url('scianimator.dark.css');
12+
@import url('scianimator.blue.css');
13+
14+
.scianimator {
15+
font-family: 'Lucida Grande', sans-serif;
16+
font-size: 12px;
17+
text-align: center;
18+
margin: auto auto 1em auto;
19+
position: relative;
20+
}
21+
.scianimator .controls,
22+
.scianimator .control {
23+
border: 1px solid;
24+
border-radius: 4px;
25+
-moz-border-radius: 4px;
26+
font-family: 'Lucida Grande', sans-serif;
27+
font-size: 10px;
28+
text-shadow: 0 1px #fff;
29+
margin: 2px;
30+
position: relative;
31+
}
32+
.scianimator .controls {
33+
margin: 0;
34+
padding: 4px;
35+
position: relative;
36+
}
37+
.scianimator a,
38+
.scianimator a:visited {
39+
cursor: pointer;
40+
display: inline-block;
41+
padding: 5px 10px;
42+
text-decoration: none;
43+
}
44+
.scianimator a.small,
45+
.scianimator a:visited.small {
46+
padding: 1px 2px;
47+
}
48+
.scianimator a:active {
49+
top: 1px;
50+
}
51+
.scianimator select.control,
52+
.scianimator label {
53+
margin: 1px;
54+
vertical-align: middle;
55+
}
56+
.scianimator .control.box {
57+
padding: 1px 5px;
58+
display: inline-block;
59+
vertical-align: top;
60+
}
61+
.scianimator .control.box .control {
62+
margin-left: 0;
63+
margin-right: 0;
64+
}
65+
66+
.scianimator .box.control.navigator {
67+
padding: 2px;
68+
}
69+
70+
.scianimator .box a.control.navigator,
71+
.scianimator .box a:hover.control.navigator {
72+
display: inline-block;
73+
height: 10px;
74+
margin: 2px;
75+
padding: 0;
76+
width: 10px;
77+
78+
background: #00FF00;
79+
box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px #00FF00;
80+
-o-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px #00FF00;
81+
-webkit-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px #00FF00;
82+
-moz-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px #00FF00;
83+
}
84+
.scianimator .box a.navigator.disabled,
85+
.scianimator .box a:hover.navigator.disabled {
86+
background: red;
87+
box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px red;
88+
-o-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px red;
89+
-webkit-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px red;
90+
-moz-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px red;
91+
}
92+
.scianimator .box a.navigator.current,
93+
.scianimator .box a:hover.navigator.current {
94+
background: blue;
95+
box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px blue;
96+
-o-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px blue;
97+
-webkit-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px blue;
98+
-moz-box-shadow: 0 0 3px 1px #666666 inset, 0 0 1px blue;
99+
}
100+
101+
.scianimator .status {
102+
background: red;
103+
color: #fff;
104+
right: 0;
105+
padding: 5px;
106+
position: absolute;
107+
top: 0;
108+
}

‎docs/simul/xgb-iterations/css/scianimator.dark.css

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
/**
2+
* SciAnimator - Scientific Image Animator Plugin for jQuery
3+
*
4+
* Copyright (c) 2010 Brent Ertz
5+
* Released under the MIT license.
6+
* http://github.com/brentertz/scianimator
7+
*/
8+
9+
/* Dark theme */
10+
.scianimator.dark,
11+
.scianimator.dark a,
12+
.scianimator.dark a:visited {
13+
color: #ccc;
14+
}
15+
.scianimator.dark {
16+
background: #333;
17+
}
18+
.scianimator.dark .controls,
19+
.scianimator.dark .control {
20+
background-color: #333;
21+
border-color: #666;
22+
color: #ccc;
23+
text-shadow: 0 1px #000;
24+
}
25+
.scianimator.dark .controls {
26+
box-shadow: inset 0 1px 3px #fff, inset 0 -17px #000, 0 0 3px #000;
27+
-o-box-shadow: inset 0 1px 3px #fff, inset 0 -17px #000, 0 0 3px #000;
28+
-webkit-box-shadow: inset 0 1px 3px #fff, inset 0 -17px #000, 0 0 3px #000;
29+
-moz-box-shadow: inset 0 1px 3px #fff, inset 0 -17px #000, 0 0 3px #000;
30+
}
31+
.scianimator.dark .control {
32+
box-shadow: inset 0 1px 3px #fff, inset 0 -10px #000, 0 0 3px #000;
33+
-o-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #000, 0 0 3px #000;
34+
-webkit-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #000, 0 0 3px #000;
35+
-moz-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #000, 0 0 3px #000;
36+
}
37+
.scianimator.dark a:hover {
38+
box-shadow: inset 0 1px 3px #fff, inset 0 -10px #000, 0 0 3px #555;
39+
-o-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #000, 0 0 3px #555;
40+
-webkit-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #000, 0 0 3px #555;
41+
-moz-box-shadow: inset 0 1px 3px #fff, inset 0 -10px #333, 0 0 3px #555;
42+
color: #fff;
43+
text-shadow: 0px 1px #000;
44+
}

0 commit comments

Comments
 (0)