Skip to content

Ch 24 - Add edge detection section #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 84 commits into from
Aug 20, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
1bbfb8a
Start converting prototype to react
jasmaa May 27, 2019
1e49a41
Finish convolution demo
jasmaa May 29, 2019
b75bd59
Improve UI, refactor code
jasmaa May 30, 2019
cbe0ee4
Add sobel image processing demo
jasmaa Jun 3, 2019
b048f1a
Improve sobel ui
jasmaa Jun 4, 2019
87778c9
Add gradient initial explanations
jasmaa Jun 4, 2019
3d99a57
Swap in image processing funcs in convolution demo, improve doc
jasmaa Jun 5, 2019
2c6de14
Fix grid input bug, generalize grid input ui
jasmaa Jun 6, 2019
cf7e77e
Start gradient demo
jasmaa Jun 6, 2019
2d963a9
Fix and improve gradient demo
jasmaa Jun 8, 2019
02d8d4c
Add topology demo
jasmaa Jun 11, 2019
967190a
Try to improve mobile support
jasmaa Jun 12, 2019
383756a
Make convolution demo more clear, replace max spread with reduce
jasmaa Jun 13, 2019
2a9a53c
Style changes, fix gradient drawing
jasmaa Jun 13, 2019
32901c3
Start suppression demo
jasmaa Jun 17, 2019
f37d358
Start pipeline demo
jasmaa Jun 19, 2019
5dda712
Add image uplaod and indicator to pipeline demo
jasmaa Jun 20, 2019
6c06ab1
Add movement control, refactor code
jasmaa Jun 20, 2019
21ce7db
UI and documentation improvements
jasmaa Jun 24, 2019
eaaec72
Start moving over explanations
jasmaa Jun 25, 2019
5f910f3
Replace third party media, fix grid input update bug
jasmaa Jun 27, 2019
b95f853
Implement remaining edge detection processes
jasmaa Jun 27, 2019
de86cc2
Tweaking explanations and reworking on convolution demo
jasmaa Jul 2, 2019
0898a19
Fix image processing bugs
jasmaa Jul 2, 2019
fbc3292
Start reworking gradient demo
jasmaa Jul 2, 2019
7e1c619
Add control UI to gradient demo
jasmaa Jul 3, 2019
c8771d9
Start replacing 3dcss with threejs in pipeline demo
jasmaa Jul 5, 2019
33f6eb0
Fix material update bug
jasmaa Jul 5, 2019
359aa9b
Re-implement pipeline demo with three
jasmaa Jul 8, 2019
3fc7e5d
Improve pipeline demo code and docs
jasmaa Jul 10, 2019
e0bd01f
Rework suppression demo
jasmaa Jul 11, 2019
c6d5b9b
Minor bug fixes, improve suppression demo
jasmaa Jul 12, 2019
fabb2e3
Add threshold section
jasmaa Jul 12, 2019
dc719d5
Add hysteresis and canny sections
jasmaa Jul 15, 2019
4512f83
Split into multiple pages, modify header injection
jasmaa Jul 17, 2019
ed46d47
Reworking convolution demo
jasmaa Jul 17, 2019
323575c
Reworking convolution topology demos
jasmaa Jul 18, 2019
4ee7950
Testing sticky footer, add breadcrumbs in injection
jasmaa Jul 19, 2019
fb296a1
Rewrite explanations
jasmaa Jul 19, 2019
a087da8
Rebuild vis with gray colorscale for surface
jasmaa Jul 19, 2019
d0f0d5b
Add opacity and boldness changes for gradient vectors
jasmaa Jul 22, 2019
72f2afd
Combine pillar and ramp diagrams in convolution demo
jasmaa Jul 22, 2019
f6637ca
Add out of bounds for convolution demo
jasmaa Jul 23, 2019
92fdba4
Extend 2d pipeline demo
jasmaa Jul 23, 2019
61fa32c
Add RGB demo
jasmaa Jul 24, 2019
05f512b
Add grayscale demo, improve readability
jasmaa Jul 25, 2019
043e9c6
Minor UI changes
jasmaa Jul 26, 2019
bcf1485
Start adding webcam
jasmaa Jul 29, 2019
cd02bb9
Clean up pipeline code and add webcam
jasmaa Jul 29, 2019
dbc1d57
Add pattern selection for sobel pipeline
jasmaa Jul 29, 2019
12dd2a1
Improving webcam performance
jasmaa Jul 29, 2019
7e5c286
Add pixel magnifier, fix image upload bug
jasmaa Jul 30, 2019
c570993
Change sobel colorscale to red-green
jasmaa Jul 31, 2019
5bc5fe1
Rehaul text for robust edge detection
jasmaa Jul 31, 2019
2aa9265
Improving mobile support
jasmaa Aug 1, 2019
dd020cf
Minor changes to suppression demo
jasmaa Aug 2, 2019
4170a2f
Separate magnifier component
jasmaa Aug 5, 2019
9a00aa4
Disable nav to section 3, pipeline improvements
jasmaa Aug 5, 2019
be66ec4
Modularize and lazy load demos
jasmaa Aug 5, 2019
ca07880
Minor changes to webcam
jasmaa Aug 6, 2019
217de34
Start condensing convolution demo
jasmaa Aug 6, 2019
88b8bc7
Minor improvements to convolution magnifier, add loading fallback
jasmaa Aug 7, 2019
bf1b0e8
Reorganize text and add noise demo
jasmaa Aug 7, 2019
899d920
Add default image reset, minor ui changes
jasmaa Aug 9, 2019
e2033f2
Try remove modules for mobile
jasmaa Aug 10, 2019
b9205aa
Fixing touch and sizing for mobile
jasmaa Aug 12, 2019
7cacfb1
Convert grids to canvas
jasmaa Aug 12, 2019
16bf6f7
Fixing mobile bugs
jasmaa Aug 12, 2019
4002101
Fixing touch scrolling bug
jasmaa Aug 12, 2019
337c19b
Fixing mobile rendering bug
jasmaa Aug 12, 2019
05cf27f
Minor ui changes
jasmaa Aug 13, 2019
0d1afdc
Improve convolution demo
jasmaa Aug 13, 2019
c7c63ee
Fix css on topology demo
jasmaa Aug 14, 2019
f9049ea
Replace RGB demo
jasmaa Aug 14, 2019
6197016
Add blurbs, rewrite grayscale func and rgb section
jasmaa Aug 14, 2019
e5ca541
Minor style changes
jasmaa Aug 15, 2019
b47dfd1
Remove unused code and assets
jasmaa Aug 16, 2019
4afd532
Merge edge detection to one page
jasmaa Aug 17, 2019
795be97
Merge filters in convolution demo
jasmaa Aug 17, 2019
9a9bec9
Add collapsable section, clean up text
jasmaa Aug 18, 2019
20673d4
Fix for mobile
jasmaa Aug 18, 2019
2343490
Make requested changes
jasmaa Aug 20, 2019
57c5136
Revert global changes
jasmaa Aug 20, 2019
8fcc95f
Fix diff problems
jasmaa Aug 20, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Split into multiple pages, modify header injection
  • Loading branch information
jasmaa committed Jul 17, 2019
commit 4512f837ab12917c0f44d7f8a5e8d28282f81ed1
87 changes: 87 additions & 0 deletions 24-Perception/Edge-Detection/1-Introduction/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
<title>24 Perception - Detecting Gradients</title>
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.8.2/css/all.css"
integrity="sha384-oS3vJWv+0UjzBfQzYUhtDYW+Pj2yciDJxpsK1OYPAYjqT085Qq/1cq5FLXAZQ7Ay" crossorigin="anonymous">
<link rel="stylesheet" href="https://code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
<link rel="stylesheet" href="/styles.css">
<link rel="stylesheet" href="../css/style.css">
<script src="https://code.jquery.com/jquery-1.12.4.js"></script>
<script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>

<script type="text/javascript" src="/main.js"></script>
</head>

<body>
<div class="container-fluid">
<div class="row">
<div class="col-sm-6 col-md-offset-3" id="content">

<h1>Introduction</h1>
<br>

<p>
For many of us, it is easy to find both the objects in a picture and their boundaries.
Humans are very good at identifying what something is and where it is given a clear line of
sight, but what if we need to find the boundaries of objects in an enormously large number photos?
While we could put a room full of toddlers with markers at work and tell them to outline the edges
in a stack of photos, we can do better in both performance and morals by making a computer do this
task
for us! In fact, in computer vision, this is a general problem known as <strong>edge
detection</strong>.
</p>
<p>
Formally, edge detection is the process of detecting the boundaries in an image that separate great
changes in brightness or <strong>intensity</strong>. One of the reasons to do edge detection is that
it is
able to reduce a complex image down to a simpler one where only these edges are marked. Because of
this,
edge detection often acts as a preliminary step to other computer vision processes.
</p>

<br>
<img src="/third-party/photo-edge-detection.png" alt="Edge detection example on photo"
class="center display-image">
<br><br>

<p>
Before we dive into edge detection, we must talk about how an image is represented on a machine.
While our toddler workforce may work with physical photographs, computers work with images as 2D
grids known as <strong>pixel arrays</strong>. A color image, for instance, is commonly represented
with 3 grids,
storing the red, green, and blue color values at each pixel. With edge detection, since we are
generally
only interested in the intensity of a pixel, or how bright or dark it is, we will often first
convert
our color images to <strong>grayscale</strong> intensities which can be calculated from the
red, green, and blue color values.
</p>
<br>

<div class="row">
<div class="col-sm-6 text-left">
</div>
<div class="col-sm-6 text-right">
<a href="../2-Detecting-Gradients/">Next >>></a>
</div>
</div>
<br>
</div>
</div>
</div>


<!-- Replace with third party -->
<script crossorigin src="https://unpkg.com/react@16/umd/react.development.js"></script>
<script crossorigin src="https://unpkg.com/react-dom@16/umd/react-dom.development.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/vis/4.21.0/vis.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/106/three.js"></script>
<script src="/third-party/OrbitControls.js"></script>

</body>

</html>
174 changes: 174 additions & 0 deletions 24-Perception/Edge-Detection/2-Detecting-Gradients/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
<title>24 Perception - Detecting Gradients</title>
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.8.2/css/all.css"
integrity="sha384-oS3vJWv+0UjzBfQzYUhtDYW+Pj2yciDJxpsK1OYPAYjqT085Qq/1cq5FLXAZQ7Ay" crossorigin="anonymous">
<link rel="stylesheet" href="https://code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
<link rel="stylesheet" href="/styles.css">
<link rel="stylesheet" href="../css/style.css">
<script src="https://code.jquery.com/jquery-1.12.4.js"></script>
<script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>

<script type="text/javascript" src="/main.js"></script>
</head>

<body>
<div class="container-fluid">
<div class="row">
<div class="col-sm-6 col-md-offset-3" id="content">

<h1>Detecting Gradients</h1>
<br>

<p>
How do we go about solving the problem of detecting edges? Since our goal is to find locations where
image
intensity changes the most, we need some way of finding out how much the intensity changes as we
move across
the pixels in our image.
</p>
<p>
For problems like this, it helps to add an extra dimension for visualization. Let us re-imagine our
pixel
array as a 3D topological map, correlating the intensity of each pixel with height on the map. Below
is an example
of how one of these maps might look. Drag it around and observe how pixel intensities are translated
into 3D
elevation.
</p>

<br>
<div id="topology-root"></div>
<br>

<p>
Picking an arbitrary point on this map, we now want to figure out the change in elevation at that
point.
Knowing the point’s height alone is relatively unhelpful since, although it may tell us how high up
it is,
we get no information about changes in elevation. What if, in addition, we also considered the
height of
land a small distance to the west and east of our chosen point? Ah-hah! Now by comparing these
nearby
heights, we can get some idea of the local elevation changes around the point!
</p>

<br>
<img src="../images/elevation.png" alt="Elevation comparison example" class="center display-image">
<br><br>

<p>
This is the driving principle behind a well-known edge detector called the <strong>Sobel
operator</strong>. The operator works
by dragging two <strong>kernels</strong> or <strong>filters</strong> called Sobel X and Sobel Y
across an
image to calculate horizontal and vertical change respectively. For each target pixel in the image,
the kernel covers and
operates on a neighborhood of surrounding pixels. We compare these neighboring pixels with each
other and use them to
assess the change of intensity at the target pixel. To demonstrate this, below is an interactive
example using the Sobel X filter. Brighter areas in the
result are indicative of large positive change while darker areas are indicative of large negative
change.
Use your mouse to drag the filter around and see how different areas reflect different intensity
changes.
</p>

<br>
<div id="convolution-root"></div>
<br>

<p>
Numerically, these filters are actually grids of values that are matched up to each neighborhood of
pixels and used in a weighted sum to calculate the change at each target pixel. In general, this
concept of
operating on a grid of values by dragging a smaller grid of values across it is a powerful
mathematical idea that
goes beyond edge detection known as <strong>convolution</strong>.
</p>

<br>
<img src="../images/filters.png" alt="Sobel X and Y filters" class="center display-image">
<br><br>

<p>
The end result of applying both the Sobel X and Sobel Y filters to our image is two grids containing
values that represent horizontal and vertical change respectively. These values can also be thought
of
as the X and Y components for a grid of 2D arrows or <strong>vectors</strong>. The vectors gotten
using the Sobel
operator are known as a <strong>gradients</strong> For a given pixel, the magnitude of the gradient
represents
how much intensity change occurs at the pixel and the angle represents the direction of the greatest
change from the pixel.
</p>

<p>
Below is a visualization of an image’s gradients. Draw edges or select one of the preset images and
observe how the direction and magnitude of the gradient vectors change in response. The magnitude of
the vectors is indicated by color with blue being small and red being large.
</p>

<br>
<div id="gradient-root"></div>
<br>

<p>
One can see why calculating gradients for intensity in our photograph using Sobel is a start to
finding edges.
Locations where the gradient magnitudes are small reflect little intensity changes and no edges
while locations
where the gradient magnitudes are large reflect great intensity changes and potential edges. To
recap, below is an
outline of the steps we have taken to edge detection so far starting from a color photograph.
</p>

<br>
<div id="pipeline-grad-root"></div>
<br>

<div class="row">
<div class="col-sm-6 text-left">
<a href="../1-Introduction/"><<< Prev</a>
</div> <div class="col-sm-6 text-right">
<a href="../3-Robust-Edge-Detection/">Next >>></a>
</div>
</div>
<br>
</div>
</div>
</div>


<!-- Replace with third party -->
<script crossorigin src="https://unpkg.com/react@16/umd/react.development.js"></script>
<script crossorigin src="https://unpkg.com/react-dom@16/umd/react-dom.development.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/vis/4.21.0/vis.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/106/three.js"></script>
<script src="/third-party/OrbitControls.js"></script>


<script type="text/javascript" src="../js/setup.js"></script>
<script type="text/javascript" src="../js/util.js"></script>
<script type="text/javascript" src="../js/ui.js"></script>
<script type="text/javascript" src="../js/imageProcessing.js"></script>

<script type="text/javascript" src="./js/demoConvolution.js"></script>
<script type="text/javascript" src="./js/demoGradient.js"></script>
<script type="text/javascript" src="./js/demoTopology.js"></script>
<script type="text/javascript" src="./js/demoPipeline3d.js"></script>

<script type="text/javascript">
// mobile setup
document.ontouchmove = function (event) {
event.preventDefault();
}
</script>

</body>

</html>
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ class GradientDemo extends React.Component {
grid: this.source,
});
};
img.src = "./images/starter.png";
img.src = "../images/starter.png";
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,7 @@ class Pipeline3dDemo extends React.Component {
e('div', {className: 'col-md-12'},
e(ImageUploader, {
imageId: this.imageId,
defaultImage: 'images/test.png',
defaultImage: '../images/test.png',
processHandler: () => this.process(),
}, null)
)
Expand Down Expand Up @@ -368,10 +368,12 @@ class Pipeline3dDemo extends React.Component {
}

// Render elements
/*
ReactDOM.render(
e(Pipeline3dDemo, {steps: 3, imageId: 'pipeline3d-grayscale' }, null),
document.getElementById('pipeline-grayscale-root')
);
*/

ReactDOM.render(
e(Pipeline3dDemo, {steps: 5, imageId: 'pipeline3d-grad'}, null),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ class TopologyDemo extends React.Component {
e('div', {className: 'col-md-12'},
e(ImageUploader, {
imageId: this.imageId,
defaultImage: 'images/test.png',
defaultImage: '../images/test.png',
processHandler: () => this.process(),
}, null)
)
Expand Down
Loading