forked from sunprinceS/ML-Assignment3
-
Notifications
You must be signed in to change notification settings - Fork 2
/
p4.html
151 lines (121 loc) · 8.99 KB
/
p4.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
<!DOCTYPE html>
<html>
<link rel="shortcut icon" href="favicon.ico">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
<link rel="stylesheet" href="highlight.css">
<meta name="author" content="ntumlta" >
<meta property="og:image" content="joy.png"/>
<title>Machine Learning (2017, Fall)</title>
<xmp theme="cerulean" style="display:none;">
# Problem 4: Analyze the Model by Plotting the Saliency Map
Problem Description:
* Given an image and its corresponding class, we would like to rank the pixels of original image based on their influence on the distribution of final output
* Use your trained CNN, get the gradient of input image and plot it, or you can use the other method mentioned in class to plot the saliency map
## 範例
* **[Note] 請不要直接使用助教的圖來當成作業交上來**
* **[Note] 請不要使用這張範例圖**
<img src="17.png" alt="Drawing" style="width: 200px;"/>
#### Heatmap
<center>原圖</center>| <center> Saliency Map</center> | <center>Mask掉heat小的部份</center>
:-------------------------:|:-------------------------:|:------------------------:
<img src="17.png" alt="Drawing" style="width: 400px;"/>|<img src="17-cmap.png" alt="Drawing" style="width: 400px;"/>|<img src="17-psee.png" alt="Drawing" style="width: 400px;"/>
## TA hour
<i class="fa fa-diamond"></i> Keywords: `keras.backend`, `gradients`
<div id="doc" class="markdown-body container-fluid" style="position: relative;"><h1 id="hw3-手把手-q4"><a class="anchor hidden-xs" href="#hw3-手把手-q4" title="hw3-手把手-q4"><span class="octicon octicon-link"></span></a>HW3 手把手 Q4</h1><pre><code class="python=3.6 hljs"><span class="hljs-comment">#!/usr/bin/env python</span>
<span class="hljs-comment"># -*- coding: utf-8 -*-</span>
<span class="hljs-keyword">import</span> os
<span class="hljs-keyword">import</span> argparse
<span class="hljs-keyword">from</span> keras.models <span class="hljs-keyword">import</span> load_model
<span class="hljs-keyword">from</span> termcolor <span class="hljs-keyword">import</span> colored,cprint
<span class="hljs-keyword">import</span> keras.backend <span class="hljs-keyword">as</span> K
<span class="hljs-keyword">from</span> utils <span class="hljs-keyword">import</span> *
<span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np
<span class="hljs-keyword">import</span> matplotlib.pyplot <span class="hljs-keyword">as</span> plt
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">deprocess_image</span><span class="hljs-params">(x)</span>:</span>
<span class="hljs-string">"""
Hint: Normalize and Clip
"""</span>
<span class="hljs-keyword">return</span> x
base_dir = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
img_dir = os.path.join(base_dir, <span class="hljs-string">'image'</span>)
<span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> os.path.exists(img_dir):
os.makedirs(img_dir)
cmap_dir = os.path.join(img_dir, <span class="hljs-string">'cmap'</span>)
<span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> os.path.exists(cmap_dir):
os.makedirs(cmap_dir)
partial_see_dir = os.path.join(img_dir,<span class="hljs-string">'partial_see'</span>)
<span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> os.path.exists(partial_see_dir):
os.makedirs(partial_see_dir)
model_dir = os.path.join(base_dir, <span class="hljs-string">'model'</span>)
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span><span class="hljs-params">()</span>:</span>
parser = argparse.ArgumentParser(prog=<span class="hljs-string">'plot_saliency.py'</span>,
description=<span class="hljs-string">'ML-Assignment3 visualize attention heat map.'</span>)
parser.add_argument(<span class="hljs-string">'--epoch'</span>, type=int, metavar=<span class="hljs-string">'<#epoch>'</span>, default=<span class="hljs-number">1</span>)
args = parser.parse_args()
model_name = <span class="hljs-string">"model-%s.h5"</span> %str(args.epoch)
model_path = os.path.join(model_dir, model_name)
emotion_classifier = load_model(model_path)
print(colored(<span class="hljs-string">"Loaded model from {}"</span>.format(model_name), <span class="hljs-string">'yellow'</span>, attrs=[<span class="hljs-string">'bold'</span>]))
private_pixels = load_pickle(<span class="hljs-string">'fer2013/test_with_ans_pixels.pkl'</span>)
private_pixels = [ np.fromstring(private_pixels[i], dtype=float, sep=<span class="hljs-string">' '</span>).reshape((<span class="hljs-number">1</span>, <span class="hljs-number">48</span>, <span class="hljs-number">48</span>, <span class="hljs-number">1</span>))
<span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(len(private_pixels)) ]
input_img = emotion_classifier.input
img_ids = [<span class="hljs-number">17</span>]
<span class="hljs-keyword">for</span> idx <span class="hljs-keyword">in</span> img_ids:
val_proba = emotion_classifier.predict(private_pixels[idx])
pred = val_proba.argmax(axis=<span class="hljs-number">-1</span>)
target = K.mean(emotion_classifier.output[:, pred])
grads = K.gradients(target, input_img)[<span class="hljs-number">0</span>]
fn = K.function([input_img, K.learning_phase()], [grads])
<span class="hljs-string">"""
Implement your heatmap processing here!
hint: Do some normalization or smoothening on grads
"""</span>
thres = <span class="hljs-number">0.5</span>
see = private_pixels[idx].reshape(<span class="hljs-number">48</span>, <span class="hljs-number">48</span>)
<span class="hljs-comment"># for i in range(48):</span>
<span class="hljs-comment"># for j in range(48):</span>
<span class="hljs-comment"># print heatmap[i][j]</span>
see[np.where(heatmap <= thres)] = np.mean(see)
plt.figure()
plt.imshow(heatmap, cmap=plt.cm.jet)
plt.colorbar()
plt.tight_layout()
fig = plt.gcf()
plt.draw()
test_dir = os.path.join(cmap_dir, <span class="hljs-string">'test'</span>)
<span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> os.path.exists(test_dir):
os.makedirs(test_dir)
fig.savefig(os.path.join(test_dir, <span class="hljs-string">'{}.png'</span>.format(idx)), dpi=<span class="hljs-number">100</span>)
plt.figure()
plt.imshow(see,cmap=<span class="hljs-string">'gray'</span>)
plt.colorbar()
plt.tight_layout()
fig = plt.gcf()
plt.draw()
test_dir = os.path.join(partial_see_dir, <span class="hljs-string">'test'</span>)
<span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> os.path.exists(test_dir):
os.makedirs(test_dir)
fig.savefig(os.path.join(test_dir, <span class="hljs-string">'{}.png'</span>.format(idx)), dpi=<span class="hljs-number">100</span>)
<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
main()
</code></pre><div class="resize-sensor" style="position: absolute; left: 0px; top: 0px; right: 0px; bottom: 0px; overflow: hidden; z-index: -1; visibility: hidden;"><div class="resize-sensor-expand" style="position: absolute; left: 0; top: 0; right: 0; bottom: 0; overflow: hidden; z-index: -1; visibility: hidden;"><div style="position: absolute; left: 0px; top: 0px; transition: 0s; width: 100000px; height: 100000px;"></div></div><div class="resize-sensor-shrink" style="position: absolute; left: 0; top: 0; right: 0; bottom: 0; overflow: hidden; z-index: -1; visibility: hidden;"><div style="position: absolute; left: 0; top: 0; transition: 0s; width: 200%; height: 200%"></div></div></div></div>
## Reference
→ [Deep Inside CNN: Visualizing Image Classification Models and Saliency Maps](https://arxiv.org/pdf/1312.6034.pdf)
</xmp>
</xmp> <script src="strapdown.js"></script> </html>
<footer>
<center><a href="./index.html"><i class="fa fa-home"></i></a></center>
<center><i class="fa fa-github"></i></a> Posted by: <a href="https://github.com/ntumlta/" target="_blank">ntumlta</a> </center>
<center><i class="fa fa-envelope"></i> Contact information: <a href="mailto:"> ntu.mlta@gmail.com </a>.</center>
<center><i class="fa fa-mortar-board"></i> Course information: <a href="http://speech.ee.ntu.edu.tw/~tlkagk/courses_ML17_2.html", target="_blank">Machine Learning (2017, Fall) @ National Taiwan University</a>.</center>
</footer>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-59748795-2', 'auto');
ga('send', 'pageview');
</script>
</html>