Skip to content

Commit 9ec1a95

Browse files
Amberlan1001Amberlan1001
committed
add chapter4
1 parent ebe1d8f commit 9ec1a95

File tree

10,032 files changed

+9084
-15
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

10,032 files changed

+9084
-15
lines changed

‎Chapter 4 Low-level API in TensorFlow/4-1Structural_Operations_of the_Tensor.ipynb

+1,302
Large diffs are not rendered by default.

‎Chapter 4 Low-level API in TensorFlow/4-2Mathematical_Operations of_the_Tensor.ipynb

+1,005
Large diffs are not rendered by default.

‎Chapter 4 Low-level API in TensorFlow/4-3Rules_of_Using_the_AutoGraph.ipynb

+281
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,281 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# 4-3 Rules of Using the AutoGraph\n",
8+
"\n",
9+
"There are three ways of constructing graph: static, dynamic and Autograph.\n",
10+
"\n",
11+
"TensorFlow 2.X uses dynamic graph and Autograph.\n",
12+
"\n",
13+
"Dynamic graph is easier for debugging with higher encoding efficiency, but with lower efficiency in execution.\n",
14+
"\n",
15+
"Static graph has high efficiency in execution, but more difficult for debugging.\n",
16+
"\n",
17+
"Autograph mechanism transforms dynamic graph into static graph, making allowance for both executing and encoding efficiencies.\n",
18+
"\n",
19+
"There are certain rules for the code that is able to converted by Autograph, or it could result in failure or unexpected results.\n",
20+
"\n",
21+
"We are going to introduce the coding rules of Autograph and its mechanism of converting into static graph, together with introduction about how to construct Autograph using `tf.Module`.\n",
22+
"\n",
23+
"This section introduce the coding rules of using Autograph. We will introduce the mechanisms of Autograph in next section and explain the logic behind the rules there.\n",
24+
"\n",
25+
"<!-- #region -->\n",
26+
"### 1. Summarization of the Coding Rules of Autograph\n",
27+
"\n",
28+
"\n",
29+
"* 1. We should use the TensorFlow-defined functions to be decorated by `@tf.function` as much as possible, instead of those Python functions. For instance, `tf.print` should be used instead of `print`; `tf.range` should be used instead of `range`; `tf.constant(True)` should be used instead of `True`.\n",
30+
"\n",
31+
"* 2. Avoid defining `tf.Variable` inside the decorator `@tf.function`.\n",
32+
"\n",
33+
"* 3. Functions that are decorated by `@tf.function` cannot modify the struct data types variables outside the function such as Python list, dictionary, etc.\n",
34+
"<!-- #endregion -->"
35+
]
36+
},
37+
{
38+
"cell_type": "markdown",
39+
"metadata": {},
40+
"source": [
41+
"### 2. Explanations to the Autograph Coding Rules\n",
42+
"\n",
43+
"\n",
44+
" **2.1 We should use the TensorFlow-defined functions to be decorated by `@tf.function` as much as possible, instead of those Python functions.**\n"
45+
]
46+
},
47+
{
48+
"cell_type": "code",
49+
"execution_count": 1,
50+
"metadata": {},
51+
"outputs": [],
52+
"source": [
53+
"import numpy as np\n",
54+
"import tensorflow as tf\n",
55+
"\n",
56+
"@tf.function\n",
57+
"def np_random():\n",
58+
" a = np.random.randn(3,3)\n",
59+
" tf.print(a)\n",
60+
" \n",
61+
"@tf.function\n",
62+
"def tf_random():\n",
63+
" a = tf.random.normal((3,3))\n",
64+
" tf.print(a)\n"
65+
]
66+
},
67+
{
68+
"cell_type": "code",
69+
"execution_count": 2,
70+
"metadata": {},
71+
"outputs": [
72+
{
73+
"name": "stdout",
74+
"output_type": "stream",
75+
"text": [
76+
"array([[ 0.25134437, -0.03228947, -0.29466093],\n",
77+
" [ 0.54150381, 0.67923698, -0.51601442],\n",
78+
" [ 0.44043714, -0.42121957, -1.00554045]])\n",
79+
"array([[ 0.25134437, -0.03228947, -0.29466093],\n",
80+
" [ 0.54150381, 0.67923698, -0.51601442],\n",
81+
" [ 0.44043714, -0.42121957, -1.00554045]])\n"
82+
]
83+
}
84+
],
85+
"source": [
86+
"# Same results after each execution of np_random\n",
87+
"np_random()\n",
88+
"np_random()"
89+
]
90+
},
91+
{
92+
"cell_type": "code",
93+
"execution_count": 3,
94+
"metadata": {},
95+
"outputs": [
96+
{
97+
"name": "stdout",
98+
"output_type": "stream",
99+
"text": [
100+
"[[-1.0011673 -0.0995076299 -2.32573843]\n",
101+
" [1.52956295 -0.982268512 -0.447938532]\n",
102+
" [-0.93382287 0.434479773 -2.08646727]]\n",
103+
"[[-0.790998399 0.0259545967 0.0513409264]\n",
104+
" [0.142200559 0.390263647 -0.902663]\n",
105+
" [-1.16874099 0.14255169 0.235685781]]\n"
106+
]
107+
}
108+
],
109+
"source": [
110+
"# New random numbers are generated after each execution of tf_random\n",
111+
"tf_random()\n",
112+
"tf_random()"
113+
]
114+
},
115+
{
116+
"cell_type": "code",
117+
"execution_count": null,
118+
"metadata": {},
119+
"outputs": [],
120+
"source": []
121+
},
122+
{
123+
"cell_type": "markdown",
124+
"metadata": {},
125+
"source": [
126+
"**2.2 Avoid defining `tf.Variable` inside the decorator `@tf.function`.**"
127+
]
128+
},
129+
{
130+
"cell_type": "code",
131+
"execution_count": 4,
132+
"metadata": {},
133+
"outputs": [
134+
{
135+
"name": "stdout",
136+
"output_type": "stream",
137+
"text": [
138+
"2\n",
139+
"3\n"
140+
]
141+
},
142+
{
143+
"data": {
144+
"text/plain": [
145+
"<tf.Tensor: shape=(), dtype=float32, numpy=3.0>"
146+
]
147+
},
148+
"execution_count": 4,
149+
"metadata": {},
150+
"output_type": "execute_result"
151+
}
152+
],
153+
"source": [
154+
"# Avoid defining tf.Variable inside the decorator @tf.function.\n",
155+
"\n",
156+
"x = tf.Variable(1.0,dtype=tf.float32)\n",
157+
"@tf.function\n",
158+
"def outer_var():\n",
159+
" x.assign_add(1.0)\n",
160+
" tf.print(x)\n",
161+
" return(x)\n",
162+
"\n",
163+
"outer_var() \n",
164+
"outer_var()\n"
165+
]
166+
},
167+
{
168+
"cell_type": "code",
169+
"execution_count": 5,
170+
"metadata": {},
171+
"outputs": [],
172+
"source": [
173+
"@tf.function\n",
174+
"def inner_var():\n",
175+
" x = tf.Variable(1.0,dtype = tf.float32)\n",
176+
" x.assign_add(1.0)\n",
177+
" tf.print(x)\n",
178+
" return(x)\n",
179+
"\n",
180+
"# Error after execution\n",
181+
"#inner_var()\n",
182+
"#inner_var()"
183+
]
184+
},
185+
{
186+
"cell_type": "code",
187+
"execution_count": null,
188+
"metadata": {},
189+
"outputs": [],
190+
"source": []
191+
},
192+
{
193+
"cell_type": "markdown",
194+
"metadata": {},
195+
"source": [
196+
"**2.3 Functions that are decorated by `@tf.function` cannot modify the struct data types variables outside the function such as Python list, dictionary, etc.**\n"
197+
]
198+
},
199+
{
200+
"cell_type": "code",
201+
"execution_count": 6,
202+
"metadata": {},
203+
"outputs": [
204+
{
205+
"name": "stdout",
206+
"output_type": "stream",
207+
"text": [
208+
"[<tf.Tensor: shape=(), dtype=float32, numpy=5.0>, <tf.Tensor: shape=(), dtype=float32, numpy=6.0>]\n"
209+
]
210+
}
211+
],
212+
"source": [
213+
"tensor_list = []\n",
214+
"\n",
215+
"#@tf.function # Autograph will result in something unexpected if executing this line\n",
216+
"def append_tensor(x):\n",
217+
" tensor_list.append(x)\n",
218+
" return tensor_list\n",
219+
"\n",
220+
"append_tensor(tf.constant(5.0))\n",
221+
"append_tensor(tf.constant(6.0))\n",
222+
"print(tensor_list)"
223+
]
224+
},
225+
{
226+
"cell_type": "code",
227+
"execution_count": 7,
228+
"metadata": {},
229+
"outputs": [
230+
{
231+
"name": "stdout",
232+
"output_type": "stream",
233+
"text": [
234+
"[<tf.Tensor 'x:0' shape=() dtype=float32>]\n"
235+
]
236+
}
237+
],
238+
"source": [
239+
"tensor_list = []\n",
240+
"\n",
241+
"@tf.function # Autograph will result in something unexpected if executing this line\n",
242+
"def append_tensor(x):\n",
243+
" tensor_list.append(x)\n",
244+
" return tensor_list\n",
245+
"\n",
246+
"\n",
247+
"append_tensor(tf.constant(5.0))\n",
248+
"append_tensor(tf.constant(6.0))\n",
249+
"print(tensor_list)"
250+
]
251+
},
252+
{
253+
"cell_type": "code",
254+
"execution_count": null,
255+
"metadata": {},
256+
"outputs": [],
257+
"source": []
258+
}
259+
],
260+
"metadata": {
261+
"kernelspec": {
262+
"display_name": "Python 3",
263+
"language": "python",
264+
"name": "python3"
265+
},
266+
"language_info": {
267+
"codemirror_mode": {
268+
"name": "ipython",
269+
"version": 3
270+
},
271+
"file_extension": ".py",
272+
"mimetype": "text/x-python",
273+
"name": "python",
274+
"nbconvert_exporter": "python",
275+
"pygments_lexer": "ipython3",
276+
"version": "3.7.6"
277+
}
278+
},
279+
"nbformat": 4,
280+
"nbformat_minor": 4
281+
}

0 commit comments

Comments
 (0)