-
Notifications
You must be signed in to change notification settings - Fork 1.8k
/
Copy path13-web.html
1004 lines (949 loc) · 63.3 KB
/
13-web.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<head>
<!--<script type="text/javascript" src="https://trinket.io/js/trinket.js"></script>-->
<link rel="stylesheet" href="trinket/base.css" type="text/css" />
<link rel="stylesheet" href="trinket/trinket.css" type="text/css" />
<link rel="stylesheet" href="trinket/font-awesome.min.css" type="text/css" />
<style type="text/css">
pre > code.sourceCode { white-space: pre; position: relative; }
pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
pre > code.sourceCode > span:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode > span { color: inherit; text-decoration: inherit; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
pre > code.sourceCode { white-space: pre-wrap; }
pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
}
pre.numberSource code
{ counter-reset: source-line 0; }
pre.numberSource code > span
{ position: relative; left: -4em; counter-increment: source-line; }
pre.numberSource code > span > a:first-child::before
{ content: counter(source-line);
position: relative; left: -1em; text-align: right; vertical-align: baseline;
border: none; display: inline-block;
-webkit-touch-callout: none; -webkit-user-select: none;
-khtml-user-select: none; -moz-user-select: none;
-ms-user-select: none; user-select: none;
padding: 0 4px; width: 4em;
color: #aaaaaa;
}
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
div.sourceCode
{ }
@media screen {
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
}
code span.al { color: #ff0000; font-weight: bold; } /* Alert */
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #7d9029; } /* Attribute */
code span.bn { color: #40a070; } /* BaseN */
code span.bu { color: #008000; } /* BuiltIn */
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4070a0; } /* Char */
code span.cn { color: #880000; } /* Constant */
code span.co { color: #60a0b0; font-style: italic; } /* Comment */
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #ba2121; font-style: italic; } /* Documentation */
code span.dt { color: #902000; } /* DataType */
code span.dv { color: #40a070; } /* DecVal */
code span.er { color: #ff0000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #40a070; } /* Float */
code span.fu { color: #06287e; } /* Function */
code span.im { color: #008000; font-weight: bold; } /* Import */
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #007020; font-weight: bold; } /* Keyword */
code span.op { color: #666666; } /* Operator */
code span.ot { color: #007020; } /* Other */
code span.pp { color: #bc7a00; } /* Preprocessor */
code span.sc { color: #4070a0; } /* SpecialChar */
code span.ss { color: #bb6688; } /* SpecialString */
code span.st { color: #4070a0; } /* String */
code span.va { color: #19177c; } /* Variable */
code span.vs { color: #4070a0; } /* VerbatimString */
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
</style>
<style>
/*Temporary Fix for Font Awesome ApplicatonManifest offline Cache */
@font-face {
font-family: 'FontAwesome';
src: url('../fonts/fontawesome-webfont.eot');
src: url('../fonts/fontawesome-webfont.eot') format('embedded-opentype'), url('../fonts/fontawesome-webfont.woff2') format('woff2'), url('../fonts/fontawesome-webfont.woff') format('woff'), url('../fonts/fontawesome-webfont.ttf') format('truetype'), url('../fonts/fontawesome-webfont.svg') format('svg');
font-weight: normal;
font-style: normal;
}
</style>
</head>
<body class="loggedout">
<div class="main-content">
<div class="nav-wrapper">
<nav class="top-bar" data-topbar data-topbar data-options="is_hover: false">
<ul class="title-area">
<li class="name">
<a href="/">
<img src="https://trinket.io/img/trinket-logo.png" alt="Hosted by Trinket" />
</a>
</li>
<li class="toggle-topbar menu-icon"><a href="#"><span></span></a></li>
</ul>
<section class="top-bar-section">
<ul class="right">
<li><a href="http://pythonlearn.com"><i class="fa fa-star"></i> PythonLearn</a></li>
<li><a href="https://trinket.io"><i class="fa fa-star"></i> Trinket</a></li>
<li><a href="https://hourofpython.com"><i class="fa fa-graduation-cap"></i> Hour of Python</a></li>
</ul>
</section>
</nav>
</div>
<div class="booktoc sticky">
<nav class="top-bar" data-topbar="" role="navigation">
<ul class="title-area">
<li class="name">
<h1 class="no-anchor"><a href="#">Python for Everyone</a></h1>
</li>
<li class="toggle-topbar"><a href="#"><span>Menu</span></a></li>
</ul>
<section class="top-bar-section">
<ul class="left">
<li class="has-dropdown not-click">
<a href="#">Chapters</a>
<ul class="dropdown">
<li class="title back js-generated">
<h5><a href="javascript:void(0)">Back</a></h5></li>
<li class="parent-link hide-for-medium-up"><a class="parent-link js-generated" href="#">Chapters</a></li>
<li><a href="index.html">See All Chapters</a></li>
<li><a href="01-intro.html">Chapter 1: Introduction</a></li>
<li><a href="02-variables.html">Chapter 2: Variables</a></li>
<li><a href="03-conditional.html">Chapter 3: Conditionals</a></li>
<li><a href="04-functions.html">Chapter 4: Functions</a></li>
<li><a href="05-iterations.html">Chapter 5: Iterations</a></li>
<li><a href="06-strings.html">Chapter 6: Strings</a></li>
<li><a href="07-files.html">Chapter 7: Files</a></li>
<li><a href="08-lists.html">Chapter 8: Lists</a></li>
<li><a href="09-dictionaries.html">Chapter 9: Dictionaries</a></li>
<li><a href="10-tuples.html">Chapter 10: Tuples</a></li>
<li><a href="11-regex.html">Chapter 11: Regex</a></li>
<li><a href="12-network.html">Chapter 12: Networked Programs</a></li>
<li><a href="13-web.html">Chapter 13: Python and Web Services</a></li>
<li><a href="14-objects.html">Chapter 14: Object Orientation</a></li>
<li><a href="15-database.html">Chapter 15: Python and Databases</a></li>
<li><a href="16-viz.html">Chapter 16: Data Vizualization</a></li>
</ul>
</li>
</ul>
</section>
</nav>
</div>
<div class="bookchapter">
<div class="row">
<div class="columns small-12">
<h1 id="using-web-services">Using Web Services</h1>
<p>Once it became easy to retrieve documents and parse documents over
HTTP using programs, it did not take long to develop an approach where
we started producing documents that were specifically designed to be
consumed by other programs (i.e., not HTML to be displayed in a
browser).</p>
<p>There are two common formats that we use when exchanging data across
the web. eXtensible Markup Language (XML) has been in use for a very
long time and is best suited for exchanging document-style data. When
programs just want to exchange dictionaries, lists, or other internal
information with each other, they use JavaScript Object Notation (JSON)
(see <a href="http://www.json.org">www.json.org</a>). We will look at
both formats.</p>
<h2 id="extensible-markup-language---xml">eXtensible Markup Language -
XML</h2>
<p>XML looks very similar to HTML, but XML is more structured than HTML.
Here is a sample of an XML document:</p>
<div class="sourceCode" id="cb1"><pre
class="sourceCode xml"><code class="sourceCode xml"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><<span class="kw">person</span>></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <<span class="kw">name</span>>Chuck</<span class="kw">name</span>></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <<span class="kw">phone</span><span class="ot"> type=</span><span class="st">"intl"</span>></span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> +1 734 303 4456</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> </<span class="kw">phone</span>></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> <<span class="kw">email</span><span class="ot"> hide=</span><span class="st">"yes"</span> /></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a></<span class="kw">person</span>></span></code></pre></div>
<p>Each pair of opening (e.g., <code><person></code>) and closing
tags (e.g., <code></person></code>) represents a <em>element</em>
or <em>node</em> with the same name as the tag (e.g.,
<code>person</code>). Each element can have some text, some attributes
(e.g., <code>hide</code>), and other nested elements. If an XML element
is empty (i.e., has no content), then it may be depicted by a
self-closing tag (e.g., <code><email /></code>).</p>
<p>Often it is helpful to think of an XML document as a tree structure
where there is a top element (here: <code>person</code>), and other tags
(e.g., <code>phone</code>) are drawn as <em>children</em> of their
<em>parent</em> elements.</p>
<figure>
<img src="../images/xml-tree.svg" alt="A Tree Representation of XML" style="height: 2.0in;"/>
<figcaption>
A Tree Representation of XML
</figcaption>
</figure>
<h2 id="parsing-xml">Parsing XML</h2>
<p> </p>
<p>Here is a simple application that parses some XML and extracts some
data elements from the XML:</p>
<script type="text/javascript">(function(d,l,s,i,c){function n(e){e=e.nextSibling;return (!e||e.nodeType!=3)?e:n(e);};function r(f){/in/.test(d.readyState) ? setTimeout(function(){r(f);},9):f()};l=d.getElementsByTagName('script');s=l[l.length-1];r(function(){i=n(s),c=n(i);i.setAttribute('data-src','https://trinket.io/tools/1.0/jekyll/embed/python3#code='+encodeURIComponent(c.nodeValue.replace(/^\s+|\s+$/g,'')));});})(document)</script>
<iframe width="100%" height="400" frameborder="0" marginwidth="0" marginheight="0" class="lazyload" allowfullscreen>
</iframe>
<!--
import xml.etree.ElementTree as ET
data = '''
<person>
<name>Chuck</name>
<phone type="intl">
+1 734 303 4456
</phone>
<email hide="yes" />
</person>'''
tree = ET.fromstring(data)
print('Name:', tree.find('name').text)
print('Attr:', tree.find('email').get('hide'))
# Code: http://www.py4e.com/code3/xml1.py
# Or select Download from this trinket's left-hand menu
-->
<p>The triple single quote (<code>'''</code>), as well as the triple
double quote (<code>"""</code>), allow for the creation of strings that
span multiple lines.</p>
<p>Calling <code>fromstring</code> converts the string representation of
the XML into a “tree” of XML elements. When the XML is in a tree, we
have a series of methods we can call to extract portions of data from
the XML string. The <code>find</code> function searches through the XML
tree and retrieves the element that matches the specified tag.</p>
<pre><code>Name: Chuck
Attr: yes</code></pre>
<p>Using an XML parser such as <code>ElementTree</code> has the
advantage that while the XML in this example is quite simple, it turns
out there are many rules regarding valid XML, and using
<code>ElementTree</code> allows us to extract data from XML without
worrying about the rules of XML syntax.</p>
<h2 id="looping-through-nodes">Looping through nodes</h2>
<p> </p>
<p>Often the XML has multiple nodes and we need to write a loop to
process all of the nodes. In the following program, we loop through all
of the <code>user</code> nodes:</p>
<script type="text/javascript">(function(d,l,s,i,c){function n(e){e=e.nextSibling;return (!e||e.nodeType!=3)?e:n(e);};function r(f){/in/.test(d.readyState) ? setTimeout(function(){r(f);},9):f()};l=d.getElementsByTagName('script');s=l[l.length-1];r(function(){i=n(s),c=n(i);i.setAttribute('data-src','https://trinket.io/tools/1.0/jekyll/embed/python3#code='+encodeURIComponent(c.nodeValue.replace(/^\s+|\s+$/g,'')));});})(document)</script>
<iframe width="100%" height="400" frameborder="0" marginwidth="0" marginheight="0" class="lazyload" allowfullscreen>
</iframe>
<!--
import xml.etree.ElementTree as ET
input = '''
<stuff>
<users>
<user x="2">
<id>001</id>
<name>Chuck</name>
</user>
<user x="7">
<id>009</id>
<name>Brent</name>
</user>
</users>
</stuff>'''
stuff = ET.fromstring(input)
lst = stuff.findall('users/user')
print('User count:', len(lst))
for item in lst:
print('Name', item.find('name').text)
print('Id', item.find('id').text)
print('Attribute', item.get('x'))
# Code: http://www.py4e.com/code3/xml2.py
# Or select Download from this trinket's left-hand menu
-->
<p>The <code>findall</code> method retrieves a Python list of subtrees
that represent the <code>user</code> structures in the XML tree. Then we
can write a <code>for</code> loop that looks at each of the user nodes,
and prints the <code>name</code> and <code>id</code> text elements as
well as the <code>x</code> attribute from the <code>user</code>
node.</p>
<pre><code>User count: 2
Name Chuck
Id 001
Attribute 2
Name Brent
Id 009
Attribute 7</code></pre>
<p>It is important to include all parent level elements in the
<code>findall</code> statement except for the top level element (e.g.,
<code>users/user</code>). Otherwise, Python will not find any desired
nodes.</p>
<div class="sourceCode" id="cb4"><pre
class="sourceCode python"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> xml.etree.ElementTree <span class="im">as</span> ET</span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a><span class="bu">input</span> <span class="op">=</span> <span class="st">'''</span></span>
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a><span class="st"><stuff></span></span>
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a><span class="st"> <users></span></span>
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a><span class="st"> <user x="2"></span></span>
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a><span class="st"> <id>001</id></span></span>
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a><span class="st"> <name>Chuck</name></span></span>
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a><span class="st"> </user></span></span>
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a><span class="st"> <user x="7"></span></span>
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a><span class="st"> <id>009</id></span></span>
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a><span class="st"> <name>Brent</name></span></span>
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a><span class="st"> </user></span></span>
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a><span class="st"> </users></span></span>
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a><span class="st"></stuff>'''</span></span>
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-17"><a href="#cb4-17" aria-hidden="true" tabindex="-1"></a>stuff <span class="op">=</span> ET.fromstring(<span class="bu">input</span>)</span>
<span id="cb4-18"><a href="#cb4-18" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-19"><a href="#cb4-19" aria-hidden="true" tabindex="-1"></a>lst <span class="op">=</span> stuff.findall(<span class="st">'users/user'</span>)</span>
<span id="cb4-20"><a href="#cb4-20" aria-hidden="true" tabindex="-1"></a><span class="bu">print</span>(<span class="st">'User count:'</span>, <span class="bu">len</span>(lst))</span>
<span id="cb4-21"><a href="#cb4-21" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb4-22"><a href="#cb4-22" aria-hidden="true" tabindex="-1"></a>lst2 <span class="op">=</span> stuff.findall(<span class="st">'user'</span>)</span>
<span id="cb4-23"><a href="#cb4-23" aria-hidden="true" tabindex="-1"></a><span class="bu">print</span>(<span class="st">'User count:'</span>, <span class="bu">len</span>(lst2))</span></code></pre></div>
<p><code>lst</code> stores all <code>user</code> elements that are
nested within their <code>users</code> parent. <code>lst2</code> looks
for <code>user</code> elements that are not nested within the top level
<code>stuff</code> element where there are none.</p>
<pre><code>User count: 2
User count: 0</code></pre>
<h2 id="javascript-object-notation---json">JavaScript Object Notation -
JSON</h2>
<p> </p>
<p>The JSON format was inspired by the object and array format used in
the JavaScript language. But since Python was invented before
JavaScript, Python’s syntax for dictionaries and lists influenced the
syntax of JSON. So the format of JSON is nearly identical to a
combination of Python lists and dictionaries.</p>
<p>Here is a JSON encoding that is roughly equivalent to the simple XML
from above:</p>
<div class="sourceCode" id="cb6"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="fu">{</span></span>
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">"name"</span> <span class="fu">:</span> <span class="st">"Chuck"</span><span class="fu">,</span></span>
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">"phone"</span> <span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">"type"</span> <span class="fu">:</span> <span class="st">"intl"</span><span class="fu">,</span></span>
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">"number"</span> <span class="fu">:</span> <span class="st">"+1 734 303 4456"</span></span>
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">"email"</span> <span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">"hide"</span> <span class="fu">:</span> <span class="st">"yes"</span></span>
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span></code></pre></div>
<p>You will notice some differences. First, in XML, we can add
attributes like “intl” to the “phone” tag. In JSON, we simply have
key-value pairs. Also the XML “person” tag is gone, replaced by a set of
outer curly braces.</p>
<p>In general, JSON structures are simpler than XML because JSON has
fewer capabilities than XML. But JSON has the advantage that it maps
<em>directly</em> to some combination of dictionaries and lists. And
since nearly all programming languages have something equivalent to
Python’s dictionaries and lists, JSON is a very natural format to have
two cooperating programs exchange data.</p>
<p>JSON is quickly becoming the format of choice for nearly all data
exchange between applications because of its relative simplicity
compared to XML.</p>
<h2 id="parsing-json">Parsing JSON</h2>
<p>We construct our JSON by nesting dictionaries and lists as needed. In
this example, we represent a list of users where each user is a set of
key-value pairs (i.e., a dictionary). So we have a list of
dictionaries.</p>
<p>In the following program, we use the built-in <code>json</code>
library to parse the JSON and read through the data. Compare this
closely to the equivalent XML data and code above. The JSON has less
detail, so we must know in advance that we are getting a list and that
the list is of users and each user is a set of key-value pairs. The JSON
is more succinct (an advantage) but also is less self-describing (a
disadvantage).</p>
<script type="text/javascript">(function(d,l,s,i,c){function n(e){e=e.nextSibling;return (!e||e.nodeType!=3)?e:n(e);};function r(f){/in/.test(d.readyState) ? setTimeout(function(){r(f);},9):f()};l=d.getElementsByTagName('script');s=l[l.length-1];r(function(){i=n(s),c=n(i);i.setAttribute('data-src','https://trinket.io/tools/1.0/jekyll/embed/python3#code='+encodeURIComponent(c.nodeValue.replace(/^\s+|\s+$/g,'')));});})(document)</script>
<iframe width="100%" height="400" frameborder="0" marginwidth="0" marginheight="0" class="lazyload" allowfullscreen>
</iframe>
<!--
import json
data = '''
[
{ "id" : "001",
"x" : "2",
"name" : "Chuck"
} ,
{ "id" : "009",
"x" : "7",
"name" : "Brent"
}
]'''
info = json.loads(data)
print('User count:', len(info))
for item in info:
print('Name', item['name'])
print('Id', item['id'])
print('Attribute', item['x'])
# Code: http://www.py4e.com/code3/json2.py
# Or select Download from this trinket's left-hand menu
-->
<p>If you compare the code to extract data from the parsed JSON and XML
you will see that what we get from <code>json.loads()</code> is a Python
list which we traverse with a <code>for</code> loop, and each item
within that list is a Python dictionary. Once the JSON has been parsed,
we can use the Python index operator to extract the various bits of data
for each user. We don’t have to use the JSON library to dig through the
parsed JSON, since the returned data is simply native Python
structures.</p>
<p>The output of this program is exactly the same as the XML version
above.</p>
<pre><code>User count: 2
Name Chuck
Id 001
Attribute 2
Name Brent
Id 009
Attribute 7</code></pre>
<p>In general, there is an industry trend away from XML and towards JSON
for web services. Because the JSON is simpler and more directly maps to
native data structures we already have in programming languages, the
parsing and data extraction code is usually simpler and more direct when
using JSON. But XML is more self-descriptive than JSON and so there are
some applications where XML retains an advantage. For example, most word
processors store documents internally using XML rather than JSON.</p>
<h2 id="application-programming-interfaces">Application Programming
Interfaces</h2>
<p>We now have the ability to exchange data between applications using
HyperText Transport Protocol (HTTP) and a way to represent complex data
that we are sending back and forth between these applications using
eXtensible Markup Language (XML) or JavaScript Object Notation
(JSON).</p>
<p>The next step is to begin to define and document “contracts” between
applications using these techniques. The general name for these
application-to-application contracts is <em>Application Program
Interfaces</em> (APIs). When we use an API, generally one program makes
a set of <em>services</em> available for use by other applications and
publishes the APIs (i.e., the “rules”) that must be followed to access
the services provided by the program.</p>
<p>When we begin to build our programs where the functionality of our
program includes access to services provided by other programs, we call
the approach a <em>Service-oriented architecture</em> (SOA). A SOA
approach is one where our overall application makes use of the services
of other applications. A non-SOA approach is where the application is a
single standalone application which contains all of the code necessary
to implement the application.</p>
<p>We see many examples of SOA when we use the web. We can go to a
single web site and book air travel, hotels, and automobiles all from a
single site. The data for hotels is not stored on the airline computers.
Instead, the airline computers contact the services on the hotel
computers and retrieve the hotel data and present it to the user. When
the user agrees to make a hotel reservation using the airline site, the
airline site uses another web service on the hotel systems to actually
make the reservation. And when it comes time to charge your credit card
for the whole transaction, still other computers become involved in the
process.</p>
<figure>
<img src="../images/soa.svg" alt="Service-oriented architecture" style="height: 3.0in;"/>
<figcaption>
Service-oriented architecture
</figcaption>
</figure>
<p>A Service-oriented architecture has many advantages, including: (1)
we always maintain only one copy of data (this is particularly important
for things like hotel reservations where we do not want to over-commit)
and (2) the owners of the data can set the rules about the use of their
data. With these advantages, an SOA system must be carefully designed to
have good performance and meet the user’s needs.</p>
<p>When an application makes a set of services in its API available over
the web, we call these <em>web services</em>.</p>
<h2 id="security-and-api-usage">Security and API usage</h2>
<p> </p>
<p>It is quite common that you need an API key to make use of a vendor’s
API. The general idea is that they want to know who is using their
services and how much each user is using. Perhaps they have free and pay
tiers of their services or have a policy that limits the number of
requests that a single individual can make during a particular time
period.</p>
<p>Sometimes once you get your API key, you simply include the key as
part of POST data or perhaps as a parameter on the URL when calling the
API.</p>
<p>Other times, the vendor wants increased assurance of the source of
the requests and so they expect you to send cryptographically signed
messages using shared keys and secrets. A very common technology that is
used to sign requests over the Internet is called <em>OAuth</em>. You
can read more about the OAuth protocol at <a
href="http://www.oauth.net">www.oauth.net</a>.</p>
<p>Thankfully there are a number of convenient and free OAuth libraries
so you can avoid writing an OAuth implementation from scratch by reading
the specification. These libraries are of varying complexity and have
varying degrees of richness. The OAuth web site has information about
various OAuth libraries.</p>
<h2 id="glossary">Glossary</h2>
<dl>
<dt>API</dt>
<dd>
Application Program Interface - A contract between applications that
defines the patterns of interaction between two application components.
</dd>
<dt>ElementTree</dt>
<dd>
A built-in Python library used to parse XML data.
</dd>
<dt>JSON</dt>
<dd>
JavaScript Object Notation - A format that allows for the markup of
structured data based on the syntax of JavaScript Objects.
</dd>
<dt>SOA</dt>
<dd>
Service-Oriented Architecture - When an application is made of
components connected across a network.
</dd>
<dt>XML</dt>
<dd>
eXtensible Markup Language - A format that allows for the markup of
structured data.
</dd>
</dl>
<h2 id="application-1-google-geocoding-web-service">Application 1:
Google geocoding web service</h2>
<p> </p>
<p>Google has an excellent web service that allows us to make use of
their large database of geographic information. We can submit a
geographical search string like “Ann Arbor, MI” to their geocoding API
and have Google return its best guess as to where on a map we might find
our search string and tell us about the landmarks nearby.</p>
<p>The geocoding service is free but rate limited so you cannot make
unlimited use of the API in a commercial application. But if you have
some survey data where an end user has entered a location in a
free-format input box, you can use this API to clean up your data quite
nicely.</p>
<p><em>When you are using a free API like Google’s geocoding API, you
need to be respectful in your use of these resources. If too many people
abuse the service, Google might drop or significantly curtail its free
service.</em></p>
<p></p>
<p>You can read the online documentation for this service, but it is
quite simple and you can even test it using a browser by typing the
following URL into your browser:</p>
<p><a
href="http://maps.googleapis.com/maps/api/geocode/json?address=Ann+Arbor%2C+MI">http://maps.googleapis.com/maps/api/geocode/json?address=Ann+Arbor%2C+MI</a></p>
<p>Make sure to unwrap the URL and remove any spaces from the URL before
pasting it into your browser.</p>
<p>The following is a simple application to prompt the user for a search
string, call the Google geocoding API, and extract information from the
returned JSON.</p>
<script type="text/javascript">(function(d,l,s,i,c){function n(e){e=e.nextSibling;return (!e||e.nodeType!=3)?e:n(e);};function r(f){/in/.test(d.readyState) ? setTimeout(function(){r(f);},9):f()};l=d.getElementsByTagName('script');s=l[l.length-1];r(function(){i=n(s),c=n(i);i.setAttribute('data-src','https://trinket.io/tools/1.0/jekyll/embed/python3#code='+encodeURIComponent(c.nodeValue.replace(/^\s+|\s+$/g,'')));});})(document)</script>
<iframe width="100%" height="400" frameborder="0" marginwidth="0" marginheight="0" class="lazyload" allowfullscreen>
</iframe>
<!--
import urllib.request, urllib.parse, urllib.error
import json
import ssl
api_key = False
# If you have a Google Places API key, enter it here
# api_key = 'AIzaSy___IDByT70'
# https://developers.google.com/maps/documentation/geocoding/intro
if api_key is False:
api_key = 42
serviceurl = 'http://py4e-data.dr-chuck.net/json?'
else :
serviceurl = 'https://maps.googleapis.com/maps/api/geocode/json?'
# Ignore SSL certificate errors
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
while True:
address = input('Enter location: ')
if len(address) < 1: break
parms = dict()
parms['address'] = address
if api_key is not False: parms['key'] = api_key
url = serviceurl + urllib.parse.urlencode(parms)
print('Retrieving', url)
uh = urllib.request.urlopen(url, context=ctx)
data = uh.read().decode()
print('Retrieved', len(data), 'characters')
try:
js = json.loads(data)
except:
js = None
if not js or 'status' not in js or js['status'] != 'OK':
print('==== Failure To Retrieve ====')
print(data)
continue
print(json.dumps(js, indent=4))
lat = js['results'][0]['geometry']['location']['lat']
lng = js['results'][0]['geometry']['location']['lng']
print('lat', lat, 'lng', lng)
location = js['results'][0]['formatted_address']
print(location)
# Code: http://www.py4e.com/code3/geojson.py
# Or select Download from this trinket's left-hand menu
-->
<p>The program takes the search string and constructs a URL with the
search string as a properly encoded parameter and then uses
<code>urllib</code> to retrieve the text from the Google geocoding API.
Unlike a fixed web page, the data we get depends on the parameters we
send and the geographical data stored in Google’s servers.</p>
<p>Once we retrieve the JSON data, we parse it with the
<code>json</code> library and do a few checks to make sure that we
received good data, then extract the information that we are looking
for.</p>
<p>The output of the program is as follows (some of the returned JSON
has been removed):</p>
<pre><code>$ python3 geojson.py
Enter location: Ann Arbor, MI
Retrieving http://py4e-data.dr-chuck.net/json?address=Ann+Arbor%2C+MI&key=42
Retrieved 1736 characters</code></pre>
<div class="sourceCode" id="cb9"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="fu">{</span></span>
<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">"results"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb9-3"><a href="#cb9-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb9-4"><a href="#cb9-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">"address_components"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb9-5"><a href="#cb9-5" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb9-6"><a href="#cb9-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">"long_name"</span><span class="fu">:</span> <span class="st">"Ann Arbor"</span><span class="fu">,</span></span>
<span id="cb9-7"><a href="#cb9-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">"short_name"</span><span class="fu">:</span> <span class="st">"Ann Arbor"</span><span class="fu">,</span></span>
<span id="cb9-8"><a href="#cb9-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">"types"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb9-9"><a href="#cb9-9" aria-hidden="true" tabindex="-1"></a> <span class="st">"locality"</span><span class="ot">,</span></span>
<span id="cb9-10"><a href="#cb9-10" aria-hidden="true" tabindex="-1"></a> <span class="st">"political"</span></span>
<span id="cb9-11"><a href="#cb9-11" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span></span>
<span id="cb9-12"><a href="#cb9-12" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span><span class="ot">,</span></span>
<span id="cb9-13"><a href="#cb9-13" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb9-14"><a href="#cb9-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">"long_name"</span><span class="fu">:</span> <span class="st">"Washtenaw County"</span><span class="fu">,</span></span>
<span id="cb9-15"><a href="#cb9-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">"short_name"</span><span class="fu">:</span> <span class="st">"Washtenaw County"</span><span class="fu">,</span></span>
<span id="cb9-16"><a href="#cb9-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">"types"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb9-17"><a href="#cb9-17" aria-hidden="true" tabindex="-1"></a> <span class="st">"administrative_area_level_2"</span><span class="ot">,</span></span>
<span id="cb9-18"><a href="#cb9-18" aria-hidden="true" tabindex="-1"></a> <span class="st">"political"</span></span>
<span id="cb9-19"><a href="#cb9-19" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span></span>
<span id="cb9-20"><a href="#cb9-20" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span><span class="ot">,</span></span>
<span id="cb9-21"><a href="#cb9-21" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb9-22"><a href="#cb9-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">"long_name"</span><span class="fu">:</span> <span class="st">"Michigan"</span><span class="fu">,</span></span>
<span id="cb9-23"><a href="#cb9-23" aria-hidden="true" tabindex="-1"></a> <span class="dt">"short_name"</span><span class="fu">:</span> <span class="st">"MI"</span><span class="fu">,</span></span>
<span id="cb9-24"><a href="#cb9-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">"types"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb9-25"><a href="#cb9-25" aria-hidden="true" tabindex="-1"></a> <span class="st">"administrative_area_level_1"</span><span class="ot">,</span></span>
<span id="cb9-26"><a href="#cb9-26" aria-hidden="true" tabindex="-1"></a> <span class="st">"political"</span></span>
<span id="cb9-27"><a href="#cb9-27" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span></span>
<span id="cb9-28"><a href="#cb9-28" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span><span class="ot">,</span></span>
<span id="cb9-29"><a href="#cb9-29" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb9-30"><a href="#cb9-30" aria-hidden="true" tabindex="-1"></a> <span class="dt">"long_name"</span><span class="fu">:</span> <span class="st">"United States"</span><span class="fu">,</span></span>
<span id="cb9-31"><a href="#cb9-31" aria-hidden="true" tabindex="-1"></a> <span class="dt">"short_name"</span><span class="fu">:</span> <span class="st">"US"</span><span class="fu">,</span></span>
<span id="cb9-32"><a href="#cb9-32" aria-hidden="true" tabindex="-1"></a> <span class="dt">"types"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb9-33"><a href="#cb9-33" aria-hidden="true" tabindex="-1"></a> <span class="st">"country"</span><span class="ot">,</span></span>
<span id="cb9-34"><a href="#cb9-34" aria-hidden="true" tabindex="-1"></a> <span class="st">"political"</span></span>
<span id="cb9-35"><a href="#cb9-35" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span></span>
<span id="cb9-36"><a href="#cb9-36" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb9-37"><a href="#cb9-37" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb9-38"><a href="#cb9-38" aria-hidden="true" tabindex="-1"></a> <span class="dt">"formatted_address"</span><span class="fu">:</span> <span class="st">"Ann Arbor, MI, USA"</span><span class="fu">,</span></span>
<span id="cb9-39"><a href="#cb9-39" aria-hidden="true" tabindex="-1"></a> <span class="dt">"geometry"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb9-40"><a href="#cb9-40" aria-hidden="true" tabindex="-1"></a> <span class="dt">"bounds"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb9-41"><a href="#cb9-41" aria-hidden="true" tabindex="-1"></a> <span class="dt">"northeast"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb9-42"><a href="#cb9-42" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lat"</span><span class="fu">:</span> <span class="fl">42.3239728</span><span class="fu">,</span></span>
<span id="cb9-43"><a href="#cb9-43" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lng"</span><span class="fu">:</span> <span class="fl">-83.6758069</span></span>
<span id="cb9-44"><a href="#cb9-44" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb9-45"><a href="#cb9-45" aria-hidden="true" tabindex="-1"></a> <span class="dt">"southwest"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb9-46"><a href="#cb9-46" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lat"</span><span class="fu">:</span> <span class="fl">42.222668</span><span class="fu">,</span></span>
<span id="cb9-47"><a href="#cb9-47" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lng"</span><span class="fu">:</span> <span class="fl">-83.799572</span></span>
<span id="cb9-48"><a href="#cb9-48" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb9-49"><a href="#cb9-49" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb9-50"><a href="#cb9-50" aria-hidden="true" tabindex="-1"></a> <span class="dt">"location"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb9-51"><a href="#cb9-51" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lat"</span><span class="fu">:</span> <span class="fl">42.2808256</span><span class="fu">,</span></span>
<span id="cb9-52"><a href="#cb9-52" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lng"</span><span class="fu">:</span> <span class="fl">-83.7430378</span></span>
<span id="cb9-53"><a href="#cb9-53" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb9-54"><a href="#cb9-54" aria-hidden="true" tabindex="-1"></a> <span class="dt">"location_type"</span><span class="fu">:</span> <span class="st">"APPROXIMATE"</span><span class="fu">,</span></span>
<span id="cb9-55"><a href="#cb9-55" aria-hidden="true" tabindex="-1"></a> <span class="dt">"viewport"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb9-56"><a href="#cb9-56" aria-hidden="true" tabindex="-1"></a> <span class="dt">"northeast"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb9-57"><a href="#cb9-57" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lat"</span><span class="fu">:</span> <span class="fl">42.3239728</span><span class="fu">,</span></span>
<span id="cb9-58"><a href="#cb9-58" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lng"</span><span class="fu">:</span> <span class="fl">-83.6758069</span></span>
<span id="cb9-59"><a href="#cb9-59" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb9-60"><a href="#cb9-60" aria-hidden="true" tabindex="-1"></a> <span class="dt">"southwest"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb9-61"><a href="#cb9-61" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lat"</span><span class="fu">:</span> <span class="fl">42.222668</span><span class="fu">,</span></span>
<span id="cb9-62"><a href="#cb9-62" aria-hidden="true" tabindex="-1"></a> <span class="dt">"lng"</span><span class="fu">:</span> <span class="fl">-83.799572</span></span>
<span id="cb9-63"><a href="#cb9-63" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb9-64"><a href="#cb9-64" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb9-65"><a href="#cb9-65" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb9-66"><a href="#cb9-66" aria-hidden="true" tabindex="-1"></a> <span class="dt">"place_id"</span><span class="fu">:</span> <span class="st">"ChIJMx9D1A2wPIgR4rXIhkb5Cds"</span><span class="fu">,</span></span>
<span id="cb9-67"><a href="#cb9-67" aria-hidden="true" tabindex="-1"></a> <span class="dt">"types"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb9-68"><a href="#cb9-68" aria-hidden="true" tabindex="-1"></a> <span class="st">"locality"</span><span class="ot">,</span></span>
<span id="cb9-69"><a href="#cb9-69" aria-hidden="true" tabindex="-1"></a> <span class="st">"political"</span></span>
<span id="cb9-70"><a href="#cb9-70" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span></span>
<span id="cb9-71"><a href="#cb9-71" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb9-72"><a href="#cb9-72" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb9-73"><a href="#cb9-73" aria-hidden="true" tabindex="-1"></a> <span class="dt">"status"</span><span class="fu">:</span> <span class="st">"OK"</span></span>
<span id="cb9-74"><a href="#cb9-74" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span>
<span id="cb9-75"><a href="#cb9-75" aria-hidden="true" tabindex="-1"></a><span class="er">lat</span> <span class="er">42.2808256</span> <span class="er">lng</span> <span class="er">-83.7430378</span></span>
<span id="cb9-76"><a href="#cb9-76" aria-hidden="true" tabindex="-1"></a><span class="er">Ann</span> <span class="er">Arbor,</span> <span class="er">MI,</span> <span class="er">USA</span></span></code></pre></div>
<pre><code>Enter location:</code></pre>
<p>You can download <a
href="http://www.py4e.com/code3/geoxml.py">www.py4e.com/code3/geoxml.py</a>
to explore the XML variant of the Google geocoding API.</p>
<p><strong>Exercise 1: Change either</strong> <a
href="http://www.py4e.com/code3/geojson.py"><strong>geojson.py</strong></a>
<strong>or</strong> <a
href="http://www.py4e.com/code3/geoxml.py"><strong>geoxml.py</strong></a>
<strong>to print out the two-character country code from the retrieved
data. Add error checking so your program does not traceback if the
country code is not there. Once you have it working, search for
“Atlantic Ocean” and make sure it can handle locations that are not in
any country.</strong></p>
<h2 id="application-2-twitter">Application 2: Twitter</h2>
<p>As the Twitter API became increasingly valuable, Twitter went from an
open and public API to an API that required the use of OAuth signatures
on each API request.</p>
<p>For this next sample program, download the files <em>twurl.py</em>,
<em>hidden.py</em>, <em>oauth.py</em>, and <em>twitter1.py</em> from <a
href="http://www.py4e.com/code3">www.py4e.com/code</a> and put them all
in a folder on your computer.</p>
<p>To make use of these programs you will need to have a Twitter
account, and authorize your Python code as an application, set up a key,
secret, token and token secret. You will edit the file
<em>hidden.py</em> and put these four strings into the appropriate
variables in the file:</p>
<script type="text/javascript">(function(d,l,s,i,c){function n(e){e=e.nextSibling;return (!e||e.nodeType!=3)?e:n(e);};function r(f){/in/.test(d.readyState) ? setTimeout(function(){r(f);},9):f()};l=d.getElementsByTagName('script');s=l[l.length-1];r(function(){i=n(s),c=n(i);i.setAttribute('data-src','https://trinket.io/tools/1.0/jekyll/embed/python3#code='+encodeURIComponent(c.nodeValue.replace(/^\s+|\s+$/g,'')));});})(document)</script>
<iframe width="100%" height="400" frameborder="0" marginwidth="0" marginheight="0" class="lazyload" allowfullscreen>
</iframe>
<!--
# Keep this file separate
# https://apps.twitter.com/
# Create new App and get the four strings
def oauth():
return {"consumer_key": "h7Lu...Ng",
"consumer_secret": "dNKenAC3New...mmn7Q",
"token_key": "10185562-eibxCp9n2...P4GEQQOSGI",
"token_secret": "H0ycCFemmC4wyf1...qoIpBo"}
# Code: http://www.py4e.com/code3/hidden.py
# Or select Download from this trinket's left-hand menu
-->
<p>The Twitter web service are accessed using a URL like this:</p>
<p><a href="https://api.twitter.com/1.1/statuses/user_timeline.json"
class="uri">https://api.twitter.com/1.1/statuses/user_timeline.json</a></p>
<p>But once all of the security information has been added, the URL will
look more like:</p>
<pre><code>https://api.twitter.com/1.1/statuses/user_timeline.json?count=2
&oauth_version=1.0&oauth_token=101...SGI&screen_name=drchuck
&oauth_nonce=09239679&oauth_timestamp=1380395644
&oauth_signature=rLK...BoD&oauth_consumer_key=h7Lu...GNg
&oauth_signature_method=HMAC-SHA1</code></pre>
<p>You can read the OAuth specification if you want to know more about
the meaning of the various parameters that are added to meet the
security requirements of OAuth.</p>
<p>For the programs we run with Twitter, we hide all the complexity in
the files <em>oauth.py</em> and <em>twurl.py</em>. We simply set the
secrets in <em>hidden.py</em> and then send the desired URL to the
<em>twurl.augment()</em> function and the library code adds all the
necessary parameters to the URL for us.</p>
<p>This program retrieves the timeline for a particular Twitter user and
returns it to us in JSON format in a string. We simply print the first
250 characters of the string:</p>
<script type="text/javascript">(function(d,l,s,i,c){function n(e){e=e.nextSibling;return (!e||e.nodeType!=3)?e:n(e);};function r(f){/in/.test(d.readyState) ? setTimeout(function(){r(f);},9):f()};l=d.getElementsByTagName('script');s=l[l.length-1];r(function(){i=n(s),c=n(i);i.setAttribute('data-src','https://trinket.io/tools/1.0/jekyll/embed/python3#code='+encodeURIComponent(c.nodeValue.replace(/^\s+|\s+$/g,'')));});})(document)</script>
<iframe width="100%" height="400" frameborder="0" marginwidth="0" marginheight="0" class="lazyload" allowfullscreen>
</iframe>
<!--
import urllib.request, urllib.parse, urllib.error
import twurl
import ssl
# https://apps.twitter.com/
# Create App and get the four strings, put them in hidden.py
TWITTER_URL = 'https://api.twitter.com/1.1/statuses/user_timeline.json'
# Ignore SSL certificate errors
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
while True:
print('')
acct = input('Enter Twitter Account:')
if (len(acct) < 1): break
url = twurl.augment(TWITTER_URL,
{'screen_name': acct, 'count': '2'})
print('Retrieving', url)
connection = urllib.request.urlopen(url, context=ctx)
data = connection.read().decode()
print(data[:250])
headers = dict(connection.getheaders())
# print headers
print('Remaining', headers['x-rate-limit-remaining'])
# Code: http://www.py4e.com/code3/twitter1.py
# Or select Download from this trinket's left-hand menu
----{twurl.py}----
import urllib.request, urllib.parse, urllib.error
import oauth
import hidden
# https://apps.twitter.com/
# Create App and get the four strings, put them in hidden.py
def augment(url, parameters):
secrets = hidden.oauth()
consumer = oauth.OAuthConsumer(secrets['consumer_key'],
secrets['consumer_secret'])
token = oauth.OAuthToken(secrets['token_key'], secrets['token_secret'])
oauth_request = oauth.OAuthRequest.from_consumer_and_token(consumer,
token=token, http_method='GET', http_url=url,
parameters=parameters)
oauth_request.sign_request(oauth.OAuthSignatureMethod_HMAC_SHA1(),
consumer, token)
return oauth_request.to_url()
def test_me():
print('* Calling Twitter...')
url = augment('https://api.twitter.com/1.1/statuses/user_timeline.json',
{'screen_name': 'drchuck', 'count': '2'})
print(url)
connection = urllib.request.urlopen(url)
data = connection.read()
print(data)
headers = dict(connection.getheaders())
print(headers)
-->
<p>When the program runs it produces the following output:</p>
<pre><code>Enter Twitter Account:drchuck
Retrieving https://api.twitter.com/1.1/ ...
[{"created_at":"Sat Sep 28 17:30:25 +0000 2013","
id":384007200990982144,"id_str":"384007200990982144",
"text":"RT @fixpert: See how the Dutch handle traffic
intersections: http:\/\/t.co\/tIiVWtEhj4\n#brilliant",
"source":"web","truncated":false,"in_rep
Remaining 178
Enter Twitter Account:fixpert
Retrieving https://api.twitter.com/1.1/ ...
[{"created_at":"Sat Sep 28 18:03:56 +0000 2013",
"id":384015634108919808,"id_str":"384015634108919808",
"text":"3 months after my freak bocce ball accident,
my wedding ring fits again! :)\n\nhttps:\/\/t.co\/2XmHPx7kgX",
"source":"web","truncated":false,
Remaining 177
Enter Twitter Account:</code></pre>
<p>Along with the returned timeline data, Twitter also returns metadata
about the request in the HTTP response headers. One header in
particular, <code>x-rate-limit-remaining</code>, informs us how many
more requests we can make before we will be shut off for a short time
period. You can see that our remaining retrievals drop by one each time
we make a request to the API.</p>
<p>In the following example, we retrieve a user’s Twitter friends, parse
the returned JSON, and extract some of the information about the
friends. We also dump the JSON after parsing and “pretty-print” it with
an indent of four characters to allow us to pore through the data when
we want to extract more fields.</p>
<script type="text/javascript">(function(d,l,s,i,c){function n(e){e=e.nextSibling;return (!e||e.nodeType!=3)?e:n(e);};function r(f){/in/.test(d.readyState) ? setTimeout(function(){r(f);},9):f()};l=d.getElementsByTagName('script');s=l[l.length-1];r(function(){i=n(s),c=n(i);i.setAttribute('data-src','https://trinket.io/tools/1.0/jekyll/embed/python3#code='+encodeURIComponent(c.nodeValue.replace(/^\s+|\s+$/g,'')));});})(document)</script>
<iframe width="100%" height="400" frameborder="0" marginwidth="0" marginheight="0" class="lazyload" allowfullscreen>
</iframe>
<!--
import urllib.request, urllib.parse, urllib.error
import twurl
import json
import ssl
# https://apps.twitter.com/
# Create App and get the four strings, put them in hidden.py
TWITTER_URL = 'https://api.twitter.com/1.1/friends/list.json'
# Ignore SSL certificate errors
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
while True:
print('')
acct = input('Enter Twitter Account:')
if (len(acct) < 1): break
url = twurl.augment(TWITTER_URL,
{'screen_name': acct, 'count': '5'})
print('Retrieving', url)
connection = urllib.request.urlopen(url, context=ctx)
data = connection.read().decode()
js = json.loads(data)
print(json.dumps(js, indent=2))
headers = dict(connection.getheaders())
print('Remaining', headers['x-rate-limit-remaining'])
for u in js['users']:
print(u['screen_name'])
if 'status' not in u:
print(' * No status found')
continue
s = u['status']['text']
print(' ', s[:50])
# Code: http://www.py4e.com/code3/twitter2.py
# Or select Download from this trinket's left-hand menu
----{twurl.py}----
import urllib.request, urllib.parse, urllib.error
import oauth
import hidden
# https://apps.twitter.com/
# Create App and get the four strings, put them in hidden.py
def augment(url, parameters):
secrets = hidden.oauth()
consumer = oauth.OAuthConsumer(secrets['consumer_key'],
secrets['consumer_secret'])
token = oauth.OAuthToken(secrets['token_key'], secrets['token_secret'])
oauth_request = oauth.OAuthRequest.from_consumer_and_token(consumer,
token=token, http_method='GET', http_url=url,
parameters=parameters)
oauth_request.sign_request(oauth.OAuthSignatureMethod_HMAC_SHA1(),
consumer, token)
return oauth_request.to_url()
def test_me():
print('* Calling Twitter...')
url = augment('https://api.twitter.com/1.1/statuses/user_timeline.json',
{'screen_name': 'drchuck', 'count': '2'})
print(url)
connection = urllib.request.urlopen(url)
data = connection.read()
print(data)
headers = dict(connection.getheaders())
print(headers)
-->
<p>Since the JSON becomes a set of nested Python lists and dictionaries,
we can use a combination of the index operation and <code>for</code>
loops to wander through the returned data structures with very little
Python code.</p>
<p>The output of the program looks as follows (some of the data items
are shortened to fit on the page):</p>
<pre><code>Enter Twitter Account:drchuck
Retrieving https://api.twitter.com/1.1/friends ...
Remaining 14</code></pre>
<div class="sourceCode" id="cb14"><pre
class="sourceCode json"><code class="sourceCode json"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="fu">{</span></span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">"next_cursor"</span><span class="fu">:</span> <span class="dv">1444171224491980205</span><span class="fu">,</span></span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">"users"</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">"id"</span><span class="fu">:</span> <span class="dv">662433</span><span class="fu">,</span></span>
<span id="cb14-6"><a href="#cb14-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">"followers_count"</span><span class="fu">:</span> <span class="dv">28725</span><span class="fu">,</span></span>
<span id="cb14-7"><a href="#cb14-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">"status"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb14-8"><a href="#cb14-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">"text"</span><span class="fu">:</span> <span class="st">"@jazzychad I just bought one .__."</span><span class="fu">,</span></span>
<span id="cb14-9"><a href="#cb14-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">"created_at"</span><span class="fu">:</span> <span class="st">"Fri Sep 20 08:36:34 +0000 2013"</span><span class="fu">,</span></span>
<span id="cb14-10"><a href="#cb14-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">"retweeted"</span><span class="fu">:</span> <span class="kw">false</span><span class="fu">,</span></span>
<span id="cb14-11"><a href="#cb14-11" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb14-12"><a href="#cb14-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">"location"</span><span class="fu">:</span> <span class="st">"San Francisco, California"</span><span class="fu">,</span></span>
<span id="cb14-13"><a href="#cb14-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">"screen_name"</span><span class="fu">:</span> <span class="st">"leahculver"</span><span class="fu">,</span></span>
<span id="cb14-14"><a href="#cb14-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">"name"</span><span class="fu">:</span> <span class="st">"Leah Culver"</span><span class="fu">,</span></span>
<span id="cb14-15"><a href="#cb14-15" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span><span class="ot">,</span></span>
<span id="cb14-16"><a href="#cb14-16" aria-hidden="true" tabindex="-1"></a> <span class="fu">{</span></span>
<span id="cb14-17"><a href="#cb14-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">"id"</span><span class="fu">:</span> <span class="dv">40426722</span><span class="fu">,</span></span>
<span id="cb14-18"><a href="#cb14-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">"followers_count"</span><span class="fu">:</span> <span class="dv">2635</span><span class="fu">,</span></span>
<span id="cb14-19"><a href="#cb14-19" aria-hidden="true" tabindex="-1"></a> <span class="dt">"status"</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb14-20"><a href="#cb14-20" aria-hidden="true" tabindex="-1"></a> <span class="dt">"text"</span><span class="fu">:</span> <span class="st">"RT @WSJ: Big employers like Google ..."</span><span class="fu">,</span></span>
<span id="cb14-21"><a href="#cb14-21" aria-hidden="true" tabindex="-1"></a> <span class="dt">"created_at"</span><span class="fu">:</span> <span class="st">"Sat Sep 28 19:36:37 +0000 2013"</span><span class="fu">,</span></span>
<span id="cb14-22"><a href="#cb14-22" aria-hidden="true" tabindex="-1"></a> <span class="fu">},</span></span>
<span id="cb14-23"><a href="#cb14-23" aria-hidden="true" tabindex="-1"></a> <span class="dt">"location"</span><span class="fu">:</span> <span class="st">"Victoria Canada"</span><span class="fu">,</span></span>
<span id="cb14-24"><a href="#cb14-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">"screen_name"</span><span class="fu">:</span> <span class="st">"_valeriei"</span><span class="fu">,</span></span>
<span id="cb14-25"><a href="#cb14-25" aria-hidden="true" tabindex="-1"></a> <span class="dt">"name"</span><span class="fu">:</span> <span class="st">"Valerie Irvine"</span><span class="fu">,</span></span>
<span id="cb14-26"><a href="#cb14-26" aria-hidden="true" tabindex="-1"></a> <span class="fu">}</span></span>
<span id="cb14-27"><a href="#cb14-27" aria-hidden="true" tabindex="-1"></a> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb14-28"><a href="#cb14-28" aria-hidden="true" tabindex="-1"></a> <span class="dt">"next_cursor_str"</span><span class="fu">:</span> <span class="st">"1444171224491980205"</span></span>
<span id="cb14-29"><a href="#cb14-29" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span></code></pre></div>
<pre><code>leahculver
@jazzychad I just bought one .__.
_valeriei
RT @WSJ: Big employers like Google, AT&amp;T are h
ericbollens
RT @lukew: sneak peek: my LONG take on the good &a
halherzog
Learning Objects is 10. We had a cake with the LO,
scweeker
@DeviceLabDC love it! Now where so I get that "etc
Enter Twitter Account:</code></pre>
<p>The last bit of the output is where we see the for loop reading the
five most recent “friends” of the <em><span class="citation"
data-cites="drchuck">@drchuck</span></em> Twitter account and printing
the most recent status for each friend. There is a great deal more data
available in the returned JSON. If you look in the output of the
program, you can also see that the “find the friends” of a particular
account has a different rate limitation than the number of timeline
queries we are allowed to run per time period.</p>
<p>These secure API keys allow Twitter to have solid confidence that
they know who is using their API and data and at what level. The
rate-limiting approach allows us to do simple, personal data retrievals
but does not allow us to build a product that pulls data from their API
millions of times per day.</p>
</div>
</div>
</div>
</div>
<script type="text/javascript" src="trinket/jquery.min.js"></script>
<script type="text/javascript" src="trinket/jquery-ui.min.js"></script>
<script type="text/javascript" src="trinket/foundation.min.js"></script>
<script type="text/javascript" src="trinket/anchor.min.js"></script>